2024-05-09T09:04:19.546 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T09:04:19.546 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/norstats}} --owner scheduled_teuthology@teuthology --verbose --name teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722 -- /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722/orig.config.yaml 2024-05-09T09:04:19.602 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T09:04:19.710 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/norstats}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7698722' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client dirsize rbytes: false client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc kclient: client.0: mount_subvol_num: 1 rbytes: false selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2334 sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc sleep_before_teardown: 0 subset: 359/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1TJHp4AedreEpQ+dOSfZdyGd4aXHtFCDSVCtTuhXw7olxvWb8TC5aTdfwox35pwIE+Y6T85DSue7DaNbPxGI8= smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHvXKNfELxEaXWsuG9xISf2pHclJSxFBQ7tF5E/7lU0Ypaj6fc14+XT3pAywBK9gnXd2a5nMZUxXM+24W0nF7nw= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPW37s12eToJm3SycTwe9gebNzwEnenw1GzxxAJcDnYTrkVhn3uCO+ViIhjnZT9yHVXUR48RwkVCgDJiaGSGwO8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/norstats teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_21:24:04 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T09:04:19.711 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa; will attempt to use it 2024-05-09T09:04:19.711 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks 2024-05-09T09:04:19.711 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T09:04:19.713 INFO:teuthology.task.internal:Checking packages... 2024-05-09T09:04:19.737 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc' 2024-05-09T09:04:19.737 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T09:04:19.737 INFO:teuthology.packaging:ref: None 2024-05-09T09:04:19.737 INFO:teuthology.packaging:tag: None 2024-05-09T09:04:19.738 INFO:teuthology.packaging:branch: squid 2024-05-09T09:04:19.738 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T09:04:19.738 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-09T09:04:20.060 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2529.gbf77c77e 2024-05-09T09:04:20.062 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T09:04:20.068 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T09:04:20.068 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T09:04:20.083 INFO:teuthology.task.internal:Saving configuration 2024-05-09T09:04:20.103 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T09:04:20.110 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T09:04:20.137 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 08:59:30.481104', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1TJHp4AedreEpQ+dOSfZdyGd4aXHtFCDSVCtTuhXw7olxvWb8TC5aTdfwox35pwIE+Y6T85DSue7DaNbPxGI8='} 2024-05-09T09:04:20.160 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 08:59:30.479658', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHvXKNfELxEaXWsuG9xISf2pHclJSxFBQ7tF5E/7lU0Ypaj6fc14+XT3pAywBK9gnXd2a5nMZUxXM+24W0nF7nw='} 2024-05-09T09:04:20.181 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 08:59:30.482286', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPW37s12eToJm3SycTwe9gebNzwEnenw1GzxxAJcDnYTrkVhn3uCO+ViIhjnZT9yHVXUR48RwkVCgDJiaGSGwO8='} 2024-05-09T09:04:20.181 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T09:04:20.188 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-09T09:04:20.188 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-09T09:04:20.188 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-09T09:04:20.188 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T09:04:20.294 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f010663b160>, signals=[15]) 2024-05-09T09:04:20.294 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T09:04:20.301 INFO:teuthology.task.internal:Opening connections... 2024-05-09T09:04:20.301 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-05-09T09:04:20.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T09:04:20.393 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-05-09T09:04:20.393 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T09:04:20.468 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-05-09T09:04:20.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T09:04:20.545 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T09:04:20.552 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-05-09T09:04:20.571 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-05-09T09:04:20.571 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-05-09T09:04:20.629 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="CentOS Stream" 2024-05-09T09:04:20.629 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="9" 2024-05-09T09:04:20.629 INFO:teuthology.orchestra.run.smithi012.stdout:ID="centos" 2024-05-09T09:04:20.629 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="rhel fedora" 2024-05-09T09:04:20.629 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="9" 2024-05-09T09:04:20.629 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el9" 2024-05-09T09:04:20.629 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T09:04:20.629 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2024-05-09T09:04:20.629 INFO:teuthology.orchestra.run.smithi012.stdout:LOGO="fedora-logo-icon" 2024-05-09T09:04:20.629 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T09:04:20.629 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://centos.org/" 2024-05-09T09:04:20.629 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T09:04:20.629 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T09:04:20.630 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T09:04:20.630 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-05-09T09:04:20.651 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-05-09T09:04:20.669 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-05-09T09:04:20.669 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-05-09T09:04:20.728 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-05-09T09:04:20.728 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-05-09T09:04:20.728 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-05-09T09:04:20.728 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-05-09T09:04:20.728 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-05-09T09:04:20.728 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-05-09T09:04:20.728 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T09:04:20.728 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-05-09T09:04:20.728 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-05-09T09:04:20.728 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T09:04:20.729 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-05-09T09:04:20.729 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T09:04:20.729 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T09:04:20.729 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T09:04:20.729 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-05-09T09:04:20.750 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-05-09T09:04:20.768 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-05-09T09:04:20.769 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-05-09T09:04:20.826 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2024-05-09T09:04:20.827 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="9" 2024-05-09T09:04:20.827 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2024-05-09T09:04:20.827 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2024-05-09T09:04:20.827 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="9" 2024-05-09T09:04:20.827 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el9" 2024-05-09T09:04:20.827 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T09:04:20.827 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-05-09T09:04:20.827 INFO:teuthology.orchestra.run.smithi170.stdout:LOGO="fedora-logo-icon" 2024-05-09T09:04:20.827 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T09:04:20.828 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2024-05-09T09:04:20.828 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T09:04:20.828 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T09:04:20.828 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T09:04:20.828 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-05-09T09:04:20.850 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T09:04:20.860 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T09:04:20.865 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T09:04:20.865 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-05-09T09:04:20.868 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-05-09T09:04:20.870 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-05-09T09:04:20.887 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T09:04:20.894 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T09:04:20.894 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-05-09T09:04:20.927 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-05-09T09:04:20.929 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-05-09T09:04:21.037 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T09:04:21.113 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T09:04:21.127 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-09T09:04:21.127 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-09T09:04:21.127 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T09:04:21.128 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T09:04:21.128 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T09:04:21.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:04:21.145 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-05-09T09:04:21.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:04:21.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:04:21.147 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-05-09T09:04:21.147 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-05-09T09:04:21.201 INFO:teuthology.orchestra.run.smithi170.stdout:5.14.0-437.el9.x86_64 2024-05-09T09:04:21.202 INFO:teuthology.task.kernel:Running kernel on smithi170: 5.14.0-437.el9.x86_64 2024-05-09T09:04:21.202 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum install -y kernel 2024-05-09T09:04:21.205 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-437.el9.x86_64 2024-05-09T09:04:21.205 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-437.el9.x86_64 2024-05-09T09:04:21.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-05-09T09:04:21.206 INFO:teuthology.orchestra.run.smithi012.stdout:5.14.0-437.el9.x86_64 2024-05-09T09:04:21.206 INFO:teuthology.task.kernel:Running kernel on smithi012: 5.14.0-437.el9.x86_64 2024-05-09T09:04:21.207 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum install -y kernel 2024-05-09T09:04:23.860 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.1 MB 00:01 2024-05-09T09:04:24.985 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.1 MB 00:02 2024-05-09T09:04:25.764 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.1 MB 00:02 2024-05-09T09:04:25.819 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-09T09:04:27.270 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-05-09T09:04:29.448 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 19 MB 00:02 2024-05-09T09:04:32.370 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.2 MB 00:02 2024-05-09T09:04:32.645 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-05-09T09:04:34.087 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-05-09T09:04:34.137 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-05-09T09:04:34.406 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.2 MB 00:01 2024-05-09T09:04:35.266 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-05-09T09:04:36.199 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 16 kB 00:00 2024-05-09T09:04:38.738 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 4.7 MB/s | 21 MB 00:04 2024-05-09T09:04:40.679 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-09T09:04:40.911 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 4.6 MB/s | 21 MB 00:04 2024-05-09T09:04:42.802 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T09:04:42.802 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T09:04:42.895 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-05-09T09:04:42.902 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-05-09T09:04:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: Package Architecture Version Repository Size 2024-05-09T09:04:42.902 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-05-09T09:04:42.903 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-05-09T09:04:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T09:04:42.903 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-05-09T09:04:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T09:04:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T09:04:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T09:04:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:04:42.903 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-05-09T09:04:42.903 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-05-09T09:04:42.903 INFO:teuthology.orchestra.run.smithi012.stdout:Install 4 Packages 2024-05-09T09:04:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:04:42.904 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 78 M 2024-05-09T09:04:42.904 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 123 M 2024-05-09T09:04:42.904 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-05-09T09:04:43.339 INFO:teuthology.orchestra.run.smithi012.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 3.0 MB/s | 484 kB 00:00 2024-05-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-09T09:04:44.239 INFO:teuthology.orchestra.run.smithi012.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 15 MB/s | 15 MB 00:01 2024-05-09T09:04:44.764 INFO:teuthology.orchestra.run.smithi012.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 20 MB/s | 28 MB 00:01 2024-05-09T09:04:45.139 INFO:teuthology.orchestra.run.smithi012.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 17 MB/s | 34 MB 00:01 2024-05-09T09:04:45.140 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-05-09T09:04:45.140 INFO:teuthology.orchestra.run.smithi012.stdout:Total 35 MB/s | 78 MB 00:02 2024-05-09T09:04:45.852 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-05-09T09:04:46.116 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-09T09:04:46.160 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T09:04:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T09:04:46.268 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-05-09T09:04:46.276 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-05-09T09:04:46.276 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-05-09T09:04:46.276 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-05-09T09:04:46.276 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-05-09T09:04:46.276 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T09:04:46.276 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-05-09T09:04:46.276 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T09:04:46.277 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T09:04:46.277 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T09:04:46.277 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:04:46.277 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-05-09T09:04:46.277 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-05-09T09:04:46.277 INFO:teuthology.orchestra.run.smithi016.stdout:Install 4 Packages 2024-05-09T09:04:46.277 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:04:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 78 M 2024-05-09T09:04:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 123 M 2024-05-09T09:04:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-05-09T09:04:46.278 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-05-09T09:04:46.278 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-05-09T09:04:46.789 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.1 MB/s | 484 kB 00:00 2024-05-09T09:04:47.246 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-05-09T09:04:47.246 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-05-09T09:04:48.156 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T09:04:48.156 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T09:04:48.263 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-05-09T09:04:48.271 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-05-09T09:04:48.272 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2024-05-09T09:04:48.272 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-05-09T09:04:48.272 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-05-09T09:04:48.272 INFO:teuthology.orchestra.run.smithi170.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T09:04:48.272 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-05-09T09:04:48.272 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T09:04:48.272 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T09:04:48.272 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T09:04:48.272 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:04:48.272 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-05-09T09:04:48.272 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-05-09T09:04:48.272 INFO:teuthology.orchestra.run.smithi170.stdout:Install 4 Packages 2024-05-09T09:04:48.273 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:04:48.273 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 78 M 2024-05-09T09:04:48.273 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 123 M 2024-05-09T09:04:48.273 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-05-09T09:04:48.948 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 6.5 MB/s | 15 MB 00:02 2024-05-09T09:04:49.043 INFO:teuthology.orchestra.run.smithi170.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 663 kB/s | 484 kB 00:00 2024-05-09T09:04:49.788 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-05-09T09:04:50.048 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 8.6 MB/s | 28 MB 00:03 2024-05-09T09:04:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T09:04:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T09:04:51.534 INFO:teuthology.orchestra.run.smithi170.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 4.8 MB/s | 15 MB 00:03 2024-05-09T09:04:52.548 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 5.7 MB/s | 34 MB 00:05 2024-05-09T09:04:52.548 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-05-09T09:04:52.548 INFO:teuthology.orchestra.run.smithi016.stdout:Total 12 MB/s | 78 MB 00:06 2024-05-09T09:04:53.253 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-05-09T09:04:53.328 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T09:04:53.452 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T09:04:53.643 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-05-09T09:04:53.643 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-05-09T09:04:54.604 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-05-09T09:04:54.604 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-05-09T09:04:57.137 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-05-09T09:04:57.847 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T09:04:57.932 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T09:04:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T09:04:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T09:05:00.860 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T09:05:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:04.133 INFO:teuthology.orchestra.run.smithi170.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 1.9 MB/s | 28 MB 00:15 2024-05-09T09:05:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T09:05:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:08.567 INFO:teuthology.orchestra.run.smithi170.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 1.7 MB/s | 34 MB 00:20 2024-05-09T09:05:08.568 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-05-09T09:05:08.568 INFO:teuthology.orchestra.run.smithi170.stdout:Total 3.8 MB/s | 78 MB 00:20 2024-05-09T09:05:09.243 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-05-09T09:05:09.587 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-05-09T09:05:09.587 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-05-09T09:05:10.509 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-05-09T09:05:10.510 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-05-09T09:05:11.074 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:12.927 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-05-09T09:05:13.543 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T09:05:13.625 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T09:05:16.313 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T09:05:16.465 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T09:05:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T09:05:21.324 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:34.944 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:34.945 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T09:05:34.945 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T09:05:34.945 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T09:05:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:05:35.739 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-05-09T09:05:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T09:05:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T09:05:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T09:05:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T09:05:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:05:35.739 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-05-09T09:05:36.130 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T09:05:36.215 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T09:05:36.216 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T09:05:36.216 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T09:05:36.216 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T09:05:36.216 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T09:05:36.216 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-09T09:05:36.216 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T09:05:36.217 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T09:05:36.268 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T09:05:36.268 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T09:05:36.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo rpm -qi grub2-tools 2024-05-09T09:05:36.299 INFO:teuthology.orchestra.run.smithi012.stdout:Name : grub2-tools 2024-05-09T09:05:36.299 INFO:teuthology.orchestra.run.smithi012.stdout:Epoch : 1 2024-05-09T09:05:36.299 INFO:teuthology.orchestra.run.smithi012.stdout:Version : 2.06 2024-05-09T09:05:36.299 INFO:teuthology.orchestra.run.smithi012.stdout:Release : 68.el9 2024-05-09T09:05:36.299 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: x86_64 2024-05-09T09:05:36.299 INFO:teuthology.orchestra.run.smithi012.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T09:05:36.299 INFO:teuthology.orchestra.run.smithi012.stdout:Group : Unspecified 2024-05-09T09:05:36.299 INFO:teuthology.orchestra.run.smithi012.stdout:Size : 8264254 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout:License : GPLv3+ 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout:Packager : builder@centos.org 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout:Vendor : CentOS 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout:Summary : Support tools for GRUB. 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout:Description : 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T09:05:36.300 INFO:teuthology.orchestra.run.smithi012.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T09:05:36.301 INFO:teuthology.orchestra.run.smithi012.stdout:hardware devices. 2024-05-09T09:05:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:05:36.301 INFO:teuthology.orchestra.run.smithi012.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T09:05:36.301 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T09:05:36.302 INFO:teuthology.task.kernel:Updating grub on smithi012 to boot 5.14.0-444.el9.x86_64 2024-05-09T09:05:36.302 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T09:05:36.932 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2024-05-09T09:05:38.287 INFO:teuthology.orchestra.run.smithi012.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T09:05:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:done 2024-05-09T09:05:38.315 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-05-09T09:05:38.334 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.mYpbZoK9hU 2024-05-09T09:05:38.334 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.mYpbZoK9hU 2024-05-09T09:05:38.400 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.mYpbZoK9hU 2024-05-09T09:05:38.545 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.mYpbZoK9hU is 6KB 2024-05-09T09:05:38.595 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.mYpbZoK9hU 2024-05-09T09:05:38.610 DEBUG:teuthology.orchestra.run.smithi012:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T09:05:38.672 INFO:teuthology.orchestra.run.smithi012.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T09:05:38.672 INFO:teuthology.orchestra.run.smithi012.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T09:05:38.672 INFO:teuthology.orchestra.run.smithi012.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T09:05:38.672 INFO:teuthology.orchestra.run.smithi012.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T09:05:38.673 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T09:05:38.839 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2024-05-09T09:05:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:42.108 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:42.420 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:42.420 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T09:05:42.420 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T09:05:42.420 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T09:05:43.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:43.231 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:05:43.231 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-05-09T09:05:43.231 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T09:05:43.231 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T09:05:43.231 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T09:05:43.231 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T09:05:43.231 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:05:43.231 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-05-09T09:05:43.586 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T09:05:43.683 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T09:05:43.683 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T09:05:43.683 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T09:05:43.683 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T09:05:43.683 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T09:05:43.683 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-09T09:05:43.683 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T09:05:43.683 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T09:05:43.735 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T09:05:43.735 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T09:05:43.735 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-05-09T09:05:43.766 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-05-09T09:05:43.766 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-05-09T09:05:43.766 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-05-09T09:05:43.766 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 68.el9 2024-05-09T09:05:43.766 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8264254 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-05-09T09:05:43.767 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:05:43.768 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T09:05:43.768 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T09:05:43.768 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T09:05:43.768 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-05-09T09:05:43.768 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:05:43.768 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T09:05:43.769 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T09:05:43.769 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 5.14.0-444.el9.x86_64 2024-05-09T09:05:43.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T09:05:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-05-09T09:05:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T09:05:45.911 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-05-09T09:05:45.913 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-05-09T09:05:45.976 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.9Y9j9C2r1N 2024-05-09T09:05:45.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9Y9j9C2r1N 2024-05-09T09:05:46.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.9Y9j9C2r1N 2024-05-09T09:05:46.238 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.9Y9j9C2r1N is 6KB 2024-05-09T09:05:46.271 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.9Y9j9C2r1N 2024-05-09T09:05:46.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T09:05:46.368 INFO:teuthology.orchestra.run.smithi016.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T09:05:46.369 INFO:teuthology.orchestra.run.smithi016.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T09:05:46.369 INFO:teuthology.orchestra.run.smithi016.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T09:05:46.369 INFO:teuthology.orchestra.run.smithi016.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T09:05:46.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T09:05:46.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-05-09T09:05:55.376 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:55.414 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:55.674 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:55.674 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T09:05:55.675 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T09:05:55.675 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T09:05:56.505 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T09:05:56.505 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:05:56.505 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-05-09T09:05:56.505 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T09:05:56.505 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T09:05:56.505 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T09:05:56.505 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T09:05:56.505 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:05:56.505 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-05-09T09:05:56.891 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T09:05:56.969 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T09:05:56.969 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T09:05:56.970 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T09:05:56.970 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T09:05:56.970 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T09:05:56.970 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-05-09T09:05:56.970 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi170.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T09:05:56.970 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T09:05:57.020 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T09:05:57.020 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T09:05:57.020 DEBUG:teuthology.orchestra.run.smithi170:> sudo rpm -qi grub2-tools 2024-05-09T09:05:57.051 INFO:teuthology.orchestra.run.smithi170.stdout:Name : grub2-tools 2024-05-09T09:05:57.051 INFO:teuthology.orchestra.run.smithi170.stdout:Epoch : 1 2024-05-09T09:05:57.051 INFO:teuthology.orchestra.run.smithi170.stdout:Version : 2.06 2024-05-09T09:05:57.051 INFO:teuthology.orchestra.run.smithi170.stdout:Release : 68.el9 2024-05-09T09:05:57.051 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: x86_64 2024-05-09T09:05:57.051 INFO:teuthology.orchestra.run.smithi170.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T09:05:57.051 INFO:teuthology.orchestra.run.smithi170.stdout:Group : Unspecified 2024-05-09T09:05:57.051 INFO:teuthology.orchestra.run.smithi170.stdout:Size : 8264254 2024-05-09T09:05:57.051 INFO:teuthology.orchestra.run.smithi170.stdout:License : GPLv3+ 2024-05-09T09:05:57.051 INFO:teuthology.orchestra.run.smithi170.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T09:05:57.051 INFO:teuthology.orchestra.run.smithi170.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T09:05:57.051 INFO:teuthology.orchestra.run.smithi170.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T09:05:57.052 INFO:teuthology.orchestra.run.smithi170.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T09:05:57.052 INFO:teuthology.orchestra.run.smithi170.stdout:Packager : builder@centos.org 2024-05-09T09:05:57.052 INFO:teuthology.orchestra.run.smithi170.stdout:Vendor : CentOS 2024-05-09T09:05:57.052 INFO:teuthology.orchestra.run.smithi170.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T09:05:57.052 INFO:teuthology.orchestra.run.smithi170.stdout:Summary : Support tools for GRUB. 2024-05-09T09:05:57.052 INFO:teuthology.orchestra.run.smithi170.stdout:Description : 2024-05-09T09:05:57.052 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:05:57.052 INFO:teuthology.orchestra.run.smithi170.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T09:05:57.052 INFO:teuthology.orchestra.run.smithi170.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T09:05:57.052 INFO:teuthology.orchestra.run.smithi170.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T09:05:57.052 INFO:teuthology.orchestra.run.smithi170.stdout:hardware devices. 2024-05-09T09:05:57.052 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:05:57.052 INFO:teuthology.orchestra.run.smithi170.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T09:05:57.053 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T09:05:57.053 INFO:teuthology.task.kernel:Updating grub on smithi170 to boot 5.14.0-444.el9.x86_64 2024-05-09T09:05:57.053 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T09:05:57.617 INFO:teuthology.orchestra.run.smithi170.stderr:Generating grub configuration file ... 2024-05-09T09:05:59.029 INFO:teuthology.orchestra.run.smithi170.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T09:05:59.058 INFO:teuthology.orchestra.run.smithi170.stderr:done 2024-05-09T09:05:59.059 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2024-05-09T09:05:59.074 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.Egp1CilFU6 2024-05-09T09:05:59.075 DEBUG:teuthology.orchestra.run.smithi170:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Egp1CilFU6 2024-05-09T09:05:59.140 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.Egp1CilFU6 2024-05-09T09:05:59.279 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.Egp1CilFU6 is 6KB 2024-05-09T09:05:59.328 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.Egp1CilFU6 2024-05-09T09:05:59.342 DEBUG:teuthology.orchestra.run.smithi170:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T09:05:59.410 INFO:teuthology.orchestra.run.smithi170.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T09:05:59.410 INFO:teuthology.orchestra.run.smithi170.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T09:05:59.410 INFO:teuthology.orchestra.run.smithi170.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T09:05:59.410 INFO:teuthology.orchestra.run.smithi170.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T09:05:59.412 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T09:05:59.571 DEBUG:teuthology.orchestra.run.smithi170:> sudo shutdown -r now 2024-05-09T09:06:08.850 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-09T09:06:08.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-05-09T09:06:08.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T09:06:16.540 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-09T09:06:16.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-05-09T09:06:16.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T09:06:29.585 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-09T09:06:29.585 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-05-09T09:06:29.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T09:07:08.890 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T09:07:16.551 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T09:07:17.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-05-09T09:07:17.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T09:07:18.196 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-05-09T09:07:18.570 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-05-09T09:07:18.571 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-09T09:07:18.571 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-05-09T09:07:18.588 INFO:teuthology.orchestra.run.smithi012.stdout:5.14.0-444.el9.x86_64 2024-05-09T09:07:18.588 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T09:07:18.588 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T09:07:18.588 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T09:07:19.590 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-09T09:07:19.590 DEBUG:teuthology.orchestra.run.smithi012:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T09:07:19.672 INFO:teuthology.orchestra.run.smithi012.stdout:ttyS1 2024-05-09T09:07:19.700 DEBUG:teuthology.parallel:result is None 2024-05-09T09:07:25.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-05-09T09:07:25.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T09:07:25.925 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-05-09T09:07:26.292 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-05-09T09:07:26.292 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-444.el9.x86_64"... 2024-05-09T09:07:26.293 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-05-09T09:07:26.309 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-444.el9.x86_64 2024-05-09T09:07:26.309 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T09:07:26.309 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T09:07:26.309 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T09:07:27.310 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-09T09:07:27.311 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T09:07:27.383 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2024-05-09T09:07:27.417 DEBUG:teuthology.parallel:result is None 2024-05-09T09:07:29.588 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T09:07:38.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-05-09T09:07:38.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T09:07:39.057 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-05-09T09:07:39.465 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-05-09T09:07:39.465 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-444.el9.x86_64"... 2024-05-09T09:07:39.465 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi170.stdout:5.14.0-444.el9.x86_64 2024-05-09T09:07:39.480 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T09:07:39.480 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T09:07:39.480 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T09:07:40.481 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-09T09:07:40.481 DEBUG:teuthology.orchestra.run.smithi170:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T09:07:40.549 INFO:teuthology.orchestra.run.smithi170.stdout:ttyS1 2024-05-09T09:07:40.580 DEBUG:teuthology.parallel:result is None 2024-05-09T09:07:40.580 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T09:07:40.587 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T09:07:40.587 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T09:07:40.594 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T09:07:40.597 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T09:07:40.614 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T09:07:40.681 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T09:07:40.689 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T09:07:40.689 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T09:07:40.692 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T09:07:40.695 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T09:07:40.743 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T09:07:40.751 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T09:07:40.751 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T09:07:40.753 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T09:07:40.768 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T09:07:40.784 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T09:07:40.795 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T09:07:40.796 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T09:07:40.796 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T09:07:40.805 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T09:07:40.809 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T09:07:40.811 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T09:07:40.818 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T09:07:40.818 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T09:07:40.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T09:07:40.853 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T09:07:40.880 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T09:07:40.890 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T09:07:40.890 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T09:07:40.911 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T09:07:40.923 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T09:07:40.938 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T09:07:41.012 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T09:07:41.057 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T09:07:41.143 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T09:07:41.183 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:07:41.184 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T09:07:41.250 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T09:07:41.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T09:07:41.349 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T09:07:41.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T09:07:41.479 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-09T09:07:41.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T09:07:41.549 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T09:07:41.590 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T09:07:41.640 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T09:07:41.722 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T09:07:41.764 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-09T09:07:41.764 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T09:07:41.829 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-05-09T09:07:41.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-05-09T09:07:41.834 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-05-09T09:07:41.878 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T09:07:41.888 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T09:07:41.930 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T09:07:42.392 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T09:07:42.400 INFO:teuthology.task.internal:Starting timer... 2024-05-09T09:07:42.400 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T09:07:42.425 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T09:07:42.433 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-09T09:07:42.434 DEBUG:teuthology.orchestra.run.smithi012:> sudo service auditd rotate 2024-05-09T09:07:42.483 INFO:teuthology.orchestra.run.smithi012.stdout:Rotating logs: 2024-05-09T09:07:42.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-05-09T09:07:42.569 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-05-09T09:07:42.571 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2024-05-09T09:07:42.668 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2024-05-09T09:07:42.672 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T09:07:42.672 DEBUG:teuthology.orchestra.run.smithi012:> /usr/sbin/getenforce 2024-05-09T09:07:42.709 INFO:teuthology.orchestra.run.smithi012.stdout:Permissive 2024-05-09T09:07:42.709 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-05-09T09:07:42.734 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-05-09T09:07:42.735 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2024-05-09T09:07:42.774 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2024-05-09T09:07:42.774 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi012.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive'} 2024-05-09T09:07:42.774 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T09:07:42.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:07:42.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T09:07:42.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:07:42.835 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T09:07:42.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:07:42.881 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T09:07:42.881 DEBUG:teuthology.orchestra.run.smithi012:> sudo /usr/sbin/setenforce permissive 2024-05-09T09:07:42.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-05-09T09:07:42.943 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2024-05-09T09:07:43.211 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T09:07:43.221 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T09:07:43.237 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T09:07:43.239 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi016.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T09:12:10.584 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2024-05-09T09:12:10.586 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-05-09T09:12:10.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T09:12:10.683 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-05-09T09:12:10.724 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-05-09T09:12:10.724 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-05-09T09:12:10.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T09:12:10.804 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-05-09T09:12:10.845 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-05-09T09:12:10.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-05-09T09:12:10.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T09:12:10.930 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-05-09T09:12:10.967 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-05-09T09:12:10.968 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T09:12:10.979 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T09:12:10.979 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T09:12:10.980 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T09:12:10.982 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T09:12:10.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T09:12:10.984 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T09:12:10.984 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T09:12:11.014 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T09:12:11.015 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T09:12:11.030 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T09:12:11.031 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T09:12:11.038 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T09:12:11.052 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T09:12:11.055 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-05-09T09:12:11.058 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: ntpd: command not found 2024-05-09T09:12:11.067 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-05-09T09:12:11.070 INFO:teuthology.orchestra.run.smithi012.stdout:506 Cannot talk to daemon 2024-05-09T09:12:11.076 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2024-05-09T09:12:11.082 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T09:12:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T09:12:11.087 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2024-05-09T09:12:11.097 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T09:12:11.101 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T09:12:11.101 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T09:12:11.115 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T09:12:11.156 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-05-09T09:12:11.160 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T09:12:11.160 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-05-09T09:12:11.160 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:12:11.160 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:12:11.160 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:12:11.160 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:12:11.163 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-05-09T09:12:11.167 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T09:12:11.167 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-05-09T09:12:11.167 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:12:11.168 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:12:11.168 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:12:11.168 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:12:11.171 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-05-09T09:12:11.172 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T09:12:11.172 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-05-09T09:12:11.172 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:12:11.173 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:12:11.173 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:12:11.173 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:12:11.174 INFO:teuthology.run_tasks:Running task install... 2024-05-09T09:12:11.182 DEBUG:teuthology.task.install:project ceph 2024-05-09T09:12:11.183 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}} 2024-05-09T09:12:11.183 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'} 2024-05-09T09:12:11.183 INFO:teuthology.task.install:Using flavor: default 2024-05-09T09:12:11.190 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T09:12:11.190 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T09:12:11.191 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T09:12:11.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T09:12:11.193 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T09:12:11.193 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T09:12:11.194 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T09:12:11.195 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T09:12:11.345 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T09:12:11.345 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T09:12:11.348 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T09:12:11.349 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T09:12:11.420 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T09:12:11.420 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T09:12:11.461 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T09:12:11.461 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:12:11.461 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T09:12:11.471 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T09:12:11.472 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-09T09:12:11.472 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T09:12:11.489 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T09:12:11.490 DEBUG:teuthology.orchestra.run.smithi012:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T09:12:11.498 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T09:12:11.499 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T09:12:11.531 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T09:12:11.531 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-09T09:12:11.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T09:12:11.558 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T09:12:11.558 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T09:12:11.559 DEBUG:teuthology.orchestra.run.smithi012:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T09:12:11.564 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T09:12:11.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T09:12:11.637 DEBUG:teuthology.orchestra.run.smithi012:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T09:12:11.639 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T09:12:11.702 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T09:12:11.703 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2024-05-09T09:12:11.704 INFO:teuthology.orchestra.run.smithi012.stdout:check_obsoletes = 1 2024-05-09T09:12:11.705 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2024-05-09T09:12:11.706 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum clean all 2024-05-09T09:12:11.767 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-05-09T09:12:11.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-05-09T09:12:11.933 INFO:teuthology.orchestra.run.smithi170.stdout:54 files removed 2024-05-09T09:12:11.947 INFO:teuthology.orchestra.run.smithi012.stdout:54 files removed 2024-05-09T09:12:11.960 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T09:12:11.974 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T09:12:12.018 INFO:teuthology.orchestra.run.smithi016.stdout:54 files removed 2024-05-09T09:12:12.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T09:12:12.594 INFO:teuthology.orchestra.run.smithi012.stdout:ceph packages for x86_64 254 kB/s | 83 kB 00:00 2024-05-09T09:12:12.637 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 207 kB/s | 83 kB 00:00 2024-05-09T09:12:12.805 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 176 kB/s | 83 kB 00:00 2024-05-09T09:12:12.986 INFO:teuthology.orchestra.run.smithi012.stdout:ceph noarch packages 34 kB/s | 11 kB 00:00 2024-05-09T09:12:13.006 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 36 kB/s | 11 kB 00:00 2024-05-09T09:12:13.205 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 33 kB/s | 11 kB 00:00 2024-05-09T09:12:13.353 INFO:teuthology.orchestra.run.smithi012.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-05-09T09:12:13.396 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-05-09T09:12:13.548 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2024-05-09T09:12:13.920 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-09T09:12:14.323 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-09T09:12:15.006 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.1 MB 00:01 2024-05-09T09:12:15.772 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-05-09T09:12:17.548 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-05-09T09:12:18.999 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 19 MB 00:03 2024-05-09T09:12:19.872 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.2 MB 00:00 2024-05-09T09:12:21.265 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 88 kB/s | 16 kB 00:00 2024-05-09T09:12:23.032 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.2 MB 00:01 2024-05-09T09:12:23.102 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-09T09:12:23.508 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.2 MB 00:00 2024-05-09T09:12:24.741 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-05-09T09:12:24.934 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 16 kB 00:00 2024-05-09T09:12:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 9.5 MB/s | 21 MB 00:02 2024-05-09T09:12:27.354 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 21 MB 00:02 2024-05-09T09:12:28.349 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-05-09T09:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T09:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T09:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T09:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T09:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T09:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T09:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T09:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T09:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T09:12:30.862 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-05-09T09:12:30.883 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-05-09T09:12:30.884 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-05-09T09:12:30.884 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-05-09T09:12:30.884 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-05-09T09:12:30.884 INFO:teuthology.orchestra.run.smithi012.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T09:12:30.884 INFO:teuthology.orchestra.run.smithi012.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T09:12:30.884 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T09:12:30.884 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T09:12:30.884 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T09:12:30.884 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T09:12:30.885 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T09:12:30.885 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T09:12:30.885 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T09:12:30.885 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T09:12:30.885 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T09:12:30.885 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T09:12:30.885 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T09:12:30.885 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T09:12:30.885 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T09:12:30.885 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T09:12:30.886 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T09:12:30.886 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T09:12:30.886 INFO:teuthology.orchestra.run.smithi012.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T09:12:30.886 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T09:12:30.886 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T09:12:30.886 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T09:12:30.886 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T09:12:30.886 INFO:teuthology.orchestra.run.smithi012.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T09:12:30.886 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T09:12:30.886 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T09:12:30.887 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T09:12:30.887 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T09:12:30.887 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T09:12:30.887 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T09:12:30.887 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T09:12:30.887 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T09:12:30.887 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T09:12:30.887 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T09:12:30.887 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T09:12:30.888 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T09:12:30.888 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T09:12:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrading: 2024-05-09T09:12:30.888 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T09:12:30.888 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T09:12:30.888 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T09:12:30.888 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T09:12:30.888 INFO:teuthology.orchestra.run.smithi012.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T09:12:30.888 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T09:12:30.889 INFO:teuthology.orchestra.run.smithi012.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T09:12:30.889 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T09:12:30.889 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-05-09T09:12:30.889 INFO:teuthology.orchestra.run.smithi012.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T09:12:30.889 INFO:teuthology.orchestra.run.smithi012.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T09:12:30.889 INFO:teuthology.orchestra.run.smithi012.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T09:12:30.889 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T09:12:30.889 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T09:12:30.889 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T09:12:30.889 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T09:12:30.889 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T09:12:30.889 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T09:12:30.890 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T09:12:30.890 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T09:12:30.890 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T09:12:30.890 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T09:12:30.890 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T09:12:30.890 INFO:teuthology.orchestra.run.smithi012.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T09:12:30.890 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T09:12:30.890 INFO:teuthology.orchestra.run.smithi012.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T09:12:30.890 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T09:12:30.890 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T09:12:30.890 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T09:12:30.891 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T09:12:30.891 INFO:teuthology.orchestra.run.smithi012.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T09:12:30.891 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T09:12:30.891 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T09:12:30.891 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T09:12:30.891 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T09:12:30.891 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T09:12:30.891 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T09:12:30.891 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T09:12:30.891 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T09:12:30.891 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T09:12:30.891 INFO:teuthology.orchestra.run.smithi012.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T09:12:30.892 INFO:teuthology.orchestra.run.smithi012.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T09:12:30.892 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T09:12:30.892 INFO:teuthology.orchestra.run.smithi012.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T09:12:30.892 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T09:12:30.892 INFO:teuthology.orchestra.run.smithi012.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T09:12:30.892 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T09:12:30.892 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T09:12:30.892 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T09:12:30.892 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T09:12:30.892 INFO:teuthology.orchestra.run.smithi012.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T09:12:30.892 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T09:12:30.892 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T09:12:30.892 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T09:12:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T09:12:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T09:12:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T09:12:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T09:12:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T09:12:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T09:12:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T09:12:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T09:12:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T09:12:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T09:12:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T09:12:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T09:12:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T09:12:30.894 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T09:12:30.894 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T09:12:30.894 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T09:12:30.894 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T09:12:30.894 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T09:12:30.894 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T09:12:30.894 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T09:12:30.894 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T09:12:30.894 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T09:12:30.894 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T09:12:30.894 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T09:12:30.894 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T09:12:30.894 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T09:12:30.895 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T09:12:30.896 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T09:12:30.896 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T09:12:30.896 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T09:12:30.896 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T09:12:30.896 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T09:12:30.896 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T09:12:30.896 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T09:12:30.896 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T09:12:30.896 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T09:12:30.896 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T09:12:30.896 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T09:12:30.896 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T09:12:30.896 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout:Installing weak dependencies: 2024-05-09T09:12:30.897 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T09:12:30.898 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:12:30.898 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-05-09T09:12:30.898 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-05-09T09:12:30.898 INFO:teuthology.orchestra.run.smithi012.stdout:Install 148 Packages 2024-05-09T09:12:30.898 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrade 8 Packages 2024-05-09T09:12:30.898 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:12:30.898 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 225 M 2024-05-09T09:12:30.898 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-05-09T09:12:31.584 INFO:teuthology.orchestra.run.smithi012.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-05-09T09:12:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 4.0 MB/s | 889 kB 00:00 2024-05-09T09:12:31.859 INFO:teuthology.orchestra.run.smithi012.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 2.4 MB/s | 145 kB 00:00 2024-05-09T09:12:32.178 INFO:teuthology.orchestra.run.smithi012.stdout:(4/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 7.5 MB/s | 2.4 MB 00:00 2024-05-09T09:12:32.410 INFO:teuthology.orchestra.run.smithi012.stdout:(5/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 6.4 MB/s | 1.5 MB 00:00 2024-05-09T09:12:32.518 INFO:teuthology.orchestra.run.smithi012.stdout:(6/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 4.8 MB/s | 5.4 MB 00:01 2024-05-09T09:12:32.559 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-05-09T09:12:32.661 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-09T09:12:33.085 INFO:teuthology.orchestra.run.smithi012.stdout:(7/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 6.9 MB/s | 4.7 MB 00:00 2024-05-09T09:12:33.311 INFO:teuthology.orchestra.run.smithi012.stdout:(8/156): ceph-common-19.0.0-2515.g96805dcf.el9. 11 MB/s | 21 MB 00:01 2024-05-09T09:12:33.361 INFO:teuthology.orchestra.run.smithi012.stdout:(9/156): ceph-selinux-19.0.0-2515.g96805dcf.el9 507 kB/s | 25 kB 00:00 2024-05-09T09:12:34.920 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T09:12:34.920 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T09:12:34.931 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T09:12:34.932 INFO:teuthology.orchestra.run.smithi170.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T09:12:34.933 INFO:teuthology.orchestra.run.smithi170.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T09:12:34.934 INFO:teuthology.orchestra.run.smithi170.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T09:12:34.936 INFO:teuthology.orchestra.run.smithi170.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T09:12:34.938 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T09:12:34.939 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T09:12:34.952 INFO:teuthology.orchestra.run.smithi012.stdout:(10/156): ceph-radosgw-19.0.0-2515.g96805dcf.el 5.5 MB/s | 10 MB 00:01 2024-05-09T09:12:35.000 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-05-09T09:12:35.024 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-05-09T09:12:35.024 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-05-09T09:12:35.025 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-05-09T09:12:35.025 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-05-09T09:12:35.025 INFO:teuthology.orchestra.run.smithi170.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T09:12:35.025 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T09:12:35.025 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T09:12:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T09:12:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T09:12:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T09:12:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T09:12:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T09:12:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T09:12:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T09:12:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T09:12:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T09:12:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T09:12:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T09:12:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T09:12:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T09:12:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T09:12:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T09:12:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T09:12:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T09:12:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T09:12:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T09:12:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T09:12:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T09:12:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T09:12:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T09:12:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T09:12:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T09:12:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T09:12:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T09:12:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T09:12:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T09:12:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T09:12:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T09:12:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T09:12:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T09:12:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T09:12:35.031 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2024-05-09T09:12:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T09:12:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T09:12:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T09:12:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T09:12:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T09:12:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T09:12:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T09:12:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T09:12:35.033 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-05-09T09:12:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T09:12:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T09:12:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T09:12:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T09:12:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T09:12:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T09:12:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T09:12:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T09:12:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T09:12:35.035 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T09:12:35.035 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T09:12:35.035 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T09:12:35.035 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T09:12:35.036 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T09:12:35.036 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T09:12:35.036 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T09:12:35.036 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T09:12:35.036 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T09:12:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T09:12:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T09:12:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T09:12:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T09:12:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T09:12:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T09:12:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T09:12:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T09:12:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T09:12:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T09:12:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T09:12:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T09:12:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T09:12:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T09:12:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T09:12:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T09:12:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T09:12:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T09:12:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T09:12:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T09:12:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T09:12:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T09:12:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T09:12:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T09:12:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T09:12:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T09:12:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T09:12:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T09:12:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T09:12:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T09:12:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T09:12:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T09:12:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T09:12:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T09:12:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T09:12:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T09:12:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T09:12:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T09:12:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T09:12:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T09:12:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T09:12:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T09:12:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T09:12:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T09:12:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T09:12:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T09:12:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T09:12:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T09:12:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T09:12:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T09:12:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T09:12:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T09:12:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T09:12:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T09:12:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T09:12:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T09:12:35.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T09:12:35.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T09:12:35.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T09:12:35.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T09:12:35.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T09:12:35.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T09:12:35.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T09:12:35.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T09:12:35.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T09:12:35.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T09:12:35.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T09:12:35.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T09:12:35.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T09:12:35.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout:Installing weak dependencies: 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-05-09T09:12:35.047 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-05-09T09:12:35.048 INFO:teuthology.orchestra.run.smithi170.stdout:Install 148 Packages 2024-05-09T09:12:35.048 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 8 Packages 2024-05-09T09:12:35.048 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:12:35.048 INFO:teuthology.orchestra.run.smithi012.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 2.4 MB/s | 222 kB 00:00 2024-05-09T09:12:35.049 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 225 M 2024-05-09T09:12:35.049 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-05-09T09:12:35.131 INFO:teuthology.orchestra.run.smithi012.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 324 kB/s | 27 kB 00:00 2024-05-09T09:12:35.139 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T09:12:35.139 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T09:12:35.151 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T09:12:35.152 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T09:12:35.152 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T09:12:35.153 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T09:12:35.155 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T09:12:35.157 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T09:12:35.158 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T09:12:35.204 INFO:teuthology.orchestra.run.smithi012.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 424 kB/s | 32 kB 00:00 2024-05-09T09:12:35.224 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-05-09T09:12:35.251 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-05-09T09:12:35.251 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-05-09T09:12:35.252 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-05-09T09:12:35.252 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-05-09T09:12:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T09:12:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T09:12:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T09:12:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T09:12:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T09:12:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T09:12:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T09:12:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T09:12:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T09:12:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T09:12:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T09:12:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T09:12:35.254 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T09:12:35.254 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T09:12:35.254 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T09:12:35.254 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T09:12:35.254 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T09:12:35.254 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T09:12:35.254 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T09:12:35.254 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T09:12:35.257 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T09:12:35.257 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T09:12:35.257 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T09:12:35.258 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T09:12:35.258 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-05-09T09:12:35.258 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T09:12:35.258 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T09:12:35.258 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T09:12:35.258 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T09:12:35.258 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T09:12:35.258 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T09:12:35.258 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T09:12:35.258 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T09:12:35.258 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-05-09T09:12:35.259 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T09:12:35.259 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T09:12:35.259 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T09:12:35.259 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T09:12:35.259 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T09:12:35.259 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T09:12:35.259 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T09:12:35.259 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T09:12:35.259 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T09:12:35.259 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T09:12:35.259 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T09:12:35.259 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T09:12:35.260 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T09:12:35.261 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T09:12:35.262 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T09:12:35.262 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T09:12:35.262 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T09:12:35.262 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T09:12:35.262 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T09:12:35.262 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T09:12:35.262 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T09:12:35.262 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T09:12:35.262 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T09:12:35.262 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T09:12:35.262 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T09:12:35.262 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T09:12:35.263 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T09:12:35.264 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T09:12:35.264 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T09:12:35.264 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T09:12:35.264 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T09:12:35.264 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T09:12:35.264 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T09:12:35.264 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T09:12:35.264 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T09:12:35.264 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T09:12:35.264 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T09:12:35.264 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T09:12:35.264 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T09:12:35.264 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T09:12:35.265 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T09:12:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T09:12:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T09:12:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T09:12:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T09:12:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T09:12:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T09:12:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T09:12:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T09:12:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T09:12:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T09:12:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T09:12:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T09:12:35.267 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T09:12:35.267 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T09:12:35.267 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T09:12:35.267 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T09:12:35.267 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T09:12:35.267 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-05-09T09:12:35.267 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T09:12:35.267 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:12:35.267 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-05-09T09:12:35.267 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-05-09T09:12:35.267 INFO:teuthology.orchestra.run.smithi016.stdout:Install 148 Packages 2024-05-09T09:12:35.267 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 8 Packages 2024-05-09T09:12:35.267 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:12:35.268 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 225 M 2024-05-09T09:12:35.268 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-05-09T09:12:35.371 INFO:teuthology.orchestra.run.smithi012.stdout:(14/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x8 5.9 MB/s | 17 MB 00:02 2024-05-09T09:12:35.404 INFO:teuthology.orchestra.run.smithi012.stdout:(15/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 3.6 MB/s | 746 kB 00:00 2024-05-09T09:12:35.421 INFO:teuthology.orchestra.run.smithi012.stdout:(16/156): libcephsqlite-19.0.0-2515.g96805dcf.e 3.2 MB/s | 163 kB 00:00 2024-05-09T09:12:35.446 INFO:teuthology.orchestra.run.smithi012.stdout:(17/156): librados-devel-19.0.0-2515.g96805dcf. 3.0 MB/s | 127 kB 00:00 2024-05-09T09:12:35.563 INFO:teuthology.orchestra.run.smithi012.stdout:(18/156): libradosstriper1-19.0.0-2515.g96805dc 3.4 MB/s | 500 kB 00:00 2024-05-09T09:12:35.611 INFO:teuthology.orchestra.run.smithi170.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-05-09T09:12:35.672 INFO:teuthology.orchestra.run.smithi012.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 417 kB/s | 45 kB 00:00 2024-05-09T09:12:35.781 INFO:teuthology.orchestra.run.smithi012.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 1.2 MB/s | 135 kB 00:00 2024-05-09T09:12:35.854 INFO:teuthology.orchestra.run.smithi016.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-05-09T09:12:35.898 INFO:teuthology.orchestra.run.smithi012.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 1.4 MB/s | 162 kB 00:00 2024-05-09T09:12:36.180 INFO:teuthology.orchestra.run.smithi016.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 2.7 MB/s | 889 kB 00:00 2024-05-09T09:12:36.234 INFO:teuthology.orchestra.run.smithi170.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 1.4 MB/s | 889 kB 00:00 2024-05-09T09:12:36.259 INFO:teuthology.orchestra.run.smithi016.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 2.1 MB/s | 145 kB 00:00 2024-05-09T09:12:36.328 INFO:teuthology.orchestra.run.smithi170.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 1.4 MB/s | 145 kB 00:00 2024-05-09T09:12:36.365 INFO:teuthology.orchestra.run.smithi012.stdout:(22/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 16 MB/s | 48 MB 00:03 2024-05-09T09:12:36.390 INFO:teuthology.orchestra.run.smithi012.stdout:(23/156): python3-rados-19.0.0-2515.g96805dcf.e 653 kB/s | 321 kB 00:00 2024-05-09T09:12:36.465 INFO:teuthology.orchestra.run.smithi012.stdout:(24/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 5.2 MB/s | 5.3 MB 00:01 2024-05-09T09:12:36.490 INFO:teuthology.orchestra.run.smithi012.stdout:(25/156): python3-rgw-19.0.0-2515.g96805dcf.el9 992 kB/s | 99 kB 00:00 2024-05-09T09:12:36.506 INFO:teuthology.orchestra.run.smithi016.stdout:(4/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 5.7 MB/s | 5.4 MB 00:00 2024-05-09T09:12:36.516 INFO:teuthology.orchestra.run.smithi012.stdout:(26/156): python3-rbd-19.0.0-2515.g96805dcf.el9 1.9 MB/s | 298 kB 00:00 2024-05-09T09:12:36.541 INFO:teuthology.orchestra.run.smithi012.stdout:(27/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-09T09:12:36.636 INFO:teuthology.orchestra.run.smithi012.stdout:(28/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 1.4 MB/s | 171 kB 00:00 2024-05-09T09:12:36.638 INFO:teuthology.orchestra.run.smithi016.stdout:(5/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 6.0 MB/s | 2.4 MB 00:00 2024-05-09T09:12:36.680 INFO:teuthology.orchestra.run.smithi016.stdout:(6/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 8.5 MB/s | 1.5 MB 00:00 2024-05-09T09:12:36.794 INFO:teuthology.orchestra.run.smithi012.stdout:(29/156): ceph-grafana-dashboards-19.0.0-2515.g 125 kB/s | 32 kB 00:00 2024-05-09T09:12:36.817 INFO:teuthology.orchestra.run.smithi012.stdout:(30/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 785 kB/s | 142 kB 00:00 2024-05-09T09:12:37.409 INFO:teuthology.orchestra.run.smithi012.stdout:(31/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 3.4 MB/s | 3.1 MB 00:00 2024-05-09T09:12:37.420 INFO:teuthology.orchestra.run.smithi170.stdout:(4/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 2.2 MB/s | 2.4 MB 00:01 2024-05-09T09:12:37.509 INFO:teuthology.orchestra.run.smithi012.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 2.4 MB/s | 249 kB 00:00 2024-05-09T09:12:37.564 INFO:teuthology.orchestra.run.smithi016.stdout:(7/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 5.1 MB/s | 4.7 MB 00:00 2024-05-09T09:12:37.601 INFO:teuthology.orchestra.run.smithi012.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 538 kB/s | 49 kB 00:00 2024-05-09T09:12:37.668 INFO:teuthology.orchestra.run.smithi012.stdout:(34/156): ceph-mgr-dashboard-19.0.0-2515.g96805 4.0 MB/s | 3.5 MB 00:00 2024-05-09T09:12:37.701 INFO:teuthology.orchestra.run.smithi012.stdout:(35/156): ceph-prometheus-alerts-19.0.0-2515.g9 164 kB/s | 16 kB 00:00 2024-05-09T09:12:37.760 INFO:teuthology.orchestra.run.smithi012.stdout:(36/156): ceph-volume-19.0.0-2515.g96805dcf.el9 2.8 MB/s | 264 kB 00:00 2024-05-09T09:12:37.860 INFO:teuthology.orchestra.run.smithi012.stdout:(37/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 182 kB/s | 18 kB 00:00 2024-05-09T09:12:37.994 INFO:teuthology.orchestra.run.smithi012.stdout:(38/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 2.5 MB/s | 763 kB 00:00 2024-05-09T09:12:38.103 INFO:teuthology.orchestra.run.smithi170.stdout:(5/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-05-09T09:12:38.294 INFO:teuthology.orchestra.run.smithi012.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 185 kB/s | 80 kB 00:00 2024-05-09T09:12:38.436 INFO:teuthology.orchestra.run.smithi012.stdout:(40/156): ceph-mgr-diskprediction-local-19.0.0- 4.6 MB/s | 7.4 MB 00:01 2024-05-09T09:12:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-05-09T09:12:38.770 INFO:teuthology.orchestra.run.smithi012.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.2 MB 00:00 2024-05-09T09:12:38.781 INFO:teuthology.orchestra.run.smithi016.stdout:(8/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x86 8.0 MB/s | 17 MB 00:02 2024-05-09T09:12:38.820 INFO:teuthology.orchestra.run.smithi170.stdout:(6/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-05-09T09:12:38.829 INFO:teuthology.orchestra.run.smithi012.stdout:(43/156): libgfortran-11.4.1-3.el9.x86_64.rpm 962 kB/s | 803 kB 00:00 2024-05-09T09:12:38.831 INFO:teuthology.orchestra.run.smithi016.stdout:(9/156): ceph-selinux-19.0.0-2515.g96805dcf.el9 507 kB/s | 25 kB 00:00 2024-05-09T09:12:38.879 INFO:teuthology.orchestra.run.smithi012.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 980 kB/s | 106 kB 00:00 2024-05-09T09:12:38.938 INFO:teuthology.orchestra.run.smithi012.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-09T09:12:38.988 INFO:teuthology.orchestra.run.smithi012.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-09T09:12:39.046 INFO:teuthology.orchestra.run.smithi012.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-09T09:12:39.072 INFO:teuthology.orchestra.run.smithi012.stdout:(48/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 399 kB/s | 253 kB 00:00 2024-05-09T09:12:39.156 INFO:teuthology.orchestra.run.smithi012.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-05-09T09:12:39.206 INFO:teuthology.orchestra.run.smithi012.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-05-09T09:12:39.231 INFO:teuthology.orchestra.run.smithi012.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 286 kB/s | 21 kB 00:00 2024-05-09T09:12:39.256 INFO:teuthology.orchestra.run.smithi012.stdout:(52/156): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-05-09T09:12:39.307 INFO:teuthology.orchestra.run.smithi012.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 436 kB/s | 33 kB 00:00 2024-05-09T09:12:39.365 INFO:teuthology.orchestra.run.smithi012.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-05-09T09:12:39.370 INFO:teuthology.orchestra.run.smithi170.stdout:(7/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 3.7 MB/s | 4.7 MB 00:01 2024-05-09T09:12:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:(10/156): ceph-radosgw-19.0.0-2515.g96805dcf.el 5.7 MB/s | 10 MB 00:01 2024-05-09T09:12:39.391 INFO:teuthology.orchestra.run.smithi012.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 217 kB/s | 18 kB 00:00 2024-05-09T09:12:39.432 INFO:teuthology.orchestra.run.smithi016.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 3.7 MB/s | 222 kB 00:00 2024-05-09T09:12:39.432 INFO:teuthology.orchestra.run.smithi012.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-05-09T09:12:39.482 INFO:teuthology.orchestra.run.smithi016.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 541 kB/s | 27 kB 00:00 2024-05-09T09:12:39.499 INFO:teuthology.orchestra.run.smithi012.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-09T09:12:39.525 INFO:teuthology.orchestra.run.smithi012.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 493 kB/s | 45 kB 00:00 2024-05-09T09:12:39.532 INFO:teuthology.orchestra.run.smithi016.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 634 kB/s | 32 kB 00:00 2024-05-09T09:12:39.633 INFO:teuthology.orchestra.run.smithi012.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.9 MB/s | 3.0 MB 00:00 2024-05-09T09:12:39.692 INFO:teuthology.orchestra.run.smithi012.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 439 kB/s | 25 kB 00:00 2024-05-09T09:12:39.699 INFO:teuthology.orchestra.run.smithi016.stdout:(14/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 4.4 MB/s | 746 kB 00:00 2024-05-09T09:12:39.751 INFO:teuthology.orchestra.run.smithi012.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2024-05-09T09:12:39.766 INFO:teuthology.orchestra.run.smithi016.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 2.4 MB/s | 163 kB 00:00 2024-05-09T09:12:39.776 INFO:teuthology.orchestra.run.smithi012.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-05-09T09:12:39.801 INFO:teuthology.orchestra.run.smithi012.stdout:(63/156): libtool-2.4.6-45.el9.x86_64.rpm 2.1 MB/s | 582 kB 00:00 2024-05-09T09:12:39.825 INFO:teuthology.orchestra.run.smithi016.stdout:(16/156): librados-devel-19.0.0-2515.g96805dcf. 2.1 MB/s | 127 kB 00:00 2024-05-09T09:12:39.851 INFO:teuthology.orchestra.run.smithi012.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-05-09T09:12:39.893 INFO:teuthology.orchestra.run.smithi012.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-05-09T09:12:39.918 INFO:teuthology.orchestra.run.smithi012.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-05-09T09:12:39.951 INFO:teuthology.orchestra.run.smithi016.stdout:(17/156): libradosstriper1-19.0.0-2515.g96805dc 3.9 MB/s | 500 kB 00:00 2024-05-09T09:12:40.269 INFO:teuthology.orchestra.run.smithi012.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 4.5 MB/s | 1.6 MB 00:00 2024-05-09T09:12:40.519 INFO:teuthology.orchestra.run.smithi012.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 3.6 MB/s | 911 kB 00:00 2024-05-09T09:12:40.628 INFO:teuthology.orchestra.run.smithi012.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-05-09T09:12:40.653 INFO:teuthology.orchestra.run.smithi012.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-05-09T09:12:40.728 INFO:teuthology.orchestra.run.smithi012.stdout:(71/156): openblas-openmp-0.3.21-2.el9.x86_64.r 5.4 MB/s | 4.7 MB 00:00 2024-05-09T09:12:40.804 INFO:teuthology.orchestra.run.smithi012.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-05-09T09:12:40.809 INFO:teuthology.orchestra.run.smithi016.stdout:(18/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 6.2 MB/s | 5.3 MB 00:00 2024-05-09T09:12:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 678 kB/s | 45 kB 00:00 2024-05-09T09:12:40.951 INFO:teuthology.orchestra.run.smithi016.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 1.8 MB/s | 135 kB 00:00 2024-05-09T09:12:40.987 INFO:teuthology.orchestra.run.smithi012.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-05-09T09:12:41.018 INFO:teuthology.orchestra.run.smithi016.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 2.4 MB/s | 162 kB 00:00 2024-05-09T09:12:41.093 INFO:teuthology.orchestra.run.smithi016.stdout:(22/156): python3-rados-19.0.0-2515.g96805dcf.e 4.2 MB/s | 321 kB 00:00 2024-05-09T09:12:41.169 INFO:teuthology.orchestra.run.smithi016.stdout:(23/156): python3-rbd-19.0.0-2515.g96805dcf.el9 3.9 MB/s | 298 kB 00:00 2024-05-09T09:12:41.187 INFO:teuthology.orchestra.run.smithi012.stdout:(74/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-05-09T09:12:41.213 INFO:teuthology.orchestra.run.smithi012.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-05-09T09:12:41.236 INFO:teuthology.orchestra.run.smithi016.stdout:(24/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.5 MB/s | 99 kB 00:00 2024-05-09T09:12:41.271 INFO:teuthology.orchestra.run.smithi012.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-05-09T09:12:41.303 INFO:teuthology.orchestra.run.smithi016.stdout:(25/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.2 MB/s | 85 kB 00:00 2024-05-09T09:12:41.401 INFO:teuthology.orchestra.run.smithi012.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 288 kB/s | 19 kB 00:00 2024-05-09T09:12:41.439 INFO:teuthology.orchestra.run.smithi012.stdout:(78/156): postgresql-server-13.14-1.el9.x86_64. 7.1 MB/s | 5.8 MB 00:00 2024-05-09T09:12:41.490 INFO:teuthology.orchestra.run.smithi012.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 697 kB/s | 35 kB 00:00 2024-05-09T09:12:41.531 INFO:teuthology.orchestra.run.smithi012.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 895 kB/s | 172 kB 00:00 2024-05-09T09:12:41.731 INFO:teuthology.orchestra.run.smithi012.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-05-09T09:12:41.845 INFO:teuthology.orchestra.run.smithi016.stdout:(26/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 5.7 MB/s | 3.1 MB 00:00 2024-05-09T09:12:41.903 INFO:teuthology.orchestra.run.smithi016.stdout:(27/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 2.9 MB/s | 171 kB 00:00 2024-05-09T09:12:41.937 INFO:teuthology.orchestra.run.smithi170.stdout:(8/156): ceph-radosgw-19.0.0-2515.g96805dcf.el9 4.0 MB/s | 10 MB 00:02 2024-05-09T09:12:41.940 INFO:teuthology.orchestra.run.smithi012.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-05-09T09:12:41.954 INFO:teuthology.orchestra.run.smithi016.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2515.g 635 kB/s | 32 kB 00:00 2024-05-09T09:12:41.957 INFO:teuthology.orchestra.run.smithi012.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 987 kB/s | 222 kB 00:00 2024-05-09T09:12:41.996 INFO:teuthology.orchestra.run.smithi170.stdout:(9/156): ceph-selinux-19.0.0-2515.g96805dcf.el9 433 kB/s | 25 kB 00:00 2024-05-09T09:12:42.013 INFO:teuthology.orchestra.run.smithi016.stdout:(29/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 2.4 MB/s | 142 kB 00:00 2024-05-09T09:12:42.032 INFO:teuthology.orchestra.run.smithi012.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 563 kB/s | 42 kB 00:00 2024-05-09T09:12:42.057 INFO:teuthology.orchestra.run.smithi012.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 659 kB/s | 77 kB 00:00 2024-05-09T09:12:42.157 INFO:teuthology.orchestra.run.smithi012.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-05-09T09:12:42.191 INFO:teuthology.orchestra.run.smithi012.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-05-09T09:12:42.258 INFO:teuthology.orchestra.run.smithi012.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 767 kB/s | 51 kB 00:00 2024-05-09T09:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-05-09T09:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 642 kB/s | 54 kB 00:00 2024-05-09T09:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 626 kB/s | 42 kB 00:00 2024-05-09T09:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 530 kB/s | 44 kB 00:00 2024-05-09T09:12:42.576 INFO:teuthology.orchestra.run.smithi012.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-09T09:12:42.638 INFO:teuthology.orchestra.run.smithi016.stdout:(30/156): ceph-mgr-dashboard-19.0.0-2515.g96805 5.6 MB/s | 3.5 MB 00:00 2024-05-09T09:12:42.643 INFO:teuthology.orchestra.run.smithi012.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 539 kB/s | 36 kB 00:00 2024-05-09T09:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 713 kB/s | 47 kB 00:00 2024-05-09T09:12:42.860 INFO:teuthology.orchestra.run.smithi012.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.0 MB/s | 305 kB 00:00 2024-05-09T09:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 695 kB/s | 58 kB 00:00 2024-05-09T09:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-05-09T09:12:43.046 INFO:teuthology.orchestra.run.smithi170.stdout:(10/156): ceph-common-19.0.0-2515.g96805dcf.el9 2.7 MB/s | 21 MB 00:07 2024-05-09T09:12:43.120 INFO:teuthology.orchestra.run.smithi012.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 766 kB/s | 64 kB 00:00 2024-05-09T09:12:43.154 INFO:teuthology.orchestra.run.smithi170.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 2.0 MB/s | 222 kB 00:00 2024-05-09T09:12:43.171 INFO:teuthology.orchestra.run.smithi016.stdout:(31/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 11 MB/s | 48 MB 00:04 2024-05-09T09:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-05-09T09:12:43.213 INFO:teuthology.orchestra.run.smithi170.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 466 kB/s | 27 kB 00:00 2024-05-09T09:12:43.230 INFO:teuthology.orchestra.run.smithi016.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 4.2 MB/s | 249 kB 00:00 2024-05-09T09:12:43.280 INFO:teuthology.orchestra.run.smithi170.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 476 kB/s | 32 kB 00:00 2024-05-09T09:12:43.280 INFO:teuthology.orchestra.run.smithi016.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 994 kB/s | 49 kB 00:00 2024-05-09T09:12:43.331 INFO:teuthology.orchestra.run.smithi016.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2515.g9 328 kB/s | 16 kB 00:00 2024-05-09T09:12:43.398 INFO:teuthology.orchestra.run.smithi016.stdout:(35/156): ceph-volume-19.0.0-2515.g96805dcf.el9 3.9 MB/s | 264 kB 00:00 2024-05-09T09:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 99 kB/s | 22 kB 00:00 2024-05-09T09:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout:(102/156): python3-jinja2-2.11.3-5.el9.noarch.r 110 kB/s | 248 kB 00:02 2024-05-09T09:12:43.523 INFO:teuthology.orchestra.run.smithi016.stdout:(36/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 6.0 MB/s | 763 kB 00:00 2024-05-09T09:12:43.530 INFO:teuthology.orchestra.run.smithi170.stdout:(14/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 2.9 MB/s | 746 kB 00:00 2024-05-09T09:12:43.565 INFO:teuthology.orchestra.run.smithi016.stdout:(37/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 440 kB/s | 18 kB 00:00 2024-05-09T09:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-05-09T09:12:43.614 INFO:teuthology.orchestra.run.smithi170.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 1.9 MB/s | 163 kB 00:00 2024-05-09T09:12:43.681 INFO:teuthology.orchestra.run.smithi170.stdout:(16/156): librados-devel-19.0.0-2515.g96805dcf. 1.9 MB/s | 127 kB 00:00 2024-05-09T09:12:43.690 INFO:teuthology.orchestra.run.smithi016.stdout:(38/156): ceph-mgr-diskprediction-local-19.0.0- 7.0 MB/s | 7.4 MB 00:01 2024-05-09T09:12:43.741 INFO:teuthology.orchestra.run.smithi016.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 458 kB/s | 80 kB 00:00 2024-05-09T09:12:43.804 INFO:teuthology.orchestra.run.smithi012.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.5 MB/s | 309 kB 00:00 2024-05-09T09:12:43.814 INFO:teuthology.orchestra.run.smithi170.stdout:(17/156): libradosstriper1-19.0.0-2515.g96805dc 3.7 MB/s | 500 kB 00:00 2024-05-09T09:12:43.829 INFO:teuthology.orchestra.run.smithi012.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-05-09T09:12:43.849 INFO:teuthology.orchestra.run.smithi016.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-05-09T09:12:43.855 INFO:teuthology.orchestra.run.smithi012.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-05-09T09:12:43.905 INFO:teuthology.orchestra.run.smithi012.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 522 kB/s | 26 kB 00:00 2024-05-09T09:12:43.955 INFO:teuthology.orchestra.run.smithi012.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 974 kB/s | 49 kB 00:00 2024-05-09T09:12:43.966 INFO:teuthology.orchestra.run.smithi016.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-05-09T09:12:44.031 INFO:teuthology.orchestra.run.smithi012.stdout:(109/156): libarrow-9.0.0-11.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-05-09T09:12:44.056 INFO:teuthology.orchestra.run.smithi012.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 674 kB/s | 67 kB 00:00 2024-05-09T09:12:44.066 INFO:teuthology.orchestra.run.smithi016.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-05-09T09:12:44.081 INFO:teuthology.orchestra.run.smithi012.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-05-09T09:12:44.106 INFO:teuthology.orchestra.run.smithi012.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-05-09T09:12:44.117 INFO:teuthology.orchestra.run.smithi016.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-09T09:12:44.140 INFO:teuthology.orchestra.run.smithi012.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.1 MB/s | 548 kB 00:00 2024-05-09T09:12:44.174 INFO:teuthology.orchestra.run.smithi012.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-05-09T09:12:44.199 INFO:teuthology.orchestra.run.smithi012.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 552 kB/s | 32 kB 00:00 2024-05-09T09:12:44.200 INFO:teuthology.orchestra.run.smithi016.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-09T09:12:44.216 INFO:teuthology.orchestra.run.smithi012.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 360 kB/s | 15 kB 00:00 2024-05-09T09:12:44.233 INFO:teuthology.orchestra.run.smithi012.stdout:(117/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 345 kB/s | 43 kB 00:00 2024-05-09T09:12:44.258 INFO:teuthology.orchestra.run.smithi012.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-09T09:12:44.292 INFO:teuthology.orchestra.run.smithi012.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-05-09T09:12:44.309 INFO:teuthology.orchestra.run.smithi012.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 704 kB/s | 35 kB 00:00 2024-05-09T09:12:44.342 INFO:teuthology.orchestra.run.smithi012.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-05-09T09:12:44.368 INFO:teuthology.orchestra.run.smithi012.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-05-09T09:12:44.393 INFO:teuthology.orchestra.run.smithi012.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-05-09T09:12:44.409 INFO:teuthology.orchestra.run.smithi016.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-05-09T09:12:44.418 INFO:teuthology.orchestra.run.smithi012.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-05-09T09:12:44.434 INFO:teuthology.orchestra.run.smithi016.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 535 kB/s | 125 kB 00:00 2024-05-09T09:12:44.444 INFO:teuthology.orchestra.run.smithi012.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 350 kB/s | 18 kB 00:00 2024-05-09T09:12:44.469 INFO:teuthology.orchestra.run.smithi012.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-05-09T09:12:44.493 INFO:teuthology.orchestra.run.smithi016.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-09T09:12:44.494 INFO:teuthology.orchestra.run.smithi012.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-05-09T09:12:44.520 INFO:teuthology.orchestra.run.smithi012.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-05-09T09:12:44.570 INFO:teuthology.orchestra.run.smithi012.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.8 MB/s | 2.0 MB 00:00 2024-05-09T09:12:44.595 INFO:teuthology.orchestra.run.smithi012.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-05-09T09:12:44.619 INFO:teuthology.orchestra.run.smithi016.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-05-09T09:12:44.620 INFO:teuthology.orchestra.run.smithi012.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-09T09:12:44.646 INFO:teuthology.orchestra.run.smithi012.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-09T09:12:44.669 INFO:teuthology.orchestra.run.smithi016.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-05-09T09:12:44.679 INFO:teuthology.orchestra.run.smithi012.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-05-09T09:12:44.702 INFO:teuthology.orchestra.run.smithi016.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-05-09T09:12:44.704 INFO:teuthology.orchestra.run.smithi012.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-05-09T09:12:44.715 INFO:teuthology.orchestra.run.smithi170.stdout:(18/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x8 2.8 MB/s | 17 MB 00:05 2024-05-09T09:12:44.728 INFO:teuthology.orchestra.run.smithi016.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 367 kB/s | 21 kB 00:00 2024-05-09T09:12:44.730 INFO:teuthology.orchestra.run.smithi012.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-05-09T09:12:44.753 INFO:teuthology.orchestra.run.smithi016.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-09T09:12:44.755 INFO:teuthology.orchestra.run.smithi012.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-09T09:12:44.778 INFO:teuthology.orchestra.run.smithi016.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-05-09T09:12:44.780 INFO:teuthology.orchestra.run.smithi012.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-05-09T09:12:44.782 INFO:teuthology.orchestra.run.smithi170.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 681 kB/s | 45 kB 00:00 2024-05-09T09:12:44.806 INFO:teuthology.orchestra.run.smithi012.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-05-09T09:12:44.823 INFO:teuthology.orchestra.run.smithi012.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2024-05-09T09:12:44.829 INFO:teuthology.orchestra.run.smithi016.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-05-09T09:12:44.848 INFO:teuthology.orchestra.run.smithi012.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 885 kB/s | 59 kB 00:00 2024-05-09T09:12:44.866 INFO:teuthology.orchestra.run.smithi170.stdout:(20/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 5.1 MB/s | 5.3 MB 00:01 2024-05-09T09:12:44.873 INFO:teuthology.orchestra.run.smithi012.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 532 kB/s | 36 kB 00:00 2024-05-09T09:12:44.879 INFO:teuthology.orchestra.run.smithi016.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-05-09T09:12:44.891 INFO:teuthology.orchestra.run.smithi170.stdout:(21/156): python3-ceph-common-19.0.0-2515.g9680 1.2 MB/s | 135 kB 00:00 2024-05-09T09:12:44.899 INFO:teuthology.orchestra.run.smithi012.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-09T09:12:44.924 INFO:teuthology.orchestra.run.smithi012.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-05-09T09:12:44.929 INFO:teuthology.orchestra.run.smithi016.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-05-09T09:12:44.933 INFO:teuthology.orchestra.run.smithi170.stdout:(22/156): python3-cephfs-19.0.0-2515.g96805dcf. 2.4 MB/s | 162 kB 00:00 2024-05-09T09:12:44.949 INFO:teuthology.orchestra.run.smithi012.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-09T09:12:44.954 INFO:teuthology.orchestra.run.smithi016.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-05-09T09:12:44.974 INFO:teuthology.orchestra.run.smithi012.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-05-09T09:12:44.979 INFO:teuthology.orchestra.run.smithi016.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-05-09T09:12:44.999 INFO:teuthology.orchestra.run.smithi170.stdout:(23/156): python3-rados-19.0.0-2515.g96805dcf.e 2.9 MB/s | 321 kB 00:00 2024-05-09T09:12:45.008 INFO:teuthology.orchestra.run.smithi012.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-05-09T09:12:45.013 INFO:teuthology.orchestra.run.smithi016.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-09T09:12:45.033 INFO:teuthology.orchestra.run.smithi012.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-09T09:12:45.056 INFO:teuthology.orchestra.run.smithi016.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 582 kB 00:00 2024-05-09T09:12:45.058 INFO:teuthology.orchestra.run.smithi170.stdout:(24/156): python3-rbd-19.0.0-2515.g96805dcf.el9 2.3 MB/s | 298 kB 00:00 2024-05-09T09:12:45.075 INFO:teuthology.orchestra.run.smithi012.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-05-09T09:12:45.081 INFO:teuthology.orchestra.run.smithi016.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 379 kB/s | 25 kB 00:00 2024-05-09T09:12:45.083 INFO:teuthology.orchestra.run.smithi170.stdout:(25/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.2 MB/s | 99 kB 00:00 2024-05-09T09:12:45.106 INFO:teuthology.orchestra.run.smithi016.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-09T09:12:45.117 INFO:teuthology.orchestra.run.smithi170.stdout:(26/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-09T09:12:45.131 INFO:teuthology.orchestra.run.smithi016.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-09T09:12:45.157 INFO:teuthology.orchestra.run.smithi016.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-09T09:12:45.174 INFO:teuthology.orchestra.run.smithi016.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 840 kB/s | 35 kB 00:00 2024-05-09T09:12:45.184 INFO:teuthology.orchestra.run.smithi170.stdout:(27/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 2.5 MB/s | 171 kB 00:00 2024-05-09T09:12:45.251 INFO:teuthology.orchestra.run.smithi170.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2515.g 476 kB/s | 32 kB 00:00 2024-05-09T09:12:45.335 INFO:teuthology.orchestra.run.smithi170.stdout:(29/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 1.7 MB/s | 142 kB 00:00 2024-05-09T09:12:45.399 INFO:teuthology.orchestra.run.smithi016.stdout:(66/156): ceph-common-19.0.0-2515.g96805dcf.el9 2.1 MB/s | 21 MB 00:09 2024-05-09T09:12:45.476 INFO:teuthology.orchestra.run.smithi012.stdout:(149/156): librados2-19.0.0-2515.g96805dcf.el9. 7.3 MB/s | 3.4 MB 00:00 2024-05-09T09:12:45.507 INFO:teuthology.orchestra.run.smithi016.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-05-09T09:12:45.591 INFO:teuthology.orchestra.run.smithi016.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-05-09T09:12:45.625 INFO:teuthology.orchestra.run.smithi016.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-05-09T09:12:45.650 INFO:teuthology.orchestra.run.smithi016.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-09T09:12:45.701 INFO:teuthology.orchestra.run.smithi016.stdout:(71/156): postgresql-13.14-1.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-05-09T09:12:45.709 INFO:teuthology.orchestra.run.smithi012.stdout:(150/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 4.6 MB/s | 3.1 MB 00:00 2024-05-09T09:12:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-05-09T09:12:45.776 INFO:teuthology.orchestra.run.smithi016.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-05-09T09:12:45.826 INFO:teuthology.orchestra.run.smithi016.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-09T09:12:45.827 INFO:teuthology.orchestra.run.smithi170.stdout:(30/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 4.2 MB/s | 3.1 MB 00:00 2024-05-09T09:12:45.851 INFO:teuthology.orchestra.run.smithi012.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 3.0 MB/s | 2.3 MB 00:00 2024-05-09T09:12:45.902 INFO:teuthology.orchestra.run.smithi016.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-05-09T09:12:45.927 INFO:teuthology.orchestra.run.smithi016.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-05-09T09:12:45.952 INFO:teuthology.orchestra.run.smithi016.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-05-09T09:12:45.959 INFO:teuthology.orchestra.run.smithi012.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.2 MB/s | 248 kB 00:00 2024-05-09T09:12:45.977 INFO:teuthology.orchestra.run.smithi016.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-05-09T09:12:46.003 INFO:teuthology.orchestra.run.smithi016.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-05-09T09:12:46.019 INFO:teuthology.orchestra.run.smithi170.stdout:(31/156): ceph-mgr-dashboard-19.0.0-2515.g96805 5.1 MB/s | 3.5 MB 00:00 2024-05-09T09:12:46.028 INFO:teuthology.orchestra.run.smithi016.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-05-09T09:12:46.051 INFO:teuthology.orchestra.run.smithi012.stdout:(153/156): NetworkManager-team-1.47.5-1.el9.x86 122 kB/s | 42 kB 00:00 2024-05-09T09:12:46.094 INFO:teuthology.orchestra.run.smithi012.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-05-09T09:12:46.119 INFO:teuthology.orchestra.run.smithi170.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 2.4 MB/s | 249 kB 00:00 2024-05-09T09:12:46.128 INFO:teuthology.orchestra.run.smithi016.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-05-09T09:12:46.154 INFO:teuthology.orchestra.run.smithi016.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-05-09T09:12:46.177 INFO:teuthology.orchestra.run.smithi170.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 849 kB/s | 49 kB 00:00 2024-05-09T09:12:46.179 INFO:teuthology.orchestra.run.smithi016.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-09T09:12:46.204 INFO:teuthology.orchestra.run.smithi016.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-09T09:12:46.229 INFO:teuthology.orchestra.run.smithi016.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-05-09T09:12:46.244 INFO:teuthology.orchestra.run.smithi170.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2515.g9 246 kB/s | 16 kB 00:00 2024-05-09T09:12:46.305 INFO:teuthology.orchestra.run.smithi016.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-05-09T09:12:46.330 INFO:teuthology.orchestra.run.smithi016.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-05-09T09:12:46.345 INFO:teuthology.orchestra.run.smithi170.stdout:(35/156): ceph-volume-19.0.0-2515.g96805dcf.el9 2.6 MB/s | 264 kB 00:00 2024-05-09T09:12:46.356 INFO:teuthology.orchestra.run.smithi016.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-05-09T09:12:46.381 INFO:teuthology.orchestra.run.smithi016.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-05-09T09:12:46.406 INFO:teuthology.orchestra.run.smithi016.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 673 kB/s | 51 kB 00:00 2024-05-09T09:12:46.419 INFO:teuthology.orchestra.run.smithi012.stdout:(155/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.9 MB/s | 1.8 MB 00:00 2024-05-09T09:12:46.441 INFO:teuthology.orchestra.run.smithi016.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 631 kB/s | 54 kB 00:00 2024-05-09T09:12:46.482 INFO:teuthology.orchestra.run.smithi016.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-05-09T09:12:46.507 INFO:teuthology.orchestra.run.smithi016.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 671 kB/s | 44 kB 00:00 2024-05-09T09:12:46.532 INFO:teuthology.orchestra.run.smithi016.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-09T09:12:46.537 INFO:teuthology.orchestra.run.smithi170.stdout:(36/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 3.9 MB/s | 763 kB 00:00 2024-05-09T09:12:46.558 INFO:teuthology.orchestra.run.smithi016.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-05-09T09:12:46.583 INFO:teuthology.orchestra.run.smithi016.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-09T09:12:46.604 INFO:teuthology.orchestra.run.smithi170.stdout:(37/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 274 kB/s | 18 kB 00:00 2024-05-09T09:12:46.608 INFO:teuthology.orchestra.run.smithi016.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-09T09:12:46.625 INFO:teuthology.orchestra.run.smithi016.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-05-09T09:12:46.659 INFO:teuthology.orchestra.run.smithi016.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-09T09:12:46.676 INFO:teuthology.orchestra.run.smithi016.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-09T09:12:46.701 INFO:teuthology.orchestra.run.smithi016.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-05-09T09:12:46.802 INFO:teuthology.orchestra.run.smithi012.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.2 MB/s | 2.4 MB 00:00 2024-05-09T09:12:46.805 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-05-09T09:12:46.805 INFO:teuthology.orchestra.run.smithi012.stdout:Total 14 MB/s | 225 MB 00:15 2024-05-09T09:12:46.935 INFO:teuthology.orchestra.run.smithi016.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-05-09T09:12:47.029 INFO:teuthology.orchestra.run.smithi170.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 189 kB/s | 80 kB 00:00 2024-05-09T09:12:47.168 INFO:teuthology.orchestra.run.smithi016.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-05-09T09:12:47.277 INFO:teuthology.orchestra.run.smithi016.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-05-09T09:12:47.321 INFO:teuthology.orchestra.run.smithi170.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 5.0 MB/s | 7.4 MB 00:01 2024-05-09T09:12:47.336 INFO:teuthology.orchestra.run.smithi016.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-05-09T09:12:47.386 INFO:teuthology.orchestra.run.smithi016.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.4 MB/s | 309 kB 00:00 2024-05-09T09:12:47.444 INFO:teuthology.orchestra.run.smithi016.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-05-09T09:12:47.470 INFO:teuthology.orchestra.run.smithi016.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-05-09T09:12:47.495 INFO:teuthology.orchestra.run.smithi016.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 964 kB/s | 49 kB 00:00 2024-05-09T09:12:47.587 INFO:teuthology.orchestra.run.smithi016.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-05-09T09:12:47.612 INFO:teuthology.orchestra.run.smithi016.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-09T09:12:47.646 INFO:teuthology.orchestra.run.smithi016.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-05-09T09:12:47.680 INFO:teuthology.orchestra.run.smithi016.stdout:(113/156): libunwind-1.6.2-1.el9.x86_64.rpm 321 kB/s | 67 kB 00:00 2024-05-09T09:12:47.705 INFO:teuthology.orchestra.run.smithi016.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-05-09T09:12:47.730 INFO:teuthology.orchestra.run.smithi016.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 519 kB/s | 43 kB 00:00 2024-05-09T09:12:47.756 INFO:teuthology.orchestra.run.smithi016.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 425 kB/s | 32 kB 00:00 2024-05-09T09:12:47.781 INFO:teuthology.orchestra.run.smithi016.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 297 kB/s | 15 kB 00:00 2024-05-09T09:12:47.806 INFO:teuthology.orchestra.run.smithi016.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-09T09:12:47.831 INFO:teuthology.orchestra.run.smithi016.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-05-09T09:12:47.855 INFO:teuthology.orchestra.run.smithi170.stdout:(40/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 8.2 MB/s | 48 MB 00:05 2024-05-09T09:12:47.856 INFO:teuthology.orchestra.run.smithi016.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 705 kB/s | 35 kB 00:00 2024-05-09T09:12:47.883 INFO:teuthology.orchestra.run.smithi016.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-05-09T09:12:47.888 INFO:teuthology.orchestra.run.smithi170.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 935 kB/s | 803 kB 00:00 2024-05-09T09:12:47.907 INFO:teuthology.orchestra.run.smithi016.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-09T09:12:47.928 INFO:teuthology.orchestra.run.smithi016.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-05-09T09:12:47.950 INFO:teuthology.orchestra.run.smithi016.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 254 kB/s | 11 kB 00:00 2024-05-09T09:12:47.975 INFO:teuthology.orchestra.run.smithi016.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-05-09T09:12:48.000 INFO:teuthology.orchestra.run.smithi016.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-05-09T09:12:48.025 INFO:teuthology.orchestra.run.smithi016.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-05-09T09:12:48.051 INFO:teuthology.orchestra.run.smithi016.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-05-09T09:12:48.060 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-05-09T09:12:48.109 INFO:teuthology.orchestra.run.smithi016.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-05-09T09:12:48.135 INFO:teuthology.orchestra.run.smithi016.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 555 kB/s | 46 kB 00:00 2024-05-09T09:12:48.161 INFO:teuthology.orchestra.run.smithi016.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-09T09:12:48.172 INFO:teuthology.orchestra.run.smithi170.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 4.3 MB/s | 1.2 MB 00:00 2024-05-09T09:12:48.185 INFO:teuthology.orchestra.run.smithi016.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-09T09:12:48.210 INFO:teuthology.orchestra.run.smithi016.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-05-09T09:12:48.226 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-05-09T09:12:48.226 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-05-09T09:12:48.236 INFO:teuthology.orchestra.run.smithi016.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-05-09T09:12:48.253 INFO:teuthology.orchestra.run.smithi016.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-05-09T09:12:48.281 INFO:teuthology.orchestra.run.smithi170.stdout:(43/156): libquadmath-11.4.1-3.el9.x86_64.rpm 204 kB/s | 195 kB 00:00 2024-05-09T09:12:48.286 INFO:teuthology.orchestra.run.smithi016.stdout:(136/156): python3-kubernetes-26.1.0-2.el9.noar 3.9 MB/s | 1.0 MB 00:00 2024-05-09T09:12:48.306 INFO:teuthology.orchestra.run.smithi170.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 800 kB/s | 106 kB 00:00 2024-05-09T09:12:48.311 INFO:teuthology.orchestra.run.smithi016.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-05-09T09:12:48.337 INFO:teuthology.orchestra.run.smithi016.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 366 kB/s | 31 kB 00:00 2024-05-09T09:12:48.362 INFO:teuthology.orchestra.run.smithi016.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-09T09:12:48.365 INFO:teuthology.orchestra.run.smithi170.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 497 kB/s | 253 kB 00:00 2024-05-09T09:12:48.379 INFO:teuthology.orchestra.run.smithi016.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 857 kB/s | 36 kB 00:00 2024-05-09T09:12:48.390 INFO:teuthology.orchestra.run.smithi170.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-09T09:12:48.396 INFO:teuthology.orchestra.run.smithi016.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-09T09:12:48.415 INFO:teuthology.orchestra.run.smithi170.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-09T09:12:48.421 INFO:teuthology.orchestra.run.smithi016.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 867 kB/s | 51 kB 00:00 2024-05-09T09:12:48.447 INFO:teuthology.orchestra.run.smithi016.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-09T09:12:48.472 INFO:teuthology.orchestra.run.smithi016.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-05-09T09:12:48.489 INFO:teuthology.orchestra.run.smithi016.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-05-09T09:12:48.514 INFO:teuthology.orchestra.run.smithi016.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-05-09T09:12:48.540 INFO:teuthology.orchestra.run.smithi016.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-05-09T09:12:48.557 INFO:teuthology.orchestra.run.smithi170.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.1 MB/s | 215 kB 00:00 2024-05-09T09:12:48.608 INFO:teuthology.orchestra.run.smithi170.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-05-09T09:12:48.615 INFO:teuthology.orchestra.run.smithi016.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-09T09:12:48.641 INFO:teuthology.orchestra.run.smithi170.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-05-09T09:12:48.667 INFO:teuthology.orchestra.run.smithi170.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 367 kB/s | 21 kB 00:00 2024-05-09T09:12:48.692 INFO:teuthology.orchestra.run.smithi170.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-09T09:12:48.717 INFO:teuthology.orchestra.run.smithi170.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-05-09T09:12:48.743 INFO:teuthology.orchestra.run.smithi170.stdout:(54/156): boost-program-options-1.75.0-8.el9.x8 579 kB/s | 107 kB 00:00 2024-05-09T09:12:48.776 INFO:teuthology.orchestra.run.smithi170.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-05-09T09:12:48.802 INFO:teuthology.orchestra.run.smithi170.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 189 kB/s | 11 kB 00:00 2024-05-09T09:12:48.860 INFO:teuthology.orchestra.run.smithi170.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-05-09T09:12:48.885 INFO:teuthology.orchestra.run.smithi170.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-05-09T09:12:48.890 INFO:teuthology.orchestra.run.smithi016.stdout:(149/156): librados2-19.0.0-2515.g96805dcf.el9. 9.0 MB/s | 3.4 MB 00:00 2024-05-09T09:12:48.911 INFO:teuthology.orchestra.run.smithi170.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-09T09:12:48.936 INFO:teuthology.orchestra.run.smithi170.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-05-09T09:12:48.961 INFO:teuthology.orchestra.run.smithi170.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-05-09T09:12:48.987 INFO:teuthology.orchestra.run.smithi170.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-09T09:12:49.012 INFO:teuthology.orchestra.run.smithi170.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-09T09:12:49.037 INFO:teuthology.orchestra.run.smithi170.stdout:(64/156): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-05-09T09:12:49.054 INFO:teuthology.orchestra.run.smithi170.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 841 kB/s | 35 kB 00:00 2024-05-09T09:12:49.079 INFO:teuthology.orchestra.run.smithi170.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-09T09:12:49.163 INFO:teuthology.orchestra.run.smithi170.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-05-09T09:12:49.255 INFO:teuthology.orchestra.run.smithi170.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-05-09T09:12:49.257 INFO:teuthology.orchestra.run.smithi016.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 3.6 MB/s | 2.3 MB 00:00 2024-05-09T09:12:49.308 INFO:teuthology.orchestra.run.smithi016.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 839 kB/s | 42 kB 00:00 2024-05-09T09:12:49.322 INFO:teuthology.orchestra.run.smithi170.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-05-09T09:12:49.347 INFO:teuthology.orchestra.run.smithi170.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 4.8 MB/s | 911 kB 00:00 2024-05-09T09:12:49.372 INFO:teuthology.orchestra.run.smithi170.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-05-09T09:12:49.400 INFO:teuthology.orchestra.run.smithi016.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.7 MB/s | 248 kB 00:00 2024-05-09T09:12:49.406 INFO:teuthology.orchestra.run.smithi170.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-05-09T09:12:49.458 INFO:teuthology.orchestra.run.smithi016.stdout:(153/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 3.4 MB/s | 3.1 MB 00:00 2024-05-09T09:12:49.492 INFO:teuthology.orchestra.run.smithi016.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 3.0 MB/s | 1.8 MB 00:00 2024-05-09T09:12:49.607 INFO:teuthology.orchestra.run.smithi170.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-05-09T09:12:49.657 INFO:teuthology.orchestra.run.smithi170.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-09T09:12:49.658 INFO:teuthology.orchestra.run.smithi016.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 1.3 MB 00:00 2024-05-09T09:12:49.716 INFO:teuthology.orchestra.run.smithi170.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-05-09T09:12:49.741 INFO:teuthology.orchestra.run.smithi170.stdout:(76/156): python3-attrs-20.3.0-7.el9.noarch.rpm 237 kB/s | 87 kB 00:00 2024-05-09T09:12:49.766 INFO:teuthology.orchestra.run.smithi170.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-09T09:12:49.816 INFO:teuthology.orchestra.run.smithi170.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-09T09:12:49.867 INFO:teuthology.orchestra.run.smithi170.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-05-09T09:12:49.892 INFO:teuthology.orchestra.run.smithi170.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 127 kB/s | 19 kB 00:00 2024-05-09T09:12:49.951 INFO:teuthology.orchestra.run.smithi170.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-05-09T09:12:50.035 INFO:teuthology.orchestra.run.smithi170.stdout:(82/156): postgresql-server-13.14-1.el9.x86_64. 8.1 MB/s | 5.8 MB 00:00 2024-05-09T09:12:50.060 INFO:teuthology.orchestra.run.smithi170.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-05-09T09:12:50.135 INFO:teuthology.orchestra.run.smithi170.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-05-09T09:12:50.178 INFO:teuthology.orchestra.run.smithi170.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 542 kB/s | 77 kB 00:00 2024-05-09T09:12:50.200 INFO:teuthology.orchestra.run.smithi016.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.2 MB/s | 2.4 MB 00:00 2024-05-09T09:12:50.202 INFO:teuthology.orchestra.run.smithi170.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-05-09T09:12:50.207 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-05-09T09:12:50.207 INFO:teuthology.orchestra.run.smithi016.stdout:Total 15 MB/s | 225 MB 00:14 2024-05-09T09:12:50.253 INFO:teuthology.orchestra.run.smithi170.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-05-09T09:12:50.295 INFO:teuthology.orchestra.run.smithi170.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-05-09T09:12:50.336 INFO:teuthology.orchestra.run.smithi170.stdout:(89/156): python3-prettytable-0.7.2-27.el9.noar 153 kB/s | 42 kB 00:00 2024-05-09T09:12:50.362 INFO:teuthology.orchestra.run.smithi170.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 802 kB/s | 54 kB 00:00 2024-05-09T09:12:50.387 INFO:teuthology.orchestra.run.smithi170.stdout:(91/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 757 kB/s | 159 kB 00:00 2024-05-09T09:12:50.412 INFO:teuthology.orchestra.run.smithi170.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 826 kB/s | 42 kB 00:00 2024-05-09T09:12:50.446 INFO:teuthology.orchestra.run.smithi170.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-05-09T09:12:50.471 INFO:teuthology.orchestra.run.smithi170.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-09T09:12:50.496 INFO:teuthology.orchestra.run.smithi170.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-05-09T09:12:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-05-09T09:12:50.504 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-05-09T09:12:50.522 INFO:teuthology.orchestra.run.smithi170.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-09T09:12:50.572 INFO:teuthology.orchestra.run.smithi170.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 305 kB 00:00 2024-05-09T09:12:50.597 INFO:teuthology.orchestra.run.smithi170.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 766 kB/s | 58 kB 00:00 2024-05-09T09:12:50.648 INFO:teuthology.orchestra.run.smithi170.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-09T09:12:50.690 INFO:teuthology.orchestra.run.smithi170.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-09T09:12:50.740 INFO:teuthology.orchestra.run.smithi170.stdout:(101/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 559 kB/s | 94 kB 00:00 2024-05-09T09:12:50.991 INFO:teuthology.orchestra.run.smithi170.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-05-09T09:12:51.416 INFO:teuthology.orchestra.run.smithi170.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-05-09T09:12:51.441 INFO:teuthology.orchestra.run.smithi170.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-05-09T09:12:51.455 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-05-09T09:12:51.516 INFO:teuthology.orchestra.run.smithi170.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-05-09T09:12:51.622 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-05-09T09:12:51.622 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-05-09T09:12:51.802 INFO:teuthology.orchestra.run.smithi170.stdout:(106/156): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-09T09:12:51.826 INFO:teuthology.orchestra.run.smithi170.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 756 kB/s | 309 kB 00:00 2024-05-09T09:12:51.860 INFO:teuthology.orchestra.run.smithi170.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 772 kB/s | 863 kB 00:01 2024-05-09T09:12:51.884 INFO:teuthology.orchestra.run.smithi170.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 309 kB/s | 26 kB 00:00 2024-05-09T09:12:51.910 INFO:teuthology.orchestra.run.smithi170.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 578 kB/s | 49 kB 00:00 2024-05-09T09:12:51.961 INFO:teuthology.orchestra.run.smithi170.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-09T09:12:52.060 INFO:teuthology.orchestra.run.smithi170.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-05-09T09:12:52.110 INFO:teuthology.orchestra.run.smithi170.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-05-09T09:12:52.219 INFO:teuthology.orchestra.run.smithi170.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 274 kB/s | 43 kB 00:00 2024-05-09T09:12:52.253 INFO:teuthology.orchestra.run.smithi170.stdout:(115/156): libunwind-1.6.2-1.el9.x86_64.rpm 171 kB/s | 67 kB 00:00 2024-05-09T09:12:52.270 INFO:teuthology.orchestra.run.smithi170.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 203 kB/s | 32 kB 00:00 2024-05-09T09:12:52.287 INFO:teuthology.orchestra.run.smithi170.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 223 kB/s | 15 kB 00:00 2024-05-09T09:12:52.346 INFO:teuthology.orchestra.run.smithi170.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-09T09:12:52.370 INFO:teuthology.orchestra.run.smithi170.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-09T09:12:52.412 INFO:teuthology.orchestra.run.smithi170.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 529 kB/s | 35 kB 00:00 2024-05-09T09:12:52.446 INFO:teuthology.orchestra.run.smithi170.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-09T09:12:52.488 INFO:teuthology.orchestra.run.smithi170.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.5 MB/s | 303 kB 00:00 2024-05-09T09:12:52.547 INFO:teuthology.orchestra.run.smithi170.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-05-09T09:12:52.572 INFO:teuthology.orchestra.run.smithi170.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-05-09T09:12:52.597 INFO:teuthology.orchestra.run.smithi170.stdout:(125/156): python3-grpcio-tools-1.46.7-10.el9.x 959 kB/s | 144 kB 00:00 2024-05-09T09:12:52.622 INFO:teuthology.orchestra.run.smithi170.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-05-09T09:12:52.647 INFO:teuthology.orchestra.run.smithi170.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 306 kB/s | 23 kB 00:00 2024-05-09T09:12:52.673 INFO:teuthology.orchestra.run.smithi170.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 256 kB/s | 19 kB 00:00 2024-05-09T09:12:52.698 INFO:teuthology.orchestra.run.smithi170.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 259 kB/s | 20 kB 00:00 2024-05-09T09:12:52.741 INFO:teuthology.orchestra.run.smithi170.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 694 kB/s | 46 kB 00:00 2024-05-09T09:12:52.766 INFO:teuthology.orchestra.run.smithi170.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-05-09T09:12:52.807 INFO:teuthology.orchestra.run.smithi170.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 862 kB/s | 58 kB 00:00 2024-05-09T09:12:52.841 INFO:teuthology.orchestra.run.smithi170.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-05-09T09:12:52.867 INFO:teuthology.orchestra.run.smithi170.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-05-09T09:12:52.892 INFO:teuthology.orchestra.run.smithi170.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 195 kB/s | 16 kB 00:00 2024-05-09T09:12:52.917 INFO:teuthology.orchestra.run.smithi170.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-09T09:12:52.942 INFO:teuthology.orchestra.run.smithi170.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-05-09T09:12:52.968 INFO:teuthology.orchestra.run.smithi170.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-05-09T09:12:52.993 INFO:teuthology.orchestra.run.smithi170.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-09T09:12:53.018 INFO:teuthology.orchestra.run.smithi170.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-05-09T09:12:53.043 INFO:teuthology.orchestra.run.smithi170.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-05-09T09:12:53.069 INFO:teuthology.orchestra.run.smithi170.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-05-09T09:12:53.094 INFO:teuthology.orchestra.run.smithi170.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-09T09:12:53.119 INFO:teuthology.orchestra.run.smithi170.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-09T09:12:53.153 INFO:teuthology.orchestra.run.smithi170.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-09T09:12:53.178 INFO:teuthology.orchestra.run.smithi170.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-05-09T09:12:53.203 INFO:teuthology.orchestra.run.smithi170.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-09T09:12:53.279 INFO:teuthology.orchestra.run.smithi170.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-09T09:12:53.546 INFO:teuthology.orchestra.run.smithi170.stdout:(149/156): librados2-19.0.0-2515.g96805dcf.el9. 9.2 MB/s | 3.4 MB 00:00 2024-05-09T09:12:53.913 INFO:teuthology.orchestra.run.smithi170.stdout:(150/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 4.4 MB/s | 3.1 MB 00:00 2024-05-09T09:12:53.942 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-05-09T09:12:53.942 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-05-09T09:12:54.196 INFO:teuthology.orchestra.run.smithi170.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.5 MB/s | 2.3 MB 00:00 2024-05-09T09:12:54.257 INFO:teuthology.orchestra.run.smithi170.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 122 kB/s | 42 kB 00:00 2024-05-09T09:12:54.330 INFO:teuthology.orchestra.run.smithi170.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.8 MB/s | 248 kB 00:00 2024-05-09T09:12:54.622 INFO:teuthology.orchestra.run.smithi170.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-05-09T09:12:54.881 INFO:teuthology.orchestra.run.smithi170.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 1.3 MB 00:00 2024-05-09T09:12:55.406 INFO:teuthology.orchestra.run.smithi170.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-05-09T09:12:55.413 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-05-09T09:12:55.413 INFO:teuthology.orchestra.run.smithi170.stdout:Total 11 MB/s | 225 MB 00:20 2024-05-09T09:12:56.623 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-05-09T09:12:56.772 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-05-09T09:12:56.773 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-05-09T09:12:56.969 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-05-09T09:12:57.057 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T09:12:57.090 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T09:12:57.324 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T09:12:57.366 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T09:12:57.392 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T09:12:57.450 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T09:12:57.799 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T09:12:57.900 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T09:12:58.017 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T09:12:58.043 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T09:12:58.120 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T09:12:58.148 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T09:12:58.339 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T09:12:58.396 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T09:12:58.422 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T09:12:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T09:12:58.557 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T09:12:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T09:12:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T09:12:58.676 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T09:12:58.877 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T09:12:58.951 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-05-09T09:12:58.952 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-05-09T09:12:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T09:12:59.172 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T09:12:59.240 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T09:12:59.357 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T09:12:59.417 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T09:12:59.614 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T09:12:59.725 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T09:12:59.821 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T09:12:59.871 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T09:12:59.907 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T09:12:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T09:13:00.024 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T09:13:00.069 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T09:13:00.100 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T09:13:00.127 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T09:13:00.198 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T09:13:00.259 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T09:13:00.289 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T09:13:00.333 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T09:13:00.382 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-05-09T09:13:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T09:13:00.443 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T09:13:00.473 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T09:13:00.493 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T09:13:00.511 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T09:13:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T09:13:00.577 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T09:13:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T09:13:00.617 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T09:13:00.640 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T09:13:00.647 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T09:13:00.696 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T09:13:00.724 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T09:13:00.870 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T09:13:00.925 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T09:13:01.052 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T09:13:01.157 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T09:13:01.291 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T09:13:01.317 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T09:13:01.410 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T09:13:01.438 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T09:13:01.485 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T09:13:01.581 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T09:13:01.615 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T09:13:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T09:13:01.659 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T09:13:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T09:13:01.697 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T09:13:01.705 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T09:13:01.739 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T09:13:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T09:13:01.839 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T09:13:01.892 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T09:13:01.902 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T09:13:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T09:13:01.958 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T09:13:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T09:13:02.014 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T09:13:02.088 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T09:13:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T09:13:02.125 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T09:13:02.162 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T09:13:02.209 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T09:13:02.262 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T09:13:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T09:13:02.305 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T09:13:02.346 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T09:13:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T09:13:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T09:13:02.489 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T09:13:02.536 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T09:13:02.566 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T09:13:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T09:13:02.616 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T09:13:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T09:13:02.714 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T09:13:02.834 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T09:13:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T09:13:02.874 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T09:13:02.915 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T09:13:02.940 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T09:13:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T09:13:03.028 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T09:13:03.078 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T09:13:03.125 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T09:13:03.166 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T09:13:03.166 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T09:13:03.205 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T09:13:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T09:13:03.240 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T09:13:03.269 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T09:13:03.281 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T09:13:03.308 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T09:13:03.324 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T09:13:03.347 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T09:13:03.357 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T09:13:03.377 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T09:13:03.384 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T09:13:03.411 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T09:13:03.444 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T09:13:03.457 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T09:13:03.487 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T09:13:03.508 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T09:13:03.520 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T09:13:03.538 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T09:13:03.581 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T09:13:03.610 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T09:13:03.659 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T09:13:03.670 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T09:13:03.691 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T09:13:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T09:13:03.739 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T09:13:03.755 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T09:13:03.772 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T09:13:03.846 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T09:13:03.866 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T09:13:03.887 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T09:13:03.926 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T09:13:03.965 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T09:13:04.110 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T09:13:04.175 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T09:13:04.686 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T09:13:04.731 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T09:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T09:13:04.764 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T09:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T09:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T09:13:04.867 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T09:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T09:13:04.915 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T09:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T09:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T09:13:05.013 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T09:13:05.050 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T09:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T09:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T09:13:05.233 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T09:13:05.264 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T09:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T09:13:05.283 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-05-09T09:13:05.345 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T09:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T09:13:05.378 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T09:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T09:13:05.415 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T09:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T09:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T09:13:05.489 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T09:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T09:13:05.529 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T09:13:05.553 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T09:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T09:13:05.604 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T09:13:05.606 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T09:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T09:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T09:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T09:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T09:13:05.938 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T09:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T09:13:06.034 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T09:13:06.046 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T09:13:06.057 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T09:13:06.158 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T09:13:06.176 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T09:13:06.206 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T09:13:06.232 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T09:13:06.263 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T09:13:06.288 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T09:13:06.297 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T09:13:06.327 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T09:13:06.382 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T09:13:06.505 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T09:13:06.516 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T09:13:06.553 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T09:13:06.565 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T09:13:06.582 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T09:13:06.593 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T09:13:06.616 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T09:13:06.656 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T09:13:06.688 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T09:13:06.695 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T09:13:06.711 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T09:13:06.724 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T09:13:06.751 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T09:13:06.773 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T09:13:06.784 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T09:13:06.795 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T09:13:06.815 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T09:13:06.826 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T09:13:06.860 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T09:13:06.951 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T09:13:06.956 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T09:13:07.000 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T09:13:07.030 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T09:13:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T09:13:07.110 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T09:13:07.189 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T09:13:07.249 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T09:13:07.251 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T09:13:07.311 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T09:13:07.431 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T09:13:07.479 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T09:13:07.513 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T09:13:07.560 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T09:13:07.587 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T09:13:07.621 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T09:13:07.731 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T09:13:07.852 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T09:13:07.957 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T09:13:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T09:13:07.996 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T09:13:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T09:13:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T09:13:08.029 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T09:13:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T09:13:08.075 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T09:13:08.087 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T09:13:08.105 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T09:13:08.113 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T09:13:08.142 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T09:13:08.145 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T09:13:08.188 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T09:13:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T09:13:08.221 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T09:13:08.248 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T09:13:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T09:13:08.317 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T09:13:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T09:13:08.370 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T09:13:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T09:13:08.401 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T09:13:08.446 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T09:13:08.523 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T09:13:08.556 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T09:13:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T09:13:08.600 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T09:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T09:13:08.628 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T09:13:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T09:13:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T09:13:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T09:13:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T09:13:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T09:13:08.759 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T09:13:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T09:13:08.834 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T09:13:08.969 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T09:13:09.029 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T09:13:09.379 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T09:13:09.506 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T09:13:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T09:13:09.650 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T09:13:09.679 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T09:13:09.714 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T09:13:09.751 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T09:13:09.794 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T09:13:09.951 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T09:13:09.996 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T09:13:10.051 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T09:13:10.124 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T09:13:10.155 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T09:13:10.192 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T09:13:10.240 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T09:13:10.274 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T09:13:10.317 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T09:13:10.360 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T09:13:10.496 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T09:13:10.540 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T09:13:10.571 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T09:13:10.613 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T09:13:10.708 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T09:13:10.755 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T09:13:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T09:13:10.828 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T09:13:10.831 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T09:13:10.853 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T09:13:10.908 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T09:13:10.936 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T09:13:10.950 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T09:13:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T09:13:11.081 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T09:13:11.119 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T09:13:11.156 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T09:13:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T09:13:11.193 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T09:13:11.195 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T09:13:11.237 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T09:13:11.269 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T09:13:11.298 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T09:13:11.303 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T09:13:11.333 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T09:13:11.348 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T09:13:11.366 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T09:13:11.408 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T09:13:11.440 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T09:13:11.442 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T09:13:11.470 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T09:13:11.507 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T09:13:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T09:13:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T09:13:11.604 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T09:13:11.652 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T09:13:11.750 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T09:13:11.770 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T09:13:11.804 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T09:13:11.806 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T09:13:11.897 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T09:13:11.897 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T09:13:11.898 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T09:13:11.898 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:13:11.931 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T09:13:12.007 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T09:13:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T09:13:12.089 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T09:13:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T09:13:12.534 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T09:13:12.578 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T09:13:12.620 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T09:13:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T09:13:12.779 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T09:13:12.867 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T09:13:12.906 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T09:13:13.084 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T09:13:13.112 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T09:13:13.191 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T09:13:13.846 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T09:13:13.969 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T09:13:14.480 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T09:13:14.516 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T09:13:15.098 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T09:13:15.136 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T09:13:15.222 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T09:13:15.222 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T09:13:15.223 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T09:13:15.223 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:13:15.226 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T09:13:15.262 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T09:13:15.300 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T09:13:15.327 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T09:13:15.362 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T09:13:15.688 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T09:13:15.732 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T09:13:15.823 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T09:13:15.854 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T09:13:15.890 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T09:13:16.365 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T09:13:16.394 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T09:13:16.465 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T09:13:16.495 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T09:13:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T09:13:18.825 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T09:13:19.378 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T09:13:19.429 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T09:13:19.516 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T09:13:19.516 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T09:13:19.517 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T09:13:19.517 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:13:19.555 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T09:13:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T09:13:24.319 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /sys 2024-05-09T09:13:24.319 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /proc 2024-05-09T09:13:24.319 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /mnt 2024-05-09T09:13:24.319 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /var/tmp 2024-05-09T09:13:24.319 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /home 2024-05-09T09:13:24.319 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /root 2024-05-09T09:13:24.319 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /tmp 2024-05-09T09:13:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:13:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T09:13:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T09:13:24.505 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:24.506 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T09:13:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:13:26.050 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T09:13:26.078 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T09:13:26.079 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:26.079 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T09:13:26.079 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T09:13:26.079 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T09:13:26.079 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:13:26.317 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T09:13:26.343 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T09:13:26.343 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:26.343 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T09:13:26.343 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T09:13:26.343 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T09:13:26.343 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:13:26.818 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T09:13:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T09:13:26.843 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:26.843 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T09:13:26.843 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T09:13:26.843 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T09:13:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:13:26.948 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T09:13:27.006 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T09:13:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T09:13:27.273 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T09:13:27.319 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T09:13:27.399 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T09:13:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T09:13:27.650 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T09:13:27.694 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T09:13:27.704 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T09:13:27.705 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-05-09T09:13:27.705 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-05-09T09:13:27.705 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-05-09T09:13:27.705 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-05-09T09:13:27.705 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-05-09T09:13:27.705 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-05-09T09:13:27.705 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-05-09T09:13:27.705 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:13:27.734 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T09:13:27.762 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T09:13:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T09:13:27.894 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T09:13:27.894 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:27.895 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T09:13:27.895 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:13:28.180 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T09:13:28.211 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T09:13:28.243 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T09:13:28.306 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T09:13:28.334 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T09:13:29.421 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T09:13:29.422 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T09:13:29.451 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T09:13:29.451 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:29.452 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T09:13:29.452 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T09:13:29.452 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T09:13:29.452 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:13:29.460 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T09:13:29.694 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T09:13:29.716 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T09:13:29.716 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:29.716 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T09:13:29.717 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T09:13:29.717 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T09:13:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:13:30.184 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T09:13:30.209 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T09:13:30.209 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:30.209 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T09:13:30.209 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T09:13:30.209 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T09:13:30.209 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:13:30.315 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T09:13:30.373 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T09:13:30.437 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T09:13:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T09:13:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T09:13:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T09:13:30.634 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T09:13:30.686 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T09:13:30.767 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T09:13:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T09:13:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T09:13:30.838 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T09:13:30.838 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:30.839 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T09:13:30.839 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T09:13:30.839 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T09:13:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:13:30.864 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T09:13:30.889 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T09:13:30.901 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T09:13:31.001 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T09:13:31.009 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T09:13:31.052 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T09:13:31.057 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T09:13:31.091 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T09:13:31.130 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T09:13:31.354 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T09:13:31.354 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2024-05-09T09:13:31.354 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2024-05-09T09:13:31.354 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2024-05-09T09:13:31.354 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2024-05-09T09:13:31.354 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2024-05-09T09:13:31.354 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /root 2024-05-09T09:13:31.354 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2024-05-09T09:13:31.354 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:13:31.520 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T09:13:31.539 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T09:13:31.539 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:31.539 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T09:13:31.539 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:13:31.578 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T09:13:31.640 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T09:13:31.674 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T09:13:31.738 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T09:13:31.782 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T09:13:32.004 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T09:13:32.041 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T09:13:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T09:13:32.066 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:32.066 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T09:13:32.066 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T09:13:32.066 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T09:13:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:13:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T09:13:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T09:13:32.141 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:32.141 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T09:13:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:13:32.192 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T09:13:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T09:13:32.217 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:32.217 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T09:13:32.217 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T09:13:32.218 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T09:13:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:13:32.535 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T09:13:32.560 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T09:13:32.560 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:32.560 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T09:13:32.560 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T09:13:32.560 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T09:13:32.560 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:13:32.876 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T09:13:32.909 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T09:13:33.013 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T09:13:33.041 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T09:13:33.041 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:33.041 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T09:13:33.041 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T09:13:33.041 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T09:13:33.041 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:13:33.275 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T09:13:33.299 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T09:13:33.300 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:33.300 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T09:13:33.300 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T09:13:33.300 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T09:13:33.300 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:13:33.758 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T09:13:33.784 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T09:13:33.785 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:33.785 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T09:13:33.785 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T09:13:33.786 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T09:13:33.786 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:13:33.882 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T09:13:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T09:13:33.931 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T09:13:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T09:13:33.994 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T09:13:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T09:13:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T09:13:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T09:13:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T09:13:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T09:13:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T09:13:34.295 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:34.295 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T09:13:34.296 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T09:13:34.296 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T09:13:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:13:34.315 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T09:13:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T09:13:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T09:13:34.410 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T09:13:34.459 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T09:13:34.516 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T09:13:34.551 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T09:13:34.603 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T09:13:34.652 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T09:13:34.679 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T09:13:35.072 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T09:13:35.103 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T09:13:35.134 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T09:13:35.197 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T09:13:35.226 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T09:13:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T09:13:35.498 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T09:13:35.523 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T09:13:35.523 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:35.524 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T09:13:35.524 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T09:13:35.524 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T09:13:35.524 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:13:35.565 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T09:13:35.590 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T09:13:35.590 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:35.590 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T09:13:35.590 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:13:35.641 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T09:13:35.666 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T09:13:35.667 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:35.667 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T09:13:35.667 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T09:13:35.667 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T09:13:35.667 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:13:35.983 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T09:13:36.009 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T09:13:36.009 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:36.009 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T09:13:36.009 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T09:13:36.009 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T09:13:36.009 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:13:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T09:13:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T09:13:37.278 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T09:13:37.303 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T09:13:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T09:13:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T09:13:37.406 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T09:13:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T09:13:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T09:13:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T09:13:37.574 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T09:13:37.605 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T09:13:37.629 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T09:13:37.629 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:37.629 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T09:13:37.629 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T09:13:37.630 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T09:13:37.630 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:13:37.689 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T09:13:37.701 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T09:13:37.713 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T09:13:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T09:13:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T09:13:37.797 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T09:13:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T09:13:37.852 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T09:13:37.857 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T09:13:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T09:13:38.513 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T09:13:38.534 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T09:13:38.745 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T09:13:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T09:13:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T09:13:38.799 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:38.799 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T09:13:38.799 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T09:13:38.799 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T09:13:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:13:38.842 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T09:13:38.865 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T09:13:38.865 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:38.866 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T09:13:38.866 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:13:38.909 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T09:13:38.933 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T09:13:38.933 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:38.934 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T09:13:38.934 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T09:13:38.934 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T09:13:38.934 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:13:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T09:13:39.259 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T09:13:39.259 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T09:13:39.259 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T09:13:39.259 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T09:13:39.259 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T09:13:39.260 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:13:39.426 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T09:13:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T09:13:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T09:13:39.649 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T09:13:39.689 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T09:13:39.715 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T09:13:39.715 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T09:13:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T09:13:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T09:13:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T09:13:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T09:13:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T09:13:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T09:13:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T09:13:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T09:13:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T09:13:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T09:13:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T09:13:40.796 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T09:13:40.866 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T09:13:40.912 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T09:13:40.943 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T09:13:40.979 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T09:13:41.176 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T09:13:41.225 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T09:13:41.255 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T09:13:41.285 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T09:13:41.331 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T09:13:41.885 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T09:13:41.929 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T09:13:41.950 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T09:13:42.873 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T09:13:42.966 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T09:13:43.013 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T09:13:43.109 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T09:13:43.145 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T09:13:43.171 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T09:13:43.171 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T09:13:43.264 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T09:13:43.305 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T09:13:43.305 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T09:13:43.390 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T09:13:43.414 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T09:13:43.420 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T09:13:43.420 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T09:13:43.608 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T09:13:43.608 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T09:13:43.675 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T09:13:43.752 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T09:13:43.797 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T09:13:43.838 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T09:13:43.888 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T09:13:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T09:13:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T09:13:44.149 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T09:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T09:13:44.223 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T09:13:44.253 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T09:13:44.380 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T09:13:44.909 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T09:13:44.972 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T09:13:44.992 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T09:13:45.832 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T09:13:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T09:13:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T09:13:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T09:13:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T09:13:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T09:13:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T09:13:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T09:13:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T09:13:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T09:13:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T09:13:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T09:13:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T09:13:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T09:13:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T09:13:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T09:13:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T09:13:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T09:13:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T09:13:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T09:13:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T09:13:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T09:13:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T09:13:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T09:13:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T09:13:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T09:13:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T09:13:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T09:13:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T09:13:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T09:13:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T09:13:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T09:13:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T09:13:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T09:13:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T09:13:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T09:13:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T09:13:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T09:13:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T09:13:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T09:13:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T09:13:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T09:13:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T09:13:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T09:13:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T09:13:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T09:13:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T09:13:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T09:13:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T09:13:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T09:13:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T09:13:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T09:13:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T09:13:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T09:13:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T09:13:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T09:13:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T09:13:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T09:13:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T09:13:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T09:13:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T09:13:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T09:13:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T09:13:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T09:13:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T09:13:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T09:13:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T09:13:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T09:13:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T09:13:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T09:13:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T09:13:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T09:13:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T09:13:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T09:13:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T09:13:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T09:13:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T09:13:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T09:13:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T09:13:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T09:13:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T09:13:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T09:13:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T09:13:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T09:13:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T09:13:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T09:13:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T09:13:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T09:13:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T09:13:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T09:13:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T09:13:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T09:13:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T09:13:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T09:13:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T09:13:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T09:13:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T09:13:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T09:13:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T09:13:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T09:13:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T09:13:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T09:13:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T09:13:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T09:13:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T09:13:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T09:13:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T09:13:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T09:13:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T09:13:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T09:13:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T09:13:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T09:13:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T09:13:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T09:13:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T09:13:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T09:13:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T09:13:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T09:13:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T09:13:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T09:13:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T09:13:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T09:13:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T09:13:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T09:13:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T09:13:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T09:13:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T09:13:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T09:13:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T09:13:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T09:13:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T09:13:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T09:13:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T09:13:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T09:13:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T09:13:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T09:13:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T09:13:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T09:13:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T09:13:45.910 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T09:13:45.961 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T09:13:46.045 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T09:13:46.079 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T09:13:46.105 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T09:13:46.106 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T09:13:46.190 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T09:13:46.215 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T09:13:46.215 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T09:13:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T09:13:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T09:13:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T09:13:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T09:13:46.518 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T09:13:46.518 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T09:13:46.610 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T09:13:46.675 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T09:13:48.653 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T09:13:48.653 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:13:48.653 INFO:teuthology.orchestra.run.smithi012.stdout:Upgraded: 2024-05-09T09:13:48.653 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T09:13:48.654 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T09:13:48.654 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T09:13:48.654 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T09:13:48.654 INFO:teuthology.orchestra.run.smithi012.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.654 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.654 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T09:13:48.654 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T09:13:48.654 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-05-09T09:13:48.654 INFO:teuthology.orchestra.run.smithi012.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T09:13:48.654 INFO:teuthology.orchestra.run.smithi012.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T09:13:48.654 INFO:teuthology.orchestra.run.smithi012.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:48.655 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:48.656 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.656 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.656 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:48.656 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.656 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.656 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.656 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:48.656 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:48.656 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.656 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.656 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:48.656 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T09:13:48.656 INFO:teuthology.orchestra.run.smithi012.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T09:13:48.657 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T09:13:48.658 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T09:13:48.658 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T09:13:48.658 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.658 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.658 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T09:13:48.658 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.658 INFO:teuthology.orchestra.run.smithi012.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T09:13:48.658 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T09:13:48.658 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T09:13:48.658 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T09:13:48.658 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T09:13:48.658 INFO:teuthology.orchestra.run.smithi012.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T09:13:48.658 INFO:teuthology.orchestra.run.smithi012.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T09:13:48.659 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T09:13:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T09:13:48.661 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T09:13:48.662 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T09:13:48.663 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:48.664 INFO:teuthology.orchestra.run.smithi012.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T09:13:48.665 INFO:teuthology.orchestra.run.smithi012.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T09:13:48.665 INFO:teuthology.orchestra.run.smithi012.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T09:13:48.665 INFO:teuthology.orchestra.run.smithi012.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T09:13:48.665 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T09:13:48.665 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T09:13:48.665 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T09:13:48.665 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:13:48.665 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-05-09T09:13:49.074 DEBUG:teuthology.parallel:result is None 2024-05-09T09:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T09:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T09:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T09:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T09:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T09:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T09:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T09:13:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T09:13:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T09:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T09:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T09:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T09:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T09:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T09:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T09:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T09:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T09:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T09:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T09:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T09:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T09:13:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T09:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T09:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T09:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T09:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T09:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T09:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T09:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T09:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T09:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T09:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T09:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T09:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T09:13:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T09:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T09:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T09:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T09:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T09:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T09:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T09:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T09:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T09:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T09:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T09:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T09:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T09:13:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T09:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T09:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T09:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T09:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T09:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T09:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T09:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T09:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T09:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T09:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T09:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T09:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T09:13:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T09:13:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T09:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T09:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T09:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T09:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T09:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T09:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T09:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T09:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T09:13:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T09:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T09:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T09:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T09:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T09:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T09:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T09:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T09:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T09:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T09:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T09:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T09:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T09:13:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T09:13:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T09:13:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T09:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T09:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T09:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T09:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T09:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T09:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T09:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T09:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T09:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T09:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T09:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T09:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T09:13:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T09:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T09:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T09:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T09:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T09:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T09:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T09:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T09:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T09:13:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T09:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T09:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T09:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T09:13:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T09:13:52.133 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T09:13:52.134 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T09:13:52.134 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T09:13:52.134 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T09:13:52.134 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T09:13:52.134 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T09:13:52.134 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T09:13:52.134 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T09:13:52.134 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T09:13:52.134 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T09:13:52.134 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T09:13:52.134 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T09:13:52.134 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T09:13:52.135 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T09:13:52.136 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T09:13:52.136 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T09:13:52.136 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T09:13:52.136 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T09:13:52.136 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T09:13:52.136 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T09:13:52.136 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T09:13:52.136 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T09:13:52.136 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T09:13:52.136 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T09:13:52.136 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T09:13:52.136 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T09:13:52.136 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T09:13:52.137 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T09:13:52.138 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T09:13:52.138 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T09:13:52.138 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T09:13:52.138 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T09:13:52.138 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T09:13:52.138 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T09:13:52.138 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T09:13:52.138 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T09:13:52.138 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T09:13:52.138 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T09:13:52.138 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T09:13:52.138 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T09:13:52.138 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T09:13:52.139 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T09:13:52.139 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T09:13:52.139 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T09:13:52.139 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T09:13:52.139 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T09:13:52.139 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T09:13:52.139 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T09:13:52.139 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T09:13:52.139 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T09:13:52.139 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T09:13:52.139 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T09:13:52.139 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T09:13:52.139 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T09:13:52.140 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T09:13:52.141 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T09:13:52.141 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T09:13:52.141 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T09:13:52.141 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T09:13:52.141 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T09:13:52.141 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T09:13:52.141 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T09:13:52.141 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T09:13:52.141 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T09:13:52.141 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T09:13:52.141 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T09:13:52.142 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T09:13:52.142 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T09:13:52.142 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T09:13:52.142 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T09:13:52.142 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T09:13:52.142 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T09:13:52.142 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T09:13:52.142 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T09:13:52.142 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T09:13:52.142 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T09:13:52.142 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T09:13:52.142 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T09:13:52.142 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T09:13:52.143 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T09:13:52.144 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T09:13:52.144 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T09:13:52.144 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T09:13:52.144 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T09:13:52.144 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T09:13:52.144 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T09:13:52.144 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T09:13:52.144 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T09:13:52.144 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T09:13:52.144 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T09:13:52.144 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T09:13:52.144 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T09:13:52.144 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T09:13:52.145 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T09:13:52.146 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T09:13:52.146 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T09:13:52.146 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T09:13:52.146 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T09:13:52.146 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T09:13:52.150 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T09:13:52.150 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:13:52.151 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-05-09T09:13:52.151 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T09:13:52.151 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T09:13:52.151 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T09:13:52.151 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T09:13:52.151 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.151 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.151 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T09:13:52.151 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T09:13:52.151 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-05-09T09:13:52.151 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T09:13:52.151 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T09:13:52.151 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:52.152 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:52.153 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.153 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.153 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:52.153 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.153 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.153 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.153 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:52.153 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:52.153 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.153 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.153 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:52.153 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T09:13:52.153 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T09:13:52.154 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T09:13:52.155 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T09:13:52.155 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T09:13:52.155 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T09:13:52.155 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.155 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.155 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T09:13:52.155 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.155 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T09:13:52.155 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T09:13:52.155 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T09:13:52.155 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T09:13:52.155 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T09:13:52.155 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T09:13:52.156 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T09:13:52.157 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T09:13:52.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T09:13:52.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T09:13:52.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T09:13:52.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T09:13:52.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T09:13:52.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T09:13:52.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T09:13:52.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T09:13:52.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T09:13:52.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T09:13:52.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T09:13:52.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T09:13:52.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T09:13:52.159 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T09:13:52.159 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T09:13:52.159 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T09:13:52.159 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T09:13:52.159 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T09:13:52.159 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T09:13:52.159 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T09:13:52.159 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T09:13:52.159 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T09:13:52.159 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T09:13:52.159 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T09:13:52.159 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T09:13:52.159 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T09:13:52.160 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.161 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.162 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:52.162 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T09:13:52.162 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T09:13:52.162 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T09:13:52.162 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T09:13:52.162 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T09:13:52.162 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T09:13:52.162 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T09:13:52.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-09T09:13:52.163 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-05-09T09:13:52.596 DEBUG:teuthology.parallel:result is None 2024-05-09T09:13:55.011 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T09:13:55.011 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:13:55.011 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2024-05-09T09:13:55.011 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T09:13:55.011 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T09:13:55.011 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T09:13:55.011 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T09:13:55.012 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.012 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.012 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T09:13:55.012 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T09:13:55.012 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-05-09T09:13:55.012 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T09:13:55.012 INFO:teuthology.orchestra.run.smithi170.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T09:13:55.012 INFO:teuthology.orchestra.run.smithi170.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T09:13:55.012 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T09:13:55.012 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.012 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.012 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.012 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.013 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.014 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.014 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:55.014 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:55.014 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.014 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.014 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T09:13:55.014 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T09:13:55.014 INFO:teuthology.orchestra.run.smithi170.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T09:13:55.014 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T09:13:55.014 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T09:13:55.014 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T09:13:55.014 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T09:13:55.014 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.015 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T09:13:55.016 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T09:13:55.017 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T09:13:55.017 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T09:13:55.017 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T09:13:55.017 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T09:13:55.017 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T09:13:55.017 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T09:13:55.017 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T09:13:55.017 INFO:teuthology.orchestra.run.smithi170.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T09:13:55.017 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T09:13:55.017 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T09:13:55.017 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T09:13:55.017 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.017 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T09:13:55.018 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T09:13:55.019 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T09:13:55.020 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T09:13:55.021 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T09:13:55.022 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T09:13:55.023 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T09:13:55.023 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T09:13:55.023 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:13:55.023 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-05-09T09:13:55.436 DEBUG:teuthology.parallel:result is None 2024-05-09T09:13:55.437 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T09:13:55.660 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T09:13:55.688 INFO:teuthology.orchestra.run.smithi012.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T09:13:55.688 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T09:13:55.688 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T09:13:55.690 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T09:13:55.912 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T09:13:55.940 INFO:teuthology.orchestra.run.smithi016.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T09:13:55.940 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T09:13:55.940 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T09:13:55.942 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T09:13:56.158 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T09:13:56.182 INFO:teuthology.orchestra.run.smithi170.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T09:13:56.182 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T09:13:56.182 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T09:13:56.184 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T09:13:56.184 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:13:56.185 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T09:13:56.218 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-09T09:13:56.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T09:13:56.253 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-09T09:13:56.253 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T09:13:56.281 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T09:13:56.281 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:13:56.281 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T09:13:56.312 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T09:13:56.383 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-09T09:13:56.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T09:13:56.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T09:13:56.486 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-09T09:13:56.486 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T09:13:56.512 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T09:13:56.579 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T09:13:56.579 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:13:56.579 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T09:13:56.607 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T09:13:56.674 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-09T09:13:56.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T09:13:56.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T09:13:56.774 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-09T09:13:56.774 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T09:13:56.800 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T09:13:56.867 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T09:13:56.868 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:13:56.868 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T09:13:56.894 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T09:13:56.967 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-09T09:13:56.967 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T09:13:56.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T09:13:57.068 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-09T09:13:57.068 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T09:13:57.094 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T09:13:57.162 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-09T09:13:57.288 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client dirsize rbytes': False, 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'subvols': {'create': 2}} 2024-05-09T09:13:57.288 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T09:13:57.289 INFO:tasks.cephadm:Cluster fsid is 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:13:57.289 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-09T09:13:57.289 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.12', 'mon.b': '172.21.15.16', 'mon.c': '172.21.15.170'} 2024-05-09T09:13:57.289 INFO:tasks.cephadm:First mon is mon.a on smithi012 2024-05-09T09:13:57.289 INFO:tasks.cephadm:First mgr is x 2024-05-09T09:13:57.289 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-09T09:13:57.289 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname $(hostname -s) 2024-05-09T09:13:57.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-05-09T09:13:57.347 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname $(hostname -s) 2024-05-09T09:13:57.375 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-09T09:13:57.375 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T09:13:57.515 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55097/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2515-g96805dcf', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2515.g96805dcf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-07 19:00:25.721799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-09T09:13:57.665 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T09:13:57.667 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/x86_64/flavors/default/cephadm 2024-05-09T09:13:57.667 DEBUG:teuthology.orchestra.run.smithi012:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T09:13:58.025 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r--. 1 ubuntu ubuntu 782039 May 9 09:13 /home/ubuntu/cephtest/cephadm 2024-05-09T09:13:58.026 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T09:13:58.584 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 782039 May 9 09:13 /home/ubuntu/cephtest/cephadm 2024-05-09T09:13:58.584 DEBUG:teuthology.orchestra.run.smithi170:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T09:13:59.150 INFO:teuthology.orchestra.run.smithi170.stdout:-rw-r--r--. 1 ubuntu ubuntu 782039 May 9 09:13 /home/ubuntu/cephtest/cephadm 2024-05-09T09:13:59.151 DEBUG:teuthology.orchestra.run.smithi012:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T09:13:59.172 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T09:13:59.192 DEBUG:teuthology.orchestra.run.smithi170:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T09:13:59.227 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc on all hosts... 2024-05-09T09:13:59.228 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc pull 2024-05-09T09:13:59.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc pull 2024-05-09T09:13:59.235 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc pull 2024-05-09T09:14:00.783 INFO:teuthology.orchestra.run.smithi012.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T09:14:00.848 INFO:teuthology.orchestra.run.smithi170.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T09:14:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T09:14:34.120 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-05-09T09:14:34.121 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T09:14:34.121 INFO:teuthology.orchestra.run.smithi170.stdout: "image_id": "d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662", 2024-05-09T09:14:34.121 INFO:teuthology.orchestra.run.smithi170.stdout: "repo_digests": [ 2024-05-09T09:14:34.121 INFO:teuthology.orchestra.run.smithi170.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac" 2024-05-09T09:14:34.121 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-05-09T09:14:34.121 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-05-09T09:14:50.018 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-09T09:14:50.018 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T09:14:50.018 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662", 2024-05-09T09:14:50.019 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-05-09T09:14:50.019 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac" 2024-05-09T09:14:50.019 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-09T09:14:50.019 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-09T09:14:53.484 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:14:53.485 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T09:14:53.485 INFO:teuthology.orchestra.run.smithi012.stdout: "image_id": "d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662", 2024-05-09T09:14:53.485 INFO:teuthology.orchestra.run.smithi012.stdout: "repo_digests": [ 2024-05-09T09:14:53.485 INFO:teuthology.orchestra.run.smithi012.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac" 2024-05-09T09:14:53.485 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T09:14:53.485 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:14:53.509 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph 2024-05-09T09:14:53.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-05-09T09:14:53.591 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph 2024-05-09T09:14:53.625 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /etc/ceph 2024-05-09T09:14:53.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-05-09T09:14:53.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /etc/ceph 2024-05-09T09:14:53.715 INFO:tasks.cephadm:Writing seed config... 2024-05-09T09:14:53.716 INFO:tasks.cephadm: override: [client] client dirsize rbytes = False 2024-05-09T09:14:53.716 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-09T09:14:53.716 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-09T09:14:53.716 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-09T09:14:53.716 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-09T09:14:53.716 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-09T09:14:53.716 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-09T09:14:53.716 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-09T09:14:53.716 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-09T09:14:53.716 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-09T09:14:53.716 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-09T09:14:53.717 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-09T09:14:53.718 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-09T09:14:53.719 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-09T09:14:53.719 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-09T09:14:53.719 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-09T09:14:53.719 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-09T09:14:53.719 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-09T09:14:53.719 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:14:53.720 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-09T09:14:53.736 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 770e12d0-0de4-11ef-bc98-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client dirsize rbytes = False client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-09T09:14:53.736 DEBUG:teuthology.orchestra.run.smithi012:mon.a> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.a.service 2024-05-09T09:14:53.779 DEBUG:teuthology.orchestra.run.smithi012:mgr.x> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.x.service 2024-05-09T09:14:53.823 INFO:tasks.cephadm:Bootstrapping... 2024-05-09T09:14:53.823 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -v bootstrap --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.12 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:14:54.021 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-05-09T09:14:54.022 INFO:teuthology.orchestra.run.smithi012.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', '-v', 'bootstrap', '--fsid', '770e12d0-0de4-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.12', '--skip-admin-label'] 2024-05-09T09:14:54.022 INFO:teuthology.orchestra.run.smithi012.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-09T09:14:54.022 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying podman|docker is present... 2024-05-09T09:14:54.048 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T09:14:54.048 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying lvm2 is present... 2024-05-09T09:14:54.049 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying time synchronization is in place... 2024-05-09T09:14:54.056 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T09:14:54.057 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T09:14:54.063 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T09:14:54.063 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-09T09:14:54.070 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-05-09T09:14:54.077 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-05-09T09:14:54.077 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-05-09T09:14:54.077 INFO:teuthology.orchestra.run.smithi012.stdout:Repeating the final host check... 2024-05-09T09:14:54.101 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T09:14:54.101 INFO:teuthology.orchestra.run.smithi012.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-09T09:14:54.101 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl is present 2024-05-09T09:14:54.101 INFO:teuthology.orchestra.run.smithi012.stdout:lvcreate is present 2024-05-09T09:14:54.108 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T09:14:54.108 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T09:14:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T09:14:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-09T09:14:54.121 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-05-09T09:14:54.128 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-05-09T09:14:54.128 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-05-09T09:14:54.128 INFO:teuthology.orchestra.run.smithi012.stdout:Host looks OK 2024-05-09T09:14:54.128 INFO:teuthology.orchestra.run.smithi012.stdout:Cluster fsid: 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:14:54.128 INFO:teuthology.orchestra.run.smithi012.stdout:Acquiring lock 139942591426512 on /run/cephadm/770e12d0-0de4-11ef-bc98-c7b262605968.lock 2024-05-09T09:14:54.128 INFO:teuthology.orchestra.run.smithi012.stdout:Lock 139942591426512 acquired on /run/cephadm/770e12d0-0de4-11ef-bc98-c7b262605968.lock 2024-05-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 3300 ... 2024-05-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 6789 ... 2024-05-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi012.stdout:Base mon IP(s) is [172.21.15.12:3300, 172.21.15.12:6789], mon addrv is [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-05-09T09:14:54.132 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.12 metric 100 2024-05-09T09:14:54.132 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.12 metric 100 2024-05-09T09:14:54.138 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-09T09:14:54.138 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-09T09:14:54.138 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-09T09:14:54.138 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-09T09:14:54.138 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-09T09:14:54.138 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T09:14:54.138 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-09T09:14:54.138 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-05-09T09:14:54.138 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T09:14:54.139 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-05-09T09:14:54.139 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-05-09T09:14:54.139 INFO:teuthology.orchestra.run.smithi012.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-09T09:14:54.139 INFO:teuthology.orchestra.run.smithi012.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-09T09:14:54.140 INFO:teuthology.orchestra.run.smithi012.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T09:14:54.629 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 2024-05-09T09:14:54.629 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T09:14:54.629 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Getting image source signatures 2024-05-09T09:14:54.629 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:9852312f87792557815c3e860a5c4f8cbf2b59bbbde9b5f86ab07b438f905b2d 2024-05-09T09:14:54.629 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-09T09:14:54.629 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying config sha256:d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 2024-05-09T09:14:54.629 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-09T09:14:55.586 INFO:teuthology.orchestra.run.smithi012.stdout:ceph: stdout ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev) 2024-05-09T09:14:55.587 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph version: ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev) 2024-05-09T09:14:55.587 INFO:teuthology.orchestra.run.smithi012.stdout:Extracting ceph user uid/gid from container image... 2024-05-09T09:14:56.550 INFO:teuthology.orchestra.run.smithi012.stdout:stat: stdout 167 167 2024-05-09T09:14:56.550 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial keys... 2024-05-09T09:14:57.550 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQAQlDxmEBgnNRAAo/slvaHVHBtCN14T2r5nFA== 2024-05-09T09:14:58.589 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQARlDxmpnT6MxAAYOXYzuGxTXkQSZKB/l36Hw== 2024-05-09T09:14:59.494 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQASlDxmLKjNNhAAPLoqh8ZgfXXI5VAI6x0Llw== 2024-05-09T09:14:59.495 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial monmap... 2024-05-09T09:15:00.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T09:15:00.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-09T09:15:00.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:15:00.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T09:15:00.384 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool for a [v2:172.21.15.12:3300,v1:172.21.15.12:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T09:15:00.384 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = quincy 2024-05-09T09:15:00.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: set fsid to 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:15:00.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T09:15:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:15:00.385 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mon... 2024-05-09T09:15:01.282 INFO:teuthology.orchestra.run.smithi012.stdout:create mon.a on 2024-05-09T09:15:01.502 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-09T09:15:01.682 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-09T09:15:01.884 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-770e12d0-0de4-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-770e12d0-0de4-11ef-bc98-c7b262605968.target. 2024-05-09T09:15:01.885 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-770e12d0-0de4-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-770e12d0-0de4-11ef-bc98-c7b262605968.target. 2024-05-09T09:15:02.207 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.a 2024-05-09T09:15:02.207 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-09T09:15:02.386 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-770e12d0-0de4-11ef-bc98-c7b262605968.target.wants/ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-770e12d0-0de4-11ef-bc98-c7b262605968@.service. 2024-05-09T09:15:02.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:02 smithi012 podman[29750]: 2024-05-09 09:15:02.657015976 +0000 UTC m=+0.115998990 container create 1183da5f1068dac3d6651086110c6a35716417431225c3e3d318a64fba1248f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc) 2024-05-09T09:15:02.939 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T09:15:02.939 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-05-09T09:15:02.946 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T09:15:02.947 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-09T09:15:02.947 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-05-09T09:15:02.947 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T09:15:02.947 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon to start... 2024-05-09T09:15:02.947 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon... 2024-05-09T09:15:03.067 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:02 smithi012 podman[29750]: 2024-05-09 09:15:02.842710337 +0000 UTC m=+0.301693357 container init 1183da5f1068dac3d6651086110c6a35716417431225c3e3d318a64fba1248f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-96805dc, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:15:03.067 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:02 smithi012 podman[29750]: 2024-05-09 09:15:02.846095227 +0000 UTC m=+0.305078240 container start 1183da5f1068dac3d6651086110c6a35716417431225c3e3d318a64fba1248f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc) 2024-05-09T09:15:03.067 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:02 smithi012 ceph-mon[29769]: mkfs 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:15:03.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:02 smithi012 ceph-mon[29769]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T09:15:03.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:02 smithi012 bash[29750]: 1183da5f1068dac3d6651086110c6a35716417431225c3e3d318a64fba1248f1 2024-05-09T09:15:03.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:02 smithi012 systemd[1]: Started Ceph mon.a for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:15:04.026 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout cluster: 2024-05-09T09:15:04.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout id: 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:15:04.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-09T09:15:04.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-09T09:15:04.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout services: 2024-05-09T09:15:04.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.566257s) 2024-05-09T09:15:04.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-09T09:15:04.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-09T09:15:04.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-09T09:15:04.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout data: 2024-05-09T09:15:04.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-09T09:15:04.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-09T09:15:04.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-09T09:15:04.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pgs: 2024-05-09T09:15:04.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-09T09:15:04.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.373+0000 7f6b0d022700 1 Processor -- start 2024-05-09T09:15:04.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.373+0000 7f6b0d022700 1 -- start start 2024-05-09T09:15:04.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.373+0000 7f6b0d022700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 0x7f6b08108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:04.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.373+0000 7f6b0d022700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081086f0 con 0x7f6b08107d00 2024-05-09T09:15:04.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.373+0000 7f6b06d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 0x7f6b08108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:04.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.373+0000 7f6b06d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 0x7f6b08108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56496/0 (socket says 172.21.15.12:56496) 2024-05-09T09:15:04.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.373+0000 7f6b06d9d700 1 -- 172.21.15.12:0/465604201 learned_addr learned my addr 172.21.15.12:0/465604201 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:04.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.374+0000 7f6b06d9d700 1 -- 172.21.15.12:0/465604201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b08108eb0 con 0x7f6b08107d00 2024-05-09T09:15:04.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.374+0000 7f6b06d9d700 1 --2- 172.21.15.12:0/465604201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 0x7f6b08108120 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6af80098a0 tx=0x7f6af8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa30bf6d240ed845 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:04.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.374+0000 7f6b05d9b700 1 -- 172.21.15.12:0/465604201 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6af8004160 con 0x7f6b08107d00 2024-05-09T09:15:04.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.374+0000 7f6b05d9b700 1 -- 172.21.15.12:0/465604201 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af8004300 con 0x7f6b08107d00 2024-05-09T09:15:04.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.374+0000 7f6b05d9b700 1 -- 172.21.15.12:0/465604201 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6af80036a0 con 0x7f6b08107d00 2024-05-09T09:15:04.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.375+0000 7f6b0d022700 1 -- 172.21.15.12:0/465604201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 msgr2=0x7f6b08108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:04.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.375+0000 7f6b0d022700 1 --2- 172.21.15.12:0/465604201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 0x7f6b08108120 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6af80098a0 tx=0x7f6af8009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:04.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.375+0000 7f6b0d022700 1 -- 172.21.15.12:0/465604201 shutdown_connections 2024-05-09T09:15:04.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.375+0000 7f6b0d022700 1 --2- 172.21.15.12:0/465604201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 0x7f6b08108120 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:04.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.375+0000 7f6b0d022700 1 -- 172.21.15.12:0/465604201 >> 172.21.15.12:0/465604201 conn(0x7f6b081032c0 msgr2=0x7f6b081056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:04.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.375+0000 7f6b0d022700 1 -- 172.21.15.12:0/465604201 shutdown_connections 2024-05-09T09:15:04.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.375+0000 7f6b0d022700 1 -- 172.21.15.12:0/465604201 wait complete. 2024-05-09T09:15:04.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.375+0000 7f6b0d022700 1 Processor -- start 2024-05-09T09:15:04.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.376+0000 7f6b0d022700 1 -- start start 2024-05-09T09:15:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.376+0000 7f6b0d022700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 0x7f6b0819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.376+0000 7f6b0d022700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081093e0 con 0x7f6b08107d00 2024-05-09T09:15:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.376+0000 7f6b06d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 0x7f6b0819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.376+0000 7f6b06d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 0x7f6b0819b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56498/0 (socket says 172.21.15.12:56498) 2024-05-09T09:15:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.376+0000 7f6b06d9d700 1 -- 172.21.15.12:0/1050327431 learned_addr learned my addr 172.21.15.12:0/1050327431 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.376+0000 7f6b06d9d700 1 -- 172.21.15.12:0/1050327431 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0819b840 con 0x7f6b08107d00 2024-05-09T09:15:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.376+0000 7f6b06d9d700 1 --2- 172.21.15.12:0/1050327431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 0x7f6b0819b300 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6af8000c00 tx=0x7f6af8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.377+0000 7f6aeffff700 1 -- 172.21.15.12:0/1050327431 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6af8003db0 con 0x7f6b08107d00 2024-05-09T09:15:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.377+0000 7f6aeffff700 1 -- 172.21.15.12:0/1050327431 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af8011630 con 0x7f6b08107d00 2024-05-09T09:15:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.377+0000 7f6aeffff700 1 -- 172.21.15.12:0/1050327431 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6af8011950 con 0x7f6b08107d00 2024-05-09T09:15:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.377+0000 7f6b0d022700 1 -- 172.21.15.12:0/1050327431 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b0819bad0 con 0x7f6b08107d00 2024-05-09T09:15:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.377+0000 7f6b0d022700 1 -- 172.21.15.12:0/1050327431 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b0819e7c0 con 0x7f6b08107d00 2024-05-09T09:15:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.377+0000 7f6aeffff700 1 -- 172.21.15.12:0/1050327431 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6af801a460 con 0x7f6b08107d00 2024-05-09T09:15:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.377+0000 7f6aeffff700 1 -- 172.21.15.12:0/1050327431 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6af801b780 con 0x7f6b08107d00 2024-05-09T09:15:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.378+0000 7f6b0d022700 1 -- 172.21.15.12:0/1050327431 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b08063980 con 0x7f6b08107d00 2024-05-09T09:15:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.380+0000 7f6aeffff700 1 -- 172.21.15.12:0/1050327431 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6af801b9d0 con 0x7f6b08107d00 2024-05-09T09:15:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.455+0000 7f6b0d022700 1 -- 172.21.15.12:0/1050327431 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f6b0802de80 con 0x7f6b08107d00 2024-05-09T09:15:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.456+0000 7f6aeffff700 1 -- 172.21.15.12:0/1050327431 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f6af801bbb0 con 0x7f6b08107d00 2024-05-09T09:15:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.457+0000 7f6b0d022700 1 -- 172.21.15.12:0/1050327431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 msgr2=0x7f6b0819b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.457+0000 7f6b0d022700 1 --2- 172.21.15.12:0/1050327431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 0x7f6b0819b300 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6af8000c00 tx=0x7f6af8004750 comp rx=0 tx=0).stop 2024-05-09T09:15:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.457+0000 7f6b0d022700 1 -- 172.21.15.12:0/1050327431 shutdown_connections 2024-05-09T09:15:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.457+0000 7f6b0d022700 1 --2- 172.21.15.12:0/1050327431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b08107d00 0x7f6b0819b300 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.457+0000 7f6b0d022700 1 -- 172.21.15.12:0/1050327431 >> 172.21.15.12:0/1050327431 conn(0x7f6b081032c0 msgr2=0x7f6b08191f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.457+0000 7f6b0d022700 1 -- 172.21.15.12:0/1050327431 shutdown_connections 2024-05-09T09:15:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:03.457+0000 7f6b0d022700 1 -- 172.21.15.12:0/1050327431 wait complete. 2024-05-09T09:15:04.079 INFO:teuthology.orchestra.run.smithi012.stdout:mon is available 2024-05-09T09:15:04.079 INFO:teuthology.orchestra.run.smithi012.stdout:Assimilating anything we can from ceph.conf... 2024-05-09T09:15:04.131 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:03 smithi012 ceph-mon[29769]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T09:15:04.132 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:03 smithi012 ceph-mon[29769]: monmap epoch 1 2024-05-09T09:15:04.132 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:03 smithi012 ceph-mon[29769]: fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:15:04.132 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:03 smithi012 ceph-mon[29769]: last_changed 2024-05-09T09:14:59.821401+0000 2024-05-09T09:15:04.132 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:03 smithi012 ceph-mon[29769]: created 2024-05-09T09:14:59.821401+0000 2024-05-09T09:15:04.132 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:03 smithi012 ceph-mon[29769]: min_mon_release 19 (squid) 2024-05-09T09:15:04.132 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:03 smithi012 ceph-mon[29769]: election_strategy: 1 2024-05-09T09:15:04.132 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:03 smithi012 ceph-mon[29769]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-09T09:15:04.132 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:03 smithi012 ceph-mon[29769]: fsmap 2024-05-09T09:15:04.132 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:03 smithi012 ceph-mon[29769]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T09:15:04.132 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:03 smithi012 ceph-mon[29769]: mgrmap e1: no daemons active 2024-05-09T09:15:04.132 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:03 smithi012 ceph-mon[29769]: from='client.? 172.21.15.12:0/1050327431' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-09T09:15:05.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-09T09:15:05.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T09:15:05.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:15:05.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-05-09T09:15:05.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T09:15:05.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T09:15:05.060 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.437+0000 7ff4a49a5700 1 Processor -- start 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.437+0000 7ff4a49a5700 1 -- start start 2024-05-09T09:15:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.438+0000 7ff4a49a5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 0x7ff4a0108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.438+0000 7ff4a49a5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a01086f0 con 0x7ff4a0107d00 2024-05-09T09:15:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.438+0000 7ff49e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 0x7ff4a0108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.438+0000 7ff49e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 0x7ff4a0108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56510/0 (socket says 172.21.15.12:56510) 2024-05-09T09:15:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.438+0000 7ff49e59c700 1 -- 172.21.15.12:0/3905133232 learned_addr learned my addr 172.21.15.12:0/3905133232 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.438+0000 7ff49e59c700 1 -- 172.21.15.12:0/3905133232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a0108eb0 con 0x7ff4a0107d00 2024-05-09T09:15:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.439+0000 7ff49e59c700 1 --2- 172.21.15.12:0/3905133232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 0x7ff4a0108120 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff4880098a0 tx=0x7ff488009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262221777cfe7d75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.439+0000 7ff49d59a700 1 -- 172.21.15.12:0/3905133232 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff488004160 con 0x7ff4a0107d00 2024-05-09T09:15:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.439+0000 7ff49d59a700 1 -- 172.21.15.12:0/3905133232 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff488004300 con 0x7ff4a0107d00 2024-05-09T09:15:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.439+0000 7ff49d59a700 1 -- 172.21.15.12:0/3905133232 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4880036a0 con 0x7ff4a0107d00 2024-05-09T09:15:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.439+0000 7ff4a49a5700 1 -- 172.21.15.12:0/3905133232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 msgr2=0x7ff4a0108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.439+0000 7ff4a49a5700 1 --2- 172.21.15.12:0/3905133232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 0x7ff4a0108120 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff4880098a0 tx=0x7ff488009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.439+0000 7ff4a49a5700 1 -- 172.21.15.12:0/3905133232 shutdown_connections 2024-05-09T09:15:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.439+0000 7ff4a49a5700 1 --2- 172.21.15.12:0/3905133232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 0x7ff4a0108120 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.439+0000 7ff4a49a5700 1 -- 172.21.15.12:0/3905133232 >> 172.21.15.12:0/3905133232 conn(0x7ff4a01032c0 msgr2=0x7ff4a01056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.439+0000 7ff4a49a5700 1 -- 172.21.15.12:0/3905133232 shutdown_connections 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.440+0000 7ff4a49a5700 1 -- 172.21.15.12:0/3905133232 wait complete. 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.440+0000 7ff4a49a5700 1 Processor -- start 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.440+0000 7ff4a49a5700 1 -- start start 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.440+0000 7ff4a49a5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 0x7ff4a019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.440+0000 7ff4a49a5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a01093e0 con 0x7ff4a0107d00 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.441+0000 7ff49e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 0x7ff4a019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.441+0000 7ff49e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 0x7ff4a019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56516/0 (socket says 172.21.15.12:56516) 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.441+0000 7ff49e59c700 1 -- 172.21.15.12:0/1886899602 learned_addr learned my addr 172.21.15.12:0/1886899602 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.441+0000 7ff49e59c700 1 -- 172.21.15.12:0/1886899602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a019b840 con 0x7ff4a0107d00 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.441+0000 7ff49e59c700 1 --2- 172.21.15.12:0/1886899602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 0x7ff4a019b300 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff488000c00 tx=0x7ff488004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.441+0000 7ff4977fe700 1 -- 172.21.15.12:0/1886899602 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff488003db0 con 0x7ff4a0107d00 2024-05-09T09:15:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.441+0000 7ff4977fe700 1 -- 172.21.15.12:0/1886899602 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff488011630 con 0x7ff4a0107d00 2024-05-09T09:15:05.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.441+0000 7ff4a49a5700 1 -- 172.21.15.12:0/1886899602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a019bad0 con 0x7ff4a0107d00 2024-05-09T09:15:05.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.441+0000 7ff4977fe700 1 -- 172.21.15.12:0/1886899602 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff488011950 con 0x7ff4a0107d00 2024-05-09T09:15:05.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.441+0000 7ff4a49a5700 1 -- 172.21.15.12:0/1886899602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a019e7c0 con 0x7ff4a0107d00 2024-05-09T09:15:05.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.441+0000 7ff4977fe700 1 -- 172.21.15.12:0/1886899602 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff48801a460 con 0x7ff4a0107d00 2024-05-09T09:15:05.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.441+0000 7ff4977fe700 1 -- 172.21.15.12:0/1886899602 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff48801b780 con 0x7ff4a0107d00 2024-05-09T09:15:05.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.442+0000 7ff4a49a5700 1 -- 172.21.15.12:0/1886899602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a0063980 con 0x7ff4a0107d00 2024-05-09T09:15:05.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.444+0000 7ff4977fe700 1 -- 172.21.15.12:0/1886899602 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff488031020 con 0x7ff4a0107d00 2024-05-09T09:15:05.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.516+0000 7ff4a49a5700 1 -- 172.21.15.12:0/1886899602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff4a019bfb0 con 0x7ff4a0107d00 2024-05-09T09:15:05.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.519+0000 7ff4977fe700 1 -- 172.21.15.12:0/1886899602 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7ff4a019bfb0 con 0x7ff4a0107d00 2024-05-09T09:15:05.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.519+0000 7ff4977fe700 1 -- 172.21.15.12:0/1886899602 <== mon.0 v2:172.21.15.12:3300/0 8 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7ff48801a8e0 con 0x7ff4a0107d00 2024-05-09T09:15:05.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.520+0000 7ff4a49a5700 1 -- 172.21.15.12:0/1886899602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 msgr2=0x7ff4a019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:05.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.520+0000 7ff4a49a5700 1 --2- 172.21.15.12:0/1886899602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 0x7ff4a019b300 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff488000c00 tx=0x7ff488004750 comp rx=0 tx=0).stop 2024-05-09T09:15:05.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.520+0000 7ff4a49a5700 1 -- 172.21.15.12:0/1886899602 shutdown_connections 2024-05-09T09:15:05.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.520+0000 7ff4a49a5700 1 --2- 172.21.15.12:0/1886899602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a0107d00 0x7ff4a019b300 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:05.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.520+0000 7ff4a49a5700 1 -- 172.21.15.12:0/1886899602 >> 172.21.15.12:0/1886899602 conn(0x7ff4a01032c0 msgr2=0x7ff4a0191f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:05.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.520+0000 7ff4a49a5700 1 -- 172.21.15.12:0/1886899602 shutdown_connections 2024-05-09T09:15:05.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:04.520+0000 7ff4a49a5700 1 -- 172.21.15.12:0/1886899602 wait complete. 2024-05-09T09:15:05.065 INFO:teuthology.orchestra.run.smithi012.stdout:Generating new minimal ceph.conf... 2024-05-09T09:15:05.162 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:04 smithi012 ceph-mon[29769]: from='client.? 172.21.15.12:0/1886899602' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T09:15:05.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:04 smithi012 ceph-mon[29769]: from='client.? 172.21.15.12:0/1886899602' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-09T09:15:06.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.437+0000 7f2a2c50f700 1 Processor -- start 2024-05-09T09:15:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.437+0000 7f2a2c50f700 1 -- start start 2024-05-09T09:15:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.438+0000 7f2a2c50f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 0x7f2a2410ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.438+0000 7f2a2c50f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2407bdf0 con 0x7f2a24108780 2024-05-09T09:15:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.438+0000 7f2a2a2ab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 0x7f2a2410ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.438+0000 7f2a2a2ab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 0x7f2a2410ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56532/0 (socket says 172.21.15.12:56532) 2024-05-09T09:15:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.438+0000 7f2a2a2ab700 1 -- 172.21.15.12:0/3570460889 learned_addr learned my addr 172.21.15.12:0/3570460889 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.438+0000 7f2a2a2ab700 1 -- 172.21.15.12:0/3570460889 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2410b0b0 con 0x7f2a24108780 2024-05-09T09:15:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.438+0000 7f2a2a2ab700 1 --2- 172.21.15.12:0/3570460889 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 0x7f2a2410ab70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2a200098a0 tx=0x7f2a20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ffe61196b737b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.439+0000 7f2a292a9700 1 -- 172.21.15.12:0/3570460889 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a20004160 con 0x7f2a24108780 2024-05-09T09:15:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.439+0000 7f2a292a9700 1 -- 172.21.15.12:0/3570460889 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f2a20004300 con 0x7f2a24108780 2024-05-09T09:15:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.439+0000 7f2a292a9700 1 -- 172.21.15.12:0/3570460889 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a20011b40 con 0x7f2a24108780 2024-05-09T09:15:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.439+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3570460889 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 msgr2=0x7f2a2410ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.439+0000 7f2a2c50f700 1 --2- 172.21.15.12:0/3570460889 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 0x7f2a2410ab70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2a200098a0 tx=0x7f2a20009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.439+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3570460889 shutdown_connections 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.439+0000 7f2a2c50f700 1 --2- 172.21.15.12:0/3570460889 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 0x7f2a2410ab70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.439+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3570460889 >> 172.21.15.12:0/3570460889 conn(0x7f2a24102170 msgr2=0x7f2a241045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.440+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3570460889 shutdown_connections 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.440+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3570460889 wait complete. 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.440+0000 7f2a2c50f700 1 Processor -- start 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.440+0000 7f2a2c50f700 1 -- start start 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.440+0000 7f2a2c50f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 0x7f2a2419bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.440+0000 7f2a2c50f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2410c0d0 con 0x7f2a24108780 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.440+0000 7f2a2a2ab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 0x7f2a2419bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.440+0000 7f2a2a2ab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 0x7f2a2419bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56548/0 (socket says 172.21.15.12:56548) 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.440+0000 7f2a2a2ab700 1 -- 172.21.15.12:0/3949512896 learned_addr learned my addr 172.21.15.12:0/3949512896 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:06.077 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.441+0000 7f2a2a2ab700 1 -- 172.21.15.12:0/3949512896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2419c100 con 0x7f2a24108780 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.441+0000 7f2a2a2ab700 1 --2- 172.21.15.12:0/3949512896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 0x7f2a2419bbc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2a20004180 tx=0x7f2a200046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.441+0000 7f2a1b7fe700 1 -- 172.21.15.12:0/3949512896 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a200039e0 con 0x7f2a24108780 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.441+0000 7f2a1b7fe700 1 -- 172.21.15.12:0/3949512896 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f2a20020070 con 0x7f2a24108780 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.441+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3949512896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2419c390 con 0x7f2a24108780 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.441+0000 7f2a1b7fe700 1 -- 172.21.15.12:0/3949512896 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a200193c0 con 0x7f2a24108780 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.441+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3949512896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a2419f080 con 0x7f2a24108780 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.441+0000 7f2a1b7fe700 1 -- 172.21.15.12:0/3949512896 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2a200199c0 con 0x7f2a24108780 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.442+0000 7f2a1b7fe700 1 -- 172.21.15.12:0/3949512896 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2a2000b720 con 0x7f2a24108780 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.443+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3949512896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a24063980 con 0x7f2a24108780 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.445+0000 7f2a1b7fe700 1 -- 172.21.15.12:0/3949512896 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2a20003b80 con 0x7f2a24108780 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.515+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3949512896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f2a2419f250 con 0x7f2a24108780 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.515+0000 7f2a1b7fe700 1 -- 172.21.15.12:0/3949512896 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f2a20019df0 con 0x7f2a24108780 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.516+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3949512896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 msgr2=0x7f2a2419bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:06.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.516+0000 7f2a2c50f700 1 --2- 172.21.15.12:0/3949512896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 0x7f2a2419bbc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2a20004180 tx=0x7f2a200046a0 comp rx=0 tx=0).stop 2024-05-09T09:15:06.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.516+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3949512896 shutdown_connections 2024-05-09T09:15:06.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.516+0000 7f2a2c50f700 1 --2- 172.21.15.12:0/3949512896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24108780 0x7f2a2419bbc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:06.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.516+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3949512896 >> 172.21.15.12:0/3949512896 conn(0x7f2a24102170 msgr2=0x7f2a24102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:06.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.516+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3949512896 shutdown_connections 2024-05-09T09:15:06.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:05.516+0000 7f2a2c50f700 1 -- 172.21.15.12:0/3949512896 wait complete. 2024-05-09T09:15:06.079 INFO:teuthology.orchestra.run.smithi012.stdout:Restarting the monitor... 2024-05-09T09:15:06.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:05 smithi012 ceph-mon[29769]: from='client.? 172.21.15.12:0/3949512896' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:15:06.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:06 smithi012 systemd[1]: Stopping Ceph mon.a for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:15:07.733 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:07 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a[29764]: 2024-05-09T09:15:07.249+0000 7f31f8b7c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:15:07.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:07 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a[29764]: 2024-05-09T09:15:07.249+0000 7f31f8b7c700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-09T09:15:07.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:07 smithi012 podman[30012]: 2024-05-09 09:15:07.270461785 +0000 UTC m=+1.026840675 container died 1183da5f1068dac3d6651086110c6a35716417431225c3e3d318a64fba1248f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T09:15:11.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:10 smithi012 podman[30012]: 2024-05-09 09:15:10.804647268 +0000 UTC m=+4.561026157 container cleanup 1183da5f1068dac3d6651086110c6a35716417431225c3e3d318a64fba1248f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-96805dc) 2024-05-09T09:15:11.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:10 smithi012 bash[30012]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a 2024-05-09T09:15:12.619 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:12 smithi012 podman[30026]: 2024-05-09 09:15:12.391540922 +0000 UTC m=+5.112011893 container remove 1183da5f1068dac3d6651086110c6a35716417431225c3e3d318a64fba1248f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:15:12.888 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:12 smithi012 systemd[1]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-09T09:15:12.888 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:12 smithi012 systemd[1]: Stopped Ceph mon.a for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:15:12.888 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:12 smithi012 systemd[1]: Starting Ceph mon.a for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:15:12.888 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:12 smithi012 podman[30121]: 2024-05-09 09:15:12.765354489 +0000 UTC m=+0.018362539 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T09:15:13.490 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 podman[30121]: 2024-05-09 09:15:13.241597206 +0000 UTC m=+0.494605257 container create 1e0b9433ce3ef6566664cf618b2ea8d6339ecf7e5aa402f846b4df79e75c0cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:15:13.490 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 podman[30121]: 2024-05-09 09:15:13.452113727 +0000 UTC m=+0.705121786 container init 1e0b9433ce3ef6566664cf618b2ea8d6339ecf7e5aa402f846b4df79e75c0cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-09T09:15:13.490 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 podman[30121]: 2024-05-09 09:15:13.456324207 +0000 UTC m=+0.709332257 container start 1e0b9433ce3ef6566664cf618b2ea8d6339ecf7e5aa402f846b4df79e75c0cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:15:13.490 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T09:15:13.490 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev), process ceph-mon, pid 2 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: pidfile_write: ignore empty --pid-file 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: load: jerasure load: lrc 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: RocksDB version: 7.9.2 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Git sha 0 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Compile date 2024-05-07 19:19:25 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: DB SUMMARY 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: DB Session ID: M32O59OI7SSUZB4QY818 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: CURRENT file: CURRENT 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104242 ; 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.error_if_exists: 0 2024-05-09T09:15:13.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.create_if_missing: 0 2024-05-09T09:15:13.492 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.paranoid_checks: 1 2024-05-09T09:15:13.492 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T09:15:13.493 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T09:15:13.493 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T09:15:13.493 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.env: 0x560c987fcc60 2024-05-09T09:15:13.493 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T09:15:13.493 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.info_log: 0x560c9afe1ac0 2024-05-09T09:15:13.493 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T09:15:13.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.statistics: (nil) 2024-05-09T09:15:13.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.use_fsync: 0 2024-05-09T09:15:13.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_log_file_size: 0 2024-05-09T09:15:13.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T09:15:13.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T09:15:13.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T09:15:13.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T09:15:13.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.allow_fallocate: 1 2024-05-09T09:15:13.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T09:15:13.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T09:15:13.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.use_direct_reads: 0 2024-05-09T09:15:13.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T09:15:13.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.db_log_dir: 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.wal_dir: 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.write_buffer_manager: 0x560c9afb41e0 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T09:15:13.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.unordered_write: 0 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.row_cache: None 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.wal_filter: None 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T09:15:13.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.two_write_queues: 0 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.wal_compression: 0 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.atomic_flush: 0 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.log_readahead_size: 0 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T09:15:13.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T09:15:13.498 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T09:15:13.498 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_background_jobs: 2 2024-05-09T09:15:13.498 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_background_compactions: -1 2024-05-09T09:15:13.498 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_subcompactions: 1 2024-05-09T09:15:13.498 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T09:15:13.498 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T09:15:13.498 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T09:15:13.498 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T09:15:13.498 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T09:15:13.498 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T09:15:13.498 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T09:15:13.498 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T09:15:13.498 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_open_files: -1 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_background_flushes: -1 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Compression algorithms supported: 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: kZSTD supported: 0 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: kXpressCompression supported: 0 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: kZlibCompression supported: 1 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: kSnappyCompression supported: 1 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: kLZ4Compression supported: 1 2024-05-09T09:15:13.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: kBZip2Compression supported: 0 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.merge_operator: 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compaction_filter: None 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560c9afe1bc0) 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks: 1 2024-05-09T09:15:13.500 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_top_level_index_and_filter: 1 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_type: 0 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_index_type: 0 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_shortening: 1 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: checksum: 4 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: no_block_cache: 0 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache: 0x560c9afe5090 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_name: BinnedLRUCache 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_options: 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: capacity : 536870912 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_shard_bits : 4 2024-05-09T09:15:13.501 INFO:journalctl@ceph.mon.a.smithi012.stdout: strict_capacity_limit : 0 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: high_pri_pool_ratio: 0.000 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_compressed: (nil) 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: persistent_cache: (nil) 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size: 4096 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size_deviation: 10 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_restart_interval: 16 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_block_restart_interval: 1 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: metadata_block_size: 4096 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: partition_filters: 0 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: use_delta_encoding: 1 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: filter_policy: bloomfilter 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: whole_key_filtering: 1 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: verify_compression: 0 2024-05-09T09:15:13.502 INFO:journalctl@ceph.mon.a.smithi012.stdout: read_amp_bytes_per_bit: 0 2024-05-09T09:15:13.503 INFO:journalctl@ceph.mon.a.smithi012.stdout: format_version: 5 2024-05-09T09:15:13.503 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_index_compression: 1 2024-05-09T09:15:13.503 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_align: 0 2024-05-09T09:15:13.503 INFO:journalctl@ceph.mon.a.smithi012.stdout: max_auto_readahead_size: 262144 2024-05-09T09:15:13.503 INFO:journalctl@ceph.mon.a.smithi012.stdout: prepopulate_block_cache: 0 2024-05-09T09:15:13.503 INFO:journalctl@ceph.mon.a.smithi012.stdout: initial_auto_readahead_size: 8192 2024-05-09T09:15:13.503 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T09:15:13.503 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T09:15:13.503 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T09:15:13.503 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compression: NoCompression 2024-05-09T09:15:13.503 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T09:15:13.503 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T09:15:13.503 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T09:15:13.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.num_levels: 7 2024-05-09T09:15:13.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T09:15:13.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T09:15:13.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T09:15:13.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T09:15:13.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T09:15:13.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T09:15:13.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T09:15:13.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T09:15:13.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T09:15:13.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T09:15:13.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T09:15:13.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T09:15:13.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T09:15:13.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T09:15:13.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T09:15:13.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T09:15:13.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T09:15:13.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T09:15:13.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T09:15:13.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T09:15:13.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T09:15:13.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T09:15:13.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T09:15:13.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T09:15:13.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T09:15:13.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T09:15:13.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T09:15:13.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T09:15:13.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T09:15:13.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T09:15:13.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T09:15:13.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T09:15:13.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T09:15:13.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T09:15:13.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T09:15:13.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T09:15:13.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T09:15:13.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T09:15:13.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T09:15:13.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T09:15:13.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T09:15:13.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T09:15:13.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T09:15:13.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T09:15:13.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T09:15:13.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T09:15:13.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T09:15:13.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.table_properties_collectors: 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.inplace_update_support: 0 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.bloom_locality: 0 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.max_successive_merges: 0 2024-05-09T09:15:13.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.ttl: 2592000 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.enable_blob_files: false 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.min_blob_size: 0 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T09:15:13.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b0141b42-9bfb-4ebc-91e9-cdba597f622f 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715246113487532, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715246113488655, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99417, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97325, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14048, "raw_average_key_size": 49, "raw_value_size": 89689, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715246113, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b0141b42-9bfb-4ebc-91e9-cdba597f622f", "db_session_id": "M32O59OI7SSUZB4QY818", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715246113488744, "job": 1, "event": "recovery_finished"} 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560c9b0d6000 2024-05-09T09:15:13.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: DB pointer 0x560c9b0c0000 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** DB Stats ** 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T09:15:13.511 INFO:journalctl@ceph.mon.a.smithi012.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: L0 2/0 98.92 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: Sum 2/0 98.92 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 134.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T09:15:13.512 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative compaction: 0.00 GB write, 28.77 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval compaction: 0.00 GB write, 28.77 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache BinnedLRUCache@0x560c9afe5090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: starting mon.a rank 0 at public addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] at bind addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: mon.a@-1(???) e1 preinit fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: mon.a@-1(???).mds e1 new map 2024-05-09T09:15:13.513 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: mon.a@-1(???).mds e1 print_map 2024-05-09T09:15:13.514 INFO:journalctl@ceph.mon.a.smithi012.stdout: e1 2024-05-09T09:15:13.514 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T09:15:13.514 INFO:journalctl@ceph.mon.a.smithi012.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T09:15:13.514 INFO:journalctl@ceph.mon.a.smithi012.stdout: legacy client fscid: -1 2024-05-09T09:15:13.514 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-09T09:15:13.514 INFO:journalctl@ceph.mon.a.smithi012.stdout: No filesystems configured 2024-05-09T09:15:13.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T09:15:13.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T09:15:13.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T09:15:13.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T09:15:13.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-09T09:15:13.564 INFO:teuthology.orchestra.run.smithi012.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-09T09:15:13.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T09:15:13.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: monmap epoch 1 2024-05-09T09:15:13.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:15:13.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: last_changed 2024-05-09T09:14:59.821401+0000 2024-05-09T09:15:13.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: created 2024-05-09T09:14:59.821401+0000 2024-05-09T09:15:13.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: min_mon_release 19 (squid) 2024-05-09T09:15:13.789 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: election_strategy: 1 2024-05-09T09:15:13.789 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-09T09:15:13.789 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: fsmap 2024-05-09T09:15:13.789 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T09:15:13.789 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 ceph-mon[30140]: mgrmap e1: no daemons active 2024-05-09T09:15:13.789 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 bash[30121]: 1e0b9433ce3ef6566664cf618b2ea8d6339ecf7e5aa402f846b4df79e75c0cc0 2024-05-09T09:15:13.789 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:13 smithi012 systemd[1]: Started Ceph mon.a for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:15:14.631 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.987+0000 7febdf54e700 1 Processor -- start 2024-05-09T09:15:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.987+0000 7febdf54e700 1 -- start start 2024-05-09T09:15:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.987+0000 7febdf54e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 0x7febd8108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.987+0000 7febdf54e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7febd81086f0 con 0x7febd8107d00 2024-05-09T09:15:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.988+0000 7febdd2ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 0x7febd8108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.988+0000 7febdd2ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 0x7febd8108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49218/0 (socket says 172.21.15.12:49218) 2024-05-09T09:15:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.988+0000 7febdd2ea700 1 -- 172.21.15.12:0/1304550604 learned_addr learned my addr 172.21.15.12:0/1304550604 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.988+0000 7febdd2ea700 1 -- 172.21.15.12:0/1304550604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd8108eb0 con 0x7febd8107d00 2024-05-09T09:15:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.988+0000 7febdd2ea700 1 --2- 172.21.15.12:0/1304550604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 0x7febd8108120 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7febd4009e80 tx=0x7febd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a53a1c5a7f5d45d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.989+0000 7febcffff700 1 -- 172.21.15.12:0/1304550604 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febd4017070 con 0x7febd8107d00 2024-05-09T09:15:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.989+0000 7febcffff700 1 -- 172.21.15.12:0/1304550604 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7febd400b8c0 con 0x7febd8107d00 2024-05-09T09:15:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.989+0000 7febcffff700 1 -- 172.21.15.12:0/1304550604 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febd40045a0 con 0x7febd8107d00 2024-05-09T09:15:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.989+0000 7febdf54e700 1 -- 172.21.15.12:0/1304550604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 msgr2=0x7febd8108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.989+0000 7febdf54e700 1 --2- 172.21.15.12:0/1304550604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 0x7febd8108120 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7febd4009e80 tx=0x7febd400b330 comp rx=0 tx=0).stop 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.989+0000 7febdf54e700 1 -- 172.21.15.12:0/1304550604 shutdown_connections 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.989+0000 7febdf54e700 1 --2- 172.21.15.12:0/1304550604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 0x7febd8108120 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.989+0000 7febdf54e700 1 -- 172.21.15.12:0/1304550604 >> 172.21.15.12:0/1304550604 conn(0x7febd81032c0 msgr2=0x7febd81056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.989+0000 7febdf54e700 1 -- 172.21.15.12:0/1304550604 shutdown_connections 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.989+0000 7febdf54e700 1 -- 172.21.15.12:0/1304550604 wait complete. 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.990+0000 7febdf54e700 1 Processor -- start 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.990+0000 7febdf54e700 1 -- start start 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.990+0000 7febdf54e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 0x7febd807daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.990+0000 7febdf54e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7febd810c260 con 0x7febd8107d00 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.990+0000 7febdd2ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 0x7febd807daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.990+0000 7febdd2ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 0x7febd807daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49230/0 (socket says 172.21.15.12:49230) 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.990+0000 7febdd2ea700 1 -- 172.21.15.12:0/2551917607 learned_addr learned my addr 172.21.15.12:0/2551917607 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.991+0000 7febdd2ea700 1 -- 172.21.15.12:0/2551917607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd807e030 con 0x7febd8107d00 2024-05-09T09:15:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.991+0000 7febdd2ea700 1 --2- 172.21.15.12:0/2551917607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 0x7febd807daf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7febd4000f80 tx=0x7febd400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:14.634 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.991+0000 7febce7fc700 1 -- 172.21.15.12:0/2551917607 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febd4021020 con 0x7febd8107d00 2024-05-09T09:15:14.634 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.991+0000 7febce7fc700 1 -- 172.21.15.12:0/2551917607 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7febd401d470 con 0x7febd8107d00 2024-05-09T09:15:14.634 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.991+0000 7febdf54e700 1 -- 172.21.15.12:0/2551917607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd807c220 con 0x7febd8107d00 2024-05-09T09:15:14.634 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.991+0000 7febce7fc700 1 -- 172.21.15.12:0/2551917607 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febd4017040 con 0x7febd8107d00 2024-05-09T09:15:14.634 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.991+0000 7febdf54e700 1 -- 172.21.15.12:0/2551917607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd807c760 con 0x7febd8107d00 2024-05-09T09:15:14.634 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.991+0000 7febce7fc700 1 -- 172.21.15.12:0/2551917607 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7febd4024020 con 0x7febd8107d00 2024-05-09T09:15:14.634 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.991+0000 7febce7fc700 1 -- 172.21.15.12:0/2551917607 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7febd400f050 con 0x7febd8107d00 2024-05-09T09:15:14.634 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.992+0000 7febdf54e700 1 -- 172.21.15.12:0/2551917607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd8063980 con 0x7febd8107d00 2024-05-09T09:15:14.634 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:13.994+0000 7febce7fc700 1 -- 172.21.15.12:0/2551917607 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7febd402b020 con 0x7febd8107d00 2024-05-09T09:15:14.634 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:14.065+0000 7febdf54e700 1 -- 172.21.15.12:0/2551917607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7febd807caa0 con 0x7febd8107d00 2024-05-09T09:15:14.634 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:14.066+0000 7febce7fc700 1 -- 172.21.15.12:0/2551917607 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7febd4026400 con 0x7febd8107d00 2024-05-09T09:15:14.634 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:14.068+0000 7febdf54e700 1 -- 172.21.15.12:0/2551917607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 msgr2=0x7febd807daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:14.634 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:14.068+0000 7febdf54e700 1 --2- 172.21.15.12:0/2551917607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 0x7febd807daf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7febd4000f80 tx=0x7febd400bd50 comp rx=0 tx=0).stop 2024-05-09T09:15:14.635 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:14.068+0000 7febdf54e700 1 -- 172.21.15.12:0/2551917607 shutdown_connections 2024-05-09T09:15:14.635 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:14.068+0000 7febdf54e700 1 --2- 172.21.15.12:0/2551917607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd8107d00 0x7febd807daf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:14.635 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:14.068+0000 7febdf54e700 1 -- 172.21.15.12:0/2551917607 >> 172.21.15.12:0/2551917607 conn(0x7febd81032c0 msgr2=0x7febd8103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:14.635 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:14.068+0000 7febdf54e700 1 -- 172.21.15.12:0/2551917607 shutdown_connections 2024-05-09T09:15:14.635 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:14.068+0000 7febdf54e700 1 -- 172.21.15.12:0/2551917607 wait complete. 2024-05-09T09:15:14.635 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-09T09:15:14.656 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:15:14.656 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mgr... 2024-05-09T09:15:14.657 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-09T09:15:14.657 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-09T09:15:14.889 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.x 2024-05-09T09:15:14.890 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-09T09:15:15.041 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-770e12d0-0de4-11ef-bc98-c7b262605968.target.wants/ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-770e12d0-0de4-11ef-bc98-c7b262605968@.service. 2024-05-09T09:15:15.356 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:15 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2551917607' entity='client.admin' 2024-05-09T09:15:15.542 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T09:15:15.542 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-05-09T09:15:15.550 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T09:15:15.550 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-09T09:15:15.550 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-05-09T09:15:15.550 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T09:15:15.559 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T09:15:15.559 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-05-09T09:15:15.567 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T09:15:15.567 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-09T09:15:15.567 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-05-09T09:15:15.567 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-09T09:15:15.567 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr to start... 2024-05-09T09:15:15.567 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr... 2024-05-09T09:15:16.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:16 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3701406394' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T09:15:16.345 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:16 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:16.256+0000 7f67811e3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T09:15:16.345 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:16 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:16.345+0000 7f67811e3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T09:15:16.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-09T09:15:16.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-09T09:15:16.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:15:16.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T09:15:16.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T09:15:16.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T09:15:16.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T09:15:16.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:16.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T09:15:16.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T09:15:16.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-05-09T09:15:16.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T09:15:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T09:15:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-09T09:15:16.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T09:15:02.892141+0000", 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.053+0000 7fbe43185700 1 Processor -- start 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.053+0000 7fbe43185700 1 -- start start 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.053+0000 7fbe43185700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 0x7fbe3c108150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.053+0000 7fbe43185700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c108720 con 0x7fbe3c107d30 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.054+0000 7fbe40f21700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 0x7fbe3c108150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.054+0000 7fbe40f21700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 0x7fbe3c108150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49248/0 (socket says 172.21.15.12:49248) 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.054+0000 7fbe40f21700 1 -- 172.21.15.12:0/1946665308 learned_addr learned my addr 172.21.15.12:0/1946665308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.054+0000 7fbe40f21700 1 -- 172.21.15.12:0/1946665308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3c108f30 con 0x7fbe3c107d30 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.054+0000 7fbe40f21700 1 --2- 172.21.15.12:0/1946665308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 0x7fbe3c108150 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbe300098a0 tx=0x7fbe30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d585396464e4700f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.054+0000 7fbe3b7fe700 1 -- 172.21.15.12:0/1946665308 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe30004160 con 0x7fbe3c107d30 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.054+0000 7fbe3b7fe700 1 -- 172.21.15.12:0/1946665308 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fbe30004300 con 0x7fbe3c107d30 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.054+0000 7fbe3b7fe700 1 -- 172.21.15.12:0/1946665308 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe30011b40 con 0x7fbe3c107d30 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.055+0000 7fbe43185700 1 -- 172.21.15.12:0/1946665308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 msgr2=0x7fbe3c108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.055+0000 7fbe43185700 1 --2- 172.21.15.12:0/1946665308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 0x7fbe3c108150 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbe300098a0 tx=0x7fbe30009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.055+0000 7fbe43185700 1 -- 172.21.15.12:0/1946665308 shutdown_connections 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.055+0000 7fbe43185700 1 --2- 172.21.15.12:0/1946665308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 0x7fbe3c108150 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.055+0000 7fbe43185700 1 -- 172.21.15.12:0/1946665308 >> 172.21.15.12:0/1946665308 conn(0x7fbe3c1032f0 msgr2=0x7fbe3c105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.055+0000 7fbe43185700 1 -- 172.21.15.12:0/1946665308 shutdown_connections 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.055+0000 7fbe43185700 1 -- 172.21.15.12:0/1946665308 wait complete. 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.056+0000 7fbe43185700 1 Processor -- start 2024-05-09T09:15:16.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.056+0000 7fbe43185700 1 -- start start 2024-05-09T09:15:16.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.056+0000 7fbe43185700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 0x7fbe3c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.056+0000 7fbe43185700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c10c2e0 con 0x7fbe3c107d30 2024-05-09T09:15:16.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.056+0000 7fbe40f21700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 0x7fbe3c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.056+0000 7fbe40f21700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 0x7fbe3c19bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49252/0 (socket says 172.21.15.12:49252) 2024-05-09T09:15:16.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.056+0000 7fbe40f21700 1 -- 172.21.15.12:0/3701406394 learned_addr learned my addr 172.21.15.12:0/3701406394 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:16.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.057+0000 7fbe40f21700 1 -- 172.21.15.12:0/3701406394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3c19c140 con 0x7fbe3c107d30 2024-05-09T09:15:16.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.057+0000 7fbe40f21700 1 --2- 172.21.15.12:0/3701406394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 0x7fbe3c19bc00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbe30011670 tx=0x7fbe30011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:16.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.057+0000 7fbe39ffb700 1 -- 172.21.15.12:0/3701406394 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe300038b0 con 0x7fbe3c107d30 2024-05-09T09:15:16.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.057+0000 7fbe43185700 1 -- 172.21.15.12:0/3701406394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3c19c3d0 con 0x7fbe3c107d30 2024-05-09T09:15:16.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.057+0000 7fbe39ffb700 1 -- 172.21.15.12:0/3701406394 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fbe3001a430 con 0x7fbe3c107d30 2024-05-09T09:15:16.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.057+0000 7fbe39ffb700 1 -- 172.21.15.12:0/3701406394 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe30019550 con 0x7fbe3c107d30 2024-05-09T09:15:16.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.057+0000 7fbe43185700 1 -- 172.21.15.12:0/3701406394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3c19f0c0 con 0x7fbe3c107d30 2024-05-09T09:15:16.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.057+0000 7fbe39ffb700 1 -- 172.21.15.12:0/3701406394 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbe30019b90 con 0x7fbe3c107d30 2024-05-09T09:15:16.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.057+0000 7fbe39ffb700 1 -- 172.21.15.12:0/3701406394 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbe30024730 con 0x7fbe3c107d30 2024-05-09T09:15:16.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.058+0000 7fbe43185700 1 -- 172.21.15.12:0/3701406394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe3c063980 con 0x7fbe3c107d30 2024-05-09T09:15:16.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.060+0000 7fbe39ffb700 1 -- 172.21.15.12:0/3701406394 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbe30043dd0 con 0x7fbe3c107d30 2024-05-09T09:15:16.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.134+0000 7fbe43185700 1 -- 172.21.15.12:0/3701406394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbe3c19f630 con 0x7fbe3c107d30 2024-05-09T09:15:16.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.134+0000 7fbe39ffb700 1 -- 172.21.15.12:0/3701406394 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fbe30032020 con 0x7fbe3c107d30 2024-05-09T09:15:16.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.135+0000 7fbe43185700 1 -- 172.21.15.12:0/3701406394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 msgr2=0x7fbe3c19bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.135+0000 7fbe43185700 1 --2- 172.21.15.12:0/3701406394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 0x7fbe3c19bc00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbe30011670 tx=0x7fbe30011b50 comp rx=0 tx=0).stop 2024-05-09T09:15:16.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.135+0000 7fbe43185700 1 -- 172.21.15.12:0/3701406394 shutdown_connections 2024-05-09T09:15:16.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.135+0000 7fbe43185700 1 --2- 172.21.15.12:0/3701406394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe3c107d30 0x7fbe3c19bc00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.135+0000 7fbe43185700 1 -- 172.21.15.12:0/3701406394 >> 172.21.15.12:0/3701406394 conn(0x7fbe3c1032f0 msgr2=0x7fbe3c103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:16.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.136+0000 7fbe43185700 1 -- 172.21.15.12:0/3701406394 shutdown_connections 2024-05-09T09:15:16.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:16.136+0000 7fbe43185700 1 -- 172.21.15.12:0/3701406394 wait complete. 2024-05-09T09:15:16.739 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (1/15)... 2024-05-09T09:15:17.976 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:17 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:17.620+0000 7f67811e3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T09:15:17.976 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:17 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:17.797+0000 7f67811e3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T09:15:17.976 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:17 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:17.975+0000 7f67811e3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T09:15:18.586 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:18 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:18.278+0000 7f67811e3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T09:15:18.586 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:18 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:18.374+0000 7f67811e3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T09:15:18.586 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:18 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:18.587+0000 7f67811e3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T09:15:18.851 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:18 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:18.790+0000 7f67811e3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T09:15:19.733 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:19 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:19.424+0000 7f67811e3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T09:15:20.174 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:19 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:19.864+0000 7f67811e3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T09:15:20.174 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:19 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:19.965+0000 7f67811e3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T09:15:20.174 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:20 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:20.174+0000 7f67811e3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T09:15:21.076 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:20 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:20.723+0000 7f67811e3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T09:15:21.076 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:20 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:20.820+0000 7f67811e3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T09:15:21.451 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:21.077+0000 7f67811e3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T09:15:21.451 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:21.216+0000 7f67811e3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T09:15:21.451 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:21.315+0000 7f67811e3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T09:15:21.451 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:21.436+0000 7f67811e3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T09:15:22.030 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:21.752+0000 7f67811e3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T09:15:22.390 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:22 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1398832688' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T09:15:22.728 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:22 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:22.391+0000 7f67811e3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T09:15:22.728 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:22 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:22.616+0000 7f67811e3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T09:15:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:23 smithi012 ceph-mon[30140]: Activating manager daemon x 2024-05-09T09:15:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:23 smithi012 ceph-mon[30140]: mgrmap e2: x(active, starting, since 0.00293937s) 2024-05-09T09:15:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:23 smithi012 ceph-mon[30140]: from='mgr.14100 172.21.15.12:0/1607477514' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T09:15:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:23 smithi012 ceph-mon[30140]: from='mgr.14100 172.21.15.12:0/1607477514' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T09:15:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:23 smithi012 ceph-mon[30140]: from='mgr.14100 172.21.15.12:0/1607477514' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T09:15:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:23 smithi012 ceph-mon[30140]: from='mgr.14100 172.21.15.12:0/1607477514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:15:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:23 smithi012 ceph-mon[30140]: from='mgr.14100 172.21.15.12:0/1607477514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T09:15:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:23 smithi012 ceph-mon[30140]: Manager daemon x is now available 2024-05-09T09:15:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:23 smithi012 ceph-mon[30140]: from='mgr.14100 172.21.15.12:0/1607477514' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T09:15:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:23 smithi012 ceph-mon[30140]: from='mgr.14100 172.21.15.12:0/1607477514' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T09:15:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:23 smithi012 ceph-mon[30140]: from='mgr.14100 172.21.15.12:0/1607477514' entity='mgr.x' 2024-05-09T09:15:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:23 smithi012 ceph-mon[30140]: from='mgr.14100 172.21.15.12:0/1607477514' entity='mgr.x' 2024-05-09T09:15:23.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:23 smithi012 ceph-mon[30140]: from='mgr.14100 172.21.15.12:0/1607477514' entity='mgr.x' 2024-05-09T09:15:23.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-09T09:15:23.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-09T09:15:23.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:15:23.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T09:15:23.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T09:15:23.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T09:15:23.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T09:15:23.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T09:15:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T09:15:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T09:15:02.892141+0000", 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.899+0000 7f5b4a3e1700 1 Processor -- start 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.899+0000 7f5b4a3e1700 1 -- start start 2024-05-09T09:15:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.899+0000 7f5b4a3e1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 0x7f5b44106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.899+0000 7f5b4a3e1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441075c0 con 0x7f5b44106bd0 2024-05-09T09:15:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.899+0000 7f5b43fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 0x7f5b44106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.899+0000 7f5b43fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 0x7f5b44106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42928/0 (socket says 172.21.15.12:42928) 2024-05-09T09:15:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.899+0000 7f5b43fff700 1 -- 172.21.15.12:0/3356999083 learned_addr learned my addr 172.21.15.12:0/3356999083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.900+0000 7f5b43fff700 1 -- 172.21.15.12:0/3356999083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b44107dd0 con 0x7f5b44106bd0 2024-05-09T09:15:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.900+0000 7f5b43fff700 1 --2- 172.21.15.12:0/3356999083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 0x7f5b44106ff0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5b340098a0 tx=0x7f5b34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a45cbafcf6a5aaa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.900+0000 7f5b42ffd700 1 -- 172.21.15.12:0/3356999083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b34004160 con 0x7f5b44106bd0 2024-05-09T09:15:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.900+0000 7f5b42ffd700 1 -- 172.21.15.12:0/3356999083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f5b34004300 con 0x7f5b44106bd0 2024-05-09T09:15:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.900+0000 7f5b42ffd700 1 -- 172.21.15.12:0/3356999083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b34011b40 con 0x7f5b44106bd0 2024-05-09T09:15:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.901+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/3356999083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 msgr2=0x7f5b44106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.901+0000 7f5b4a3e1700 1 --2- 172.21.15.12:0/3356999083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 0x7f5b44106ff0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5b340098a0 tx=0x7f5b34009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.901+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/3356999083 shutdown_connections 2024-05-09T09:15:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.901+0000 7f5b4a3e1700 1 --2- 172.21.15.12:0/3356999083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 0x7f5b44106ff0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.901+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/3356999083 >> 172.21.15.12:0/3356999083 conn(0x7f5b44102170 msgr2=0x7f5b441045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.901+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/3356999083 shutdown_connections 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.901+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/3356999083 wait complete. 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.901+0000 7f5b4a3e1700 1 Processor -- start 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.901+0000 7f5b4a3e1700 1 -- start start 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.902+0000 7f5b4a3e1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 0x7f5b4419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.902+0000 7f5b4a3e1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4410b180 con 0x7f5b44106bd0 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.902+0000 7f5b43fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 0x7f5b4419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.902+0000 7f5b43fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 0x7f5b4419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42942/0 (socket says 172.21.15.12:42942) 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.902+0000 7f5b43fff700 1 -- 172.21.15.12:0/1398832688 learned_addr learned my addr 172.21.15.12:0/1398832688 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.902+0000 7f5b43fff700 1 -- 172.21.15.12:0/1398832688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4419b080 con 0x7f5b44106bd0 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.902+0000 7f5b43fff700 1 --2- 172.21.15.12:0/1398832688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 0x7f5b4419ab40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b34011670 tx=0x7f5b34011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.902+0000 7f5b417fa700 1 -- 172.21.15.12:0/1398832688 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b340038b0 con 0x7f5b44106bd0 2024-05-09T09:15:23.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.903+0000 7f5b417fa700 1 -- 172.21.15.12:0/1398832688 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f5b3401a430 con 0x7f5b44106bd0 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.903+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/1398832688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4419b310 con 0x7f5b44106bd0 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.903+0000 7f5b417fa700 1 -- 172.21.15.12:0/1398832688 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b34019550 con 0x7f5b44106bd0 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.903+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/1398832688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4419e000 con 0x7f5b44106bd0 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.903+0000 7f5b417fa700 1 -- 172.21.15.12:0/1398832688 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5b34019be0 con 0x7f5b44106bd0 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.903+0000 7f5b417fa700 1 -- 172.21.15.12:0/1398832688 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5b34024730 con 0x7f5b44106bd0 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.903+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/1398832688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b44063980 con 0x7f5b44106bd0 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.906+0000 7f5b417fa700 1 -- 172.21.15.12:0/1398832688 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5b34043dd0 con 0x7f5b44106bd0 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.980+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/1398832688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5b4419e580 con 0x7f5b44106bd0 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.980+0000 7f5b417fa700 1 -- 172.21.15.12:0/1398832688 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5b34032030 con 0x7f5b44106bd0 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.981+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/1398832688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 msgr2=0x7f5b4419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.981+0000 7f5b4a3e1700 1 --2- 172.21.15.12:0/1398832688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 0x7f5b4419ab40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b34011670 tx=0x7f5b34011b50 comp rx=0 tx=0).stop 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.981+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/1398832688 shutdown_connections 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.981+0000 7f5b4a3e1700 1 --2- 172.21.15.12:0/1398832688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44106bd0 0x7f5b4419ab40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.981+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/1398832688 >> 172.21.15.12:0/1398832688 conn(0x7f5b44102170 msgr2=0x7f5b44102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.981+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/1398832688 shutdown_connections 2024-05-09T09:15:23.823 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:21.981+0000 7f5b4a3e1700 1 -- 172.21.15.12:0/1398832688 wait complete. 2024-05-09T09:15:23.823 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (2/15)... 2024-05-09T09:15:24.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:24 smithi012 ceph-mon[30140]: mgrmap e3: x(active, since 1.00801s) 2024-05-09T09:15:25.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:25 smithi012 ceph-mon[30140]: mgrmap e4: x(active, since 2s) 2024-05-09T09:15:26.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:26 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1140295002' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T09:15:27.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-09T09:15:27.029 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-09T09:15:27.029 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:15:27.029 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T09:15:27.029 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T09:15:27.029 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T09:15:27.029 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T09:15:27.029 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:27.029 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T09:15:27.029 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T09:15:27.029 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-05-09T09:15:27.029 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-09T09:15:27.029 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T09:15:27.030 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T09:15:27.030 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-09T09:15:27.030 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-05-09T09:15:27.030 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T09:15:27.030 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T09:15:27.030 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T09:15:27.030 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T09:15:27.030 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:27.030 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T09:15:27.030 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T09:15:27.030 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T09:15:27.030 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T09:15:27.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T09:15:27.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T09:15:27.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T09:15:27.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T09:15:27.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T09:15:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T09:15:02.892141+0000", 2024-05-09T09:15:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.213+0000 7fc0f94b7700 1 Processor -- start 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.213+0000 7fc0f94b7700 1 -- start start 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.213+0000 7fc0f94b7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 0x7fc0f4108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.213+0000 7fc0f94b7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f41086f0 con 0x7fc0f4107d00 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.213+0000 7fc0f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 0x7fc0f4108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.213+0000 7fc0f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 0x7fc0f4108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43020/0 (socket says 172.21.15.12:43020) 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.213+0000 7fc0f2ffd700 1 -- 172.21.15.12:0/2383336828 learned_addr learned my addr 172.21.15.12:0/2383336828 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.213+0000 7fc0f2ffd700 1 -- 172.21.15.12:0/2383336828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f4108eb0 con 0x7fc0f4107d00 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.214+0000 7fc0f2ffd700 1 --2- 172.21.15.12:0/2383336828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 0x7fc0f4108120 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40098a0 tx=0x7fc0e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e510ddd8d869a72e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.214+0000 7fc0f27fc700 1 -- 172.21.15.12:0/2383336828 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0e4004160 con 0x7fc0f4107d00 2024-05-09T09:15:27.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.214+0000 7fc0f27fc700 1 -- 172.21.15.12:0/2383336828 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fc0e4004300 con 0x7fc0f4107d00 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.214+0000 7fc0f27fc700 1 -- 172.21.15.12:0/2383336828 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0e4011b40 con 0x7fc0f4107d00 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.214+0000 7fc0f94b7700 1 -- 172.21.15.12:0/2383336828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 msgr2=0x7fc0f4108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.214+0000 7fc0f94b7700 1 --2- 172.21.15.12:0/2383336828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 0x7fc0f4108120 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40098a0 tx=0x7fc0e4009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.214+0000 7fc0f94b7700 1 -- 172.21.15.12:0/2383336828 shutdown_connections 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.214+0000 7fc0f94b7700 1 --2- 172.21.15.12:0/2383336828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 0x7fc0f4108120 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.214+0000 7fc0f94b7700 1 -- 172.21.15.12:0/2383336828 >> 172.21.15.12:0/2383336828 conn(0x7fc0f41032c0 msgr2=0x7fc0f41056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.214+0000 7fc0f94b7700 1 -- 172.21.15.12:0/2383336828 shutdown_connections 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.215+0000 7fc0f94b7700 1 -- 172.21.15.12:0/2383336828 wait complete. 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.215+0000 7fc0f94b7700 1 Processor -- start 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.215+0000 7fc0f94b7700 1 -- start start 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.215+0000 7fc0f94b7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 0x7fc0f419bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.216+0000 7fc0f94b7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f410c260 con 0x7fc0f4107d00 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.216+0000 7fc0f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 0x7fc0f419bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:27.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.216+0000 7fc0f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 0x7fc0f419bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43036/0 (socket says 172.21.15.12:43036) 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.216+0000 7fc0f2ffd700 1 -- 172.21.15.12:0/1140295002 learned_addr learned my addr 172.21.15.12:0/1140295002 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.216+0000 7fc0f2ffd700 1 -- 172.21.15.12:0/1140295002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f419c180 con 0x7fc0f4107d00 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.216+0000 7fc0f2ffd700 1 --2- 172.21.15.12:0/1140295002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 0x7fc0f419bc40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4011670 tx=0x7fc0e4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.216+0000 7fc0f0ff9700 1 -- 172.21.15.12:0/1140295002 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0e40038b0 con 0x7fc0f4107d00 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.216+0000 7fc0f94b7700 1 -- 172.21.15.12:0/1140295002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f419c410 con 0x7fc0f4107d00 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.216+0000 7fc0f0ff9700 1 -- 172.21.15.12:0/1140295002 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fc0e401a430 con 0x7fc0f4107d00 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.216+0000 7fc0f0ff9700 1 -- 172.21.15.12:0/1140295002 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0e4019550 con 0x7fc0f4107d00 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.217+0000 7fc0f94b7700 1 -- 172.21.15.12:0/1140295002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f419f100 con 0x7fc0f4107d00 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.217+0000 7fc0f0ff9700 1 -- 172.21.15.12:0/1140295002 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fc0e40198b0 con 0x7fc0f4107d00 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.217+0000 7fc0f0ff9700 1 --2- 172.21.15.12:0/1140295002 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fc0e0039070 0x7fc0e003b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.217+0000 7fc0f0ff9700 1 -- 172.21.15.12:0/1140295002 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc0e404d1c0 con 0x7fc0f4107d00 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.217+0000 7fc0ea7fc700 1 --2- 172.21.15.12:0/1140295002 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fc0e0039070 0x7fc0e003b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.218+0000 7fc0f94b7700 1 -- 172.21.15.12:0/1140295002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f4063980 con 0x7fc0f4107d00 2024-05-09T09:15:27.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.218+0000 7fc0ea7fc700 1 --2- 172.21.15.12:0/1140295002 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fc0e0039070 0x7fc0e003b530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc006fd0 tx=0x7fc0dc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.223+0000 7fc0f0ff9700 1 -- 172.21.15.12:0/1140295002 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc0e400fda0 con 0x7fc0f4107d00 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.513+0000 7fc0f94b7700 1 -- 172.21.15.12:0/1140295002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc0f419f6b0 con 0x7fc0f4107d00 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.513+0000 7fc0f0ff9700 1 -- 172.21.15.12:0/1140295002 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc0e402d960 con 0x7fc0f4107d00 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.515+0000 7fc0f94b7700 1 -- 172.21.15.12:0/1140295002 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fc0e0039070 msgr2=0x7fc0e003b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.515+0000 7fc0f94b7700 1 --2- 172.21.15.12:0/1140295002 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fc0e0039070 0x7fc0e003b530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc006fd0 tx=0x7fc0dc006e40 comp rx=0 tx=0).stop 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.515+0000 7fc0f94b7700 1 -- 172.21.15.12:0/1140295002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 msgr2=0x7fc0f419bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.515+0000 7fc0f94b7700 1 --2- 172.21.15.12:0/1140295002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 0x7fc0f419bc40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4011670 tx=0x7fc0e4011b50 comp rx=0 tx=0).stop 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.515+0000 7fc0f94b7700 1 -- 172.21.15.12:0/1140295002 shutdown_connections 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.515+0000 7fc0f94b7700 1 --2- 172.21.15.12:0/1140295002 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fc0e0039070 0x7fc0e003b530 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.515+0000 7fc0f94b7700 1 --2- 172.21.15.12:0/1140295002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0f4107d00 0x7fc0f419bc40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.515+0000 7fc0f94b7700 1 -- 172.21.15.12:0/1140295002 >> 172.21.15.12:0/1140295002 conn(0x7fc0f41032c0 msgr2=0x7fc0f4103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.515+0000 7fc0f94b7700 1 -- 172.21.15.12:0/1140295002 shutdown_connections 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:26.515+0000 7fc0f94b7700 1 -- 172.21.15.12:0/1140295002 wait complete. 2024-05-09T09:15:27.045 INFO:teuthology.orchestra.run.smithi012.stdout:mgr is available 2024-05-09T09:15:27.929 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:27 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2264820172' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T09:15:28.190 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-09T09:15:28.190 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T09:15:28.190 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:15:28.190 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T09:15:28.190 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T09:15:28.190 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T09:15:28.190 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T09:15:28.190 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-09T09:15:28.190 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T09:15:28.190 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T09:15:28.190 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.409+0000 7fc815e9d700 1 Processor -- start 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.409+0000 7fc815e9d700 1 -- start start 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.410+0000 7fc815e9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 0x7fc81007c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.410+0000 7fc815e9d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc81007c830 con 0x7fc81007de00 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.410+0000 7fc80f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 0x7fc81007c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.410+0000 7fc80f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 0x7fc81007c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43048/0 (socket says 172.21.15.12:43048) 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.410+0000 7fc80f7fe700 1 -- 172.21.15.12:0/325068491 learned_addr learned my addr 172.21.15.12:0/325068491 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:28.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.410+0000 7fc80f7fe700 1 -- 172.21.15.12:0/325068491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81007c9b0 con 0x7fc81007de00 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.410+0000 7fc80f7fe700 1 --2- 172.21.15.12:0/325068491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 0x7fc81007c260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc8000098a0 tx=0x7fc800009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4787f71c40c21210 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.410+0000 7fc80effd700 1 -- 172.21.15.12:0/325068491 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc800004160 con 0x7fc81007de00 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.410+0000 7fc80effd700 1 -- 172.21.15.12:0/325068491 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fc800004300 con 0x7fc81007de00 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.410+0000 7fc80effd700 1 -- 172.21.15.12:0/325068491 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc800011b40 con 0x7fc81007de00 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.411+0000 7fc815e9d700 1 -- 172.21.15.12:0/325068491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 msgr2=0x7fc81007c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.411+0000 7fc815e9d700 1 --2- 172.21.15.12:0/325068491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 0x7fc81007c260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc8000098a0 tx=0x7fc800009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.411+0000 7fc815e9d700 1 -- 172.21.15.12:0/325068491 shutdown_connections 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.411+0000 7fc815e9d700 1 --2- 172.21.15.12:0/325068491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 0x7fc81007c260 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.411+0000 7fc815e9d700 1 -- 172.21.15.12:0/325068491 >> 172.21.15.12:0/325068491 conn(0x7fc8101032f0 msgr2=0x7fc810105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.411+0000 7fc815e9d700 1 -- 172.21.15.12:0/325068491 shutdown_connections 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.411+0000 7fc815e9d700 1 -- 172.21.15.12:0/325068491 wait complete. 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.412+0000 7fc815e9d700 1 Processor -- start 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.412+0000 7fc815e9d700 1 -- start start 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.412+0000 7fc815e9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 0x7fc81019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:28.192 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.412+0000 7fc815e9d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc81010e300 con 0x7fc81007de00 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.412+0000 7fc80f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 0x7fc81019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.412+0000 7fc80f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 0x7fc81019ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43052/0 (socket says 172.21.15.12:43052) 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.412+0000 7fc80f7fe700 1 -- 172.21.15.12:0/2264820172 learned_addr learned my addr 172.21.15.12:0/2264820172 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.412+0000 7fc80f7fe700 1 -- 172.21.15.12:0/2264820172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81019e2f0 con 0x7fc81007de00 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.413+0000 7fc80f7fe700 1 --2- 172.21.15.12:0/2264820172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 0x7fc81019ddb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc800011670 tx=0x7fc800011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.413+0000 7fc80d7fa700 1 -- 172.21.15.12:0/2264820172 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8000038b0 con 0x7fc81007de00 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.413+0000 7fc80d7fa700 1 -- 172.21.15.12:0/2264820172 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fc80001a430 con 0x7fc81007de00 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.413+0000 7fc815e9d700 1 -- 172.21.15.12:0/2264820172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81019e580 con 0x7fc81007de00 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.413+0000 7fc80d7fa700 1 -- 172.21.15.12:0/2264820172 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc800019550 con 0x7fc81007de00 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.413+0000 7fc815e9d700 1 -- 172.21.15.12:0/2264820172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8101a1230 con 0x7fc81007de00 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.414+0000 7fc815e9d700 1 -- 172.21.15.12:0/2264820172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc810063980 con 0x7fc81007de00 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.414+0000 7fc80d7fa700 1 -- 172.21.15.12:0/2264820172 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fc800003a50 con 0x7fc81007de00 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.414+0000 7fc80d7fa700 1 --2- 172.21.15.12:0/2264820172 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fc7fc0390c0 0x7fc7fc03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:28.193 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.414+0000 7fc80d7fa700 1 -- 172.21.15.12:0/2264820172 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc80004d210 con 0x7fc81007de00 2024-05-09T09:15:28.194 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.414+0000 7fc806ffd700 1 --2- 172.21.15.12:0/2264820172 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fc7fc0390c0 0x7fc7fc03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:28.194 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.415+0000 7fc806ffd700 1 --2- 172.21.15.12:0/2264820172 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fc7fc0390c0 0x7fc7fc03b580 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8006fd0 tx=0x7fc7f8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:28.194 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.419+0000 7fc80d7fa700 1 -- 172.21.15.12:0/2264820172 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc800003d40 con 0x7fc81007de00 2024-05-09T09:15:28.194 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.623+0000 7fc815e9d700 1 -- 172.21.15.12:0/2264820172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc8101a1450 con 0x7fc81007de00 2024-05-09T09:15:28.194 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.624+0000 7fc80d7fa700 1 -- 172.21.15.12:0/2264820172 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fc8101a1450 con 0x7fc81007de00 2024-05-09T09:15:28.194 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.626+0000 7fc815e9d700 1 -- 172.21.15.12:0/2264820172 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fc7fc0390c0 msgr2=0x7fc7fc03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:28.194 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.626+0000 7fc815e9d700 1 --2- 172.21.15.12:0/2264820172 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fc7fc0390c0 0x7fc7fc03b580 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8006fd0 tx=0x7fc7f8006e40 comp rx=0 tx=0).stop 2024-05-09T09:15:28.194 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.626+0000 7fc815e9d700 1 -- 172.21.15.12:0/2264820172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 msgr2=0x7fc81019ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:28.194 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.626+0000 7fc815e9d700 1 --2- 172.21.15.12:0/2264820172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 0x7fc81019ddb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc800011670 tx=0x7fc800011b50 comp rx=0 tx=0).stop 2024-05-09T09:15:28.195 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.626+0000 7fc815e9d700 1 -- 172.21.15.12:0/2264820172 shutdown_connections 2024-05-09T09:15:28.195 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.626+0000 7fc815e9d700 1 --2- 172.21.15.12:0/2264820172 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fc7fc0390c0 0x7fc7fc03b580 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:28.195 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.626+0000 7fc815e9d700 1 --2- 172.21.15.12:0/2264820172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc81007de00 0x7fc81019ddb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:28.195 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.626+0000 7fc815e9d700 1 -- 172.21.15.12:0/2264820172 >> 172.21.15.12:0/2264820172 conn(0x7fc8101032f0 msgr2=0x7fc810103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:28.195 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.626+0000 7fc815e9d700 1 -- 172.21.15.12:0/2264820172 shutdown_connections 2024-05-09T09:15:28.195 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:27.626+0000 7fc815e9d700 1 -- 172.21.15.12:0/2264820172 wait complete. 2024-05-09T09:15:28.195 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling cephadm module... 2024-05-09T09:15:29.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:28 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4226349296' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-09T09:15:29.234 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:28 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: ignoring --setuser ceph since I am not root 2024-05-09T09:15:29.234 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:28 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: ignoring --setgroup ceph since I am not root 2024-05-09T09:15:29.234 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:29 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:29.120+0000 7fc3f53b6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T09:15:29.528 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:29 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:29.315+0000 7fc3f53b6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T09:15:29.528 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:29 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:29.422+0000 7fc3f53b6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T09:15:29.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.618+0000 7fe43f9d8700 1 Processor -- start 2024-05-09T09:15:29.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.618+0000 7fe43f9d8700 1 -- start start 2024-05-09T09:15:29.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.618+0000 7fe43f9d8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 0x7fe43807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:29.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.618+0000 7fe43f9d8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe43807c890 con 0x7fe43807de10 2024-05-09T09:15:29.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.618+0000 7fe43d774700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 0x7fe43807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:29.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.618+0000 7fe43d774700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 0x7fe43807c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43054/0 (socket says 172.21.15.12:43054) 2024-05-09T09:15:29.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.618+0000 7fe43d774700 1 -- 172.21.15.12:0/2486915350 learned_addr learned my addr 172.21.15.12:0/2486915350 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:29.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.619+0000 7fe43d774700 1 -- 172.21.15.12:0/2486915350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43807ca10 con 0x7fe43807de10 2024-05-09T09:15:29.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.619+0000 7fe43d774700 1 --2- 172.21.15.12:0/2486915350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 0x7fe43807c2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe4340098a0 tx=0x7fe434009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16bdea35c1f3be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:29.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.619+0000 7fe42ffff700 1 -- 172.21.15.12:0/2486915350 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe434004160 con 0x7fe43807de10 2024-05-09T09:15:29.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.619+0000 7fe42ffff700 1 -- 172.21.15.12:0/2486915350 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fe434004300 con 0x7fe43807de10 2024-05-09T09:15:29.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.619+0000 7fe42ffff700 1 -- 172.21.15.12:0/2486915350 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe434011b40 con 0x7fe43807de10 2024-05-09T09:15:29.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.620+0000 7fe43f9d8700 1 -- 172.21.15.12:0/2486915350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 msgr2=0x7fe43807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:29.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.620+0000 7fe43f9d8700 1 --2- 172.21.15.12:0/2486915350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 0x7fe43807c2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe4340098a0 tx=0x7fe434009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:29.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.620+0000 7fe43f9d8700 1 -- 172.21.15.12:0/2486915350 shutdown_connections 2024-05-09T09:15:29.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.620+0000 7fe43f9d8700 1 --2- 172.21.15.12:0/2486915350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 0x7fe43807c2c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:29.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.620+0000 7fe43f9d8700 1 -- 172.21.15.12:0/2486915350 >> 172.21.15.12:0/2486915350 conn(0x7fe4381032c0 msgr2=0x7fe4381056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:29.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.620+0000 7fe43f9d8700 1 -- 172.21.15.12:0/2486915350 shutdown_connections 2024-05-09T09:15:29.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.620+0000 7fe43f9d8700 1 -- 172.21.15.12:0/2486915350 wait complete. 2024-05-09T09:15:29.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.620+0000 7fe43f9d8700 1 Processor -- start 2024-05-09T09:15:29.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.620+0000 7fe43f9d8700 1 -- start start 2024-05-09T09:15:29.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.621+0000 7fe43f9d8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 0x7fe43819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:29.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.621+0000 7fe43f9d8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe43810e2d0 con 0x7fe43807de10 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.621+0000 7fe43d774700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 0x7fe43819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.621+0000 7fe43d774700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 0x7fe43819ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43068/0 (socket says 172.21.15.12:43068) 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.621+0000 7fe43d774700 1 -- 172.21.15.12:0/4226349296 learned_addr learned my addr 172.21.15.12:0/4226349296 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.621+0000 7fe43d774700 1 -- 172.21.15.12:0/4226349296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43819e300 con 0x7fe43807de10 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.621+0000 7fe43d774700 1 --2- 172.21.15.12:0/4226349296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 0x7fe43819ddc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe434004180 tx=0x7fe4340046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.621+0000 7fe42e7fc700 1 -- 172.21.15.12:0/4226349296 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4340039e0 con 0x7fe43807de10 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.621+0000 7fe42e7fc700 1 -- 172.21.15.12:0/4226349296 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fe434020070 con 0x7fe43807de10 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.621+0000 7fe43f9d8700 1 -- 172.21.15.12:0/4226349296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43819e590 con 0x7fe43807de10 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.621+0000 7fe42e7fc700 1 -- 172.21.15.12:0/4226349296 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4340193c0 con 0x7fe43807de10 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.621+0000 7fe43f9d8700 1 -- 172.21.15.12:0/4226349296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4381a1280 con 0x7fe43807de10 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.622+0000 7fe42e7fc700 1 -- 172.21.15.12:0/4226349296 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fe43401a430 con 0x7fe43807de10 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.622+0000 7fe43f9d8700 1 -- 172.21.15.12:0/4226349296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe438063980 con 0x7fe43807de10 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.623+0000 7fe42e7fc700 1 --2- 172.21.15.12:0/4226349296 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fe4240390c0 0x7fe42403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:29.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.623+0000 7fe42e7fc700 1 -- 172.21.15.12:0/4226349296 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe43404d070 con 0x7fe43807de10 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.623+0000 7fe43cf73700 1 --2- 172.21.15.12:0/4226349296 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fe4240390c0 0x7fe42403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.623+0000 7fe43cf73700 1 --2- 172.21.15.12:0/4226349296 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fe4240390c0 0x7fe42403b580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe428006fd0 tx=0x7fe428006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.628+0000 7fe42e7fc700 1 -- 172.21.15.12:0/4226349296 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe43400e260 con 0x7fe43807de10 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.893+0000 7fe43f9d8700 1 -- 172.21.15.12:0/4226349296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fe4381a1450 con 0x7fe43807de10 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.945+0000 7fe42e7fc700 1 -- 172.21.15.12:0/4226349296 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fe4381a1450 con 0x7fe43807de10 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.946+0000 7fe42e7fc700 1 -- 172.21.15.12:0/4226349296 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fe434019680 con 0x7fe43807de10 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.948+0000 7fe43f9d8700 1 -- 172.21.15.12:0/4226349296 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fe4240390c0 msgr2=0x7fe42403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.948+0000 7fe43f9d8700 1 --2- 172.21.15.12:0/4226349296 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fe4240390c0 0x7fe42403b580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe428006fd0 tx=0x7fe428006e40 comp rx=0 tx=0).stop 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.948+0000 7fe43f9d8700 1 -- 172.21.15.12:0/4226349296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 msgr2=0x7fe43819ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.948+0000 7fe43f9d8700 1 --2- 172.21.15.12:0/4226349296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 0x7fe43819ddc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe434004180 tx=0x7fe4340046a0 comp rx=0 tx=0).stop 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.948+0000 7fe43f9d8700 1 -- 172.21.15.12:0/4226349296 shutdown_connections 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.948+0000 7fe43f9d8700 1 --2- 172.21.15.12:0/4226349296 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fe4240390c0 0x7fe42403b580 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.948+0000 7fe43f9d8700 1 --2- 172.21.15.12:0/4226349296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe43807de10 0x7fe43819ddc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.948+0000 7fe43f9d8700 1 -- 172.21.15.12:0/4226349296 >> 172.21.15.12:0/4226349296 conn(0x7fe4381032c0 msgr2=0x7fe438103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.948+0000 7fe43f9d8700 1 -- 172.21.15.12:0/4226349296 shutdown_connections 2024-05-09T09:15:29.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:28.948+0000 7fe43f9d8700 1 -- 172.21.15.12:0/4226349296 wait complete. 2024-05-09T09:15:29.803 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:29 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:29.751+0000 7fc3f53b6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T09:15:30.201 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:29 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4226349296' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-09T09:15:30.201 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:29 smithi012 ceph-mon[30140]: mgrmap e5: x(active, since 6s) 2024-05-09T09:15:30.201 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:29 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:29.843+0000 7fc3f53b6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T09:15:30.885 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-09T09:15:30.886 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-09T09:15:30.886 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T09:15:30.886 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T09:15:30.886 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T09:15:30.886 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-09T09:15:30.886 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.995+0000 7fd81c367700 1 Processor -- start 2024-05-09T09:15:30.886 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.995+0000 7fd81c367700 1 -- start start 2024-05-09T09:15:30.886 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.995+0000 7fd81c367700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 0x7fd81410abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:30.886 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.995+0000 7fd81c367700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd81407bdb0 con 0x7fd8141087e0 2024-05-09T09:15:30.886 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.996+0000 7fd81a103700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 0x7fd81410abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:30.886 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.996+0000 7fd81a103700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 0x7fd81410abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43102/0 (socket says 172.21.15.12:43102) 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.996+0000 7fd81a103700 1 -- 172.21.15.12:0/3296116007 learned_addr learned my addr 172.21.15.12:0/3296116007 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.996+0000 7fd81a103700 1 -- 172.21.15.12:0/3296116007 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81410b110 con 0x7fd8141087e0 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.996+0000 7fd81a103700 1 --2- 172.21.15.12:0/3296116007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 0x7fd81410abd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd8100098a0 tx=0x7fd810009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10abd666c0630187 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.996+0000 7fd819101700 1 -- 172.21.15.12:0/3296116007 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd810004160 con 0x7fd8141087e0 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.996+0000 7fd819101700 1 -- 172.21.15.12:0/3296116007 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fd810004300 con 0x7fd8141087e0 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.996+0000 7fd819101700 1 -- 172.21.15.12:0/3296116007 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd810011b40 con 0x7fd8141087e0 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.997+0000 7fd81c367700 1 -- 172.21.15.12:0/3296116007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 msgr2=0x7fd81410abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.997+0000 7fd81c367700 1 --2- 172.21.15.12:0/3296116007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 0x7fd81410abd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd8100098a0 tx=0x7fd810009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.997+0000 7fd81c367700 1 -- 172.21.15.12:0/3296116007 shutdown_connections 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.997+0000 7fd81c367700 1 --2- 172.21.15.12:0/3296116007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 0x7fd81410abd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.997+0000 7fd81c367700 1 -- 172.21.15.12:0/3296116007 >> 172.21.15.12:0/3296116007 conn(0x7fd814102190 msgr2=0x7fd8141045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.997+0000 7fd81c367700 1 -- 172.21.15.12:0/3296116007 shutdown_connections 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.997+0000 7fd81c367700 1 -- 172.21.15.12:0/3296116007 wait complete. 2024-05-09T09:15:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.998+0000 7fd81c367700 1 Processor -- start 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.998+0000 7fd81c367700 1 -- start start 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.998+0000 7fd81c367700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 0x7fd81419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.998+0000 7fd81c367700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd81410c0d0 con 0x7fd8141087e0 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.998+0000 7fd81a103700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 0x7fd81419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.999+0000 7fd81a103700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 0x7fd81419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43104/0 (socket says 172.21.15.12:43104) 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.999+0000 7fd81a103700 1 -- 172.21.15.12:0/2420990528 learned_addr learned my addr 172.21.15.12:0/2420990528 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.999+0000 7fd81a103700 1 -- 172.21.15.12:0/2420990528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81419c120 con 0x7fd8141087e0 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.999+0000 7fd81a103700 1 --2- 172.21.15.12:0/2420990528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 0x7fd81419bbe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd810004180 tx=0x7fd8100046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.999+0000 7fd80b7fe700 1 -- 172.21.15.12:0/2420990528 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8100039e0 con 0x7fd8141087e0 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.999+0000 7fd80b7fe700 1 -- 172.21.15.12:0/2420990528 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fd810020070 con 0x7fd8141087e0 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.999+0000 7fd80b7fe700 1 -- 172.21.15.12:0/2420990528 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8100193c0 con 0x7fd8141087e0 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.999+0000 7fd81c367700 1 -- 172.21.15.12:0/2420990528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81419c3b0 con 0x7fd8141087e0 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:29.999+0000 7fd81c367700 1 -- 172.21.15.12:0/2420990528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81419f0a0 con 0x7fd8141087e0 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.000+0000 7fd81c367700 1 -- 172.21.15.12:0/2420990528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd814063980 con 0x7fd8141087e0 2024-05-09T09:15:30.888 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.001+0000 7fd80b7fe700 1 -- 172.21.15.12:0/2420990528 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fd81001a430 con 0x7fd8141087e0 2024-05-09T09:15:30.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.001+0000 7fd80b7fe700 1 --2- 172.21.15.12:0/2420990528 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fd800039160 0x7fd80003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:30.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.001+0000 7fd80b7fe700 1 -- 172.21.15.12:0/2420990528 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd81004d0b0 con 0x7fd8141087e0 2024-05-09T09:15:30.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.001+0000 7fd819902700 1 -- 172.21.15.12:0/2420990528 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fd800039160 msgr2=0x7fd80003b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1876427478 2024-05-09T09:15:30.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.001+0000 7fd819902700 1 --2- 172.21.15.12:0/2420990528 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fd800039160 0x7fd80003b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T09:15:30.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.006+0000 7fd80b7fe700 1 -- 172.21.15.12:0/2420990528 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd81000e260 con 0x7fd8141087e0 2024-05-09T09:15:30.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.202+0000 7fd819902700 1 -- 172.21.15.12:0/2420990528 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fd800039160 msgr2=0x7fd80003b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1876427478 2024-05-09T09:15:30.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.202+0000 7fd819902700 1 --2- 172.21.15.12:0/2420990528 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fd800039160 0x7fd80003b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T09:15:30.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.316+0000 7fd81c367700 1 -- 172.21.15.12:0/2420990528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd81419f310 con 0x7fd8141087e0 2024-05-09T09:15:30.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.317+0000 7fd80b7fe700 1 -- 172.21.15.12:0/2420990528 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fd81001a720 con 0x7fd8141087e0 2024-05-09T09:15:30.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.319+0000 7fd81c367700 1 -- 172.21.15.12:0/2420990528 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fd800039160 msgr2=0x7fd80003b620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:15:30.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.319+0000 7fd81c367700 1 --2- 172.21.15.12:0/2420990528 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fd800039160 0x7fd80003b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:30.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.319+0000 7fd81c367700 1 -- 172.21.15.12:0/2420990528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 msgr2=0x7fd81419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:30.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.319+0000 7fd81c367700 1 --2- 172.21.15.12:0/2420990528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 0x7fd81419bbe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd810004180 tx=0x7fd8100046a0 comp rx=0 tx=0).stop 2024-05-09T09:15:30.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.319+0000 7fd81c367700 1 -- 172.21.15.12:0/2420990528 shutdown_connections 2024-05-09T09:15:30.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.319+0000 7fd81c367700 1 --2- 172.21.15.12:0/2420990528 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7fd800039160 0x7fd80003b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:30.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.319+0000 7fd81c367700 1 --2- 172.21.15.12:0/2420990528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8141087e0 0x7fd81419bbe0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:30.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.319+0000 7fd81c367700 1 -- 172.21.15.12:0/2420990528 >> 172.21.15.12:0/2420990528 conn(0x7fd814102190 msgr2=0x7fd814102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:30.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.320+0000 7fd81c367700 1 -- 172.21.15.12:0/2420990528 shutdown_connections 2024-05-09T09:15:30.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:30.320+0000 7fd81c367700 1 -- 172.21.15.12:0/2420990528 wait complete. 2024-05-09T09:15:30.890 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-05-09T09:15:30.890 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 5... 2024-05-09T09:15:31.131 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:30 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2420990528' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T09:15:31.483 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:31 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:31.132+0000 7fc3f53b6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T09:15:31.484 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:31 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:31.315+0000 7fc3f53b6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T09:15:31.797 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:31 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:31.494+0000 7fc3f53b6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T09:15:31.797 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:31 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:31.797+0000 7fc3f53b6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T09:15:32.104 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:31 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:31.894+0000 7fc3f53b6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T09:15:32.104 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:32 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:32.105+0000 7fc3f53b6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T09:15:32.483 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:32 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:32.306+0000 7fc3f53b6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T09:15:33.233 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:32 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:32.941+0000 7fc3f53b6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T09:15:33.695 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:33 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:33.383+0000 7fc3f53b6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T09:15:33.695 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:33 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:33.484+0000 7fc3f53b6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T09:15:33.695 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:33 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:33.696+0000 7fc3f53b6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T09:15:34.612 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:34.260+0000 7fc3f53b6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T09:15:34.612 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:34.356+0000 7fc3f53b6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T09:15:34.868 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:34.612+0000 7fc3f53b6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T09:15:34.868 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:34.751+0000 7fc3f53b6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T09:15:34.868 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:34.868+0000 7fc3f53b6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T09:15:35.233 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:34.990+0000 7fc3f53b6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T09:15:35.734 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:35 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:35.315+0000 7fc3f53b6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T09:15:36.232 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:35 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:35.951+0000 7fc3f53b6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T09:15:36.232 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:36 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:36.180+0000 7fc3f53b6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T09:15:36.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: Active manager daemon x restarted 2024-05-09T09:15:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: Activating manager daemon x 2024-05-09T09:15:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: osdmap e2: 0 total, 0 up, 0 in 2024-05-09T09:15:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: mgrmap e6: x(active, starting, since 0.00333025s) 2024-05-09T09:15:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:15:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T09:15:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T09:15:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T09:15:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T09:15:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: Manager daemon x is now available 2024-05-09T09:15:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:15:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:36 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:15:37.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:37 smithi012 ceph-mon[30140]: Found migration_current of "None". Setting to last migration. 2024-05-09T09:15:37.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:37 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T09:15:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:37 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T09:15:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:37 smithi012 ceph-mon[30140]: mgrmap e7: x(active, since 1.00536s) 2024-05-09T09:15:37.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-09T09:15:37.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-09T09:15:37.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T09:15:37.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-09T09:15:37.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.343+0000 7f3cdc761700 1 Processor -- start 2024-05-09T09:15:37.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.343+0000 7f3cdc761700 1 -- start start 2024-05-09T09:15:37.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.344+0000 7f3cdc761700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 0x7f3cd41080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:37.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.344+0000 7f3cdc761700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd4108690 con 0x7f3cd4107ca0 2024-05-09T09:15:37.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.344+0000 7f3cda4fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 0x7f3cd41080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:37.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.344+0000 7f3cda4fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 0x7f3cd41080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43112/0 (socket says 172.21.15.12:43112) 2024-05-09T09:15:37.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.344+0000 7f3cda4fd700 1 -- 172.21.15.12:0/1879775320 learned_addr learned my addr 172.21.15.12:0/1879775320 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:37.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.344+0000 7f3cda4fd700 1 -- 172.21.15.12:0/1879775320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd4108ea0 con 0x7f3cd4107ca0 2024-05-09T09:15:37.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.344+0000 7f3cda4fd700 1 --2- 172.21.15.12:0/1879775320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 0x7f3cd41080c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00098a0 tx=0x7f3cd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec343738ff5de5e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:37.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.345+0000 7f3cd94fb700 1 -- 172.21.15.12:0/1879775320 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd0004160 con 0x7f3cd4107ca0 2024-05-09T09:15:37.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.345+0000 7f3cd94fb700 1 -- 172.21.15.12:0/1879775320 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f3cd0004300 con 0x7f3cd4107ca0 2024-05-09T09:15:37.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.345+0000 7f3cd94fb700 1 -- 172.21.15.12:0/1879775320 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd0011b40 con 0x7f3cd4107ca0 2024-05-09T09:15:37.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.345+0000 7f3cdc761700 1 -- 172.21.15.12:0/1879775320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 msgr2=0x7f3cd41080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:37.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.345+0000 7f3cdc761700 1 --2- 172.21.15.12:0/1879775320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 0x7f3cd41080c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00098a0 tx=0x7f3cd0009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:37.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.345+0000 7f3cdc761700 1 -- 172.21.15.12:0/1879775320 shutdown_connections 2024-05-09T09:15:37.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.345+0000 7f3cdc761700 1 --2- 172.21.15.12:0/1879775320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 0x7f3cd41080c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:37.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.345+0000 7f3cdc761700 1 -- 172.21.15.12:0/1879775320 >> 172.21.15.12:0/1879775320 conn(0x7f3cd41032a0 msgr2=0x7f3cd41056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:37.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.345+0000 7f3cdc761700 1 -- 172.21.15.12:0/1879775320 shutdown_connections 2024-05-09T09:15:37.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.346+0000 7f3cdc761700 1 -- 172.21.15.12:0/1879775320 wait complete. 2024-05-09T09:15:37.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.346+0000 7f3cdc761700 1 Processor -- start 2024-05-09T09:15:37.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.346+0000 7f3cdc761700 1 -- start start 2024-05-09T09:15:37.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.346+0000 7f3cdc761700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 0x7f3cd419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.346+0000 7f3cdc761700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd410c250 con 0x7f3cd4107ca0 2024-05-09T09:15:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.346+0000 7f3cda4fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 0x7f3cd419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.346+0000 7f3cda4fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 0x7f3cd419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43118/0 (socket says 172.21.15.12:43118) 2024-05-09T09:15:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.346+0000 7f3cda4fd700 1 -- 172.21.15.12:0/2123414344 learned_addr learned my addr 172.21.15.12:0/2123414344 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.347+0000 7f3cda4fd700 1 -- 172.21.15.12:0/2123414344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd419c0e0 con 0x7f3cd4107ca0 2024-05-09T09:15:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.347+0000 7f3cda4fd700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 0x7f3cd419bba0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0006af0 tx=0x7f3cd0011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.347+0000 7f3ccb7fe700 1 -- 172.21.15.12:0/2123414344 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd0003970 con 0x7f3cd4107ca0 2024-05-09T09:15:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.347+0000 7f3cdc761700 1 -- 172.21.15.12:0/2123414344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd419c370 con 0x7f3cd4107ca0 2024-05-09T09:15:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.347+0000 7f3ccb7fe700 1 -- 172.21.15.12:0/2123414344 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f3cd0003b10 con 0x7f3cd4107ca0 2024-05-09T09:15:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.347+0000 7f3cdc761700 1 -- 172.21.15.12:0/2123414344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd419f060 con 0x7f3cd4107ca0 2024-05-09T09:15:37.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.347+0000 7f3ccb7fe700 1 -- 172.21.15.12:0/2123414344 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd00196b0 con 0x7f3cd4107ca0 2024-05-09T09:15:37.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.348+0000 7f3ccb7fe700 1 -- 172.21.15.12:0/2123414344 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f3cd0019960 con 0x7f3cd4107ca0 2024-05-09T09:15:37.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.349+0000 7f3ccb7fe700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 0x7f3cc003b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:37.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.349+0000 7f3ccb7fe700 1 -- 172.21.15.12:0/2123414344 --> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3cc003bce0 con 0x7f3cc0039110 2024-05-09T09:15:37.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.349+0000 7f3ccb7fe700 1 -- 172.21.15.12:0/2123414344 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3cd004d510 con 0x7f3cd4107ca0 2024-05-09T09:15:37.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.349+0000 7f3cd9cfc700 1 -- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 msgr2=0x7f3cc003b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1876427478 2024-05-09T09:15:37.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.349+0000 7f3cd9cfc700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 0x7f3cc003b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T09:15:37.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.550+0000 7f3cd9cfc700 1 -- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 msgr2=0x7f3cc003b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1876427478 2024-05-09T09:15:37.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.550+0000 7f3cd9cfc700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 0x7f3cc003b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T09:15:37.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.951+0000 7f3cd9cfc700 1 -- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 msgr2=0x7f3cc003b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1876427478 2024-05-09T09:15:37.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:31.951+0000 7f3cd9cfc700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 0x7f3cc003b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T09:15:37.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:32.752+0000 7f3cd9cfc700 1 -- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 msgr2=0x7f3cc003b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1876427478 2024-05-09T09:15:37.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:32.752+0000 7f3cd9cfc700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 0x7f3cc003b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:34.353+0000 7f3cd9cfc700 1 -- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 msgr2=0x7f3cc003b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1876427478 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:34.353+0000 7f3cd9cfc700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 0x7f3cc003b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:36.183+0000 7f3ccb7fe700 1 -- 172.21.15.12:0/2123414344 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f3cd0019cf0 con 0x7f3cd4107ca0 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:36.184+0000 7f3ccb7fe700 1 -- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 msgr2=0x7f3cc003b5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:36.184+0000 7f3ccb7fe700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 0x7f3cc003b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.185+0000 7f3ccb7fe700 1 -- 172.21.15.12:0/2123414344 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f3cd004ee00 con 0x7f3cd4107ca0 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.185+0000 7f3ccb7fe700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f3cc003ca40 0x7f3cc003ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.185+0000 7f3ccb7fe700 1 -- 172.21.15.12:0/2123414344 --> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3cd0008ea0 con 0x7f3cc003ca40 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.186+0000 7f3cd9cfc700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f3cc003ca40 0x7f3cc003ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.186+0000 7f3cd9cfc700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f3cc003ca40 0x7f3cc003ee30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4003a10 tx=0x7f3cc40092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.187+0000 7f3ccb7fe700 1 -- 172.21.15.12:0/2123414344 <== mgr.14118 v2:172.21.15.12:6800/2168557623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f3cd0008ea0 con 0x7f3cc003ca40 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.194+0000 7f3cdc761700 1 -- 172.21.15.12:0/2123414344 --> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3cd4063980 con 0x7f3cc003ca40 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.194+0000 7f3ccb7fe700 1 -- 172.21.15.12:0/2123414344 <== mgr.14118 v2:172.21.15.12:6800/2168557623 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f3cd4063980 con 0x7f3cc003ca40 2024-05-09T09:15:37.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.194+0000 7f3cdc761700 1 -- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f3cc003ca40 msgr2=0x7f3cc003ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:37.740 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.194+0000 7f3cdc761700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f3cc003ca40 0x7f3cc003ee30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4003a10 tx=0x7f3cc40092b0 comp rx=0 tx=0).stop 2024-05-09T09:15:37.740 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.194+0000 7f3cdc761700 1 -- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 msgr2=0x7f3cd419bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:37.740 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.194+0000 7f3cdc761700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 0x7f3cd419bba0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0006af0 tx=0x7f3cd0011c00 comp rx=0 tx=0).stop 2024-05-09T09:15:37.740 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.194+0000 7f3cdc761700 1 -- 172.21.15.12:0/2123414344 shutdown_connections 2024-05-09T09:15:37.740 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.194+0000 7f3cdc761700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f3cc003ca40 0x7f3cc003ee30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:37.740 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.194+0000 7f3cdc761700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd4107ca0 0x7f3cd419bba0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:37.740 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.194+0000 7f3cdc761700 1 --2- 172.21.15.12:0/2123414344 >> [v2:172.21.15.12:6800/1876427478,v1:172.21.15.12:6801/1876427478] conn(0x7f3cc0039110 0x7f3cc003b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:37.740 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.194+0000 7f3cdc761700 1 -- 172.21.15.12:0/2123414344 >> 172.21.15.12:0/2123414344 conn(0x7f3cd41032a0 msgr2=0x7f3cd4103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:37.740 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.194+0000 7f3cdc761700 1 -- 172.21.15.12:0/2123414344 shutdown_connections 2024-05-09T09:15:37.740 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:37.194+0000 7f3cdc761700 1 -- 172.21.15.12:0/2123414344 wait complete. 2024-05-09T09:15:37.740 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 5 is available 2024-05-09T09:15:37.740 INFO:teuthology.orchestra.run.smithi012.stdout:Setting orchestrator backend to cephadm... 2024-05-09T09:15:38.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:38 smithi012 ceph-mon[30140]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T09:15:38.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:38 smithi012 ceph-mon[30140]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T09:15:38.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:38 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:38.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:38 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:38.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:38 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:38.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:38 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:15:39.264 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.380+0000 7f4fd2275700 1 Processor -- start 2024-05-09T09:15:39.264 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.380+0000 7f4fd2275700 1 -- start start 2024-05-09T09:15:39.264 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.381+0000 7f4fd2275700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 0x7f4fcc106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:39.264 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.381+0000 7f4fd2275700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1075c0 con 0x7f4fcc106bd0 2024-05-09T09:15:39.264 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.381+0000 7f4fcbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 0x7f4fcc106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:39.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.381+0000 7f4fcbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 0x7f4fcc106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46516/0 (socket says 172.21.15.12:46516) 2024-05-09T09:15:39.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.381+0000 7f4fcbfff700 1 -- 172.21.15.12:0/1954132654 learned_addr learned my addr 172.21.15.12:0/1954132654 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:39.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.381+0000 7f4fcbfff700 1 -- 172.21.15.12:0/1954132654 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc107dd0 con 0x7f4fcc106bd0 2024-05-09T09:15:39.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.381+0000 7f4fcbfff700 1 --2- 172.21.15.12:0/1954132654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 0x7f4fcc106ff0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0098a0 tx=0x7f4fbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3611b4cab0a2de71 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:39.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.382+0000 7f4fcaffd700 1 -- 172.21.15.12:0/1954132654 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fbc004160 con 0x7f4fcc106bd0 2024-05-09T09:15:39.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.382+0000 7f4fcaffd700 1 -- 172.21.15.12:0/1954132654 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f4fbc004300 con 0x7f4fcc106bd0 2024-05-09T09:15:39.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.382+0000 7f4fcaffd700 1 -- 172.21.15.12:0/1954132654 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fbc011b40 con 0x7f4fcc106bd0 2024-05-09T09:15:39.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.382+0000 7f4fd2275700 1 -- 172.21.15.12:0/1954132654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 msgr2=0x7f4fcc106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:39.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.382+0000 7f4fd2275700 1 --2- 172.21.15.12:0/1954132654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 0x7f4fcc106ff0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0098a0 tx=0x7f4fbc009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:39.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.382+0000 7f4fd2275700 1 -- 172.21.15.12:0/1954132654 shutdown_connections 2024-05-09T09:15:39.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.382+0000 7f4fd2275700 1 --2- 172.21.15.12:0/1954132654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 0x7f4fcc106ff0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:39.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.382+0000 7f4fd2275700 1 -- 172.21.15.12:0/1954132654 >> 172.21.15.12:0/1954132654 conn(0x7f4fcc102170 msgr2=0x7f4fcc1045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:39.266 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.382+0000 7f4fd2275700 1 -- 172.21.15.12:0/1954132654 shutdown_connections 2024-05-09T09:15:39.266 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.382+0000 7f4fd2275700 1 -- 172.21.15.12:0/1954132654 wait complete. 2024-05-09T09:15:39.266 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.383+0000 7f4fd2275700 1 Processor -- start 2024-05-09T09:15:39.266 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.383+0000 7f4fd2275700 1 -- start start 2024-05-09T09:15:39.266 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.383+0000 7f4fd2275700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 0x7f4fcc19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:39.266 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.383+0000 7f4fd2275700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc10b180 con 0x7f4fcc106bd0 2024-05-09T09:15:39.266 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.384+0000 7f4fcbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 0x7f4fcc19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:39.266 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.384+0000 7f4fcbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 0x7f4fcc19bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46526/0 (socket says 172.21.15.12:46526) 2024-05-09T09:15:39.266 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.384+0000 7f4fcbfff700 1 -- 172.21.15.12:0/699750766 learned_addr learned my addr 172.21.15.12:0/699750766 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.384+0000 7f4fcbfff700 1 -- 172.21.15.12:0/699750766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc19c1a0 con 0x7f4fcc106bd0 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.384+0000 7f4fcbfff700 1 --2- 172.21.15.12:0/699750766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 0x7f4fcc19bc60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc011670 tx=0x7f4fbc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.384+0000 7f4fc97fa700 1 -- 172.21.15.12:0/699750766 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fbc0038b0 con 0x7f4fcc106bd0 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.384+0000 7f4fc97fa700 1 -- 172.21.15.12:0/699750766 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f4fbc01a430 con 0x7f4fcc106bd0 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.384+0000 7f4fd2275700 1 -- 172.21.15.12:0/699750766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fcc19c430 con 0x7f4fcc106bd0 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.384+0000 7f4fc97fa700 1 -- 172.21.15.12:0/699750766 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fbc019550 con 0x7f4fcc106bd0 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.384+0000 7f4fd2275700 1 -- 172.21.15.12:0/699750766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fcc19f120 con 0x7f4fcc106bd0 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.385+0000 7f4fd2275700 1 -- 172.21.15.12:0/699750766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fcc063980 con 0x7f4fcc106bd0 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.386+0000 7f4fc97fa700 1 -- 172.21.15.12:0/699750766 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f4fbc01aa50 con 0x7f4fcc106bd0 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.386+0000 7f4fc97fa700 1 --2- 172.21.15.12:0/699750766 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f4fb4039000 0x7f4fb403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.386+0000 7f4fc97fa700 1 -- 172.21.15.12:0/699750766 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4fbc04d910 con 0x7f4fcc106bd0 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.386+0000 7f4fcb7fe700 1 --2- 172.21.15.12:0/699750766 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f4fb4039000 0x7f4fb403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.387+0000 7f4fcb7fe700 1 --2- 172.21.15.12:0/699750766 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f4fb4039000 0x7f4fb403b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006fd0 tx=0x7f4fc0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:39.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.391+0000 7f4fc97fa700 1 -- 172.21.15.12:0/699750766 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4fbc016260 con 0x7f4fcc106bd0 2024-05-09T09:15:39.268 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.613+0000 7f4fd2275700 1 -- 172.21.15.12:0/699750766 --> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f4fcc02def0 con 0x7f4fb4039000 2024-05-09T09:15:39.268 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.618+0000 7f4fc97fa700 1 -- 172.21.15.12:0/699750766 <== mgr.14118 v2:172.21.15.12:6800/2168557623 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4fcc02def0 con 0x7f4fb4039000 2024-05-09T09:15:39.268 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.621+0000 7f4fd2275700 1 -- 172.21.15.12:0/699750766 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f4fb4039000 msgr2=0x7f4fb403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:39.268 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.621+0000 7f4fd2275700 1 --2- 172.21.15.12:0/699750766 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f4fb4039000 0x7f4fb403b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006fd0 tx=0x7f4fc0006e40 comp rx=0 tx=0).stop 2024-05-09T09:15:39.268 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.621+0000 7f4fd2275700 1 -- 172.21.15.12:0/699750766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 msgr2=0x7f4fcc19bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:39.268 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.621+0000 7f4fd2275700 1 --2- 172.21.15.12:0/699750766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 0x7f4fcc19bc60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc011670 tx=0x7f4fbc011b50 comp rx=0 tx=0).stop 2024-05-09T09:15:39.268 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.621+0000 7f4fd2275700 1 -- 172.21.15.12:0/699750766 shutdown_connections 2024-05-09T09:15:39.268 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.621+0000 7f4fd2275700 1 --2- 172.21.15.12:0/699750766 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f4fb4039000 0x7f4fb403b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:39.268 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.621+0000 7f4fd2275700 1 --2- 172.21.15.12:0/699750766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fcc106bd0 0x7f4fcc19bc60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:39.268 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.621+0000 7f4fd2275700 1 -- 172.21.15.12:0/699750766 >> 172.21.15.12:0/699750766 conn(0x7f4fcc102170 msgr2=0x7f4fcc102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:39.268 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.621+0000 7f4fd2275700 1 -- 172.21.15.12:0/699750766 shutdown_connections 2024-05-09T09:15:39.268 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:38.621+0000 7f4fd2275700 1 -- 172.21.15.12:0/699750766 wait complete. 2024-05-09T09:15:39.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:39 smithi012 ceph-mon[30140]: [09/May/2024:09:15:38] ENGINE Bus STARTING 2024-05-09T09:15:39.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:39 smithi012 ceph-mon[30140]: [09/May/2024:09:15:38] ENGINE Serving on https://172.21.15.12:7150 2024-05-09T09:15:39.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:39 smithi012 ceph-mon[30140]: [09/May/2024:09:15:38] ENGINE Serving on http://172.21.15.12:8765 2024-05-09T09:15:39.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:39 smithi012 ceph-mon[30140]: [09/May/2024:09:15:38] ENGINE Bus STARTED 2024-05-09T09:15:39.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:39 smithi012 ceph-mon[30140]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:15:39.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:39 smithi012 ceph-mon[30140]: mgrmap e8: x(active, since 2s) 2024-05-09T09:15:40.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-09T09:15:40.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.698+0000 7f9bfb1b5700 1 Processor -- start 2024-05-09T09:15:40.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.698+0000 7f9bfb1b5700 1 -- start start 2024-05-09T09:15:40.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.698+0000 7f9bfb1b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 0x7f9bf4108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:40.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.698+0000 7f9bfb1b5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf41086f0 con 0x7f9bf4107d00 2024-05-09T09:15:40.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.698+0000 7f9bf8f51700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 0x7f9bf4108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:40.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.698+0000 7f9bf8f51700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 0x7f9bf4108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46528/0 (socket says 172.21.15.12:46528) 2024-05-09T09:15:40.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.698+0000 7f9bf8f51700 1 -- 172.21.15.12:0/2976498815 learned_addr learned my addr 172.21.15.12:0/2976498815 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:40.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.699+0000 7f9bf8f51700 1 -- 172.21.15.12:0/2976498815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf4108eb0 con 0x7f9bf4107d00 2024-05-09T09:15:40.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.699+0000 7f9bf8f51700 1 --2- 172.21.15.12:0/2976498815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 0x7f9bf4108120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9be80098a0 tx=0x7f9be8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2686b8a893704506 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:40.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.699+0000 7f9bf37fe700 1 -- 172.21.15.12:0/2976498815 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9be8004160 con 0x7f9bf4107d00 2024-05-09T09:15:40.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.699+0000 7f9bf37fe700 1 -- 172.21.15.12:0/2976498815 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f9be8004300 con 0x7f9bf4107d00 2024-05-09T09:15:40.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.699+0000 7f9bf37fe700 1 -- 172.21.15.12:0/2976498815 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9be8011b40 con 0x7f9bf4107d00 2024-05-09T09:15:40.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.700+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/2976498815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 msgr2=0x7f9bf4108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:40.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.700+0000 7f9bfb1b5700 1 --2- 172.21.15.12:0/2976498815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 0x7f9bf4108120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9be80098a0 tx=0x7f9be8009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:40.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.700+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/2976498815 shutdown_connections 2024-05-09T09:15:40.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.700+0000 7f9bfb1b5700 1 --2- 172.21.15.12:0/2976498815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 0x7f9bf4108120 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:40.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.700+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/2976498815 >> 172.21.15.12:0/2976498815 conn(0x7f9bf4103300 msgr2=0x7f9bf4105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.700+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/2976498815 shutdown_connections 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.700+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/2976498815 wait complete. 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.700+0000 7f9bfb1b5700 1 Processor -- start 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.700+0000 7f9bfb1b5700 1 -- start start 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.700+0000 7f9bfb1b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 0x7f9bf407daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.700+0000 7f9bfb1b5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf410c260 con 0x7f9bf4107d00 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.701+0000 7f9bf8f51700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 0x7f9bf407daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.701+0000 7f9bf8f51700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 0x7f9bf407daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46536/0 (socket says 172.21.15.12:46536) 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.701+0000 7f9bf8f51700 1 -- 172.21.15.12:0/1786647007 learned_addr learned my addr 172.21.15.12:0/1786647007 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.701+0000 7f9bf8f51700 1 -- 172.21.15.12:0/1786647007 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf407e030 con 0x7f9bf4107d00 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.701+0000 7f9bf8f51700 1 --2- 172.21.15.12:0/1786647007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 0x7f9bf407daf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006af0 tx=0x7f9be8011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.701+0000 7f9bf1ffb700 1 -- 172.21.15.12:0/1786647007 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9be8003970 con 0x7f9bf4107d00 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.701+0000 7f9bf1ffb700 1 -- 172.21.15.12:0/1786647007 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f9be8003b10 con 0x7f9bf4107d00 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.701+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/1786647007 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf407c220 con 0x7f9bf4107d00 2024-05-09T09:15:40.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.701+0000 7f9bf1ffb700 1 -- 172.21.15.12:0/1786647007 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9be80196b0 con 0x7f9bf4107d00 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.701+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/1786647007 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf407c760 con 0x7f9bf4107d00 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.702+0000 7f9bf1ffb700 1 -- 172.21.15.12:0/1786647007 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9be801a430 con 0x7f9bf4107d00 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.702+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/1786647007 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf4063980 con 0x7f9bf4107d00 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.703+0000 7f9bf1ffb700 1 --2- 172.21.15.12:0/1786647007 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f9be4039110 0x7f9be403b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.703+0000 7f9bf1ffb700 1 -- 172.21.15.12:0/1786647007 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9be804d3a0 con 0x7f9bf4107d00 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.703+0000 7f9bf3fff700 1 --2- 172.21.15.12:0/1786647007 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f9be4039110 0x7f9be403b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.703+0000 7f9bf3fff700 1 --2- 172.21.15.12:0/1786647007 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f9be4039110 0x7f9be403b5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9be0006fd0 tx=0x7f9be0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.708+0000 7f9bf1ffb700 1 -- 172.21.15.12:0/1786647007 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9be8004300 con 0x7f9bf4107d00 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.916+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/1786647007 --> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9bf407cfb0 con 0x7f9be4039110 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.917+0000 7f9bf1ffb700 1 -- 172.21.15.12:0/1786647007 <== mgr.14118 v2:172.21.15.12:6800/2168557623 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9bf407cfb0 con 0x7f9be4039110 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.920+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/1786647007 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f9be4039110 msgr2=0x7f9be403b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.920+0000 7f9bfb1b5700 1 --2- 172.21.15.12:0/1786647007 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f9be4039110 0x7f9be403b5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9be0006fd0 tx=0x7f9be0006e40 comp rx=0 tx=0).stop 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.920+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/1786647007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 msgr2=0x7f9bf407daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:40.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.920+0000 7f9bfb1b5700 1 --2- 172.21.15.12:0/1786647007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 0x7f9bf407daf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006af0 tx=0x7f9be8011c00 comp rx=0 tx=0).stop 2024-05-09T09:15:40.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.920+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/1786647007 shutdown_connections 2024-05-09T09:15:40.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.920+0000 7f9bfb1b5700 1 --2- 172.21.15.12:0/1786647007 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f9be4039110 0x7f9be403b5d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:40.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.920+0000 7f9bfb1b5700 1 --2- 172.21.15.12:0/1786647007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf4107d00 0x7f9bf407daf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:40.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.920+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/1786647007 >> 172.21.15.12:0/1786647007 conn(0x7f9bf4103300 msgr2=0x7f9bf4103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:40.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.921+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/1786647007 shutdown_connections 2024-05-09T09:15:40.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:39.921+0000 7f9bfb1b5700 1 -- 172.21.15.12:0/1786647007 wait complete. 2024-05-09T09:15:40.473 INFO:teuthology.orchestra.run.smithi012.stdout:Generating ssh key... 2024-05-09T09:15:40.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:40 smithi012 ceph-mon[30140]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:15:41.942 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: Generating public/private rsa key pair. 2024-05-09T09:15:41.942 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: Your identification has been saved in /tmp/tmpez14w9d2/key. 2024-05-09T09:15:41.942 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: Your public key has been saved in /tmp/tmpez14w9d2/key.pub. 2024-05-09T09:15:41.942 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: The key fingerprint is: 2024-05-09T09:15:41.942 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: SHA256:+Labb9LfN9Ul7Ef+MzDuV3uICp3hm+K5dD2RAQ4/2Z0 ceph-770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:15:41.942 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: The key's randomart image is: 2024-05-09T09:15:41.943 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: +---[RSA 3072]----+ 2024-05-09T09:15:41.943 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: | . . | 2024-05-09T09:15:41.943 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: | + + . . | 2024-05-09T09:15:41.943 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: | = o.E | 2024-05-09T09:15:41.943 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: | . . oo o| 2024-05-09T09:15:41.943 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: | . S . o. +o| 2024-05-09T09:15:41.943 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: | . o + +. *| 2024-05-09T09:15:41.943 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: | =.= +.oo=| 2024-05-09T09:15:41.943 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: | oo*oo.+.*+| 2024-05-09T09:15:41.943 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: | .BB=oo.o.=| 2024-05-09T09:15:41.943 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: +----[SHA256]-----+ 2024-05-09T09:15:42.233 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.902+0000 7f52095af700 1 Processor -- start 2024-05-09T09:15:42.233 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.902+0000 7f52095af700 1 -- start start 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.902+0000 7f52095af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 0x7f5204108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.902+0000 7f52095af700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f52041086f0 con 0x7f5204107d00 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.902+0000 7f5202ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 0x7f5204108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.902+0000 7f5202ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 0x7f5204108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46550/0 (socket says 172.21.15.12:46550) 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.902+0000 7f5202ffd700 1 -- 172.21.15.12:0/163242957 learned_addr learned my addr 172.21.15.12:0/163242957 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.902+0000 7f5202ffd700 1 -- 172.21.15.12:0/163242957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5204108eb0 con 0x7f5204107d00 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.902+0000 7f5202ffd700 1 --2- 172.21.15.12:0/163242957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 0x7f5204108120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f51f40098a0 tx=0x7f51f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30756d801936ddc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.903+0000 7f5201ffb700 1 -- 172.21.15.12:0/163242957 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51f4004160 con 0x7f5204107d00 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.903+0000 7f5201ffb700 1 -- 172.21.15.12:0/163242957 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f51f4004300 con 0x7f5204107d00 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.903+0000 7f5201ffb700 1 -- 172.21.15.12:0/163242957 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51f4011b40 con 0x7f5204107d00 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.903+0000 7f52095af700 1 -- 172.21.15.12:0/163242957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 msgr2=0x7f5204108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.903+0000 7f52095af700 1 --2- 172.21.15.12:0/163242957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 0x7f5204108120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f51f40098a0 tx=0x7f51f4009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.903+0000 7f52095af700 1 -- 172.21.15.12:0/163242957 shutdown_connections 2024-05-09T09:15:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.903+0000 7f52095af700 1 --2- 172.21.15.12:0/163242957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 0x7f5204108120 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.903+0000 7f52095af700 1 -- 172.21.15.12:0/163242957 >> 172.21.15.12:0/163242957 conn(0x7f52041032c0 msgr2=0x7f52041056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.903+0000 7f52095af700 1 -- 172.21.15.12:0/163242957 shutdown_connections 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.903+0000 7f52095af700 1 -- 172.21.15.12:0/163242957 wait complete. 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.904+0000 7f52095af700 1 Processor -- start 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.904+0000 7f52095af700 1 -- start start 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.904+0000 7f52095af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 0x7f520419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.904+0000 7f52095af700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f520410c260 con 0x7f5204107d00 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.905+0000 7f5202ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 0x7f520419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.905+0000 7f5202ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 0x7f520419bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46556/0 (socket says 172.21.15.12:46556) 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.905+0000 7f5202ffd700 1 -- 172.21.15.12:0/782649633 learned_addr learned my addr 172.21.15.12:0/782649633 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.905+0000 7f5202ffd700 1 -- 172.21.15.12:0/782649633 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520419c150 con 0x7f5204107d00 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.905+0000 7f5202ffd700 1 --2- 172.21.15.12:0/782649633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 0x7f520419bc10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f51f4011840 tx=0x7f51f4004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.905+0000 7f51fbfff700 1 -- 172.21.15.12:0/782649633 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51f40038b0 con 0x7f5204107d00 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.905+0000 7f52095af700 1 -- 172.21.15.12:0/782649633 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520419c3e0 con 0x7f5204107d00 2024-05-09T09:15:42.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.905+0000 7f51fbfff700 1 -- 172.21.15.12:0/782649633 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f51f401a430 con 0x7f5204107d00 2024-05-09T09:15:42.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.905+0000 7f51fbfff700 1 -- 172.21.15.12:0/782649633 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51f4024650 con 0x7f5204107d00 2024-05-09T09:15:42.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.905+0000 7f52095af700 1 -- 172.21.15.12:0/782649633 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520419f0d0 con 0x7f5204107d00 2024-05-09T09:15:42.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.906+0000 7f51fbfff700 1 -- 172.21.15.12:0/782649633 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f51f4022020 con 0x7f5204107d00 2024-05-09T09:15:42.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.906+0000 7f51fbfff700 1 --2- 172.21.15.12:0/782649633 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f51f0039110 0x7f51f003b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:42.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.906+0000 7f51fbfff700 1 -- 172.21.15.12:0/782649633 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f51f404cf20 con 0x7f5204107d00 2024-05-09T09:15:42.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.906+0000 7f52095af700 1 -- 172.21.15.12:0/782649633 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5204063980 con 0x7f5204107d00 2024-05-09T09:15:42.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.907+0000 7f52027fc700 1 --2- 172.21.15.12:0/782649633 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f51f0039110 0x7f51f003b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:42.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.907+0000 7f52027fc700 1 --2- 172.21.15.12:0/782649633 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f51f0039110 0x7f51f003b5d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f51ec006fd0 tx=0x7f51ec006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:42.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:40.912+0000 7f51fbfff700 1 -- 172.21.15.12:0/782649633 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f51f402c7a0 con 0x7f5204107d00 2024-05-09T09:15:42.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:41.118+0000 7f52095af700 1 -- 172.21.15.12:0/782649633 --> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f5204104420 con 0x7f51f0039110 2024-05-09T09:15:42.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:41.642+0000 7f51fbfff700 1 -- 172.21.15.12:0/782649633 <== mgr.14118 v2:172.21.15.12:6800/2168557623 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5204104420 con 0x7f51f0039110 2024-05-09T09:15:42.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:41.644+0000 7f52095af700 1 -- 172.21.15.12:0/782649633 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f51f0039110 msgr2=0x7f51f003b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:42.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:41.644+0000 7f52095af700 1 --2- 172.21.15.12:0/782649633 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f51f0039110 0x7f51f003b5d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f51ec006fd0 tx=0x7f51ec006e40 comp rx=0 tx=0).stop 2024-05-09T09:15:42.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:41.644+0000 7f52095af700 1 -- 172.21.15.12:0/782649633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 msgr2=0x7f520419bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:42.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:41.644+0000 7f52095af700 1 --2- 172.21.15.12:0/782649633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 0x7f520419bc10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f51f4011840 tx=0x7f51f4004230 comp rx=0 tx=0).stop 2024-05-09T09:15:42.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:41.644+0000 7f52095af700 1 -- 172.21.15.12:0/782649633 shutdown_connections 2024-05-09T09:15:42.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:41.644+0000 7f52095af700 1 --2- 172.21.15.12:0/782649633 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f51f0039110 0x7f51f003b5d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:42.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:41.644+0000 7f52095af700 1 --2- 172.21.15.12:0/782649633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5204107d00 0x7f520419bc10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:42.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:41.644+0000 7f52095af700 1 -- 172.21.15.12:0/782649633 >> 172.21.15.12:0/782649633 conn(0x7f52041032c0 msgr2=0x7f5204103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:42.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:41.644+0000 7f52095af700 1 -- 172.21.15.12:0/782649633 shutdown_connections 2024-05-09T09:15:42.239 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:41.644+0000 7f52095af700 1 -- 172.21.15.12:0/782649633 wait complete. 2024-05-09T09:15:42.889 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:42 smithi012 ceph-mon[30140]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:15:42.889 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:42 smithi012 ceph-mon[30140]: Generating ssh key... 2024-05-09T09:15:42.889 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:42 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:42.889 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:42 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:43.465 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEreAC0MgbXwawxTqBnsofiiZrXHdPozHxaWgW43L6LSDRtHBS8XzZ61w9nvQh3ngjE21rdFKLj8L22AE3qaB+SlxSZQRramPs2cE4MeGUC9VFEr5Ff+yCR8NZsHe6mxJ2nEi+ufYOkOH/q5eSOIIqw4ahRk8lJwnKSMYnyIZ+dW7jvOsSK6qb1VB3VvNEBzp5tyCnfX9fRnQrF3N0lxpN0Ixw5wSLshKWnwFDm2iZv4VleQZzMdhzw0MYknLAEzapPfmkqJ8TGAcKmgXsGcJx2Bl4GA3FUBLLLCSMaaMNwI68iPoidcDh22Z9p+baaUQUvkO1p4kRWVE7XDQ8e0iUL+1PvIxJfMRO25Ofs2b1tuTQpLlMuIn+8no1oJF4uE7gKr2fpooWke5zjnDB5QTrqZ3sS9prrSGVWRgcYa3n8bguMDklRlyP3UTfYUuxk7Fw18TYQ5g3FLPTB74c2Xa78VdvSv5w9JYin5SO9M+OEwr1T7LeMKPDxOlQrh4s7Pc= ceph-770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:15:43.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.669+0000 7fa075210700 1 Processor -- start 2024-05-09T09:15:43.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.669+0000 7fa075210700 1 -- start start 2024-05-09T09:15:43.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.669+0000 7fa075210700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 0x7fa07010ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:43.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.669+0000 7fa075210700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa07007bdf0 con 0x7fa070108780 2024-05-09T09:15:43.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.670+0000 7fa06ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 0x7fa07010ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:43.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.670+0000 7fa06ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 0x7fa07010ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34766/0 (socket says 172.21.15.12:34766) 2024-05-09T09:15:43.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.670+0000 7fa06ed9d700 1 -- 172.21.15.12:0/3151561924 learned_addr learned my addr 172.21.15.12:0/3151561924 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:43.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.670+0000 7fa06ed9d700 1 -- 172.21.15.12:0/3151561924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07010b0b0 con 0x7fa070108780 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.670+0000 7fa06ed9d700 1 --2- 172.21.15.12:0/3151561924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 0x7fa07010ab70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa0600098a0 tx=0x7fa060009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7085d389021945b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.670+0000 7fa06dd9b700 1 -- 172.21.15.12:0/3151561924 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa060004160 con 0x7fa070108780 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.670+0000 7fa06dd9b700 1 -- 172.21.15.12:0/3151561924 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fa060004300 con 0x7fa070108780 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.670+0000 7fa06dd9b700 1 -- 172.21.15.12:0/3151561924 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa060011b40 con 0x7fa070108780 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.671+0000 7fa075210700 1 -- 172.21.15.12:0/3151561924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 msgr2=0x7fa07010ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.671+0000 7fa075210700 1 --2- 172.21.15.12:0/3151561924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 0x7fa07010ab70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa0600098a0 tx=0x7fa060009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.671+0000 7fa075210700 1 -- 172.21.15.12:0/3151561924 shutdown_connections 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.671+0000 7fa075210700 1 --2- 172.21.15.12:0/3151561924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 0x7fa07010ab70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.671+0000 7fa075210700 1 -- 172.21.15.12:0/3151561924 >> 172.21.15.12:0/3151561924 conn(0x7fa070102170 msgr2=0x7fa0701045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.671+0000 7fa075210700 1 -- 172.21.15.12:0/3151561924 shutdown_connections 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.671+0000 7fa075210700 1 -- 172.21.15.12:0/3151561924 wait complete. 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.672+0000 7fa075210700 1 Processor -- start 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.672+0000 7fa075210700 1 -- start start 2024-05-09T09:15:43.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.672+0000 7fa075210700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 0x7fa070108000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.672+0000 7fa075210700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa07010c0d0 con 0x7fa070108780 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.672+0000 7fa06ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 0x7fa070108000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.672+0000 7fa06ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 0x7fa070108000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34768/0 (socket says 172.21.15.12:34768) 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.672+0000 7fa06ed9d700 1 -- 172.21.15.12:0/3265727593 learned_addr learned my addr 172.21.15.12:0/3265727593 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.673+0000 7fa06ed9d700 1 -- 172.21.15.12:0/3265727593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa070108540 con 0x7fa070108780 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.673+0000 7fa06ed9d700 1 --2- 172.21.15.12:0/3265727593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 0x7fa070108000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa060011670 tx=0x7fa060011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.673+0000 7fa057fff700 1 -- 172.21.15.12:0/3265727593 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0600038b0 con 0x7fa070108780 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.673+0000 7fa057fff700 1 -- 172.21.15.12:0/3265727593 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fa06001a430 con 0x7fa070108780 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.673+0000 7fa075210700 1 -- 172.21.15.12:0/3265727593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa070106730 con 0x7fa070108780 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.673+0000 7fa057fff700 1 -- 172.21.15.12:0/3265727593 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa060019550 con 0x7fa070108780 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.673+0000 7fa075210700 1 -- 172.21.15.12:0/3265727593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa070106c70 con 0x7fa070108780 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.674+0000 7fa057fff700 1 -- 172.21.15.12:0/3265727593 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa060019850 con 0x7fa070108780 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.674+0000 7fa075210700 1 -- 172.21.15.12:0/3265727593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa070063980 con 0x7fa070108780 2024-05-09T09:15:43.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.674+0000 7fa057fff700 1 --2- 172.21.15.12:0/3265727593 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fa0580390c0 0x7fa05803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.674+0000 7fa057fff700 1 -- 172.21.15.12:0/3265727593 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa06004d500 con 0x7fa070108780 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.674+0000 7fa06e59c700 1 --2- 172.21.15.12:0/3265727593 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fa0580390c0 0x7fa05803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.675+0000 7fa06e59c700 1 --2- 172.21.15.12:0/3265727593 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fa0580390c0 0x7fa05803b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa064006fd0 tx=0x7fa064006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.679+0000 7fa057fff700 1 -- 172.21.15.12:0/3265727593 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa060019c40 con 0x7fa070108780 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.889+0000 7fa075210700 1 -- 172.21.15.12:0/3265727593 --> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fa070103340 con 0x7fa0580390c0 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.890+0000 7fa057fff700 1 -- 172.21.15.12:0/3265727593 <== mgr.14118 v2:172.21.15.12:6800/2168557623 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fa070103340 con 0x7fa0580390c0 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.892+0000 7fa075210700 1 -- 172.21.15.12:0/3265727593 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fa0580390c0 msgr2=0x7fa05803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.892+0000 7fa075210700 1 --2- 172.21.15.12:0/3265727593 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fa0580390c0 0x7fa05803b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa064006fd0 tx=0x7fa064006e40 comp rx=0 tx=0).stop 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.892+0000 7fa075210700 1 -- 172.21.15.12:0/3265727593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 msgr2=0x7fa070108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.892+0000 7fa075210700 1 --2- 172.21.15.12:0/3265727593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 0x7fa070108000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa060011670 tx=0x7fa060011b50 comp rx=0 tx=0).stop 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.892+0000 7fa075210700 1 -- 172.21.15.12:0/3265727593 shutdown_connections 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.892+0000 7fa075210700 1 --2- 172.21.15.12:0/3265727593 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fa0580390c0 0x7fa05803b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.892+0000 7fa075210700 1 --2- 172.21.15.12:0/3265727593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa070108780 0x7fa070108000 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.892+0000 7fa075210700 1 -- 172.21.15.12:0/3265727593 >> 172.21.15.12:0/3265727593 conn(0x7fa070102170 msgr2=0x7fa070102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:43.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.892+0000 7fa075210700 1 -- 172.21.15.12:0/3265727593 shutdown_connections 2024-05-09T09:15:43.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:42.892+0000 7fa075210700 1 -- 172.21.15.12:0/3265727593 wait complete. 2024-05-09T09:15:43.470 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-09T09:15:43.470 INFO:teuthology.orchestra.run.smithi012.stdout:Adding key to root@localhost authorized_keys... 2024-05-09T09:15:43.470 INFO:teuthology.orchestra.run.smithi012.stdout:Adding host smithi012... 2024-05-09T09:15:43.733 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:43 smithi012 ceph-mon[30140]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:15:45.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:45 smithi012 ceph-mon[30140]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:15:45.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:45 smithi012 ceph-mon[30140]: Deploying cephadm binary to smithi012 2024-05-09T09:15:48.981 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Added host 'smithi012' with addr '172.21.15.12' 2024-05-09T09:15:48.981 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.967+0000 7f4072277700 1 Processor -- start 2024-05-09T09:15:48.981 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.967+0000 7f4072277700 1 -- start start 2024-05-09T09:15:48.981 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.967+0000 7f4072277700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 0x7f406c07c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:48.981 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.967+0000 7f4072277700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f406c07c830 con 0x7f406c07de00 2024-05-09T09:15:48.981 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.967+0000 7f406bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 0x7f406c07c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:48.981 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.967+0000 7f406bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 0x7f406c07c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34780/0 (socket says 172.21.15.12:34780) 2024-05-09T09:15:48.981 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.967+0000 7f406bfff700 1 -- 172.21.15.12:0/1669466294 learned_addr learned my addr 172.21.15.12:0/1669466294 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:48.981 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.968+0000 7f406bfff700 1 -- 172.21.15.12:0/1669466294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c07c9b0 con 0x7f406c07de00 2024-05-09T09:15:48.981 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.968+0000 7f406bfff700 1 --2- 172.21.15.12:0/1669466294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 0x7f406c07c260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f405c0098a0 tx=0x7f405c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a72ee2f95305b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:48.981 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.968+0000 7f406affd700 1 -- 172.21.15.12:0/1669466294 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f405c004160 con 0x7f406c07de00 2024-05-09T09:15:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.968+0000 7f406affd700 1 -- 172.21.15.12:0/1669466294 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f405c004300 con 0x7f406c07de00 2024-05-09T09:15:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.968+0000 7f406affd700 1 -- 172.21.15.12:0/1669466294 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f405c011b40 con 0x7f406c07de00 2024-05-09T09:15:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.969+0000 7f4072277700 1 -- 172.21.15.12:0/1669466294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 msgr2=0x7f406c07c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.969+0000 7f4072277700 1 --2- 172.21.15.12:0/1669466294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 0x7f406c07c260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f405c0098a0 tx=0x7f405c009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.969+0000 7f4072277700 1 -- 172.21.15.12:0/1669466294 shutdown_connections 2024-05-09T09:15:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.969+0000 7f4072277700 1 --2- 172.21.15.12:0/1669466294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 0x7f406c07c260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.969+0000 7f4072277700 1 -- 172.21.15.12:0/1669466294 >> 172.21.15.12:0/1669466294 conn(0x7f406c1032f0 msgr2=0x7f406c105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.969+0000 7f4072277700 1 -- 172.21.15.12:0/1669466294 shutdown_connections 2024-05-09T09:15:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.969+0000 7f4072277700 1 -- 172.21.15.12:0/1669466294 wait complete. 2024-05-09T09:15:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.969+0000 7f4072277700 1 Processor -- start 2024-05-09T09:15:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.970+0000 7f4072277700 1 -- start start 2024-05-09T09:15:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.970+0000 7f4072277700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 0x7f406c19ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.970+0000 7f4072277700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f406c10e300 con 0x7f406c07de00 2024-05-09T09:15:48.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.970+0000 7f406bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 0x7f406c19ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:48.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.970+0000 7f406bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 0x7f406c19ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34790/0 (socket says 172.21.15.12:34790) 2024-05-09T09:15:48.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.970+0000 7f406bfff700 1 -- 172.21.15.12:0/3516586586 learned_addr learned my addr 172.21.15.12:0/3516586586 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:48.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.970+0000 7f406bfff700 1 -- 172.21.15.12:0/3516586586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c19e2f0 con 0x7f406c07de00 2024-05-09T09:15:48.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.971+0000 7f406bfff700 1 --2- 172.21.15.12:0/3516586586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 0x7f406c19ddb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f405c004180 tx=0x7f405c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:48.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.971+0000 7f40697fa700 1 -- 172.21.15.12:0/3516586586 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f405c0039e0 con 0x7f406c07de00 2024-05-09T09:15:48.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.971+0000 7f40697fa700 1 -- 172.21.15.12:0/3516586586 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f405c020070 con 0x7f406c07de00 2024-05-09T09:15:48.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.971+0000 7f4072277700 1 -- 172.21.15.12:0/3516586586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406c19e580 con 0x7f406c07de00 2024-05-09T09:15:48.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.971+0000 7f40697fa700 1 -- 172.21.15.12:0/3516586586 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f405c0193c0 con 0x7f406c07de00 2024-05-09T09:15:48.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.971+0000 7f4072277700 1 -- 172.21.15.12:0/3516586586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406c1a1230 con 0x7f406c07de00 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.972+0000 7f40697fa700 1 -- 172.21.15.12:0/3516586586 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f405c019630 con 0x7f406c07de00 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.972+0000 7f4072277700 1 -- 172.21.15.12:0/3516586586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406c063980 con 0x7f406c07de00 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.972+0000 7f40697fa700 1 --2- 172.21.15.12:0/3516586586 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f4054039160 0x7f405403b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.972+0000 7f40697fa700 1 -- 172.21.15.12:0/3516586586 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f405c04d340 con 0x7f406c07de00 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.972+0000 7f406b7fe700 1 --2- 172.21.15.12:0/3516586586 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f4054039160 0x7f405403b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.972+0000 7f406b7fe700 1 --2- 172.21.15.12:0/3516586586 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f4054039160 0x7f405403b620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4060006fd0 tx=0x7f4060006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:43.977+0000 7f40697fa700 1 -- 172.21.15.12:0/3516586586 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f405c02dde0 con 0x7f406c07de00 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:44.187+0000 7f4072277700 1 -- 172.21.15.12:0/3516586586 --> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}) -- 0x7f406c104490 con 0x7f4054039160 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:48.418+0000 7f40697fa700 1 -- 172.21.15.12:0/3516586586 <== mgr.14118 v2:172.21.15.12:6800/2168557623 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f406c104490 con 0x7f4054039160 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:48.421+0000 7f4072277700 1 -- 172.21.15.12:0/3516586586 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f4054039160 msgr2=0x7f405403b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:48.421+0000 7f4072277700 1 --2- 172.21.15.12:0/3516586586 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f4054039160 0x7f405403b620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4060006fd0 tx=0x7f4060006e40 comp rx=0 tx=0).stop 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:48.421+0000 7f4072277700 1 -- 172.21.15.12:0/3516586586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 msgr2=0x7f406c19ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:48.421+0000 7f4072277700 1 --2- 172.21.15.12:0/3516586586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 0x7f406c19ddb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f405c004180 tx=0x7f405c0046a0 comp rx=0 tx=0).stop 2024-05-09T09:15:48.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:48.422+0000 7f4072277700 1 -- 172.21.15.12:0/3516586586 shutdown_connections 2024-05-09T09:15:48.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:48.422+0000 7f4072277700 1 --2- 172.21.15.12:0/3516586586 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f4054039160 0x7f405403b620 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:48.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:48.422+0000 7f4072277700 1 --2- 172.21.15.12:0/3516586586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c07de00 0x7f406c19ddb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:48.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:48.422+0000 7f4072277700 1 -- 172.21.15.12:0/3516586586 >> 172.21.15.12:0/3516586586 conn(0x7f406c1032f0 msgr2=0x7f406c103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:48.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:48.422+0000 7f4072277700 1 -- 172.21.15.12:0/3516586586 shutdown_connections 2024-05-09T09:15:48.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:48.422+0000 7f4072277700 1 -- 172.21.15.12:0/3516586586 wait complete. 2024-05-09T09:15:48.985 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mon service... 2024-05-09T09:15:49.696 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:49 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:49.697 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:49 smithi012 ceph-mon[30140]: Added host smithi012 2024-05-09T09:15:49.697 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:49 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:15:50.417 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-09T09:15:50.417 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.459+0000 7f7823490700 1 Processor -- start 2024-05-09T09:15:50.417 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.460+0000 7f7823490700 1 -- start start 2024-05-09T09:15:50.417 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.460+0000 7f7823490700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 0x7f781c106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.417 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.460+0000 7f7823490700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1075c0 con 0x7f781c106bd0 2024-05-09T09:15:50.417 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.460+0000 7f782122c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 0x7f781c106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.460+0000 7f782122c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 0x7f781c106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34792/0 (socket says 172.21.15.12:34792) 2024-05-09T09:15:50.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.460+0000 7f782122c700 1 -- 172.21.15.12:0/653583608 learned_addr learned my addr 172.21.15.12:0/653583608 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:50.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.460+0000 7f782122c700 1 -- 172.21.15.12:0/653583608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c107dd0 con 0x7f781c106bd0 2024-05-09T09:15:50.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.461+0000 7f782122c700 1 --2- 172.21.15.12:0/653583608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 0x7f781c106ff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f78180098a0 tx=0x7f7818009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8becc9db9dc00705 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.461+0000 7f7813fff700 1 -- 172.21.15.12:0/653583608 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7818004160 con 0x7f781c106bd0 2024-05-09T09:15:50.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.461+0000 7f7813fff700 1 -- 172.21.15.12:0/653583608 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f7818004300 con 0x7f781c106bd0 2024-05-09T09:15:50.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.461+0000 7f7813fff700 1 -- 172.21.15.12:0/653583608 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7818011b40 con 0x7f781c106bd0 2024-05-09T09:15:50.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.462+0000 7f7823490700 1 -- 172.21.15.12:0/653583608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 msgr2=0x7f781c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.462+0000 7f7823490700 1 --2- 172.21.15.12:0/653583608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 0x7f781c106ff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f78180098a0 tx=0x7f7818009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.462+0000 7f7823490700 1 -- 172.21.15.12:0/653583608 shutdown_connections 2024-05-09T09:15:50.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.462+0000 7f7823490700 1 --2- 172.21.15.12:0/653583608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 0x7f781c106ff0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.462+0000 7f7823490700 1 -- 172.21.15.12:0/653583608 >> 172.21.15.12:0/653583608 conn(0x7f781c102170 msgr2=0x7f781c1045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.462+0000 7f7823490700 1 -- 172.21.15.12:0/653583608 shutdown_connections 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.462+0000 7f7823490700 1 -- 172.21.15.12:0/653583608 wait complete. 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.463+0000 7f7823490700 1 Processor -- start 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.463+0000 7f7823490700 1 -- start start 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.463+0000 7f7823490700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 0x7f781c07b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.463+0000 7f7823490700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f781c10b180 con 0x7f781c106bd0 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.463+0000 7f782122c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 0x7f781c07b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.463+0000 7f782122c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 0x7f781c07b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34806/0 (socket says 172.21.15.12:34806) 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.463+0000 7f782122c700 1 -- 172.21.15.12:0/318212150 learned_addr learned my addr 172.21.15.12:0/318212150 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.463+0000 7f782122c700 1 -- 172.21.15.12:0/318212150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c19e950 con 0x7f781c106bd0 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.464+0000 7f782122c700 1 --2- 172.21.15.12:0/318212150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 0x7f781c07b9d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7818011840 tx=0x7f7818004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.464+0000 7f78127fc700 1 -- 172.21.15.12:0/318212150 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78180038b0 con 0x7f781c106bd0 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.464+0000 7f78127fc700 1 -- 172.21.15.12:0/318212150 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f781801a430 con 0x7f781c106bd0 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.464+0000 7f7823490700 1 -- 172.21.15.12:0/318212150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781c19eb20 con 0x7f781c106bd0 2024-05-09T09:15:50.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.464+0000 7f78127fc700 1 -- 172.21.15.12:0/318212150 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7818024650 con 0x7f781c106bd0 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.464+0000 7f7823490700 1 -- 172.21.15.12:0/318212150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f781c19eec0 con 0x7f781c106bd0 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.465+0000 7f7823490700 1 -- 172.21.15.12:0/318212150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f781c063980 con 0x7f781c106bd0 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.465+0000 7f78127fc700 1 -- 172.21.15.12:0/318212150 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7818022020 con 0x7f781c106bd0 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.465+0000 7f78127fc700 1 --2- 172.21.15.12:0/318212150 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f7808039160 0x7f780803b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.465+0000 7f78127fc700 1 -- 172.21.15.12:0/318212150 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f781804ce80 con 0x7f781c106bd0 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.465+0000 7f7820a2b700 1 --2- 172.21.15.12:0/318212150 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f7808039160 0x7f780803b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.466+0000 7f7820a2b700 1 --2- 172.21.15.12:0/318212150 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f7808039160 0x7f780803b620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f780c006fd0 tx=0x7f780c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.470+0000 7f78127fc700 1 -- 172.21.15.12:0/318212150 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f781802a2a0 con 0x7f781c106bd0 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.697+0000 7f7823490700 1 -- 172.21.15.12:0/318212150 --> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f781c103320 con 0x7f7808039160 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.702+0000 7f78127fc700 1 -- 172.21.15.12:0/318212150 <== mgr.14118 v2:172.21.15.12:6800/2168557623 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f781c103320 con 0x7f7808039160 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.705+0000 7f7823490700 1 -- 172.21.15.12:0/318212150 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f7808039160 msgr2=0x7f780803b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.705+0000 7f7823490700 1 --2- 172.21.15.12:0/318212150 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f7808039160 0x7f780803b620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f780c006fd0 tx=0x7f780c006e40 comp rx=0 tx=0).stop 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.705+0000 7f7823490700 1 -- 172.21.15.12:0/318212150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 msgr2=0x7f781c07b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.705+0000 7f7823490700 1 --2- 172.21.15.12:0/318212150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 0x7f781c07b9d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7818011840 tx=0x7f7818004230 comp rx=0 tx=0).stop 2024-05-09T09:15:50.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.705+0000 7f7823490700 1 -- 172.21.15.12:0/318212150 shutdown_connections 2024-05-09T09:15:50.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.705+0000 7f7823490700 1 --2- 172.21.15.12:0/318212150 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f7808039160 0x7f780803b620 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.705+0000 7f7823490700 1 --2- 172.21.15.12:0/318212150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f781c106bd0 0x7f781c07b9d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.705+0000 7f7823490700 1 -- 172.21.15.12:0/318212150 >> 172.21.15.12:0/318212150 conn(0x7f781c102170 msgr2=0x7f781c102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.705+0000 7f7823490700 1 -- 172.21.15.12:0/318212150 shutdown_connections 2024-05-09T09:15:50.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:49.705+0000 7f7823490700 1 -- 172.21.15.12:0/318212150 wait complete. 2024-05-09T09:15:50.421 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mgr service... 2024-05-09T09:15:50.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:50 smithi012 ceph-mon[30140]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:15:50.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:50 smithi012 ceph-mon[30140]: Saving service mon spec with placement count:5 2024-05-09T09:15:50.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:50 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:51.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-09T09:15:51.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.807+0000 7f07e4108700 1 Processor -- start 2024-05-09T09:15:51.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.807+0000 7f07e4108700 1 -- start start 2024-05-09T09:15:51.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.807+0000 7f07e4108700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 0x7f07dc07c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.807+0000 7f07e4108700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc07c7a0 con 0x7f07dc07ddb0 2024-05-09T09:15:51.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.807+0000 7f07e1ea4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 0x7f07dc07c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.807+0000 7f07e1ea4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 0x7f07dc07c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34820/0 (socket says 172.21.15.12:34820) 2024-05-09T09:15:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.807+0000 7f07e1ea4700 1 -- 172.21.15.12:0/1456984993 learned_addr learned my addr 172.21.15.12:0/1456984993 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.808+0000 7f07e1ea4700 1 -- 172.21.15.12:0/1456984993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc07c920 con 0x7f07dc07ddb0 2024-05-09T09:15:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.808+0000 7f07e1ea4700 1 --2- 172.21.15.12:0/1456984993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 0x7f07dc07c260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f07d80098a0 tx=0x7f07d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaefb84c149ded17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.808+0000 7f07e0ea2700 1 -- 172.21.15.12:0/1456984993 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07d8004160 con 0x7f07dc07ddb0 2024-05-09T09:15:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.808+0000 7f07e0ea2700 1 -- 172.21.15.12:0/1456984993 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f07d8004300 con 0x7f07dc07ddb0 2024-05-09T09:15:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.808+0000 7f07e0ea2700 1 -- 172.21.15.12:0/1456984993 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07d8011b40 con 0x7f07dc07ddb0 2024-05-09T09:15:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.808+0000 7f07e4108700 1 -- 172.21.15.12:0/1456984993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 msgr2=0x7f07dc07c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.808+0000 7f07e4108700 1 --2- 172.21.15.12:0/1456984993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 0x7f07dc07c260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f07d80098a0 tx=0x7f07d8009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.809+0000 7f07e4108700 1 -- 172.21.15.12:0/1456984993 shutdown_connections 2024-05-09T09:15:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.809+0000 7f07e4108700 1 --2- 172.21.15.12:0/1456984993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 0x7f07dc07c260 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.809+0000 7f07e4108700 1 -- 172.21.15.12:0/1456984993 >> 172.21.15.12:0/1456984993 conn(0x7f07dc1032c0 msgr2=0x7f07dc1056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.809+0000 7f07e4108700 1 -- 172.21.15.12:0/1456984993 shutdown_connections 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.809+0000 7f07e4108700 1 -- 172.21.15.12:0/1456984993 wait complete. 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.809+0000 7f07e4108700 1 Processor -- start 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.809+0000 7f07e4108700 1 -- start start 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.809+0000 7f07e4108700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 0x7f07dc19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.809+0000 7f07e4108700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc10e2d0 con 0x7f07dc07ddb0 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.810+0000 7f07e1ea4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 0x7f07dc19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.810+0000 7f07e1ea4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 0x7f07dc19ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34834/0 (socket says 172.21.15.12:34834) 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.810+0000 7f07e1ea4700 1 -- 172.21.15.12:0/3297378034 learned_addr learned my addr 172.21.15.12:0/3297378034 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.810+0000 7f07e1ea4700 1 -- 172.21.15.12:0/3297378034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc19e300 con 0x7f07dc07ddb0 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.810+0000 7f07e1ea4700 1 --2- 172.21.15.12:0/3297378034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 0x7f07dc19ddc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f07d8011670 tx=0x7f07d8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.810+0000 7f07d2ffd700 1 -- 172.21.15.12:0/3297378034 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07d80038b0 con 0x7f07dc07ddb0 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.810+0000 7f07d2ffd700 1 -- 172.21.15.12:0/3297378034 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f07d801a430 con 0x7f07dc07ddb0 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.810+0000 7f07e4108700 1 -- 172.21.15.12:0/3297378034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07dc19e590 con 0x7f07dc07ddb0 2024-05-09T09:15:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.810+0000 7f07d2ffd700 1 -- 172.21.15.12:0/3297378034 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07d8019550 con 0x7f07dc07ddb0 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.810+0000 7f07e4108700 1 -- 172.21.15.12:0/3297378034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07dc1a1280 con 0x7f07dc07ddb0 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.811+0000 7f07d2ffd700 1 -- 172.21.15.12:0/3297378034 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f07d8003a50 con 0x7f07dc07ddb0 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.811+0000 7f07e4108700 1 -- 172.21.15.12:0/3297378034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07dc063980 con 0x7f07dc07ddb0 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.811+0000 7f07d2ffd700 1 --2- 172.21.15.12:0/3297378034 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f07c80390c0 0x7f07c803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.812+0000 7f07d2ffd700 1 -- 172.21.15.12:0/3297378034 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f07d804d340 con 0x7f07dc07ddb0 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.812+0000 7f07e16a3700 1 --2- 172.21.15.12:0/3297378034 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f07c80390c0 0x7f07c803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.812+0000 7f07e16a3700 1 --2- 172.21.15.12:0/3297378034 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f07c80390c0 0x7f07c803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f07cc006fd0 tx=0x7f07cc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:50.816+0000 7f07d2ffd700 1 -- 172.21.15.12:0/3297378034 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f07d8003d40 con 0x7f07dc07ddb0 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:51.038+0000 7f07e4108700 1 -- 172.21.15.12:0/3297378034 --> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f07dc104400 con 0x7f07c80390c0 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:51.042+0000 7f07d2ffd700 1 -- 172.21.15.12:0/3297378034 <== mgr.14118 v2:172.21.15.12:6800/2168557623 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f07dc104400 con 0x7f07c80390c0 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:51.044+0000 7f07e4108700 1 -- 172.21.15.12:0/3297378034 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f07c80390c0 msgr2=0x7f07c803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:51.044+0000 7f07e4108700 1 --2- 172.21.15.12:0/3297378034 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f07c80390c0 0x7f07c803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f07cc006fd0 tx=0x7f07cc006e40 comp rx=0 tx=0).stop 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:51.044+0000 7f07e4108700 1 -- 172.21.15.12:0/3297378034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 msgr2=0x7f07dc19ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:51.044+0000 7f07e4108700 1 --2- 172.21.15.12:0/3297378034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 0x7f07dc19ddc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f07d8011670 tx=0x7f07d8011b50 comp rx=0 tx=0).stop 2024-05-09T09:15:51.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:51.044+0000 7f07e4108700 1 -- 172.21.15.12:0/3297378034 shutdown_connections 2024-05-09T09:15:51.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:51.044+0000 7f07e4108700 1 --2- 172.21.15.12:0/3297378034 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f07c80390c0 0x7f07c803b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:51.044+0000 7f07e4108700 1 --2- 172.21.15.12:0/3297378034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07dc07ddb0 0x7f07dc19ddc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:51.044+0000 7f07e4108700 1 -- 172.21.15.12:0/3297378034 >> 172.21.15.12:0/3297378034 conn(0x7f07dc1032c0 msgr2=0x7f07dc103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:51.044+0000 7f07e4108700 1 -- 172.21.15.12:0/3297378034 shutdown_connections 2024-05-09T09:15:51.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:51.045+0000 7f07e4108700 1 -- 172.21.15.12:0/3297378034 wait complete. 2024-05-09T09:15:52.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:52 smithi012 ceph-mon[30140]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:15:52.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:52 smithi012 ceph-mon[30140]: Saving service mgr spec with placement count:2 2024-05-09T09:15:52.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:52 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:53.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:53 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:53.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:53 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1854162363' entity='client.admin' 2024-05-09T09:15:53.672 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:53 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.884+0000 7f85b7a3c700 1 Processor -- start 2024-05-09T09:15:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.884+0000 7f85b7a3c700 1 -- start start 2024-05-09T09:15:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.884+0000 7f85b7a3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 0x7f85b0107010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.884+0000 7f85b7a3c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01075e0 con 0x7f85b0106bf0 2024-05-09T09:15:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.884+0000 7f85b57d8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 0x7f85b0107010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.884+0000 7f85b57d8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 0x7f85b0107010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46604/0 (socket says 172.21.15.12:46604) 2024-05-09T09:15:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.884+0000 7f85b57d8700 1 -- 172.21.15.12:0/3238807248 learned_addr learned my addr 172.21.15.12:0/3238807248 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.884+0000 7f85b57d8700 1 -- 172.21.15.12:0/3238807248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b0107df0 con 0x7f85b0106bf0 2024-05-09T09:15:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.885+0000 7f85b57d8700 1 --2- 172.21.15.12:0/3238807248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 0x7f85b0107010 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0098a0 tx=0x7f85ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175fa784345f936e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.885+0000 7f85a7fff700 1 -- 172.21.15.12:0/3238807248 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85ac004160 con 0x7f85b0106bf0 2024-05-09T09:15:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.885+0000 7f85a7fff700 1 -- 172.21.15.12:0/3238807248 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f85ac004300 con 0x7f85b0106bf0 2024-05-09T09:15:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.885+0000 7f85a7fff700 1 -- 172.21.15.12:0/3238807248 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85ac011b10 con 0x7f85b0106bf0 2024-05-09T09:15:53.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.885+0000 7f85b7a3c700 1 -- 172.21.15.12:0/3238807248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 msgr2=0x7f85b0107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:53.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.885+0000 7f85b7a3c700 1 --2- 172.21.15.12:0/3238807248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 0x7f85b0107010 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0098a0 tx=0x7f85ac009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:53.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.886+0000 7f85b7a3c700 1 -- 172.21.15.12:0/3238807248 shutdown_connections 2024-05-09T09:15:53.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.886+0000 7f85b7a3c700 1 --2- 172.21.15.12:0/3238807248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 0x7f85b0107010 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:53.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.886+0000 7f85b7a3c700 1 -- 172.21.15.12:0/3238807248 >> 172.21.15.12:0/3238807248 conn(0x7f85b0102190 msgr2=0x7f85b01045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:53.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.886+0000 7f85b7a3c700 1 -- 172.21.15.12:0/3238807248 shutdown_connections 2024-05-09T09:15:53.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.886+0000 7f85b7a3c700 1 -- 172.21.15.12:0/3238807248 wait complete. 2024-05-09T09:15:53.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.886+0000 7f85b7a3c700 1 Processor -- start 2024-05-09T09:15:53.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.886+0000 7f85b7a3c700 1 -- start start 2024-05-09T09:15:53.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.887+0000 7f85b7a3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 0x7f85b01044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:53.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.887+0000 7f85b7a3c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f85b010b1a0 con 0x7f85b0106bf0 2024-05-09T09:15:53.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.887+0000 7f85b57d8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 0x7f85b01044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:53.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.887+0000 7f85b57d8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 0x7f85b01044d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46612/0 (socket says 172.21.15.12:46612) 2024-05-09T09:15:53.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.887+0000 7f85b57d8700 1 -- 172.21.15.12:0/1854162363 learned_addr learned my addr 172.21.15.12:0/1854162363 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:53.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.887+0000 7f85b57d8700 1 -- 172.21.15.12:0/1854162363 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b0104a10 con 0x7f85b0106bf0 2024-05-09T09:15:53.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.887+0000 7f85b57d8700 1 --2- 172.21.15.12:0/1854162363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 0x7f85b01044d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85ac006af0 tx=0x7f85ac011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:53.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.887+0000 7f85a67fc700 1 -- 172.21.15.12:0/1854162363 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85ac003af0 con 0x7f85b0106bf0 2024-05-09T09:15:53.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.887+0000 7f85a67fc700 1 -- 172.21.15.12:0/1854162363 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f85ac003c90 con 0x7f85b0106bf0 2024-05-09T09:15:53.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.887+0000 7f85b7a3c700 1 -- 172.21.15.12:0/1854162363 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b0102c50 con 0x7f85b0106bf0 2024-05-09T09:15:53.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.887+0000 7f85a67fc700 1 -- 172.21.15.12:0/1854162363 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85ac024830 con 0x7f85b0106bf0 2024-05-09T09:15:53.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.887+0000 7f85b7a3c700 1 -- 172.21.15.12:0/1854162363 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b0103130 con 0x7f85b0106bf0 2024-05-09T09:15:53.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.888+0000 7f85a67fc700 1 -- 172.21.15.12:0/1854162363 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f85ac022020 con 0x7f85b0106bf0 2024-05-09T09:15:53.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.888+0000 7f85b7a3c700 1 -- 172.21.15.12:0/1854162363 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b0063980 con 0x7f85b0106bf0 2024-05-09T09:15:53.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.889+0000 7f85a67fc700 1 --2- 172.21.15.12:0/1854162363 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f859c039110 0x7f859c03b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:53.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.889+0000 7f85a67fc700 1 -- 172.21.15.12:0/1854162363 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f85ac04d0c0 con 0x7f85b0106bf0 2024-05-09T09:15:53.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.889+0000 7f85b4fd7700 1 --2- 172.21.15.12:0/1854162363 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f859c039110 0x7f859c03b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.889+0000 7f85b4fd7700 1 --2- 172.21.15.12:0/1854162363 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f859c039110 0x7f859c03b5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f85a0006fd0 tx=0x7f85a0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:52.894+0000 7f85a67fc700 1 -- 172.21.15.12:0/1854162363 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f85ac00f2e0 con 0x7f85b0106bf0 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:53.098+0000 7f85b7a3c700 1 -- 172.21.15.12:0/1854162363 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f85b0195490 con 0x7f85b0106bf0 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:53.104+0000 7f85a67fc700 1 -- 172.21.15.12:0/1854162363 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f85ac04ca10 con 0x7f85b0106bf0 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:53.107+0000 7f85b7a3c700 1 -- 172.21.15.12:0/1854162363 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f859c039110 msgr2=0x7f859c03b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:53.107+0000 7f85b7a3c700 1 --2- 172.21.15.12:0/1854162363 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f859c039110 0x7f859c03b5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f85a0006fd0 tx=0x7f85a0006e40 comp rx=0 tx=0).stop 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:53.107+0000 7f85b7a3c700 1 -- 172.21.15.12:0/1854162363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 msgr2=0x7f85b01044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:53.107+0000 7f85b7a3c700 1 --2- 172.21.15.12:0/1854162363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 0x7f85b01044d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85ac006af0 tx=0x7f85ac011bd0 comp rx=0 tx=0).stop 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:53.107+0000 7f85b7a3c700 1 -- 172.21.15.12:0/1854162363 shutdown_connections 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:53.107+0000 7f85b7a3c700 1 --2- 172.21.15.12:0/1854162363 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f859c039110 0x7f859c03b5d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:53.107+0000 7f85b7a3c700 1 --2- 172.21.15.12:0/1854162363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b0106bf0 0x7f85b01044d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:53.107+0000 7f85b7a3c700 1 -- 172.21.15.12:0/1854162363 >> 172.21.15.12:0/1854162363 conn(0x7f85b0102190 msgr2=0x7f85b0192800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:53.107+0000 7f85b7a3c700 1 -- 172.21.15.12:0/1854162363 shutdown_connections 2024-05-09T09:15:53.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:53.107+0000 7f85b7a3c700 1 -- 172.21.15.12:0/1854162363 wait complete. 2024-05-09T09:15:55.466 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:55 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3529788022' entity='client.admin' 2024-05-09T09:15:55.466 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:55 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:55.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.132+0000 7fb6e7acf700 1 Processor -- start 2024-05-09T09:15:55.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.132+0000 7fb6e7acf700 1 -- start start 2024-05-09T09:15:55.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.132+0000 7fb6e7acf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 0x7fb6e010abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:55.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.132+0000 7fb6e7acf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e007bdd0 con 0x7fb6e0108800 2024-05-09T09:15:55.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.133+0000 7fb6e586b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 0x7fb6e010abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:55.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.133+0000 7fb6e586b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 0x7fb6e010abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46618/0 (socket says 172.21.15.12:46618) 2024-05-09T09:15:55.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.133+0000 7fb6e586b700 1 -- 172.21.15.12:0/2661757190 learned_addr learned my addr 172.21.15.12:0/2661757190 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:55.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.133+0000 7fb6e586b700 1 -- 172.21.15.12:0/2661757190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e010b130 con 0x7fb6e0108800 2024-05-09T09:15:55.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.133+0000 7fb6e586b700 1 --2- 172.21.15.12:0/2661757190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 0x7fb6e010abf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0098a0 tx=0x7fb6dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d85ed19d10f72b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:55.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.133+0000 7fb6e4869700 1 -- 172.21.15.12:0/2661757190 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6dc004160 con 0x7fb6e0108800 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.133+0000 7fb6e4869700 1 -- 172.21.15.12:0/2661757190 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb6dc004300 con 0x7fb6e0108800 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.133+0000 7fb6e4869700 1 -- 172.21.15.12:0/2661757190 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6dc011b10 con 0x7fb6e0108800 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.134+0000 7fb6e7acf700 1 -- 172.21.15.12:0/2661757190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 msgr2=0x7fb6e010abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.134+0000 7fb6e7acf700 1 --2- 172.21.15.12:0/2661757190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 0x7fb6e010abf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0098a0 tx=0x7fb6dc009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.134+0000 7fb6e7acf700 1 -- 172.21.15.12:0/2661757190 shutdown_connections 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.134+0000 7fb6e7acf700 1 --2- 172.21.15.12:0/2661757190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 0x7fb6e010abf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.134+0000 7fb6e7acf700 1 -- 172.21.15.12:0/2661757190 >> 172.21.15.12:0/2661757190 conn(0x7fb6e01021d0 msgr2=0x7fb6e0104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.134+0000 7fb6e7acf700 1 -- 172.21.15.12:0/2661757190 shutdown_connections 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.134+0000 7fb6e7acf700 1 -- 172.21.15.12:0/2661757190 wait complete. 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.135+0000 7fb6e7acf700 1 Processor -- start 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.135+0000 7fb6e7acf700 1 -- start start 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.135+0000 7fb6e7acf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 0x7fb6e019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.135+0000 7fb6e7acf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e010c140 con 0x7fb6e0108800 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.135+0000 7fb6e586b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 0x7fb6e019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:55.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.135+0000 7fb6e586b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 0x7fb6e019bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46620/0 (socket says 172.21.15.12:46620) 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.135+0000 7fb6e586b700 1 -- 172.21.15.12:0/3529788022 learned_addr learned my addr 172.21.15.12:0/3529788022 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.136+0000 7fb6e586b700 1 -- 172.21.15.12:0/3529788022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e019c100 con 0x7fb6e0108800 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.136+0000 7fb6e586b700 1 --2- 172.21.15.12:0/3529788022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 0x7fb6e019bbc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc006af0 tx=0x7fb6dc011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.136+0000 7fb6d6ffd700 1 -- 172.21.15.12:0/3529788022 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6dc003af0 con 0x7fb6e0108800 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.136+0000 7fb6e7acf700 1 -- 172.21.15.12:0/3529788022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e019c390 con 0x7fb6e0108800 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.136+0000 7fb6d6ffd700 1 -- 172.21.15.12:0/3529788022 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb6dc003c90 con 0x7fb6e0108800 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.136+0000 7fb6d6ffd700 1 -- 172.21.15.12:0/3529788022 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6dc024830 con 0x7fb6e0108800 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.136+0000 7fb6e7acf700 1 -- 172.21.15.12:0/3529788022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e019f080 con 0x7fb6e0108800 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.137+0000 7fb6e7acf700 1 -- 172.21.15.12:0/3529788022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e0063980 con 0x7fb6e0108800 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.137+0000 7fb6d6ffd700 1 -- 172.21.15.12:0/3529788022 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb6dc022020 con 0x7fb6e0108800 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.138+0000 7fb6d6ffd700 1 --2- 172.21.15.12:0/3529788022 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fb6cc039110 0x7fb6cc03b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.138+0000 7fb6d6ffd700 1 -- 172.21.15.12:0/3529788022 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb6dc04d050 con 0x7fb6e0108800 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.138+0000 7fb6e506a700 1 --2- 172.21.15.12:0/3529788022 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fb6cc039110 0x7fb6cc03b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.138+0000 7fb6e506a700 1 --2- 172.21.15.12:0/3529788022 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fb6cc039110 0x7fb6cc03b5d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0006fd0 tx=0x7fb6d0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.143+0000 7fb6d6ffd700 1 -- 172.21.15.12:0/3529788022 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb6dc00f2e0 con 0x7fb6e0108800 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.361+0000 7fb6e7acf700 1 -- 172.21.15.12:0/3529788022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fb6e019f250 con 0x7fb6e0108800 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.364+0000 7fb6d6ffd700 1 -- 172.21.15.12:0/3529788022 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fb6dc02d770 con 0x7fb6e0108800 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.366+0000 7fb6e7acf700 1 -- 172.21.15.12:0/3529788022 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fb6cc039110 msgr2=0x7fb6cc03b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.366+0000 7fb6e7acf700 1 --2- 172.21.15.12:0/3529788022 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fb6cc039110 0x7fb6cc03b5d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0006fd0 tx=0x7fb6d0006e40 comp rx=0 tx=0).stop 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.366+0000 7fb6e7acf700 1 -- 172.21.15.12:0/3529788022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 msgr2=0x7fb6e019bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.366+0000 7fb6e7acf700 1 --2- 172.21.15.12:0/3529788022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 0x7fb6e019bbc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc006af0 tx=0x7fb6dc011bd0 comp rx=0 tx=0).stop 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.366+0000 7fb6e7acf700 1 -- 172.21.15.12:0/3529788022 shutdown_connections 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.366+0000 7fb6e7acf700 1 --2- 172.21.15.12:0/3529788022 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fb6cc039110 0x7fb6cc03b5d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.366+0000 7fb6e7acf700 1 --2- 172.21.15.12:0/3529788022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6e0108800 0x7fb6e019bbc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.366+0000 7fb6e7acf700 1 -- 172.21.15.12:0/3529788022 >> 172.21.15.12:0/3529788022 conn(0x7fb6e01021d0 msgr2=0x7fb6e0102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.366+0000 7fb6e7acf700 1 -- 172.21.15.12:0/3529788022 shutdown_connections 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:54.366+0000 7fb6e7acf700 1 -- 172.21.15.12:0/3529788022 wait complete. 2024-05-09T09:15:55.477 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling the dashboard module... 2024-05-09T09:15:57.209 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:57 smithi012 ceph-mon[30140]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:15:57.210 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:57 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1714338274' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-09T09:15:57.210 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:57 smithi012 ceph-mon[30140]: from='mgr.14118 172.21.15.12:0/2612851750' entity='mgr.x' 2024-05-09T09:15:57.483 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:57 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: ignoring --setuser ceph since I am not root 2024-05-09T09:15:57.484 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:57 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: ignoring --setgroup ceph since I am not root 2024-05-09T09:15:57.484 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:57 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:57.378+0000 7f0462fbd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T09:15:57.972 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:57 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:57.570+0000 7f0462fbd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T09:15:57.973 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:57 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:57.677+0000 7f0462fbd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T09:15:58.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:58 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1714338274' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-09T09:15:58.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:58 smithi012 ceph-mon[30140]: mgrmap e9: x(active, since 21s) 2024-05-09T09:15:58.234 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:58 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:58.002+0000 7f0462fbd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T09:15:58.234 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:58 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:58.092+0000 7f0462fbd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T09:15:58.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.929+0000 7fbdf211f700 1 Processor -- start 2024-05-09T09:15:58.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.929+0000 7fbdf211f700 1 -- start start 2024-05-09T09:15:58.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.929+0000 7fbdf211f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 0x7fbdec109760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:58.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.930+0000 7fbdf211f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec109d30 con 0x7fbdec109340 2024-05-09T09:15:58.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.930+0000 7fbdeb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 0x7fbdec109760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:58.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.930+0000 7fbdeb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 0x7fbdec109760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46636/0 (socket says 172.21.15.12:46636) 2024-05-09T09:15:58.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.930+0000 7fbdeb7fe700 1 -- 172.21.15.12:0/3911710531 learned_addr learned my addr 172.21.15.12:0/3911710531 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:58.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.931+0000 7fbdeb7fe700 1 -- 172.21.15.12:0/3911710531 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdec109eb0 con 0x7fbdec109340 2024-05-09T09:15:58.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.931+0000 7fbdeb7fe700 1 --2- 172.21.15.12:0/3911710531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 0x7fbdec109760 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0098a0 tx=0x7fbddc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc0ece91f0317d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:58.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.931+0000 7fbdea7fc700 1 -- 172.21.15.12:0/3911710531 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbddc004160 con 0x7fbdec109340 2024-05-09T09:15:58.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.931+0000 7fbdea7fc700 1 -- 172.21.15.12:0/3911710531 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbddc004300 con 0x7fbdec109340 2024-05-09T09:15:58.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.931+0000 7fbdea7fc700 1 -- 172.21.15.12:0/3911710531 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbddc011b10 con 0x7fbdec109340 2024-05-09T09:15:58.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.932+0000 7fbdf211f700 1 -- 172.21.15.12:0/3911710531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 msgr2=0x7fbdec109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:58.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.932+0000 7fbdf211f700 1 --2- 172.21.15.12:0/3911710531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 0x7fbdec109760 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0098a0 tx=0x7fbddc009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:58.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.932+0000 7fbdf211f700 1 -- 172.21.15.12:0/3911710531 shutdown_connections 2024-05-09T09:15:58.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.932+0000 7fbdf211f700 1 --2- 172.21.15.12:0/3911710531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 0x7fbdec109760 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:58.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.932+0000 7fbdf211f700 1 -- 172.21.15.12:0/3911710531 >> 172.21.15.12:0/3911710531 conn(0x7fbdec101980 msgr2=0x7fbdec103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:58.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.932+0000 7fbdf211f700 1 -- 172.21.15.12:0/3911710531 shutdown_connections 2024-05-09T09:15:58.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.932+0000 7fbdf211f700 1 -- 172.21.15.12:0/3911710531 wait complete. 2024-05-09T09:15:58.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.933+0000 7fbdf211f700 1 Processor -- start 2024-05-09T09:15:58.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.933+0000 7fbdf211f700 1 -- start start 2024-05-09T09:15:58.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.933+0000 7fbdf211f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 0x7fbdec19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.933+0000 7fbdf211f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec10c310 con 0x7fbdec109340 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.933+0000 7fbdeb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 0x7fbdec19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.933+0000 7fbdeb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 0x7fbdec19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46638/0 (socket says 172.21.15.12:46638) 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.933+0000 7fbdeb7fe700 1 -- 172.21.15.12:0/1714338274 learned_addr learned my addr 172.21.15.12:0/1714338274 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.934+0000 7fbdeb7fe700 1 -- 172.21.15.12:0/1714338274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdec19c0d0 con 0x7fbdec109340 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.934+0000 7fbdeb7fe700 1 --2- 172.21.15.12:0/1714338274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 0x7fbdec19bb90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbddc006af0 tx=0x7fbddc011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.934+0000 7fbde8ff9700 1 -- 172.21.15.12:0/1714338274 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbddc003af0 con 0x7fbdec109340 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.934+0000 7fbde8ff9700 1 -- 172.21.15.12:0/1714338274 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbddc003c90 con 0x7fbdec109340 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.934+0000 7fbdf211f700 1 -- 172.21.15.12:0/1714338274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdec19c3c0 con 0x7fbdec109340 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.934+0000 7fbde8ff9700 1 -- 172.21.15.12:0/1714338274 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbddc024830 con 0x7fbdec109340 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.934+0000 7fbdf211f700 1 -- 172.21.15.12:0/1714338274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdec19f050 con 0x7fbdec109340 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.935+0000 7fbde8ff9700 1 -- 172.21.15.12:0/1714338274 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbddc022020 con 0x7fbdec109340 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.935+0000 7fbdf211f700 1 -- 172.21.15.12:0/1714338274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdec10c490 con 0x7fbdec109340 2024-05-09T09:15:58.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.935+0000 7fbde8ff9700 1 --2- 172.21.15.12:0/1714338274 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fbdd4039160 0x7fbdd403b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:58.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.935+0000 7fbde8ff9700 1 -- 172.21.15.12:0/1714338274 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbddc04cf20 con 0x7fbdec109340 2024-05-09T09:15:58.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.936+0000 7fbdeaffd700 1 --2- 172.21.15.12:0/1714338274 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fbdd4039160 0x7fbdd403b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:58.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.936+0000 7fbdeaffd700 1 --2- 172.21.15.12:0/1714338274 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fbdd4039160 0x7fbdd403b620 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbde0006fd0 tx=0x7fbde0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:58.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:55.940+0000 7fbde8ff9700 1 -- 172.21.15.12:0/1714338274 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbddc02cdf0 con 0x7fbdec109340 2024-05-09T09:15:58.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:56.209+0000 7fbdf211f700 1 -- 172.21.15.12:0/1714338274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fbdec19f7a0 con 0x7fbdec109340 2024-05-09T09:15:58.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:57.194+0000 7fbde8ff9700 1 -- 172.21.15.12:0/1714338274 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fbddc00b5a0 con 0x7fbdec109340 2024-05-09T09:15:58.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:57.196+0000 7fbde8ff9700 1 -- 172.21.15.12:0/1714338274 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fbddc02dc90 con 0x7fbdec109340 2024-05-09T09:15:58.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:57.197+0000 7fbdf211f700 1 -- 172.21.15.12:0/1714338274 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fbdd4039160 msgr2=0x7fbdd403b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:58.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:57.197+0000 7fbdf211f700 1 --2- 172.21.15.12:0/1714338274 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fbdd4039160 0x7fbdd403b620 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbde0006fd0 tx=0x7fbde0006e40 comp rx=0 tx=0).stop 2024-05-09T09:15:58.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:57.197+0000 7fbdf211f700 1 -- 172.21.15.12:0/1714338274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 msgr2=0x7fbdec19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:58.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:57.197+0000 7fbdf211f700 1 --2- 172.21.15.12:0/1714338274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 0x7fbdec19bb90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbddc006af0 tx=0x7fbddc011bd0 comp rx=0 tx=0).stop 2024-05-09T09:15:58.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:57.198+0000 7fbdf211f700 1 -- 172.21.15.12:0/1714338274 shutdown_connections 2024-05-09T09:15:58.529 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:57.198+0000 7fbdf211f700 1 --2- 172.21.15.12:0/1714338274 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7fbdd4039160 0x7fbdd403b620 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:58.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:57.198+0000 7fbdf211f700 1 --2- 172.21.15.12:0/1714338274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdec109340 0x7fbdec19bb90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:58.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:57.198+0000 7fbdf211f700 1 -- 172.21.15.12:0/1714338274 >> 172.21.15.12:0/1714338274 conn(0x7fbdec101980 msgr2=0x7fbdec1023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:58.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:57.198+0000 7fbdf211f700 1 -- 172.21.15.12:0/1714338274 shutdown_connections 2024-05-09T09:15:58.530 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:57.198+0000 7fbdf211f700 1 -- 172.21.15.12:0/1714338274 wait complete. 2024-05-09T09:15:59.552 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:15:59 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/672418749' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T09:15:59.552 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:59 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:59.372+0000 7f0462fbd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T09:15:59.552 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:59 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:59.553+0000 7f0462fbd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T09:15:59.853 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:15:59 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:15:59.731+0000 7f0462fbd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T09:15:59.863 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-09T09:15:59.863 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-09T09:15:59.863 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T09:15:59.863 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T09:15:59.863 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T09:15:59.863 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-09T09:15:59.863 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.979+0000 7f2c16e08700 1 Processor -- start 2024-05-09T09:15:59.863 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.979+0000 7f2c16e08700 1 -- start start 2024-05-09T09:15:59.863 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.979+0000 7f2c16e08700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 0x7f2c1007c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:59.863 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.979+0000 7f2c16e08700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1007c850 con 0x7f2c1007de20 2024-05-09T09:15:59.863 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.980+0000 7f2c14ba4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 0x7f2c1007c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:59.864 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.980+0000 7f2c14ba4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 0x7f2c1007c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46668/0 (socket says 172.21.15.12:46668) 2024-05-09T09:15:59.864 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.980+0000 7f2c14ba4700 1 -- 172.21.15.12:0/2502427889 learned_addr learned my addr 172.21.15.12:0/2502427889 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:59.864 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.980+0000 7f2c14ba4700 1 -- 172.21.15.12:0/2502427889 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1007c9d0 con 0x7f2c1007de20 2024-05-09T09:15:59.864 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.980+0000 7f2c14ba4700 1 --2- 172.21.15.12:0/2502427889 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 0x7f2c1007c280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2c040098a0 tx=0x7f2c04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a7a45b7f93b23c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:59.864 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.980+0000 7f2c0f7fe700 1 -- 172.21.15.12:0/2502427889 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c04004160 con 0x7f2c1007de20 2024-05-09T09:15:59.864 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.980+0000 7f2c0f7fe700 1 -- 172.21.15.12:0/2502427889 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2c04004300 con 0x7f2c1007de20 2024-05-09T09:15:59.864 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.980+0000 7f2c0f7fe700 1 -- 172.21.15.12:0/2502427889 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c04011b10 con 0x7f2c1007de20 2024-05-09T09:15:59.864 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.981+0000 7f2c16e08700 1 -- 172.21.15.12:0/2502427889 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 msgr2=0x7f2c1007c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:59.864 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.981+0000 7f2c16e08700 1 --2- 172.21.15.12:0/2502427889 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 0x7f2c1007c280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2c040098a0 tx=0x7f2c04009bb0 comp rx=0 tx=0).stop 2024-05-09T09:15:59.864 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.981+0000 7f2c16e08700 1 -- 172.21.15.12:0/2502427889 shutdown_connections 2024-05-09T09:15:59.864 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.981+0000 7f2c16e08700 1 --2- 172.21.15.12:0/2502427889 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 0x7f2c1007c280 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:59.864 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.981+0000 7f2c16e08700 1 -- 172.21.15.12:0/2502427889 >> 172.21.15.12:0/2502427889 conn(0x7f2c101032f0 msgr2=0x7f2c10105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.981+0000 7f2c16e08700 1 -- 172.21.15.12:0/2502427889 shutdown_connections 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.981+0000 7f2c16e08700 1 -- 172.21.15.12:0/2502427889 wait complete. 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.982+0000 7f2c16e08700 1 Processor -- start 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.982+0000 7f2c16e08700 1 -- start start 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.982+0000 7f2c16e08700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 0x7f2c1019de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.982+0000 7f2c16e08700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1010e390 con 0x7f2c1007de20 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.982+0000 7f2c14ba4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 0x7f2c1019de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.982+0000 7f2c14ba4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 0x7f2c1019de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46676/0 (socket says 172.21.15.12:46676) 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.982+0000 7f2c14ba4700 1 -- 172.21.15.12:0/672418749 learned_addr learned my addr 172.21.15.12:0/672418749 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.982+0000 7f2c14ba4700 1 -- 172.21.15.12:0/672418749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1019e370 con 0x7f2c1007de20 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.982+0000 7f2c14ba4700 1 --2- 172.21.15.12:0/672418749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 0x7f2c1019de30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c04006af0 tx=0x7f2c04011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.983+0000 7f2c0dffb700 1 -- 172.21.15.12:0/672418749 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c040038b0 con 0x7f2c1007de20 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.983+0000 7f2c0dffb700 1 -- 172.21.15.12:0/672418749 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2c0401a430 con 0x7f2c1007de20 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.983+0000 7f2c16e08700 1 -- 172.21.15.12:0/672418749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1019e600 con 0x7f2c1007de20 2024-05-09T09:15:59.865 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.983+0000 7f2c0dffb700 1 -- 172.21.15.12:0/672418749 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c04019530 con 0x7f2c1007de20 2024-05-09T09:15:59.866 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.983+0000 7f2c16e08700 1 -- 172.21.15.12:0/672418749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c101a12f0 con 0x7f2c1007de20 2024-05-09T09:15:59.866 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.984+0000 7f2c0dffb700 1 -- 172.21.15.12:0/672418749 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f2c040197a0 con 0x7f2c1007de20 2024-05-09T09:15:59.866 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.984+0000 7f2c16e08700 1 -- 172.21.15.12:0/672418749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c10063980 con 0x7f2c1007de20 2024-05-09T09:15:59.866 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.984+0000 7f2c0dffb700 1 --2- 172.21.15.12:0/672418749 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f2c00039170 0x7f2c0003b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:59.866 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.984+0000 7f2c0dffb700 1 -- 172.21.15.12:0/672418749 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2c0404d360 con 0x7f2c1007de20 2024-05-09T09:15:59.866 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.984+0000 7f2c0ffff700 1 -- 172.21.15.12:0/672418749 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f2c00039170 msgr2=0x7f2c0003b630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2168557623 2024-05-09T09:15:59.866 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.984+0000 7f2c0ffff700 1 --2- 172.21.15.12:0/672418749 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f2c00039170 0x7f2c0003b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T09:15:59.866 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:58.989+0000 7f2c0dffb700 1 -- 172.21.15.12:0/672418749 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2c04019dc0 con 0x7f2c1007de20 2024-05-09T09:15:59.866 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.185+0000 7f2c0ffff700 1 -- 172.21.15.12:0/672418749 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f2c00039170 msgr2=0x7f2c0003b630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2168557623 2024-05-09T09:15:59.866 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.185+0000 7f2c0ffff700 1 --2- 172.21.15.12:0/672418749 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f2c00039170 0x7f2c0003b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T09:15:59.866 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.253+0000 7f2c16e08700 1 -- 172.21.15.12:0/672418749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f2c101a18a0 con 0x7f2c1007de20 2024-05-09T09:15:59.866 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.254+0000 7f2c0dffb700 1 -- 172.21.15.12:0/672418749 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f2c0400f2e0 con 0x7f2c1007de20 2024-05-09T09:15:59.866 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.256+0000 7f2c16e08700 1 -- 172.21.15.12:0/672418749 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f2c00039170 msgr2=0x7f2c0003b630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:15:59.867 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.256+0000 7f2c16e08700 1 --2- 172.21.15.12:0/672418749 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f2c00039170 0x7f2c0003b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:59.867 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.256+0000 7f2c16e08700 1 -- 172.21.15.12:0/672418749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 msgr2=0x7f2c1019de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:59.867 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.256+0000 7f2c16e08700 1 --2- 172.21.15.12:0/672418749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 0x7f2c1019de30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c04006af0 tx=0x7f2c04011bd0 comp rx=0 tx=0).stop 2024-05-09T09:15:59.867 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.256+0000 7f2c16e08700 1 -- 172.21.15.12:0/672418749 shutdown_connections 2024-05-09T09:15:59.867 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.256+0000 7f2c16e08700 1 --2- 172.21.15.12:0/672418749 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f2c00039170 0x7f2c0003b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:59.867 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.256+0000 7f2c16e08700 1 --2- 172.21.15.12:0/672418749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c1007de20 0x7f2c1019de30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:59.867 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.256+0000 7f2c16e08700 1 -- 172.21.15.12:0/672418749 >> 172.21.15.12:0/672418749 conn(0x7f2c101032f0 msgr2=0x7f2c10103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:59.867 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.256+0000 7f2c16e08700 1 -- 172.21.15.12:0/672418749 shutdown_connections 2024-05-09T09:15:59.867 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:15:59.256+0000 7f2c16e08700 1 -- 172.21.15.12:0/672418749 wait complete. 2024-05-09T09:15:59.867 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-05-09T09:15:59.867 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 9... 2024-05-09T09:16:00.143 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:00 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:00.041+0000 7f0462fbd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T09:16:00.143 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:00 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:00.144+0000 7f0462fbd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T09:16:00.483 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:00 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:00.361+0000 7f0462fbd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T09:16:00.906 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:00 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:00.565+0000 7f0462fbd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T09:16:01.483 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:01 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:01.202+0000 7f0462fbd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T09:16:01.965 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:01 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:01.650+0000 7f0462fbd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T09:16:01.966 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:01 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:01.752+0000 7f0462fbd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T09:16:01.966 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:01 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:01.966+0000 7f0462fbd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T09:16:02.896 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:02 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:02.538+0000 7f0462fbd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T09:16:02.896 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:02 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:02.637+0000 7f0462fbd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T09:16:03.233 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:02 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:02.897+0000 7f0462fbd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T09:16:03.234 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:03 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:03.038+0000 7f0462fbd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T09:16:03.234 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:03 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:03.140+0000 7f0462fbd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T09:16:03.577 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:03 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:03.263+0000 7f0462fbd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T09:16:03.577 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:03 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:03.578+0000 7f0462fbd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T09:16:04.484 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:04 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:04.233+0000 7f0462fbd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T09:16:04.484 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:16:04 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:16:04.464+0000 7f0462fbd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T09:16:04.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:04 smithi012 ceph-mon[30140]: Active manager daemon x restarted 2024-05-09T09:16:04.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:04 smithi012 ceph-mon[30140]: Activating manager daemon x 2024-05-09T09:16:04.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:04 smithi012 ceph-mon[30140]: osdmap e3: 0 total, 0 up, 0 in 2024-05-09T09:16:04.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:04 smithi012 ceph-mon[30140]: mgrmap e10: x(active, starting, since 0.00364271s) 2024-05-09T09:16:04.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:16:04.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T09:16:04.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T09:16:04.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T09:16:04.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T09:16:04.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:04 smithi012 ceph-mon[30140]: Manager daemon x is now available 2024-05-09T09:16:04.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:16:05.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T09:16:05.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T09:16:05.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:05 smithi012 ceph-mon[30140]: [09/May/2024:09:16:05] ENGINE Bus STARTING 2024-05-09T09:16:05.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:05 smithi012 ceph-mon[30140]: [09/May/2024:09:16:05] ENGINE Serving on https://172.21.15.12:7150 2024-05-09T09:16:05.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:05 smithi012 ceph-mon[30140]: mgrmap e11: x(active, since 1.00614s) 2024-05-09T09:16:06.340 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-09T09:16:06.340 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-09T09:16:06.340 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T09:16:06.340 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-09T09:16:06.340 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.300+0000 7f92e3e33700 1 Processor -- start 2024-05-09T09:16:06.340 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.300+0000 7f92e3e33700 1 -- start start 2024-05-09T09:16:06.341 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.301+0000 7f92e3e33700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 0x7f92dc1069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:06.341 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.301+0000 7f92e3e33700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc07bdf0 con 0x7f92dc106580 2024-05-09T09:16:06.341 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.301+0000 7f92e1bcf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 0x7f92dc1069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:06.341 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.301+0000 7f92e1bcf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 0x7f92dc1069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46678/0 (socket says 172.21.15.12:46678) 2024-05-09T09:16:06.341 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.301+0000 7f92e1bcf700 1 -- 172.21.15.12:0/1789473339 learned_addr learned my addr 172.21.15.12:0/1789473339 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:06.341 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.301+0000 7f92e1bcf700 1 -- 172.21.15.12:0/1789473339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc106ee0 con 0x7f92dc106580 2024-05-09T09:16:06.341 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.301+0000 7f92e1bcf700 1 --2- 172.21.15.12:0/1789473339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 0x7f92dc1069a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f92d80098a0 tx=0x7f92d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257c10ead2858d08 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:06.341 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.301+0000 7f92e0bcd700 1 -- 172.21.15.12:0/1789473339 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d8004160 con 0x7f92dc106580 2024-05-09T09:16:06.341 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.301+0000 7f92e0bcd700 1 -- 172.21.15.12:0/1789473339 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f92d8004300 con 0x7f92dc106580 2024-05-09T09:16:06.341 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.301+0000 7f92e0bcd700 1 -- 172.21.15.12:0/1789473339 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d8011b10 con 0x7f92dc106580 2024-05-09T09:16:06.341 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.302+0000 7f92e3e33700 1 -- 172.21.15.12:0/1789473339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 msgr2=0x7f92dc1069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:06.341 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.302+0000 7f92e3e33700 1 --2- 172.21.15.12:0/1789473339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 0x7f92dc1069a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f92d80098a0 tx=0x7f92d8009bb0 comp rx=0 tx=0).stop 2024-05-09T09:16:06.341 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.302+0000 7f92e3e33700 1 -- 172.21.15.12:0/1789473339 shutdown_connections 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.302+0000 7f92e3e33700 1 --2- 172.21.15.12:0/1789473339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 0x7f92dc1069a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.302+0000 7f92e3e33700 1 -- 172.21.15.12:0/1789473339 >> 172.21.15.12:0/1789473339 conn(0x7f92dc102180 msgr2=0x7f92dc1045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.302+0000 7f92e3e33700 1 -- 172.21.15.12:0/1789473339 shutdown_connections 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.302+0000 7f92e3e33700 1 -- 172.21.15.12:0/1789473339 wait complete. 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.303+0000 7f92e3e33700 1 Processor -- start 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.303+0000 7f92e3e33700 1 -- start start 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.303+0000 7f92e3e33700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 0x7f92dc19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.303+0000 7f92e3e33700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc107fb0 con 0x7f92dc106580 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.303+0000 7f92e1bcf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 0x7f92dc19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.303+0000 7f92e1bcf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 0x7f92dc19dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46684/0 (socket says 172.21.15.12:46684) 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.303+0000 7f92e1bcf700 1 -- 172.21.15.12:0/3356212804 learned_addr learned my addr 172.21.15.12:0/3356212804 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.303+0000 7f92e1bcf700 1 -- 172.21.15.12:0/3356212804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc19e2e0 con 0x7f92dc106580 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.304+0000 7f92e1bcf700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 0x7f92dc19dda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009fb0 tx=0x7f92d8011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:06.342 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.304+0000 7f92d2ffd700 1 -- 172.21.15.12:0/3356212804 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d8003b20 con 0x7f92dc106580 2024-05-09T09:16:06.343 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.304+0000 7f92d2ffd700 1 -- 172.21.15.12:0/3356212804 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f92d8003cc0 con 0x7f92dc106580 2024-05-09T09:16:06.343 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.304+0000 7f92e3e33700 1 -- 172.21.15.12:0/3356212804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92dc19e570 con 0x7f92dc106580 2024-05-09T09:16:06.343 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.304+0000 7f92d2ffd700 1 -- 172.21.15.12:0/3356212804 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d8024830 con 0x7f92dc106580 2024-05-09T09:16:06.343 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.304+0000 7f92e3e33700 1 -- 172.21.15.12:0/3356212804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92dc1a1220 con 0x7f92dc106580 2024-05-09T09:16:06.343 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.305+0000 7f92d2ffd700 1 -- 172.21.15.12:0/3356212804 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f92d8022020 con 0x7f92dc106580 2024-05-09T09:16:06.343 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.305+0000 7f92d2ffd700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 0x7f92c803b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:06.343 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.305+0000 7f92e3e33700 1 -- 172.21.15.12:0/3356212804 --> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f92dc063980 con 0x7f92c8039160 2024-05-09T09:16:06.343 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.305+0000 7f92d2ffd700 1 -- 172.21.15.12:0/3356212804 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f92d804d130 con 0x7f92dc106580 2024-05-09T09:16:06.343 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.305+0000 7f92e13ce700 1 -- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 msgr2=0x7f92c803b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2168557623 2024-05-09T09:16:06.343 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.305+0000 7f92e13ce700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 0x7f92c803b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T09:16:06.343 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.506+0000 7f92e13ce700 1 -- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 msgr2=0x7f92c803b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2168557623 2024-05-09T09:16:06.343 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.506+0000 7f92e13ce700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 0x7f92c803b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T09:16:06.344 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.907+0000 7f92e13ce700 1 -- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 msgr2=0x7f92c803b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2168557623 2024-05-09T09:16:06.344 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:00.907+0000 7f92e13ce700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 0x7f92c803b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T09:16:06.344 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:01.708+0000 7f92e13ce700 1 -- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 msgr2=0x7f92c803b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2168557623 2024-05-09T09:16:06.344 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:01.708+0000 7f92e13ce700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 0x7f92c803b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T09:16:06.344 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:03.309+0000 7f92e13ce700 1 -- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 msgr2=0x7f92c803b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2168557623 2024-05-09T09:16:06.344 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:03.309+0000 7f92e13ce700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 0x7f92c803b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T09:16:06.344 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:04.468+0000 7f92d2ffd700 1 -- 172.21.15.12:0/3356212804 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f92d802dc90 con 0x7f92dc106580 2024-05-09T09:16:06.344 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:04.468+0000 7f92d2ffd700 1 -- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 msgr2=0x7f92c803b620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:16:06.344 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:04.468+0000 7f92d2ffd700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 0x7f92c803b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:06.344 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.469+0000 7f92d2ffd700 1 -- 172.21.15.12:0/3356212804 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f92d804f350 con 0x7f92dc106580 2024-05-09T09:16:06.344 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.470+0000 7f92d2ffd700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f92c803ca30 0x7f92c803ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:06.344 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.470+0000 7f92d2ffd700 1 -- 172.21.15.12:0/3356212804 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f92dc063980 con 0x7f92c803ca30 2024-05-09T09:16:06.344 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.470+0000 7f92e13ce700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f92c803ca30 0x7f92c803ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.471+0000 7f92e13ce700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f92c803ca30 0x7f92c803ee20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f92cc003a10 tx=0x7f92cc0092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.471+0000 7f92d2ffd700 1 -- 172.21.15.12:0/3356212804 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f92dc063980 con 0x7f92c803ca30 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.478+0000 7f92e3e33700 1 -- 172.21.15.12:0/3356212804 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f92dc062760 con 0x7f92c803ca30 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.479+0000 7f92d2ffd700 1 -- 172.21.15.12:0/3356212804 <== mgr.14150 v2:172.21.15.12:6800/3809515815 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f92dc062760 con 0x7f92c803ca30 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.479+0000 7f92e3e33700 1 -- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f92c803ca30 msgr2=0x7f92c803ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.479+0000 7f92e3e33700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f92c803ca30 0x7f92c803ee20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f92cc003a10 tx=0x7f92cc0092b0 comp rx=0 tx=0).stop 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.479+0000 7f92e3e33700 1 -- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 msgr2=0x7f92dc19dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.479+0000 7f92e3e33700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 0x7f92dc19dda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009fb0 tx=0x7f92d8011bd0 comp rx=0 tx=0).stop 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.479+0000 7f92e3e33700 1 -- 172.21.15.12:0/3356212804 shutdown_connections 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.479+0000 7f92e3e33700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f92c803ca30 0x7f92c803ee20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.479+0000 7f92e3e33700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92dc106580 0x7f92dc19dda0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.479+0000 7f92e3e33700 1 --2- 172.21.15.12:0/3356212804 >> [v2:172.21.15.12:6800/2168557623,v1:172.21.15.12:6801/2168557623] conn(0x7f92c8039160 0x7f92c803b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.479+0000 7f92e3e33700 1 -- 172.21.15.12:0/3356212804 >> 172.21.15.12:0/3356212804 conn(0x7f92dc102180 msgr2=0x7f92dc1949f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.479+0000 7f92e3e33700 1 -- 172.21.15.12:0/3356212804 shutdown_connections 2024-05-09T09:16:06.346 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:05.479+0000 7f92e3e33700 1 -- 172.21.15.12:0/3356212804 wait complete. 2024-05-09T09:16:06.346 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 9 is available 2024-05-09T09:16:06.346 INFO:teuthology.orchestra.run.smithi012.stdout:Generating a dashboard self-signed certificate... 2024-05-09T09:16:07.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:07 smithi012 ceph-mon[30140]: [09/May/2024:09:16:05] ENGINE Serving on http://172.21.15.12:8765 2024-05-09T09:16:07.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:07 smithi012 ceph-mon[30140]: [09/May/2024:09:16:05] ENGINE Bus STARTED 2024-05-09T09:16:07.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:07 smithi012 ceph-mon[30140]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T09:16:07.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:07 smithi012 ceph-mon[30140]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T09:16:07.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:07.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-09T09:16:07.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.812+0000 7fb7fc22a700 1 Processor -- start 2024-05-09T09:16:07.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.812+0000 7fb7fc22a700 1 -- start start 2024-05-09T09:16:07.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.812+0000 7fb7fc22a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 0x7fb7f410abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:07.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.812+0000 7fb7fc22a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f407bdd0 con 0x7fb7f41087c0 2024-05-09T09:16:07.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.812+0000 7fb7f9fc6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 0x7fb7f410abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:07.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.812+0000 7fb7f9fc6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 0x7fb7f410abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40988/0 (socket says 172.21.15.12:40988) 2024-05-09T09:16:07.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.812+0000 7fb7f9fc6700 1 -- 172.21.15.12:0/465269726 learned_addr learned my addr 172.21.15.12:0/465269726 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:07.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.812+0000 7fb7f9fc6700 1 -- 172.21.15.12:0/465269726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f410b0f0 con 0x7fb7f41087c0 2024-05-09T09:16:07.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.813+0000 7fb7f9fc6700 1 --2- 172.21.15.12:0/465269726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 0x7fb7f410abb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00098a0 tx=0x7fb7f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd6eb77daa24f33 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:07.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.813+0000 7fb7f8fc4700 1 -- 172.21.15.12:0/465269726 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7f0004160 con 0x7fb7f41087c0 2024-05-09T09:16:07.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.813+0000 7fb7f8fc4700 1 -- 172.21.15.12:0/465269726 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb7f0004300 con 0x7fb7f41087c0 2024-05-09T09:16:07.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.813+0000 7fb7f8fc4700 1 -- 172.21.15.12:0/465269726 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7f0011b10 con 0x7fb7f41087c0 2024-05-09T09:16:07.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.813+0000 7fb7fc22a700 1 -- 172.21.15.12:0/465269726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 msgr2=0x7fb7f410abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:07.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.813+0000 7fb7fc22a700 1 --2- 172.21.15.12:0/465269726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 0x7fb7f410abb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00098a0 tx=0x7fb7f0009bb0 comp rx=0 tx=0).stop 2024-05-09T09:16:07.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.814+0000 7fb7fc22a700 1 -- 172.21.15.12:0/465269726 shutdown_connections 2024-05-09T09:16:07.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.814+0000 7fb7fc22a700 1 --2- 172.21.15.12:0/465269726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 0x7fb7f410abb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.814+0000 7fb7fc22a700 1 -- 172.21.15.12:0/465269726 >> 172.21.15.12:0/465269726 conn(0x7fb7f4102190 msgr2=0x7fb7f41045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.814+0000 7fb7fc22a700 1 -- 172.21.15.12:0/465269726 shutdown_connections 2024-05-09T09:16:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.814+0000 7fb7fc22a700 1 -- 172.21.15.12:0/465269726 wait complete. 2024-05-09T09:16:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.815+0000 7fb7fc22a700 1 Processor -- start 2024-05-09T09:16:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.815+0000 7fb7fc22a700 1 -- start start 2024-05-09T09:16:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.815+0000 7fb7fc22a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 0x7fb7f419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.815+0000 7fb7fc22a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f410c100 con 0x7fb7f41087c0 2024-05-09T09:16:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.815+0000 7fb7f9fc6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 0x7fb7f419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.815+0000 7fb7f9fc6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 0x7fb7f419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40996/0 (socket says 172.21.15.12:40996) 2024-05-09T09:16:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.815+0000 7fb7f9fc6700 1 -- 172.21.15.12:0/953049428 learned_addr learned my addr 172.21.15.12:0/953049428 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.815+0000 7fb7f9fc6700 1 -- 172.21.15.12:0/953049428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f419c0e0 con 0x7fb7f41087c0 2024-05-09T09:16:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.815+0000 7fb7f9fc6700 1 --2- 172.21.15.12:0/953049428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 0x7fb7f419bba0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0003fa0 tx=0x7fb7f00046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:07.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.816+0000 7fb7eaffd700 1 -- 172.21.15.12:0/953049428 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7f00038b0 con 0x7fb7f41087c0 2024-05-09T09:16:07.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.816+0000 7fb7eaffd700 1 -- 172.21.15.12:0/953049428 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb7f001a430 con 0x7fb7f41087c0 2024-05-09T09:16:07.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.816+0000 7fb7fc22a700 1 -- 172.21.15.12:0/953049428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f419c370 con 0x7fb7f41087c0 2024-05-09T09:16:07.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.816+0000 7fb7fc22a700 1 -- 172.21.15.12:0/953049428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f419f060 con 0x7fb7f41087c0 2024-05-09T09:16:07.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.816+0000 7fb7eaffd700 1 -- 172.21.15.12:0/953049428 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7f0019550 con 0x7fb7f41087c0 2024-05-09T09:16:07.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.817+0000 7fb7eaffd700 1 -- 172.21.15.12:0/953049428 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fb7f00196f0 con 0x7fb7f41087c0 2024-05-09T09:16:07.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.817+0000 7fb7fc22a700 1 -- 172.21.15.12:0/953049428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f4063980 con 0x7fb7f41087c0 2024-05-09T09:16:07.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.817+0000 7fb7eaffd700 1 --2- 172.21.15.12:0/953049428 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb7e0039050 0x7fb7e003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:07.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.817+0000 7fb7eaffd700 1 -- 172.21.15.12:0/953049428 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb7f004d2f0 con 0x7fb7f41087c0 2024-05-09T09:16:07.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.817+0000 7fb7f97c5700 1 --2- 172.21.15.12:0/953049428 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb7e0039050 0x7fb7e003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:07.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.817+0000 7fb7f97c5700 1 --2- 172.21.15.12:0/953049428 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb7e0039050 0x7fb7e003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4006fd0 tx=0x7fb7e4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:07.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:06.822+0000 7fb7eaffd700 1 -- 172.21.15.12:0/953049428 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb7f000bc70 con 0x7fb7f41087c0 2024-05-09T09:16:07.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:07.031+0000 7fb7eaffd700 1 -- 172.21.15.12:0/953049428 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fb7f002d7c0 con 0x7fb7f41087c0 2024-05-09T09:16:07.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:07.039+0000 7fb7fc22a700 1 -- 172.21.15.12:0/953049428 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb7f41032c0 con 0x7fb7e0039050 2024-05-09T09:16:07.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:07.131+0000 7fb7eaffd700 1 -- 172.21.15.12:0/953049428 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb7f41032c0 con 0x7fb7e0039050 2024-05-09T09:16:07.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:07.134+0000 7fb7fc22a700 1 -- 172.21.15.12:0/953049428 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb7e0039050 msgr2=0x7fb7e003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:07.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:07.134+0000 7fb7fc22a700 1 --2- 172.21.15.12:0/953049428 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb7e0039050 0x7fb7e003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4006fd0 tx=0x7fb7e4006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:07.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:07.134+0000 7fb7fc22a700 1 -- 172.21.15.12:0/953049428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 msgr2=0x7fb7f419bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:07.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:07.134+0000 7fb7fc22a700 1 --2- 172.21.15.12:0/953049428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 0x7fb7f419bba0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0003fa0 tx=0x7fb7f00046a0 comp rx=0 tx=0).stop 2024-05-09T09:16:07.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:07.134+0000 7fb7fc22a700 1 -- 172.21.15.12:0/953049428 shutdown_connections 2024-05-09T09:16:07.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:07.134+0000 7fb7fc22a700 1 --2- 172.21.15.12:0/953049428 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb7e0039050 0x7fb7e003b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:07.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:07.134+0000 7fb7fc22a700 1 --2- 172.21.15.12:0/953049428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f41087c0 0x7fb7f419bba0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:07.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:07.134+0000 7fb7fc22a700 1 -- 172.21.15.12:0/953049428 >> 172.21.15.12:0/953049428 conn(0x7fb7f4102190 msgr2=0x7fb7f4102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:07.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:07.134+0000 7fb7fc22a700 1 -- 172.21.15.12:0/953049428 shutdown_connections 2024-05-09T09:16:07.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:07.134+0000 7fb7fc22a700 1 -- 172.21.15.12:0/953049428 wait complete. 2024-05-09T09:16:07.727 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial admin user... 2024-05-09T09:16:08.295 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:08 smithi012 ceph-mon[30140]: mgrmap e12: x(active, since 2s) 2024-05-09T09:16:08.295 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:08 smithi012 ceph-mon[30140]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:16:08.295 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:08 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:08.295 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:08 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:08.295 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:08 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:09.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$uO0MviN6QsC7TGJcvfeRguUvDvB470zhuRrG1MvjrSjATVbzdwXLS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715246168, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-09T09:16:09.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.132+0000 7fe9b7f14700 1 Processor -- start 2024-05-09T09:16:09.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.132+0000 7fe9b7f14700 1 -- start start 2024-05-09T09:16:09.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.133+0000 7fe9b7f14700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 0x7fe9b01069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:09.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.133+0000 7fe9b7f14700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b007bdb0 con 0x7fe9b01065b0 2024-05-09T09:16:09.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.133+0000 7fe9b5cb0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 0x7fe9b01069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:09.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.133+0000 7fe9b5cb0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 0x7fe9b01069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41004/0 (socket says 172.21.15.12:41004) 2024-05-09T09:16:09.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.133+0000 7fe9b5cb0700 1 -- 172.21.15.12:0/1332981681 learned_addr learned my addr 172.21.15.12:0/1332981681 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.133+0000 7fe9b5cb0700 1 -- 172.21.15.12:0/1332981681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b0106f10 con 0x7fe9b01065b0 2024-05-09T09:16:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.133+0000 7fe9b5cb0700 1 --2- 172.21.15.12:0/1332981681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 0x7fe9b01069d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009e80 tx=0x7fe9ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71143796087c8a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.133+0000 7fe9b4cae700 1 -- 172.21.15.12:0/1332981681 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9ac017070 con 0x7fe9b01065b0 2024-05-09T09:16:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.133+0000 7fe9b4cae700 1 -- 172.21.15.12:0/1332981681 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe9ac00b8c0 con 0x7fe9b01065b0 2024-05-09T09:16:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.133+0000 7fe9b4cae700 1 -- 172.21.15.12:0/1332981681 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9ac004520 con 0x7fe9b01065b0 2024-05-09T09:16:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.134+0000 7fe9b7f14700 1 -- 172.21.15.12:0/1332981681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 msgr2=0x7fe9b01069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.134+0000 7fe9b7f14700 1 --2- 172.21.15.12:0/1332981681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 0x7fe9b01069d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009e80 tx=0x7fe9ac00b330 comp rx=0 tx=0).stop 2024-05-09T09:16:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.134+0000 7fe9b7f14700 1 -- 172.21.15.12:0/1332981681 shutdown_connections 2024-05-09T09:16:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.134+0000 7fe9b7f14700 1 --2- 172.21.15.12:0/1332981681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 0x7fe9b01069d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.134+0000 7fe9b7f14700 1 -- 172.21.15.12:0/1332981681 >> 172.21.15.12:0/1332981681 conn(0x7fe9b0102170 msgr2=0x7fe9b01045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.134+0000 7fe9b7f14700 1 -- 172.21.15.12:0/1332981681 shutdown_connections 2024-05-09T09:16:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.134+0000 7fe9b7f14700 1 -- 172.21.15.12:0/1332981681 wait complete. 2024-05-09T09:16:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.135+0000 7fe9b7f14700 1 Processor -- start 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.135+0000 7fe9b7f14700 1 -- start start 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.135+0000 7fe9b7f14700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 0x7fe9b019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.135+0000 7fe9b7f14700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b0107fe0 con 0x7fe9b01065b0 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.136+0000 7fe9b5cb0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 0x7fe9b019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.136+0000 7fe9b5cb0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 0x7fe9b019ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41006/0 (socket says 172.21.15.12:41006) 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.136+0000 7fe9b5cb0700 1 -- 172.21.15.12:0/516305451 learned_addr learned my addr 172.21.15.12:0/516305451 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.136+0000 7fe9b5cb0700 1 -- 172.21.15.12:0/516305451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b019e2f0 con 0x7fe9b01065b0 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.136+0000 7fe9b5cb0700 1 --2- 172.21.15.12:0/516305451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 0x7fe9b019ddb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00b7b0 tx=0x7fe9ac00bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.136+0000 7fe9a6ffd700 1 -- 172.21.15.12:0/516305451 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9ac017070 con 0x7fe9b01065b0 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.136+0000 7fe9a6ffd700 1 -- 172.21.15.12:0/516305451 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe9ac022070 con 0x7fe9b01065b0 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.136+0000 7fe9b7f14700 1 -- 172.21.15.12:0/516305451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b019e580 con 0x7fe9b01065b0 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.136+0000 7fe9b7f14700 1 -- 172.21.15.12:0/516305451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b01a1230 con 0x7fe9b01065b0 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.137+0000 7fe9a6ffd700 1 -- 172.21.15.12:0/516305451 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9ac01d690 con 0x7fe9b01065b0 2024-05-09T09:16:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.137+0000 7fe9a6ffd700 1 -- 172.21.15.12:0/516305451 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fe9ac01d940 con 0x7fe9b01065b0 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.138+0000 7fe9b7f14700 1 -- 172.21.15.12:0/516305451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b0063980 con 0x7fe9b01065b0 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.138+0000 7fe9a6ffd700 1 --2- 172.21.15.12:0/516305451 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe99c039170 0x7fe99c03b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.138+0000 7fe9a6ffd700 1 -- 172.21.15.12:0/516305451 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe9ac00f050 con 0x7fe9b01065b0 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.138+0000 7fe9b54af700 1 --2- 172.21.15.12:0/516305451 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe99c039170 0x7fe99c03b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.138+0000 7fe9b54af700 1 --2- 172.21.15.12:0/516305451 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe99c039170 0x7fe99c03b630 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0006fd0 tx=0x7fe9a0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.143+0000 7fe9a6ffd700 1 -- 172.21.15.12:0/516305451 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe9ac025360 con 0x7fe9b01065b0 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.383+0000 7fe9b7f14700 1 -- 172.21.15.12:0/516305451 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fe9b0197890 con 0x7fe99c039170 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.629+0000 7fe9a6ffd700 1 -- 172.21.15.12:0/516305451 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fe9b0197890 con 0x7fe99c039170 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.632+0000 7fe9b7f14700 1 -- 172.21.15.12:0/516305451 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe99c039170 msgr2=0x7fe99c03b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.632+0000 7fe9b7f14700 1 --2- 172.21.15.12:0/516305451 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe99c039170 0x7fe99c03b630 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0006fd0 tx=0x7fe9a0006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.632+0000 7fe9b7f14700 1 -- 172.21.15.12:0/516305451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 msgr2=0x7fe9b019ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.632+0000 7fe9b7f14700 1 --2- 172.21.15.12:0/516305451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 0x7fe9b019ddb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00b7b0 tx=0x7fe9ac00bdc0 comp rx=0 tx=0).stop 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.632+0000 7fe9b7f14700 1 -- 172.21.15.12:0/516305451 shutdown_connections 2024-05-09T09:16:09.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.632+0000 7fe9b7f14700 1 --2- 172.21.15.12:0/516305451 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe99c039170 0x7fe99c03b630 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:09.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.632+0000 7fe9b7f14700 1 --2- 172.21.15.12:0/516305451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01065b0 0x7fe9b019ddb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:09.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.632+0000 7fe9b7f14700 1 -- 172.21.15.12:0/516305451 >> 172.21.15.12:0/516305451 conn(0x7fe9b0102170 msgr2=0x7fe9b01949d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:09.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.632+0000 7fe9b7f14700 1 -- 172.21.15.12:0/516305451 shutdown_connections 2024-05-09T09:16:09.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:08.632+0000 7fe9b7f14700 1 -- 172.21.15.12:0/516305451 wait complete. 2024-05-09T09:16:09.353 INFO:teuthology.orchestra.run.smithi012.stdout:Fetching dashboard port number... 2024-05-09T09:16:09.927 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:09 smithi012 ceph-mon[30140]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:16:09.927 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:10.832 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:10 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3096769328' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-09T09:16:11.099 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 8443 2024-05-09T09:16:11.100 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.317+0000 7f590b639700 1 Processor -- start 2024-05-09T09:16:11.100 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.317+0000 7f590b639700 1 -- start start 2024-05-09T09:16:11.100 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.317+0000 7f590b639700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 0x7f590410abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:11.100 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.318+0000 7f590b639700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f590407bdd0 con 0x7f59041087c0 2024-05-09T09:16:11.100 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.318+0000 7f59093d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 0x7f590410abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:11.100 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.318+0000 7f59093d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 0x7f590410abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41014/0 (socket says 172.21.15.12:41014) 2024-05-09T09:16:11.100 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.318+0000 7f59093d5700 1 -- 172.21.15.12:0/1554725080 learned_addr learned my addr 172.21.15.12:0/1554725080 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:11.100 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.318+0000 7f59093d5700 1 -- 172.21.15.12:0/1554725080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590410b0f0 con 0x7f59041087c0 2024-05-09T09:16:11.100 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.318+0000 7f59093d5700 1 --2- 172.21.15.12:0/1554725080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 0x7f590410abb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f59000098a0 tx=0x7f5900009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf0df91ac0bd7af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:11.100 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.318+0000 7f58fbfff700 1 -- 172.21.15.12:0/1554725080 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5900004160 con 0x7f59041087c0 2024-05-09T09:16:11.100 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.318+0000 7f58fbfff700 1 -- 172.21.15.12:0/1554725080 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5900004300 con 0x7f59041087c0 2024-05-09T09:16:11.100 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.318+0000 7f58fbfff700 1 -- 172.21.15.12:0/1554725080 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5900011b10 con 0x7f59041087c0 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.319+0000 7f590b639700 1 -- 172.21.15.12:0/1554725080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 msgr2=0x7f590410abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.319+0000 7f590b639700 1 --2- 172.21.15.12:0/1554725080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 0x7f590410abb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f59000098a0 tx=0x7f5900009bb0 comp rx=0 tx=0).stop 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.319+0000 7f590b639700 1 -- 172.21.15.12:0/1554725080 shutdown_connections 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.319+0000 7f590b639700 1 --2- 172.21.15.12:0/1554725080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 0x7f590410abb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.319+0000 7f590b639700 1 -- 172.21.15.12:0/1554725080 >> 172.21.15.12:0/1554725080 conn(0x7f5904102190 msgr2=0x7f59041045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.319+0000 7f590b639700 1 -- 172.21.15.12:0/1554725080 shutdown_connections 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.319+0000 7f590b639700 1 -- 172.21.15.12:0/1554725080 wait complete. 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.320+0000 7f590b639700 1 Processor -- start 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.320+0000 7f590b639700 1 -- start start 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.320+0000 7f590b639700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 0x7f590419bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.320+0000 7f590b639700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f590410c100 con 0x7f59041087c0 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.320+0000 7f59093d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 0x7f590419bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.320+0000 7f59093d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 0x7f590419bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41018/0 (socket says 172.21.15.12:41018) 2024-05-09T09:16:11.101 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.320+0000 7f59093d5700 1 -- 172.21.15.12:0/3096769328 learned_addr learned my addr 172.21.15.12:0/3096769328 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.320+0000 7f59093d5700 1 -- 172.21.15.12:0/3096769328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590419c100 con 0x7f59041087c0 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.320+0000 7f59093d5700 1 --2- 172.21.15.12:0/3096769328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 0x7f590419bbc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5900006af0 tx=0x7f5900011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.321+0000 7f58fa7fc700 1 -- 172.21.15.12:0/3096769328 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5900003af0 con 0x7f59041087c0 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.321+0000 7f58fa7fc700 1 -- 172.21.15.12:0/3096769328 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5900003c90 con 0x7f59041087c0 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.321+0000 7f590b639700 1 -- 172.21.15.12:0/3096769328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590419c390 con 0x7f59041087c0 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.321+0000 7f58fa7fc700 1 -- 172.21.15.12:0/3096769328 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5900024830 con 0x7f59041087c0 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.321+0000 7f590b639700 1 -- 172.21.15.12:0/3096769328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f590419f080 con 0x7f59041087c0 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.321+0000 7f58fa7fc700 1 -- 172.21.15.12:0/3096769328 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f5900022020 con 0x7f59041087c0 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.322+0000 7f590b639700 1 -- 172.21.15.12:0/3096769328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5904063980 con 0x7f59041087c0 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.322+0000 7f58fa7fc700 1 --2- 172.21.15.12:0/3096769328 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f58f0039160 0x7f58f003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.322+0000 7f58fa7fc700 1 -- 172.21.15.12:0/3096769328 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f590004d340 con 0x7f59041087c0 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.322+0000 7f5908bd4700 1 --2- 172.21.15.12:0/3096769328 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f58f0039160 0x7f58f003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.322+0000 7f5908bd4700 1 --2- 172.21.15.12:0/3096769328 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f58f0039160 0x7f58f003b620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f58f4006fd0 tx=0x7f58f4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.327+0000 7f58fa7fc700 1 -- 172.21.15.12:0/3096769328 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f590000b5a0 con 0x7f59041087c0 2024-05-09T09:16:11.103 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.533+0000 7f590b639700 1 -- 172.21.15.12:0/3096769328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f590419f630 con 0x7f59041087c0 2024-05-09T09:16:11.103 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.533+0000 7f58fa7fc700 1 -- 172.21.15.12:0/3096769328 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f590001ac90 con 0x7f59041087c0 2024-05-09T09:16:11.103 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.535+0000 7f590b639700 1 -- 172.21.15.12:0/3096769328 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f58f0039160 msgr2=0x7f58f003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:11.103 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.535+0000 7f590b639700 1 --2- 172.21.15.12:0/3096769328 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f58f0039160 0x7f58f003b620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f58f4006fd0 tx=0x7f58f4006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:11.103 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.535+0000 7f590b639700 1 -- 172.21.15.12:0/3096769328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 msgr2=0x7f590419bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:11.103 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.535+0000 7f590b639700 1 --2- 172.21.15.12:0/3096769328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 0x7f590419bbc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5900006af0 tx=0x7f5900011bd0 comp rx=0 tx=0).stop 2024-05-09T09:16:11.103 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.535+0000 7f590b639700 1 -- 172.21.15.12:0/3096769328 shutdown_connections 2024-05-09T09:16:11.103 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.535+0000 7f590b639700 1 --2- 172.21.15.12:0/3096769328 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f58f0039160 0x7f58f003b620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:11.103 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.535+0000 7f590b639700 1 --2- 172.21.15.12:0/3096769328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59041087c0 0x7f590419bbc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:11.103 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.535+0000 7f590b639700 1 -- 172.21.15.12:0/3096769328 >> 172.21.15.12:0/3096769328 conn(0x7f5904102190 msgr2=0x7f5904102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:11.103 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.535+0000 7f590b639700 1 -- 172.21.15.12:0/3096769328 shutdown_connections 2024-05-09T09:16:11.103 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:10.535+0000 7f590b639700 1 -- 172.21.15.12:0/3096769328 wait complete. 2024-05-09T09:16:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T09:16:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-05-09T09:16:11.115 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T09:16:11.115 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-09T09:16:11.115 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-05-09T09:16:11.115 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-09T09:16:11.117 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph Dashboard is now available at: 2024-05-09T09:16:11.117 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:16:11.117 INFO:teuthology.orchestra.run.smithi012.stdout: URL: https://smithi012.front.sepia.ceph.com:8443/ 2024-05-09T09:16:11.117 INFO:teuthology.orchestra.run.smithi012.stdout: User: admin 2024-05-09T09:16:11.117 INFO:teuthology.orchestra.run.smithi012.stdout: Password: 1wmoc6iuo0 2024-05-09T09:16:11.117 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:16:11.118 INFO:teuthology.orchestra.run.smithi012.stdout:Saving cluster configuration to /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config directory 2024-05-09T09:16:11.118 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling autotune for osd_memory_target 2024-05-09T09:16:11.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:11 smithi012 ceph-mon[30140]: mgrmap e13: x(active, since 6s) 2024-05-09T09:16:12.331 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.553+0000 7f3a56183700 1 Processor -- start 2024-05-09T09:16:12.331 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.553+0000 7f3a56183700 1 -- start start 2024-05-09T09:16:12.331 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.554+0000 7f3a56183700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 0x7f3a50107050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.554+0000 7f3a56183700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3a50107620 con 0x7f3a50106c30 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.554+0000 7f3a4f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 0x7f3a50107050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.554+0000 7f3a4f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 0x7f3a50107050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41028/0 (socket says 172.21.15.12:41028) 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.554+0000 7f3a4f7fe700 1 -- 172.21.15.12:0/891074481 learned_addr learned my addr 172.21.15.12:0/891074481 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.554+0000 7f3a4f7fe700 1 -- 172.21.15.12:0/891074481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a50107e30 con 0x7f3a50106c30 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.554+0000 7f3a4f7fe700 1 --2- 172.21.15.12:0/891074481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 0x7f3a50107050 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009e80 tx=0x7f3a4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f640116bf7984b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.554+0000 7f3a4effd700 1 -- 172.21.15.12:0/891074481 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a40017070 con 0x7f3a50106c30 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.555+0000 7f3a4effd700 1 -- 172.21.15.12:0/891074481 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3a4000b8c0 con 0x7f3a50106c30 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.555+0000 7f3a4effd700 1 -- 172.21.15.12:0/891074481 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a40004520 con 0x7f3a50106c30 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.555+0000 7f3a56183700 1 -- 172.21.15.12:0/891074481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 msgr2=0x7f3a50107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.555+0000 7f3a56183700 1 --2- 172.21.15.12:0/891074481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 0x7f3a50107050 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009e80 tx=0x7f3a4000b330 comp rx=0 tx=0).stop 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.555+0000 7f3a56183700 1 -- 172.21.15.12:0/891074481 shutdown_connections 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.555+0000 7f3a56183700 1 --2- 172.21.15.12:0/891074481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 0x7f3a50107050 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:12.332 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.555+0000 7f3a56183700 1 -- 172.21.15.12:0/891074481 >> 172.21.15.12:0/891074481 conn(0x7f3a501021d0 msgr2=0x7f3a50104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.555+0000 7f3a56183700 1 -- 172.21.15.12:0/891074481 shutdown_connections 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.555+0000 7f3a56183700 1 -- 172.21.15.12:0/891074481 wait complete. 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.556+0000 7f3a56183700 1 Processor -- start 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.556+0000 7f3a56183700 1 -- start start 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.556+0000 7f3a56183700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 0x7f3a5019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.556+0000 7f3a56183700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5010b1e0 con 0x7f3a50106c30 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.556+0000 7f3a4f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 0x7f3a5019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.556+0000 7f3a4f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 0x7f3a5019bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41042/0 (socket says 172.21.15.12:41042) 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.556+0000 7f3a4f7fe700 1 -- 172.21.15.12:0/3813817867 learned_addr learned my addr 172.21.15.12:0/3813817867 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.556+0000 7f3a4f7fe700 1 -- 172.21.15.12:0/3813817867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5019c0e0 con 0x7f3a50106c30 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.557+0000 7f3a4f7fe700 1 --2- 172.21.15.12:0/3813817867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 0x7f3a5019bba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a40000f80 tx=0x7f3a4000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.557+0000 7f3a4d7fa700 1 -- 172.21.15.12:0/3813817867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a40017070 con 0x7f3a50106c30 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.557+0000 7f3a4d7fa700 1 -- 172.21.15.12:0/3813817867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3a40022070 con 0x7f3a50106c30 2024-05-09T09:16:12.333 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.557+0000 7f3a56183700 1 -- 172.21.15.12:0/3813817867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5019c370 con 0x7f3a50106c30 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.557+0000 7f3a4d7fa700 1 -- 172.21.15.12:0/3813817867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a4001d5b0 con 0x7f3a50106c30 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.557+0000 7f3a56183700 1 -- 172.21.15.12:0/3813817867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5019f060 con 0x7f3a50106c30 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.558+0000 7f3a4d7fa700 1 -- 172.21.15.12:0/3813817867 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3a40012990 con 0x7f3a50106c30 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.558+0000 7f3a56183700 1 -- 172.21.15.12:0/3813817867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a50063980 con 0x7f3a50106c30 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.558+0000 7f3a4d7fa700 1 --2- 172.21.15.12:0/3813817867 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f3a3c039200 0x7f3a3c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.558+0000 7f3a4d7fa700 1 -- 172.21.15.12:0/3813817867 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3a4000f050 con 0x7f3a50106c30 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.558+0000 7f3a46ffd700 1 --2- 172.21.15.12:0/3813817867 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f3a3c039200 0x7f3a3c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.558+0000 7f3a46ffd700 1 --2- 172.21.15.12:0/3813817867 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f3a3c039200 0x7f3a3c03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3a38006fd0 tx=0x7f3a38006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.563+0000 7f3a4d7fa700 1 -- 172.21.15.12:0/3813817867 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3a40024310 con 0x7f3a50106c30 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.769+0000 7f3a56183700 1 -- 172.21.15.12:0/3813817867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f3a5002de80 con 0x7f3a50106c30 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.769+0000 7f3a4d7fa700 1 -- 172.21.15.12:0/3813817867 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f3a4001bd80 con 0x7f3a50106c30 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.771+0000 7f3a56183700 1 -- 172.21.15.12:0/3813817867 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f3a3c039200 msgr2=0x7f3a3c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.771+0000 7f3a56183700 1 --2- 172.21.15.12:0/3813817867 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f3a3c039200 0x7f3a3c03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3a38006fd0 tx=0x7f3a38006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:12.334 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.771+0000 7f3a56183700 1 -- 172.21.15.12:0/3813817867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 msgr2=0x7f3a5019bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:12.335 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.771+0000 7f3a56183700 1 --2- 172.21.15.12:0/3813817867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 0x7f3a5019bba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a40000f80 tx=0x7f3a4000bd50 comp rx=0 tx=0).stop 2024-05-09T09:16:12.335 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.771+0000 7f3a56183700 1 -- 172.21.15.12:0/3813817867 shutdown_connections 2024-05-09T09:16:12.335 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.771+0000 7f3a56183700 1 --2- 172.21.15.12:0/3813817867 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f3a3c039200 0x7f3a3c03b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:12.335 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.771+0000 7f3a56183700 1 --2- 172.21.15.12:0/3813817867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a50106c30 0x7f3a5019bba0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:12.335 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.771+0000 7f3a56183700 1 -- 172.21.15.12:0/3813817867 >> 172.21.15.12:0/3813817867 conn(0x7f3a501021d0 msgr2=0x7f3a50102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:12.335 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.772+0000 7f3a56183700 1 -- 172.21.15.12:0/3813817867 shutdown_connections 2024-05-09T09:16:12.335 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:11.772+0000 7f3a56183700 1 -- 172.21.15.12:0/3813817867 wait complete. 2024-05-09T09:16:13.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.750+0000 7f7282225700 1 Processor -- start 2024-05-09T09:16:13.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.750+0000 7f7282225700 1 -- start start 2024-05-09T09:16:13.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.751+0000 7f7282225700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 0x7f727c10ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:13.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.751+0000 7f7282225700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f727c07bdf0 con 0x7f727c108780 2024-05-09T09:16:13.680 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.751+0000 7f727b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 0x7f727c10ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:13.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.751+0000 7f727b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 0x7f727c10ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38638/0 (socket says 172.21.15.12:38638) 2024-05-09T09:16:13.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.751+0000 7f727b7fe700 1 -- 172.21.15.12:0/758929715 learned_addr learned my addr 172.21.15.12:0/758929715 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:13.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.751+0000 7f727b7fe700 1 -- 172.21.15.12:0/758929715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c10b0b0 con 0x7f727c108780 2024-05-09T09:16:13.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.751+0000 7f727b7fe700 1 --2- 172.21.15.12:0/758929715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 0x7f727c10ab70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f726c009e80 tx=0x7f726c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9184e0350f264f21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:13.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.751+0000 7f727a7fc700 1 -- 172.21.15.12:0/758929715 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f726c017070 con 0x7f727c108780 2024-05-09T09:16:13.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.752+0000 7f727a7fc700 1 -- 172.21.15.12:0/758929715 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f726c00b8c0 con 0x7f727c108780 2024-05-09T09:16:13.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.752+0000 7f727a7fc700 1 -- 172.21.15.12:0/758929715 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f726c004520 con 0x7f727c108780 2024-05-09T09:16:13.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.752+0000 7f7282225700 1 -- 172.21.15.12:0/758929715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 msgr2=0x7f727c10ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:13.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.752+0000 7f7282225700 1 --2- 172.21.15.12:0/758929715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 0x7f727c10ab70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f726c009e80 tx=0x7f726c00b330 comp rx=0 tx=0).stop 2024-05-09T09:16:13.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.752+0000 7f7282225700 1 -- 172.21.15.12:0/758929715 shutdown_connections 2024-05-09T09:16:13.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.752+0000 7f7282225700 1 --2- 172.21.15.12:0/758929715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 0x7f727c10ab70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:13.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.752+0000 7f7282225700 1 -- 172.21.15.12:0/758929715 >> 172.21.15.12:0/758929715 conn(0x7f727c102170 msgr2=0x7f727c1045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:13.681 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.752+0000 7f7282225700 1 -- 172.21.15.12:0/758929715 shutdown_connections 2024-05-09T09:16:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.752+0000 7f7282225700 1 -- 172.21.15.12:0/758929715 wait complete. 2024-05-09T09:16:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.753+0000 7f7282225700 1 Processor -- start 2024-05-09T09:16:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.753+0000 7f7282225700 1 -- start start 2024-05-09T09:16:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.753+0000 7f7282225700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 0x7f727c19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.753+0000 7f7282225700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f727c10c110 con 0x7f727c108780 2024-05-09T09:16:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.753+0000 7f727b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 0x7f727c19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.753+0000 7f727b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 0x7f727c19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38646/0 (socket says 172.21.15.12:38646) 2024-05-09T09:16:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.753+0000 7f727b7fe700 1 -- 172.21.15.12:0/2298568614 learned_addr learned my addr 172.21.15.12:0/2298568614 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.754+0000 7f727b7fe700 1 -- 172.21.15.12:0/2298568614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c19c0c0 con 0x7f727c108780 2024-05-09T09:16:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.754+0000 7f727b7fe700 1 --2- 172.21.15.12:0/2298568614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 0x7f727c19bb80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f726c0042b0 tx=0x7f726c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.754+0000 7f7278ff9700 1 -- 172.21.15.12:0/2298568614 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f726c017070 con 0x7f727c108780 2024-05-09T09:16:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.754+0000 7f7278ff9700 1 -- 172.21.15.12:0/2298568614 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f726c022070 con 0x7f727c108780 2024-05-09T09:16:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.754+0000 7f7278ff9700 1 -- 172.21.15.12:0/2298568614 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f726c01d680 con 0x7f727c108780 2024-05-09T09:16:13.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.754+0000 7f7282225700 1 -- 172.21.15.12:0/2298568614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c19c350 con 0x7f727c108780 2024-05-09T09:16:13.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.754+0000 7f7282225700 1 -- 172.21.15.12:0/2298568614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c19f040 con 0x7f727c108780 2024-05-09T09:16:13.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.755+0000 7f7278ff9700 1 -- 172.21.15.12:0/2298568614 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f726c012430 con 0x7f727c108780 2024-05-09T09:16:13.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.755+0000 7f7282225700 1 -- 172.21.15.12:0/2298568614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727c063980 con 0x7f727c108780 2024-05-09T09:16:13.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.755+0000 7f7278ff9700 1 --2- 172.21.15.12:0/2298568614 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7264039260 0x7f726403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:13.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.755+0000 7f7278ff9700 1 -- 172.21.15.12:0/2298568614 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f726c00f050 con 0x7f727c108780 2024-05-09T09:16:13.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.756+0000 7f727affd700 1 --2- 172.21.15.12:0/2298568614 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7264039260 0x7f726403b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:13.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.756+0000 7f727affd700 1 --2- 172.21.15.12:0/2298568614 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7264039260 0x7f726403b720 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7270006fd0 tx=0x7f7270006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:13.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:12.760+0000 7f7278ff9700 1 -- 172.21.15.12:0/2298568614 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f726c022c60 con 0x7f727c108780 2024-05-09T09:16:13.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:13.078+0000 7f7282225700 1 -- 172.21.15.12:0/2298568614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f727c19f5f0 con 0x7f727c108780 2024-05-09T09:16:13.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:13.081+0000 7f7278ff9700 1 -- 172.21.15.12:0/2298568614 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f726c01bd80 con 0x7f727c108780 2024-05-09T09:16:13.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:13.083+0000 7f7282225700 1 -- 172.21.15.12:0/2298568614 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7264039260 msgr2=0x7f726403b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:13.683 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:13.083+0000 7f7282225700 1 --2- 172.21.15.12:0/2298568614 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7264039260 0x7f726403b720 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7270006fd0 tx=0x7f7270006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:13.083+0000 7f7282225700 1 -- 172.21.15.12:0/2298568614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 msgr2=0x7f727c19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:13.083+0000 7f7282225700 1 --2- 172.21.15.12:0/2298568614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 0x7f727c19bb80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f726c0042b0 tx=0x7f726c00bd50 comp rx=0 tx=0).stop 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:13.083+0000 7f7282225700 1 -- 172.21.15.12:0/2298568614 shutdown_connections 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:13.083+0000 7f7282225700 1 --2- 172.21.15.12:0/2298568614 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7264039260 0x7f726403b720 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:13.083+0000 7f7282225700 1 --2- 172.21.15.12:0/2298568614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f727c108780 0x7f727c19bb80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:13.083+0000 7f7282225700 1 -- 172.21.15.12:0/2298568614 >> 172.21.15.12:0/2298568614 conn(0x7f727c102170 msgr2=0x7f727c102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:13.083+0000 7f7282225700 1 -- 172.21.15.12:0/2298568614 shutdown_connections 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-09T09:16:13.083+0000 7f7282225700 1 -- 172.21.15.12:0/2298568614 wait complete. 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:16:13.684 INFO:teuthology.orchestra.run.smithi012.stdout:Or, if you are only running a single cluster on this host: 2024-05-09T09:16:13.685 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:16:13.685 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-09T09:16:13.685 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:16:13.685 INFO:teuthology.orchestra.run.smithi012.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-09T09:16:13.685 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:16:13.685 INFO:teuthology.orchestra.run.smithi012.stdout: ceph telemetry on 2024-05-09T09:16:13.685 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:16:13.685 INFO:teuthology.orchestra.run.smithi012.stdout:For more information see: 2024-05-09T09:16:13.685 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:16:13.685 INFO:teuthology.orchestra.run.smithi012.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-09T09:16:13.685 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:16:13.685 INFO:teuthology.orchestra.run.smithi012.stdout:Bootstrap complete. 2024-05-09T09:16:13.723 INFO:tasks.cephadm:Fetching config... 2024-05-09T09:16:13.724 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:16:13.724 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-09T09:16:13.741 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-09T09:16:13.742 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:16:13.742 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-09T09:16:13.799 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-09T09:16:13.799 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:16:13.799 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-09T09:16:13.866 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-09T09:16:13.866 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:16:13.866 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-09T09:16:13.923 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-09T09:16:13.923 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEreAC0MgbXwawxTqBnsofiiZrXHdPozHxaWgW43L6LSDRtHBS8XzZ61w9nvQh3ngjE21rdFKLj8L22AE3qaB+SlxSZQRramPs2cE4MeGUC9VFEr5Ff+yCR8NZsHe6mxJ2nEi+ufYOkOH/q5eSOIIqw4ahRk8lJwnKSMYnyIZ+dW7jvOsSK6qb1VB3VvNEBzp5tyCnfX9fRnQrF3N0lxpN0Ixw5wSLshKWnwFDm2iZv4VleQZzMdhzw0MYknLAEzapPfmkqJ8TGAcKmgXsGcJx2Bl4GA3FUBLLLCSMaaMNwI68iPoidcDh22Z9p+baaUQUvkO1p4kRWVE7XDQ8e0iUL+1PvIxJfMRO25Ofs2b1tuTQpLlMuIn+8no1oJF4uE7gKr2fpooWke5zjnDB5QTrqZ3sS9prrSGVWRgcYa3n8bguMDklRlyP3UTfYUuxk7Fw18TYQ5g3FLPTB74c2Xa78VdvSv5w9JYin5SO9M+OEwr1T7LeMKPDxOlQrh4s7Pc= ceph-770e12d0-0de4-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T09:16:14.016 INFO:teuthology.orchestra.run.smithi012.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEreAC0MgbXwawxTqBnsofiiZrXHdPozHxaWgW43L6LSDRtHBS8XzZ61w9nvQh3ngjE21rdFKLj8L22AE3qaB+SlxSZQRramPs2cE4MeGUC9VFEr5Ff+yCR8NZsHe6mxJ2nEi+ufYOkOH/q5eSOIIqw4ahRk8lJwnKSMYnyIZ+dW7jvOsSK6qb1VB3VvNEBzp5tyCnfX9fRnQrF3N0lxpN0Ixw5wSLshKWnwFDm2iZv4VleQZzMdhzw0MYknLAEzapPfmkqJ8TGAcKmgXsGcJx2Bl4GA3FUBLLLCSMaaMNwI68iPoidcDh22Z9p+baaUQUvkO1p4kRWVE7XDQ8e0iUL+1PvIxJfMRO25Ofs2b1tuTQpLlMuIn+8no1oJF4uE7gKr2fpooWke5zjnDB5QTrqZ3sS9prrSGVWRgcYa3n8bguMDklRlyP3UTfYUuxk7Fw18TYQ5g3FLPTB74c2Xa78VdvSv5w9JYin5SO9M+OEwr1T7LeMKPDxOlQrh4s7Pc= ceph-770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:16:14.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEreAC0MgbXwawxTqBnsofiiZrXHdPozHxaWgW43L6LSDRtHBS8XzZ61w9nvQh3ngjE21rdFKLj8L22AE3qaB+SlxSZQRramPs2cE4MeGUC9VFEr5Ff+yCR8NZsHe6mxJ2nEi+ufYOkOH/q5eSOIIqw4ahRk8lJwnKSMYnyIZ+dW7jvOsSK6qb1VB3VvNEBzp5tyCnfX9fRnQrF3N0lxpN0Ixw5wSLshKWnwFDm2iZv4VleQZzMdhzw0MYknLAEzapPfmkqJ8TGAcKmgXsGcJx2Bl4GA3FUBLLLCSMaaMNwI68iPoidcDh22Z9p+baaUQUvkO1p4kRWVE7XDQ8e0iUL+1PvIxJfMRO25Ofs2b1tuTQpLlMuIn+8no1oJF4uE7gKr2fpooWke5zjnDB5QTrqZ3sS9prrSGVWRgcYa3n8bguMDklRlyP3UTfYUuxk7Fw18TYQ5g3FLPTB74c2Xa78VdvSv5w9JYin5SO9M+OEwr1T7LeMKPDxOlQrh4s7Pc= ceph-770e12d0-0de4-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T09:16:14.078 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEreAC0MgbXwawxTqBnsofiiZrXHdPozHxaWgW43L6LSDRtHBS8XzZ61w9nvQh3ngjE21rdFKLj8L22AE3qaB+SlxSZQRramPs2cE4MeGUC9VFEr5Ff+yCR8NZsHe6mxJ2nEi+ufYOkOH/q5eSOIIqw4ahRk8lJwnKSMYnyIZ+dW7jvOsSK6qb1VB3VvNEBzp5tyCnfX9fRnQrF3N0lxpN0Ixw5wSLshKWnwFDm2iZv4VleQZzMdhzw0MYknLAEzapPfmkqJ8TGAcKmgXsGcJx2Bl4GA3FUBLLLCSMaaMNwI68iPoidcDh22Z9p+baaUQUvkO1p4kRWVE7XDQ8e0iUL+1PvIxJfMRO25Ofs2b1tuTQpLlMuIn+8no1oJF4uE7gKr2fpooWke5zjnDB5QTrqZ3sS9prrSGVWRgcYa3n8bguMDklRlyP3UTfYUuxk7Fw18TYQ5g3FLPTB74c2Xa78VdvSv5w9JYin5SO9M+OEwr1T7LeMKPDxOlQrh4s7Pc= ceph-770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:16:14.091 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEreAC0MgbXwawxTqBnsofiiZrXHdPozHxaWgW43L6LSDRtHBS8XzZ61w9nvQh3ngjE21rdFKLj8L22AE3qaB+SlxSZQRramPs2cE4MeGUC9VFEr5Ff+yCR8NZsHe6mxJ2nEi+ufYOkOH/q5eSOIIqw4ahRk8lJwnKSMYnyIZ+dW7jvOsSK6qb1VB3VvNEBzp5tyCnfX9fRnQrF3N0lxpN0Ixw5wSLshKWnwFDm2iZv4VleQZzMdhzw0MYknLAEzapPfmkqJ8TGAcKmgXsGcJx2Bl4GA3FUBLLLCSMaaMNwI68iPoidcDh22Z9p+baaUQUvkO1p4kRWVE7XDQ8e0iUL+1PvIxJfMRO25Ofs2b1tuTQpLlMuIn+8no1oJF4uE7gKr2fpooWke5zjnDB5QTrqZ3sS9prrSGVWRgcYa3n8bguMDklRlyP3UTfYUuxk7Fw18TYQ5g3FLPTB74c2Xa78VdvSv5w9JYin5SO9M+OEwr1T7LeMKPDxOlQrh4s7Pc= ceph-770e12d0-0de4-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T09:16:14.133 INFO:teuthology.orchestra.run.smithi170.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEreAC0MgbXwawxTqBnsofiiZrXHdPozHxaWgW43L6LSDRtHBS8XzZ61w9nvQh3ngjE21rdFKLj8L22AE3qaB+SlxSZQRramPs2cE4MeGUC9VFEr5Ff+yCR8NZsHe6mxJ2nEi+ufYOkOH/q5eSOIIqw4ahRk8lJwnKSMYnyIZ+dW7jvOsSK6qb1VB3VvNEBzp5tyCnfX9fRnQrF3N0lxpN0Ixw5wSLshKWnwFDm2iZv4VleQZzMdhzw0MYknLAEzapPfmkqJ8TGAcKmgXsGcJx2Bl4GA3FUBLLLCSMaaMNwI68iPoidcDh22Z9p+baaUQUvkO1p4kRWVE7XDQ8e0iUL+1PvIxJfMRO25Ofs2b1tuTQpLlMuIn+8no1oJF4uE7gKr2fpooWke5zjnDB5QTrqZ3sS9prrSGVWRgcYa3n8bguMDklRlyP3UTfYUuxk7Fw18TYQ5g3FLPTB74c2Xa78VdvSv5w9JYin5SO9M+OEwr1T7LeMKPDxOlQrh4s7Pc= ceph-770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:16:14.145 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-09T09:16:14.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:14 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2298568614' entity='client.admin' 2024-05-09T09:16:14.395 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:16:17.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.018+0000 7fe273024700 1 -- 172.21.15.12:0/613565197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe26c101dc0 msgr2=0x7fe26c1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:17.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.019+0000 7fe273024700 1 --2- 172.21.15.12:0/613565197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe26c101dc0 0x7fe26c1021e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe260009910 tx=0x7fe260009c20 comp rx=0 tx=0).stop 2024-05-09T09:16:17.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.019+0000 7fe273024700 1 -- 172.21.15.12:0/613565197 shutdown_connections 2024-05-09T09:16:17.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.019+0000 7fe273024700 1 --2- 172.21.15.12:0/613565197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe26c101dc0 0x7fe26c1021e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:17.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.019+0000 7fe273024700 1 -- 172.21.15.12:0/613565197 >> 172.21.15.12:0/613565197 conn(0x7fe26c0fd340 msgr2=0x7fe26c0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:17.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.019+0000 7fe273024700 1 -- 172.21.15.12:0/613565197 shutdown_connections 2024-05-09T09:16:17.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.019+0000 7fe273024700 1 -- 172.21.15.12:0/613565197 wait complete. 2024-05-09T09:16:17.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.019+0000 7fe273024700 1 Processor -- start 2024-05-09T09:16:17.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.019+0000 7fe273024700 1 -- start start 2024-05-09T09:16:17.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.020+0000 7fe273024700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe26c101dc0 0x7fe26c197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:17.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.020+0000 7fe273024700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c107e40 con 0x7fe26c101dc0 2024-05-09T09:16:17.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.020+0000 7fe270dc0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe26c101dc0 0x7fe26c197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:17.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.020+0000 7fe270dc0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe26c101dc0 0x7fe26c197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38668/0 (socket says 172.21.15.12:38668) 2024-05-09T09:16:17.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.020+0000 7fe270dc0700 1 -- 172.21.15.12:0/1114977930 learned_addr learned my addr 172.21.15.12:0/1114977930 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:17.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.020+0000 7fe270dc0700 1 -- 172.21.15.12:0/1114977930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c197990 con 0x7fe26c101dc0 2024-05-09T09:16:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.021+0000 7fe270dc0700 1 --2- 172.21.15.12:0/1114977930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe26c101dc0 0x7fe26c197450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe260004f80 tx=0x7fe260009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.021+0000 7fe269ffb700 1 -- 172.21.15.12:0/1114977930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe260027070 con 0x7fe26c101dc0 2024-05-09T09:16:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.021+0000 7fe269ffb700 1 -- 172.21.15.12:0/1114977930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe26000be30 con 0x7fe26c101dc0 2024-05-09T09:16:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.021+0000 7fe269ffb700 1 -- 172.21.15.12:0/1114977930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe26000f460 con 0x7fe26c101dc0 2024-05-09T09:16:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.021+0000 7fe273024700 1 -- 172.21.15.12:0/1114977930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c197c20 con 0x7fe26c101dc0 2024-05-09T09:16:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.022+0000 7fe273024700 1 -- 172.21.15.12:0/1114977930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c074550 con 0x7fe26c101dc0 2024-05-09T09:16:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.022+0000 7fe269ffb700 1 -- 172.21.15.12:0/1114977930 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe26002a070 con 0x7fe26c101dc0 2024-05-09T09:16:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.023+0000 7fe269ffb700 1 --2- 172.21.15.12:0/1114977930 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe25c039150 0x7fe25c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.023+0000 7fe269ffb700 1 -- 172.21.15.12:0/1114977930 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe26001e030 con 0x7fe26c101dc0 2024-05-09T09:16:17.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.023+0000 7fe273024700 1 -- 172.21.15.12:0/1114977930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26c1085c0 con 0x7fe26c101dc0 2024-05-09T09:16:17.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.023+0000 7fe26bfff700 1 --2- 172.21.15.12:0/1114977930 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe25c039150 0x7fe25c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:17.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.023+0000 7fe26bfff700 1 --2- 172.21.15.12:0/1114977930 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe25c039150 0x7fe25c03b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe258006fd0 tx=0x7fe258006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:17.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.028+0000 7fe269ffb700 1 -- 172.21.15.12:0/1114977930 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe260013070 con 0x7fe26c101dc0 2024-05-09T09:16:17.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.237+0000 7fe273024700 1 -- 172.21.15.12:0/1114977930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fe26c02db80 con 0x7fe26c101dc0 2024-05-09T09:16:17.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.240+0000 7fe269ffb700 1 -- 172.21.15.12:0/1114977930 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fe2600292c0 con 0x7fe26c101dc0 2024-05-09T09:16:17.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.243+0000 7fe273024700 1 -- 172.21.15.12:0/1114977930 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe25c039150 msgr2=0x7fe25c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:17.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.243+0000 7fe273024700 1 --2- 172.21.15.12:0/1114977930 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe25c039150 0x7fe25c03b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe258006fd0 tx=0x7fe258006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:17.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.243+0000 7fe273024700 1 -- 172.21.15.12:0/1114977930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe26c101dc0 msgr2=0x7fe26c197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:17.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.243+0000 7fe273024700 1 --2- 172.21.15.12:0/1114977930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe26c101dc0 0x7fe26c197450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe260004f80 tx=0x7fe260009f90 comp rx=0 tx=0).stop 2024-05-09T09:16:17.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.243+0000 7fe273024700 1 -- 172.21.15.12:0/1114977930 shutdown_connections 2024-05-09T09:16:17.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.243+0000 7fe273024700 1 --2- 172.21.15.12:0/1114977930 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe25c039150 0x7fe25c03b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:17.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.243+0000 7fe273024700 1 --2- 172.21.15.12:0/1114977930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe26c101dc0 0x7fe26c197450 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:17.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.243+0000 7fe273024700 1 -- 172.21.15.12:0/1114977930 >> 172.21.15.12:0/1114977930 conn(0x7fe26c0fd340 msgr2=0x7fe26c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:17.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.243+0000 7fe273024700 1 -- 172.21.15.12:0/1114977930 shutdown_connections 2024-05-09T09:16:17.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:17.243+0000 7fe273024700 1 -- 172.21.15.12:0/1114977930 wait complete. 2024-05-09T09:16:17.931 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-09T09:16:17.932 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-09T09:16:18.371 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:16:18.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:18 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1114977930' entity='client.admin' 2024-05-09T09:16:19.624 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:19 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:19.624 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:19 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:19.624 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:19 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi012", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:19.625 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:19 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:19.625 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:19 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:16:19.625 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:19 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:19.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.985+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3714957961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5cc100cd0 msgr2=0x7fc5cc1010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:19.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.985+0000 7fc5d2a91700 1 --2- 172.21.15.12:0/3714957961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5cc100cd0 0x7fc5cc1010f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009910 tx=0x7fc5c0009c20 comp rx=0 tx=0).stop 2024-05-09T09:16:19.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.985+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3714957961 shutdown_connections 2024-05-09T09:16:19.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.985+0000 7fc5d2a91700 1 --2- 172.21.15.12:0/3714957961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5cc100cd0 0x7fc5cc1010f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:19.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.985+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3714957961 >> 172.21.15.12:0/3714957961 conn(0x7fc5cc0fc230 msgr2=0x7fc5cc0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:19.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.986+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3714957961 shutdown_connections 2024-05-09T09:16:19.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.986+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3714957961 wait complete. 2024-05-09T09:16:19.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.986+0000 7fc5d2a91700 1 Processor -- start 2024-05-09T09:16:19.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.986+0000 7fc5d2a91700 1 -- start start 2024-05-09T09:16:19.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.986+0000 7fc5d2a91700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5cc100cd0 0x7fc5cc06ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:19.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.986+0000 7fc5d2a91700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc106d50 con 0x7fc5cc100cd0 2024-05-09T09:16:19.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.987+0000 7fc5d082d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5cc100cd0 0x7fc5cc06ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:19.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.987+0000 7fc5d082d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5cc100cd0 0x7fc5cc06ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38684/0 (socket says 172.21.15.12:38684) 2024-05-09T09:16:19.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.987+0000 7fc5d082d700 1 -- 172.21.15.12:0/3807856650 learned_addr learned my addr 172.21.15.12:0/3807856650 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:19.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.987+0000 7fc5d082d700 1 -- 172.21.15.12:0/3807856650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc19a190 con 0x7fc5cc100cd0 2024-05-09T09:16:19.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.987+0000 7fc5d082d700 1 --2- 172.21.15.12:0/3807856650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5cc100cd0 0x7fc5cc06ac70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0005950 tx=0x7fc5c0009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:19.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.987+0000 7fc5c9ffb700 1 -- 172.21.15.12:0/3807856650 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5c0027070 con 0x7fc5cc100cd0 2024-05-09T09:16:19.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.987+0000 7fc5c9ffb700 1 -- 172.21.15.12:0/3807856650 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc5c000be30 con 0x7fc5cc100cd0 2024-05-09T09:16:19.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.987+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3807856650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc19a360 con 0x7fc5cc100cd0 2024-05-09T09:16:19.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.987+0000 7fc5c9ffb700 1 -- 172.21.15.12:0/3807856650 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5c000f460 con 0x7fc5cc100cd0 2024-05-09T09:16:19.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.987+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3807856650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc19a780 con 0x7fc5cc100cd0 2024-05-09T09:16:19.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.988+0000 7fc5c9ffb700 1 -- 172.21.15.12:0/3807856650 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc5c002a070 con 0x7fc5cc100cd0 2024-05-09T09:16:19.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.988+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3807856650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5cc1074d0 con 0x7fc5cc100cd0 2024-05-09T09:16:19.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.989+0000 7fc5c9ffb700 1 --2- 172.21.15.12:0/3807856650 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc5bc0390f0 0x7fc5bc03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:19.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.989+0000 7fc5c9ffb700 1 -- 172.21.15.12:0/3807856650 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc5c001e030 con 0x7fc5cc100cd0 2024-05-09T09:16:19.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.989+0000 7fc5cbfff700 1 --2- 172.21.15.12:0/3807856650 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc5bc0390f0 0x7fc5bc03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:19.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.989+0000 7fc5cbfff700 1 --2- 172.21.15.12:0/3807856650 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc5bc0390f0 0x7fc5bc03b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8006fd0 tx=0x7fc5b8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:19.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:19.994+0000 7fc5c9ffb700 1 -- 172.21.15.12:0/3807856650 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc5c0013070 con 0x7fc5cc100cd0 2024-05-09T09:16:20.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:20.203+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3807856650 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fc5cc02dd40 con 0x7fc5bc0390f0 2024-05-09T09:16:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:20.206+0000 7fc5c9ffb700 1 -- 172.21.15.12:0/3807856650 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc5cc02dd40 con 0x7fc5bc0390f0 2024-05-09T09:16:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:20.208+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3807856650 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc5bc0390f0 msgr2=0x7fc5bc03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:20.208+0000 7fc5d2a91700 1 --2- 172.21.15.12:0/3807856650 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc5bc0390f0 0x7fc5bc03b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8006fd0 tx=0x7fc5b8006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:20.208+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3807856650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5cc100cd0 msgr2=0x7fc5cc06ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:20.208+0000 7fc5d2a91700 1 --2- 172.21.15.12:0/3807856650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5cc100cd0 0x7fc5cc06ac70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0005950 tx=0x7fc5c0009f10 comp rx=0 tx=0).stop 2024-05-09T09:16:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:20.208+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3807856650 shutdown_connections 2024-05-09T09:16:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:20.208+0000 7fc5d2a91700 1 --2- 172.21.15.12:0/3807856650 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc5bc0390f0 0x7fc5bc03b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:20.208+0000 7fc5d2a91700 1 --2- 172.21.15.12:0/3807856650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5cc100cd0 0x7fc5cc06ac70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:20.208+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3807856650 >> 172.21.15.12:0/3807856650 conn(0x7fc5cc0fc230 msgr2=0x7fc5cc0fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:20.208+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3807856650 shutdown_connections 2024-05-09T09:16:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:20.209+0000 7fc5d2a91700 1 -- 172.21.15.12:0/3807856650 wait complete. 2024-05-09T09:16:21.026 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi016 2024-05-09T09:16:21.026 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-09T09:16:21.026 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/etc/ceph/ceph.conf 2024-05-09T09:16:21.046 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-09T09:16:21.046 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:16:21.105 INFO:tasks.cephadm:Adding host smithi016 to orchestrator... 2024-05-09T09:16:21.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch host add smithi016 2024-05-09T09:16:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:16:21.468 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:21 smithi012 ceph-mon[30140]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:16:21.468 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:21.468 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:16:21.468 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:21.468 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:16:21.468 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:21 smithi012 ceph-mon[30140]: Updating smithi012:/etc/ceph/ceph.conf 2024-05-09T09:16:22.471 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:22 smithi012 ceph-mon[30140]: Updating smithi012:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:16:22.471 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:22 smithi012 ceph-mon[30140]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:16:22.471 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:22 smithi012 ceph-mon[30140]: Updating smithi012:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T09:16:22.471 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:22 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:22.471 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:22 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:22.471 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:22 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:22.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.867+0000 7fc3c92a4700 1 -- 172.21.15.12:0/3988487942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3c4107dc0 msgr2=0x7fc3c41081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:22.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.867+0000 7fc3c92a4700 1 --2- 172.21.15.12:0/3988487942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3c4107dc0 0x7fc3c41081a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009910 tx=0x7fc3ac009c20 comp rx=0 tx=0).stop 2024-05-09T09:16:22.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.868+0000 7fc3c92a4700 1 -- 172.21.15.12:0/3988487942 shutdown_connections 2024-05-09T09:16:22.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.868+0000 7fc3c92a4700 1 --2- 172.21.15.12:0/3988487942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3c4107dc0 0x7fc3c41081a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:22.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.868+0000 7fc3c92a4700 1 -- 172.21.15.12:0/3988487942 >> 172.21.15.12:0/3988487942 conn(0x7fc3c4076780 msgr2=0x7fc3c4076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:22.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.868+0000 7fc3c92a4700 1 -- 172.21.15.12:0/3988487942 shutdown_connections 2024-05-09T09:16:22.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.868+0000 7fc3c92a4700 1 -- 172.21.15.12:0/3988487942 wait complete. 2024-05-09T09:16:22.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.868+0000 7fc3c92a4700 1 Processor -- start 2024-05-09T09:16:22.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.869+0000 7fc3c92a4700 1 -- start start 2024-05-09T09:16:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.869+0000 7fc3c92a4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3c4107dc0 0x7fc3c419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.869+0000 7fc3c92a4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c410c340 con 0x7fc3c4107dc0 2024-05-09T09:16:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.869+0000 7fc3c2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3c4107dc0 0x7fc3c419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.869+0000 7fc3c2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3c4107dc0 0x7fc3c419d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59794/0 (socket says 172.21.15.12:59794) 2024-05-09T09:16:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.869+0000 7fc3c2ffd700 1 -- 172.21.15.12:0/2093161668 learned_addr learned my addr 172.21.15.12:0/2093161668 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.869+0000 7fc3c2ffd700 1 -- 172.21.15.12:0/2093161668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c419db20 con 0x7fc3c4107dc0 2024-05-09T09:16:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.869+0000 7fc3c2ffd700 1 --2- 172.21.15.12:0/2093161668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3c4107dc0 0x7fc3c419d5e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac006010 tx=0x7fc3ac005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.870+0000 7fc3bbfff700 1 -- 172.21.15.12:0/2093161668 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3ac026070 con 0x7fc3c4107dc0 2024-05-09T09:16:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.870+0000 7fc3bbfff700 1 -- 172.21.15.12:0/2093161668 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc3ac00bbc0 con 0x7fc3c4107dc0 2024-05-09T09:16:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.870+0000 7fc3c92a4700 1 -- 172.21.15.12:0/2093161668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c419de10 con 0x7fc3c4107dc0 2024-05-09T09:16:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.870+0000 7fc3bbfff700 1 -- 172.21.15.12:0/2093161668 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3ac0038a0 con 0x7fc3c4107dc0 2024-05-09T09:16:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.870+0000 7fc3c92a4700 1 -- 172.21.15.12:0/2093161668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c41978d0 con 0x7fc3c4107dc0 2024-05-09T09:16:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.870+0000 7fc3bbfff700 1 -- 172.21.15.12:0/2093161668 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc3ac029070 con 0x7fc3c4107dc0 2024-05-09T09:16:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.871+0000 7fc3c92a4700 1 -- 172.21.15.12:0/2093161668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c4103470 con 0x7fc3c4107dc0 2024-05-09T09:16:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.871+0000 7fc3bbfff700 1 --2- 172.21.15.12:0/2093161668 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc3b00390f0 0x7fc3b003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.871+0000 7fc3bbfff700 1 -- 172.21.15.12:0/2093161668 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc3ac01e070 con 0x7fc3c4107dc0 2024-05-09T09:16:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.871+0000 7fc3c27fc700 1 --2- 172.21.15.12:0/2093161668 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc3b00390f0 0x7fc3b003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.871+0000 7fc3c27fc700 1 --2- 172.21.15.12:0/2093161668 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc3b00390f0 0x7fc3b003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4006fd0 tx=0x7fc3b4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:22.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:22.876+0000 7fc3bbfff700 1 -- 172.21.15.12:0/2093161668 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc3ac013070 con 0x7fc3c4107dc0 2024-05-09T09:16:23.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:23.085+0000 7fc3c92a4700 1 -- 172.21.15.12:0/2093161668 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi016", "target": ["mon-mgr", ""]}) -- 0x7fc3c4198c10 con 0x7fc3b00390f0 2024-05-09T09:16:23.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:23 smithi012 ceph-mon[30140]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:16:24.733 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:24 smithi012 ceph-mon[30140]: Deploying cephadm binary to smithi016 2024-05-09T09:16:25.733 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:25 smithi012 ceph-mon[30140]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:27.428+0000 7fc3bbfff700 1 -- 172.21.15.12:0/2093161668 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc3c4198c10 con 0x7fc3b00390f0 2024-05-09T09:16:27.427 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi016' with addr '172.21.15.16' 2024-05-09T09:16:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:27.431+0000 7fc3c92a4700 1 -- 172.21.15.12:0/2093161668 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc3b00390f0 msgr2=0x7fc3b003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:27.431+0000 7fc3c92a4700 1 --2- 172.21.15.12:0/2093161668 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc3b00390f0 0x7fc3b003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4006fd0 tx=0x7fc3b4006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:27.431+0000 7fc3c92a4700 1 -- 172.21.15.12:0/2093161668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3c4107dc0 msgr2=0x7fc3c419d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:27.431+0000 7fc3c92a4700 1 --2- 172.21.15.12:0/2093161668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3c4107dc0 0x7fc3c419d5e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac006010 tx=0x7fc3ac005810 comp rx=0 tx=0).stop 2024-05-09T09:16:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:27.431+0000 7fc3c92a4700 1 -- 172.21.15.12:0/2093161668 shutdown_connections 2024-05-09T09:16:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:27.431+0000 7fc3c92a4700 1 --2- 172.21.15.12:0/2093161668 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc3b00390f0 0x7fc3b003b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:27.431+0000 7fc3c92a4700 1 --2- 172.21.15.12:0/2093161668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3c4107dc0 0x7fc3c419d5e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:27.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:27.431+0000 7fc3c92a4700 1 -- 172.21.15.12:0/2093161668 >> 172.21.15.12:0/2093161668 conn(0x7fc3c4076780 msgr2=0x7fc3c40fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:27.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:27.431+0000 7fc3c92a4700 1 -- 172.21.15.12:0/2093161668 shutdown_connections 2024-05-09T09:16:27.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:27.431+0000 7fc3c92a4700 1 -- 172.21.15.12:0/2093161668 wait complete. 2024-05-09T09:16:27.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:27 smithi012 ceph-mon[30140]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:27.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:27 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:27.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:27 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:16:28.040 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T09:16:28.294 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:16:29.354 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:29 smithi012 ceph-mon[30140]: Added host smithi016 2024-05-09T09:16:29.354 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:29.354 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:29.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.872+0000 7fc7485ce700 1 -- 172.21.15.12:0/1873971710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc740100ed0 msgr2=0x7fc7401012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:29.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.872+0000 7fc7485ce700 1 --2- 172.21.15.12:0/1873971710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc740100ed0 0x7fc7401012b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009910 tx=0x7fc73c009c20 comp rx=0 tx=0).stop 2024-05-09T09:16:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.872+0000 7fc7485ce700 1 -- 172.21.15.12:0/1873971710 shutdown_connections 2024-05-09T09:16:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.872+0000 7fc7485ce700 1 --2- 172.21.15.12:0/1873971710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc740100ed0 0x7fc7401012b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.872+0000 7fc7485ce700 1 -- 172.21.15.12:0/1873971710 >> 172.21.15.12:0/1873971710 conn(0x7fc740076780 msgr2=0x7fc740076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.872+0000 7fc7485ce700 1 -- 172.21.15.12:0/1873971710 shutdown_connections 2024-05-09T09:16:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.872+0000 7fc7485ce700 1 -- 172.21.15.12:0/1873971710 wait complete. 2024-05-09T09:16:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.873+0000 7fc7485ce700 1 Processor -- start 2024-05-09T09:16:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.873+0000 7fc7485ce700 1 -- start start 2024-05-09T09:16:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.873+0000 7fc7485ce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc740100ed0 0x7fc74019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.873+0000 7fc7485ce700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc74010b240 con 0x7fc740100ed0 2024-05-09T09:16:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.873+0000 7fc74636a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc740100ed0 0x7fc74019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.873+0000 7fc74636a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc740100ed0 0x7fc74019d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59814/0 (socket says 172.21.15.12:59814) 2024-05-09T09:16:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.873+0000 7fc74636a700 1 -- 172.21.15.12:0/1972419649 learned_addr learned my addr 172.21.15.12:0/1972419649 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.874+0000 7fc74636a700 1 -- 172.21.15.12:0/1972419649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74019db00 con 0x7fc740100ed0 2024-05-09T09:16:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.874+0000 7fc74636a700 1 --2- 172.21.15.12:0/1972419649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc740100ed0 0x7fc74019d5c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc73c006010 tx=0x7fc73c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.874+0000 7fc7377fe700 1 -- 172.21.15.12:0/1972419649 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc73c027070 con 0x7fc740100ed0 2024-05-09T09:16:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.874+0000 7fc7377fe700 1 -- 172.21.15.12:0/1972419649 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc73c00be30 con 0x7fc740100ed0 2024-05-09T09:16:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.874+0000 7fc7485ce700 1 -- 172.21.15.12:0/1972419649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc74019ddf0 con 0x7fc740100ed0 2024-05-09T09:16:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.874+0000 7fc7377fe700 1 -- 172.21.15.12:0/1972419649 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc73c022d70 con 0x7fc740100ed0 2024-05-09T09:16:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.874+0000 7fc7485ce700 1 -- 172.21.15.12:0/1972419649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc740197860 con 0x7fc740100ed0 2024-05-09T09:16:29.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.875+0000 7fc7377fe700 1 -- 172.21.15.12:0/1972419649 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc73c02a070 con 0x7fc740100ed0 2024-05-09T09:16:29.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.875+0000 7fc7485ce700 1 -- 172.21.15.12:0/1972419649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc740102360 con 0x7fc740100ed0 2024-05-09T09:16:29.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.875+0000 7fc7377fe700 1 --2- 172.21.15.12:0/1972419649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc72c0390f0 0x7fc72c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:29.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.875+0000 7fc7377fe700 1 -- 172.21.15.12:0/1972419649 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc73c01e030 con 0x7fc740100ed0 2024-05-09T09:16:29.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.875+0000 7fc745b69700 1 --2- 172.21.15.12:0/1972419649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc72c0390f0 0x7fc72c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.876+0000 7fc745b69700 1 --2- 172.21.15.12:0/1972419649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc72c0390f0 0x7fc72c03b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc730006fd0 tx=0x7fc730006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:29.880+0000 7fc7377fe700 1 -- 172.21.15.12:0/1972419649 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc73c013070 con 0x7fc740100ed0 2024-05-09T09:16:30.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:30.094+0000 7fc7485ce700 1 -- 172.21.15.12:0/1972419649 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc740198bf0 con 0x7fc72c0390f0 2024-05-09T09:16:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:30.095+0000 7fc7377fe700 1 -- 172.21.15.12:0/1972419649 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fc740198bf0 con 0x7fc72c0390f0 2024-05-09T09:16:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:16:30.094 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}] 2024-05-09T09:16:30.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:30.097+0000 7fc7485ce700 1 -- 172.21.15.12:0/1972419649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc72c0390f0 msgr2=0x7fc72c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:30.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:30.097+0000 7fc7485ce700 1 --2- 172.21.15.12:0/1972419649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc72c0390f0 0x7fc72c03b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc730006fd0 tx=0x7fc730006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:30.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:30.097+0000 7fc7485ce700 1 -- 172.21.15.12:0/1972419649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc740100ed0 msgr2=0x7fc74019d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:30.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:30.097+0000 7fc7485ce700 1 --2- 172.21.15.12:0/1972419649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc740100ed0 0x7fc74019d5c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc73c006010 tx=0x7fc73c005810 comp rx=0 tx=0).stop 2024-05-09T09:16:30.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:30.097+0000 7fc7485ce700 1 -- 172.21.15.12:0/1972419649 shutdown_connections 2024-05-09T09:16:30.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:30.097+0000 7fc7485ce700 1 --2- 172.21.15.12:0/1972419649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc72c0390f0 0x7fc72c03b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:30.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:30.097+0000 7fc7485ce700 1 --2- 172.21.15.12:0/1972419649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc740100ed0 0x7fc74019d5c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:30.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:30.097+0000 7fc7485ce700 1 -- 172.21.15.12:0/1972419649 >> 172.21.15.12:0/1972419649 conn(0x7fc740076780 msgr2=0x7fc740069740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:30.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:30.097+0000 7fc7485ce700 1 -- 172.21.15.12:0/1972419649 shutdown_connections 2024-05-09T09:16:30.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:30.097+0000 7fc7485ce700 1 -- 172.21.15.12:0/1972419649 wait complete. 2024-05-09T09:16:30.393 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:30 smithi012 ceph-mon[30140]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:30.688 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi170 2024-05-09T09:16:30.688 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-09T09:16:30.689 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.conf 2024-05-09T09:16:30.710 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-09T09:16:30.710 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:16:30.768 INFO:tasks.cephadm:Adding host smithi170 to orchestrator... 2024-05-09T09:16:30.769 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch host add smithi170 2024-05-09T09:16:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:16:31.413 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:31 smithi012 ceph-mon[30140]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:16:31.413 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:31 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:32.209 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:32 smithi012 ceph-mon[30140]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.428+0000 7fce24daa700 1 -- 172.21.15.12:0/1854335376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce20100ed0 msgr2=0x7fce201012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.428+0000 7fce24daa700 1 --2- 172.21.15.12:0/1854335376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce20100ed0 0x7fce201012b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fce10009910 tx=0x7fce10009c20 comp rx=0 tx=0).stop 2024-05-09T09:16:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.428+0000 7fce24daa700 1 -- 172.21.15.12:0/1854335376 shutdown_connections 2024-05-09T09:16:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.428+0000 7fce24daa700 1 --2- 172.21.15.12:0/1854335376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce20100ed0 0x7fce201012b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.428+0000 7fce24daa700 1 -- 172.21.15.12:0/1854335376 >> 172.21.15.12:0/1854335376 conn(0x7fce20076780 msgr2=0x7fce20076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.428+0000 7fce24daa700 1 -- 172.21.15.12:0/1854335376 shutdown_connections 2024-05-09T09:16:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.428+0000 7fce24daa700 1 -- 172.21.15.12:0/1854335376 wait complete. 2024-05-09T09:16:32.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.429+0000 7fce24daa700 1 Processor -- start 2024-05-09T09:16:32.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.429+0000 7fce24daa700 1 -- start start 2024-05-09T09:16:32.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.429+0000 7fce24daa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce20100ed0 0x7fce20102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:32.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.429+0000 7fce24daa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fce2010b240 con 0x7fce20100ed0 2024-05-09T09:16:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.429+0000 7fce1e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce20100ed0 0x7fce20102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.429+0000 7fce1e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce20100ed0 0x7fce20102920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59270/0 (socket says 172.21.15.12:59270) 2024-05-09T09:16:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.429+0000 7fce1e59c700 1 -- 172.21.15.12:0/1517188942 learned_addr learned my addr 172.21.15.12:0/1517188942 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.429+0000 7fce1e59c700 1 -- 172.21.15.12:0/1517188942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce20104810 con 0x7fce20100ed0 2024-05-09T09:16:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.430+0000 7fce1e59c700 1 --2- 172.21.15.12:0/1517188942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce20100ed0 0x7fce20102920 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fce10006010 tx=0x7fce10005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.430+0000 7fce0f7fe700 1 -- 172.21.15.12:0/1517188942 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce10027070 con 0x7fce20100ed0 2024-05-09T09:16:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.430+0000 7fce0f7fe700 1 -- 172.21.15.12:0/1517188942 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fce1000be30 con 0x7fce20100ed0 2024-05-09T09:16:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.430+0000 7fce24daa700 1 -- 172.21.15.12:0/1517188942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce20102ec0 con 0x7fce20100ed0 2024-05-09T09:16:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.430+0000 7fce0f7fe700 1 -- 172.21.15.12:0/1517188942 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce10022d70 con 0x7fce20100ed0 2024-05-09T09:16:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.430+0000 7fce24daa700 1 -- 172.21.15.12:0/1517188942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce20103380 con 0x7fce20100ed0 2024-05-09T09:16:32.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.431+0000 7fce0f7fe700 1 -- 172.21.15.12:0/1517188942 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fce1002a070 con 0x7fce20100ed0 2024-05-09T09:16:32.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.431+0000 7fce24daa700 1 -- 172.21.15.12:0/1517188942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce20102360 con 0x7fce20100ed0 2024-05-09T09:16:32.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.431+0000 7fce0f7fe700 1 --2- 172.21.15.12:0/1517188942 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fce080390f0 0x7fce0803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:32.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.431+0000 7fce0f7fe700 1 -- 172.21.15.12:0/1517188942 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fce1001e030 con 0x7fce20100ed0 2024-05-09T09:16:32.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.431+0000 7fce1dd9b700 1 --2- 172.21.15.12:0/1517188942 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fce080390f0 0x7fce0803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:32.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.431+0000 7fce1dd9b700 1 --2- 172.21.15.12:0/1517188942 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fce080390f0 0x7fce0803b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fce14006fd0 tx=0x7fce14006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:32.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.436+0000 7fce0f7fe700 1 -- 172.21.15.12:0/1517188942 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fce10013070 con 0x7fce20100ed0 2024-05-09T09:16:32.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:32.646+0000 7fce24daa700 1 -- 172.21.15.12:0/1517188942 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}) -- 0x7fce20103e50 con 0x7fce080390f0 2024-05-09T09:16:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:34 smithi012 ceph-mon[30140]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:34 smithi012 ceph-mon[30140]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:16:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:34 smithi012 ceph-mon[30140]: Deploying cephadm binary to smithi170 2024-05-09T09:16:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:16:35.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:35 smithi012 ceph-mon[30140]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-09T09:16:35.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:35 smithi012 ceph-mon[30140]: Updating smithi016:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:16:36.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:36 smithi012 ceph-mon[30140]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:36.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:36.892+0000 7fce0f7fe700 1 -- 172.21.15.12:0/1517188942 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fce20103e50 con 0x7fce080390f0 2024-05-09T09:16:36.892 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi170' with addr '172.21.15.170' 2024-05-09T09:16:36.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:36.894+0000 7fce24daa700 1 -- 172.21.15.12:0/1517188942 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fce080390f0 msgr2=0x7fce0803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:36.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:36.894+0000 7fce24daa700 1 --2- 172.21.15.12:0/1517188942 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fce080390f0 0x7fce0803b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fce14006fd0 tx=0x7fce14006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:36.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:36.894+0000 7fce24daa700 1 -- 172.21.15.12:0/1517188942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce20100ed0 msgr2=0x7fce20102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:36.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:36.894+0000 7fce24daa700 1 --2- 172.21.15.12:0/1517188942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce20100ed0 0x7fce20102920 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fce10006010 tx=0x7fce10005810 comp rx=0 tx=0).stop 2024-05-09T09:16:36.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:36.894+0000 7fce24daa700 1 -- 172.21.15.12:0/1517188942 shutdown_connections 2024-05-09T09:16:36.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:36.894+0000 7fce24daa700 1 --2- 172.21.15.12:0/1517188942 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fce080390f0 0x7fce0803b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:36.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:36.894+0000 7fce24daa700 1 --2- 172.21.15.12:0/1517188942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce20100ed0 0x7fce20102920 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:36.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:36.894+0000 7fce24daa700 1 -- 172.21.15.12:0/1517188942 >> 172.21.15.12:0/1517188942 conn(0x7fce20076780 msgr2=0x7fce20069740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:36.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:36.895+0000 7fce24daa700 1 -- 172.21.15.12:0/1517188942 shutdown_connections 2024-05-09T09:16:36.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:36.895+0000 7fce24daa700 1 -- 172.21.15.12:0/1517188942 wait complete. 2024-05-09T09:16:37.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:37 smithi012 ceph-mon[30140]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:16:37.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:37.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:37.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:37.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:37.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:16:37.450 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T09:16:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:16:38.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:38 smithi012 ceph-mon[30140]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:38 smithi012 ceph-mon[30140]: Updating smithi016:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T09:16:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:38 smithi012 ceph-mon[30140]: Added host smithi170 2024-05-09T09:16:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:39.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.241+0000 7f8ecfa21700 1 -- 172.21.15.12:0/54148945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8100ed0 msgr2=0x7f8ec81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:39.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.241+0000 7f8ecfa21700 1 --2- 172.21.15.12:0/54148945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8100ed0 0x7f8ec81012b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009910 tx=0x7f8ec4009c20 comp rx=0 tx=0).stop 2024-05-09T09:16:39.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.241+0000 7f8ecfa21700 1 -- 172.21.15.12:0/54148945 shutdown_connections 2024-05-09T09:16:39.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.241+0000 7f8ecfa21700 1 --2- 172.21.15.12:0/54148945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8100ed0 0x7f8ec81012b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:39.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.241+0000 7f8ecfa21700 1 -- 172.21.15.12:0/54148945 >> 172.21.15.12:0/54148945 conn(0x7f8ec8076780 msgr2=0x7f8ec8076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:39.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.241+0000 7f8ecfa21700 1 -- 172.21.15.12:0/54148945 shutdown_connections 2024-05-09T09:16:39.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.241+0000 7f8ecfa21700 1 -- 172.21.15.12:0/54148945 wait complete. 2024-05-09T09:16:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.242+0000 7f8ecfa21700 1 Processor -- start 2024-05-09T09:16:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.242+0000 7f8ecfa21700 1 -- start start 2024-05-09T09:16:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.242+0000 7f8ecfa21700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8100ed0 0x7f8ec8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.242+0000 7f8ecfa21700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec810b240 con 0x7f8ec8100ed0 2024-05-09T09:16:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.242+0000 7f8ecd7bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8100ed0 0x7f8ec8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.242+0000 7f8ecd7bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8100ed0 0x7f8ec8198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59290/0 (socket says 172.21.15.12:59290) 2024-05-09T09:16:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.243+0000 7f8ecd7bd700 1 -- 172.21.15.12:0/3124098276 learned_addr learned my addr 172.21.15.12:0/3124098276 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.243+0000 7f8ecd7bd700 1 -- 172.21.15.12:0/3124098276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec806a670 con 0x7f8ec8100ed0 2024-05-09T09:16:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.243+0000 7f8ecd7bd700 1 --2- 172.21.15.12:0/3124098276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8100ed0 0x7f8ec8198750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4006010 tx=0x7f8ec4005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:39.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.243+0000 7f8ebe7fc700 1 -- 172.21.15.12:0/3124098276 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ec4027070 con 0x7f8ec8100ed0 2024-05-09T09:16:39.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.243+0000 7f8ebe7fc700 1 -- 172.21.15.12:0/3124098276 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8ec400be30 con 0x7f8ec8100ed0 2024-05-09T09:16:39.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.243+0000 7f8ecfa21700 1 -- 172.21.15.12:0/3124098276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec8198cf0 con 0x7f8ec8100ed0 2024-05-09T09:16:39.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.243+0000 7f8ebe7fc700 1 -- 172.21.15.12:0/3124098276 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ec4022d70 con 0x7f8ec8100ed0 2024-05-09T09:16:39.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.243+0000 7f8ecfa21700 1 -- 172.21.15.12:0/3124098276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec81991b0 con 0x7f8ec8100ed0 2024-05-09T09:16:39.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.244+0000 7f8ebe7fc700 1 -- 172.21.15.12:0/3124098276 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8ec402a070 con 0x7f8ec8100ed0 2024-05-09T09:16:39.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.244+0000 7f8ecfa21700 1 -- 172.21.15.12:0/3124098276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec8102360 con 0x7f8ec8100ed0 2024-05-09T09:16:39.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.244+0000 7f8ebe7fc700 1 --2- 172.21.15.12:0/3124098276 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8eb4039150 0x7f8eb403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:39.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.244+0000 7f8ebe7fc700 1 -- 172.21.15.12:0/3124098276 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8ec401e030 con 0x7f8ec8100ed0 2024-05-09T09:16:39.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.244+0000 7f8eccfbc700 1 --2- 172.21.15.12:0/3124098276 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8eb4039150 0x7f8eb403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:39.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.245+0000 7f8eccfbc700 1 --2- 172.21.15.12:0/3124098276 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8eb4039150 0x7f8eb403b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8006fd0 tx=0x7f8eb8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.249+0000 7f8ebe7fc700 1 -- 172.21.15.12:0/3124098276 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8ec806a670 con 0x7f8ec8100ed0 2024-05-09T09:16:39.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.460+0000 7f8ecfa21700 1 -- 172.21.15.12:0/3124098276 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ec81994f0 con 0x7f8eb4039150 2024-05-09T09:16:39.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.461+0000 7f8ebe7fc700 1 -- 172.21.15.12:0/3124098276 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f8ec81994f0 con 0x7f8eb4039150 2024-05-09T09:16:39.461 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:16:39.461 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}] 2024-05-09T09:16:39.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.464+0000 7f8ecfa21700 1 -- 172.21.15.12:0/3124098276 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8eb4039150 msgr2=0x7f8eb403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:39.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.464+0000 7f8ecfa21700 1 --2- 172.21.15.12:0/3124098276 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8eb4039150 0x7f8eb403b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8006fd0 tx=0x7f8eb8006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:39.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.464+0000 7f8ecfa21700 1 -- 172.21.15.12:0/3124098276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8100ed0 msgr2=0x7f8ec8198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:39.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.464+0000 7f8ecfa21700 1 --2- 172.21.15.12:0/3124098276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8100ed0 0x7f8ec8198750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4006010 tx=0x7f8ec4005810 comp rx=0 tx=0).stop 2024-05-09T09:16:39.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.464+0000 7f8ecfa21700 1 -- 172.21.15.12:0/3124098276 shutdown_connections 2024-05-09T09:16:39.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.464+0000 7f8ecfa21700 1 --2- 172.21.15.12:0/3124098276 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8eb4039150 0x7f8eb403b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:39.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.464+0000 7f8ecfa21700 1 --2- 172.21.15.12:0/3124098276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8100ed0 0x7f8ec8198750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:39.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.464+0000 7f8ecfa21700 1 -- 172.21.15.12:0/3124098276 >> 172.21.15.12:0/3124098276 conn(0x7f8ec8076780 msgr2=0x7f8ec8102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:39.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.464+0000 7f8ecfa21700 1 -- 172.21.15.12:0/3124098276 shutdown_connections 2024-05-09T09:16:39.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:39.464+0000 7f8ecfa21700 1 -- 172.21.15.12:0/3124098276 wait complete. 2024-05-09T09:16:40.047 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-09T09:16:40.048 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-09T09:16:40.296 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:16:40.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:40 smithi012 ceph-mon[30140]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:40.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:40 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:41.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:41 smithi012 ceph-mon[30140]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:16:41.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.899+0000 7f6799dcd700 1 -- 172.21.15.12:0/3868992025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6794100ed0 msgr2=0x7f67941012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:41.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.899+0000 7f6799dcd700 1 --2- 172.21.15.12:0/3868992025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6794100ed0 0x7f67941012b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f678400b0a0 tx=0x7f678400b3b0 comp rx=0 tx=0).stop 2024-05-09T09:16:41.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.899+0000 7f6799dcd700 1 -- 172.21.15.12:0/3868992025 shutdown_connections 2024-05-09T09:16:41.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.899+0000 7f6799dcd700 1 --2- 172.21.15.12:0/3868992025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6794100ed0 0x7f67941012b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:41.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.899+0000 7f6799dcd700 1 -- 172.21.15.12:0/3868992025 >> 172.21.15.12:0/3868992025 conn(0x7f67940fc780 msgr2=0x7f67940feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:41.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.899+0000 7f6799dcd700 1 -- 172.21.15.12:0/3868992025 shutdown_connections 2024-05-09T09:16:41.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.899+0000 7f6799dcd700 1 -- 172.21.15.12:0/3868992025 wait complete. 2024-05-09T09:16:41.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.899+0000 7f6799dcd700 1 Processor -- start 2024-05-09T09:16:41.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.900+0000 7f6799dcd700 1 -- start start 2024-05-09T09:16:41.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.900+0000 7f6799dcd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6794100ed0 0x7f679419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:41.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.900+0000 7f6799dcd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f679410b290 con 0x7f6794100ed0 2024-05-09T09:16:41.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.900+0000 7f67937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6794100ed0 0x7f679419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:41.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.900+0000 7f67937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6794100ed0 0x7f679419d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56696/0 (socket says 172.21.15.12:56696) 2024-05-09T09:16:41.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.900+0000 7f67937fe700 1 -- 172.21.15.12:0/3334774078 learned_addr learned my addr 172.21.15.12:0/3334774078 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:16:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.900+0000 7f67937fe700 1 -- 172.21.15.12:0/3334774078 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679419dae0 con 0x7f6794100ed0 2024-05-09T09:16:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.900+0000 7f67937fe700 1 --2- 172.21.15.12:0/3334774078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6794100ed0 0x7f679419d5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6784015040 tx=0x7f67840078b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.901+0000 7f6790ff9700 1 -- 172.21.15.12:0/3334774078 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6784017050 con 0x7f6794100ed0 2024-05-09T09:16:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.901+0000 7f6790ff9700 1 -- 172.21.15.12:0/3334774078 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6784018040 con 0x7f6794100ed0 2024-05-09T09:16:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.901+0000 7f6799dcd700 1 -- 172.21.15.12:0/3334774078 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679419dd70 con 0x7f6794100ed0 2024-05-09T09:16:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.901+0000 7f6790ff9700 1 -- 172.21.15.12:0/3334774078 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f678401c440 con 0x7f6794100ed0 2024-05-09T09:16:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.901+0000 7f6799dcd700 1 -- 172.21.15.12:0/3334774078 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6794197670 con 0x7f6794100ed0 2024-05-09T09:16:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.902+0000 7f6799dcd700 1 -- 172.21.15.12:0/3334774078 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67941023b0 con 0x7f6794100ed0 2024-05-09T09:16:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.902+0000 7f6790ff9700 1 -- 172.21.15.12:0/3334774078 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6784003b30 con 0x7f6794100ed0 2024-05-09T09:16:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.902+0000 7f6790ff9700 1 --2- 172.21.15.12:0/3334774078 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6780039150 0x7f678003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.902+0000 7f6790ff9700 1 -- 172.21.15.12:0/3334774078 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f678400f050 con 0x7f6794100ed0 2024-05-09T09:16:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.902+0000 7f6792ffd700 1 --2- 172.21.15.12:0/3334774078 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6780039150 0x7f678003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:41.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.903+0000 7f6792ffd700 1 --2- 172.21.15.12:0/3334774078 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6780039150 0x7f678003b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f677c006fd0 tx=0x7f677c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:41.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:41.907+0000 7f6790ff9700 1 -- 172.21.15.12:0/3334774078 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f678401b9d0 con 0x7f6794100ed0 2024-05-09T09:16:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:42.120+0000 7f6799dcd700 1 -- 172.21.15.12:0/3334774078 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f6794198ba0 con 0x7f6794100ed0 2024-05-09T09:16:42.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:42.189+0000 7f6790ff9700 1 -- 172.21.15.12:0/3334774078 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f678401c6f0 con 0x7f6794100ed0 2024-05-09T09:16:42.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:42.192+0000 7f6799dcd700 1 -- 172.21.15.12:0/3334774078 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6780039150 msgr2=0x7f678003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:42.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:42.192+0000 7f6799dcd700 1 --2- 172.21.15.12:0/3334774078 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6780039150 0x7f678003b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f677c006fd0 tx=0x7f677c006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:42.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:42.192+0000 7f6799dcd700 1 -- 172.21.15.12:0/3334774078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6794100ed0 msgr2=0x7f679419d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:42.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:42.192+0000 7f6799dcd700 1 --2- 172.21.15.12:0/3334774078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6794100ed0 0x7f679419d5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6784015040 tx=0x7f67840078b0 comp rx=0 tx=0).stop 2024-05-09T09:16:42.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:42.192+0000 7f6799dcd700 1 -- 172.21.15.12:0/3334774078 shutdown_connections 2024-05-09T09:16:42.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:42.192+0000 7f6799dcd700 1 --2- 172.21.15.12:0/3334774078 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6780039150 0x7f678003b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:42.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:42.192+0000 7f6799dcd700 1 --2- 172.21.15.12:0/3334774078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6794100ed0 0x7f679419d5a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:42.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:42.192+0000 7f6799dcd700 1 -- 172.21.15.12:0/3334774078 >> 172.21.15.12:0/3334774078 conn(0x7f67940fc780 msgr2=0x7f679406a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:42.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:42.193+0000 7f6799dcd700 1 -- 172.21.15.12:0/3334774078 shutdown_connections 2024-05-09T09:16:42.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:16:42.193+0000 7f6799dcd700 1 -- 172.21.15.12:0/3334774078 wait complete. 2024-05-09T09:16:42.193 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-05-09T09:16:42.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:42 smithi012 ceph-mon[30140]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:42 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3334774078' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T09:16:42.793 INFO:tasks.cephadm:Adding mon.a on smithi012 2024-05-09T09:16:42.793 INFO:tasks.cephadm:Adding mon.b on smithi016 2024-05-09T09:16:42.794 INFO:tasks.cephadm:Adding mon.c on smithi170 2024-05-09T09:16:42.794 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi012:172.21.15.12=a;smithi016:172.21.15.16=b;smithi170:172.21.15.170=c' 2024-05-09T09:16:43.029 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:16:43.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:43 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3334774078' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T09:16:43.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:43 smithi012 ceph-mon[30140]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:16:43.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:43 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:43.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:43 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:43.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:43 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:43.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:43 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:43.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:43 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:43.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:43 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:43.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:43 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:16:44.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.443+0000 7fa336df8700 1 -- 172.21.15.170:0/815267528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa330100e90 msgr2=0x7fa330101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:44.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.443+0000 7fa336df8700 1 --2- 172.21.15.170:0/815267528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa330100e90 0x7fa330101270 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa32000b0a0 tx=0x7fa32000b3b0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.443+0000 7fa336df8700 1 -- 172.21.15.170:0/815267528 shutdown_connections 2024-05-09T09:16:44.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.443+0000 7fa336df8700 1 --2- 172.21.15.170:0/815267528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa330100e90 0x7fa330101270 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.443+0000 7fa336df8700 1 -- 172.21.15.170:0/815267528 >> 172.21.15.170:0/815267528 conn(0x7fa3300fc740 msgr2=0x7fa3300feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:44.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.443+0000 7fa336df8700 1 -- 172.21.15.170:0/815267528 shutdown_connections 2024-05-09T09:16:44.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.443+0000 7fa336df8700 1 -- 172.21.15.170:0/815267528 wait complete. 2024-05-09T09:16:44.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.443+0000 7fa336df8700 1 Processor -- start 2024-05-09T09:16:44.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.443+0000 7fa336df8700 1 -- start start 2024-05-09T09:16:44.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.443+0000 7fa336df8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa330100e90 0x7fa3301964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.444+0000 7fa336df8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa33010b250 con 0x7fa330100e90 2024-05-09T09:16:44.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.444+0000 7fa334b94700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa330100e90 0x7fa3301964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.444+0000 7fa334b94700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa330100e90 0x7fa3301964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.170:41658/0 (socket says 172.21.15.170:41658) 2024-05-09T09:16:44.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.444+0000 7fa334b94700 1 -- 172.21.15.170:0/1269407144 learned_addr learned my addr 172.21.15.170:0/1269407144 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-09T09:16:44.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.444+0000 7fa334b94700 1 -- 172.21.15.170:0/1269407144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33006afb0 con 0x7fa330100e90 2024-05-09T09:16:44.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.444+0000 7fa334b94700 1 --2- 172.21.15.170:0/1269407144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa330100e90 0x7fa3301964c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa320009030 tx=0x7fa32000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:44.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.444+0000 7fa32dffb700 1 -- 172.21.15.170:0/1269407144 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa320017050 con 0x7fa330100e90 2024-05-09T09:16:44.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.444+0000 7fa336df8700 1 -- 172.21.15.170:0/1269407144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa330196a00 con 0x7fa330100e90 2024-05-09T09:16:44.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.444+0000 7fa32dffb700 1 -- 172.21.15.170:0/1269407144 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa320018040 con 0x7fa330100e90 2024-05-09T09:16:44.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.444+0000 7fa32dffb700 1 -- 172.21.15.170:0/1269407144 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa32001c5d0 con 0x7fa330100e90 2024-05-09T09:16:44.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.445+0000 7fa336df8700 1 -- 172.21.15.170:0/1269407144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa330196ec0 con 0x7fa330100e90 2024-05-09T09:16:44.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.445+0000 7fa336df8700 1 -- 172.21.15.170:0/1269407144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa330102370 con 0x7fa330100e90 2024-05-09T09:16:44.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.446+0000 7fa32dffb700 1 -- 172.21.15.170:0/1269407144 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa320004040 con 0x7fa330100e90 2024-05-09T09:16:44.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.446+0000 7fa32dffb700 1 --2- 172.21.15.170:0/1269407144 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa31c039140 0x7fa31c03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.446+0000 7fa32dffb700 1 -- 172.21.15.170:0/1269407144 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa32000f050 con 0x7fa330100e90 2024-05-09T09:16:44.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.446+0000 7fa32ffff700 1 --2- 172.21.15.170:0/1269407144 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa31c039140 0x7fa31c03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.447+0000 7fa32ffff700 1 --2- 172.21.15.170:0/1269407144 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa31c039140 0x7fa31c03b600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa318006fd0 tx=0x7fa318006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:44.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.451+0000 7fa32dffb700 1 -- 172.21.15.170:0/1269407144 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa32001be90 con 0x7fa330100e90 2024-05-09T09:16:44.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:44 smithi012 ceph-mon[30140]: Updating smithi170:/etc/ceph/ceph.conf 2024-05-09T09:16:44.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:44 smithi012 ceph-mon[30140]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:44.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:44 smithi012 ceph-mon[30140]: Updating smithi170:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:16:44.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:44 smithi012 ceph-mon[30140]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:16:44.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:44 smithi012 ceph-mon[30140]: Updating smithi170:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T09:16:44.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:44 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:44.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:44 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:44.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:44 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:44.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.660+0000 7fa336df8700 1 -- 172.21.15.170:0/1269407144 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi016:172.21.15.16=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}) -- 0x7fa33006a7a0 con 0x7fa31c039140 2024-05-09T09:16:44.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.670+0000 7fa32dffb700 1 -- 172.21.15.170:0/1269407144 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa33006a7a0 con 0x7fa31c039140 2024-05-09T09:16:44.669 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mon update... 2024-05-09T09:16:44.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.672+0000 7fa336df8700 1 -- 172.21.15.170:0/1269407144 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa31c039140 msgr2=0x7fa31c03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:44.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.672+0000 7fa336df8700 1 --2- 172.21.15.170:0/1269407144 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa31c039140 0x7fa31c03b600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa318006fd0 tx=0x7fa318006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:44.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.672+0000 7fa336df8700 1 -- 172.21.15.170:0/1269407144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa330100e90 msgr2=0x7fa3301964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:44.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.672+0000 7fa336df8700 1 --2- 172.21.15.170:0/1269407144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa330100e90 0x7fa3301964c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa320009030 tx=0x7fa32000bb10 comp rx=0 tx=0).stop 2024-05-09T09:16:44.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.672+0000 7fa336df8700 1 -- 172.21.15.170:0/1269407144 shutdown_connections 2024-05-09T09:16:44.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.672+0000 7fa336df8700 1 --2- 172.21.15.170:0/1269407144 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa31c039140 0x7fa31c03b600 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.672+0000 7fa336df8700 1 --2- 172.21.15.170:0/1269407144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa330100e90 0x7fa3301964c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.672+0000 7fa336df8700 1 -- 172.21.15.170:0/1269407144 >> 172.21.15.170:0/1269407144 conn(0x7fa3300fc740 msgr2=0x7fa330102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:44.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.672+0000 7fa336df8700 1 -- 172.21.15.170:0/1269407144 shutdown_connections 2024-05-09T09:16:44.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:44.672+0000 7fa336df8700 1 -- 172.21.15.170:0/1269407144 wait complete. 2024-05-09T09:16:45.300 DEBUG:teuthology.orchestra.run.smithi016:mon.b> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.b.service 2024-05-09T09:16:45.303 DEBUG:teuthology.orchestra.run.smithi170:mon.c> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.c.service 2024-05-09T09:16:45.305 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T09:16:45.306 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-09T09:16:45.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:45 smithi012 ceph-mon[30140]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:45.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:45 smithi012 ceph-mon[30140]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi016:172.21.15.16=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:16:45.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:45 smithi012 ceph-mon[30140]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi016:172.21.15.16=b;smithi170:172.21.15.170=c;count:3 2024-05-09T09:16:45.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:45.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:16:45.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:45.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:16:45.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:45.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:16:45.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:45.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:45 smithi012 ceph-mon[30140]: Deploying daemon mon.c on smithi170 2024-05-09T09:16:47.271 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:16:47.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:47 smithi012 ceph-mon[30140]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:48.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.958+0000 7f6961951700 1 -- 172.21.15.170:0/891876500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f695c100f70 msgr2=0x7f695c101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:48.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.958+0000 7f6961951700 1 --2- 172.21.15.170:0/891876500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f695c100f70 0x7f695c101350 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f694c009910 tx=0x7f694c009c20 comp rx=0 tx=0).stop 2024-05-09T09:16:48.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.958+0000 7f6961951700 1 -- 172.21.15.170:0/891876500 shutdown_connections 2024-05-09T09:16:48.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.958+0000 7f6961951700 1 --2- 172.21.15.170:0/891876500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f695c100f70 0x7f695c101350 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:48.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.958+0000 7f6961951700 1 -- 172.21.15.170:0/891876500 >> 172.21.15.170:0/891876500 conn(0x7f695c0fc800 msgr2=0x7f695c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:48.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.959+0000 7f6961951700 1 -- 172.21.15.170:0/891876500 shutdown_connections 2024-05-09T09:16:48.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.959+0000 7f6961951700 1 -- 172.21.15.170:0/891876500 wait complete. 2024-05-09T09:16:48.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.959+0000 7f6961951700 1 Processor -- start 2024-05-09T09:16:48.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.959+0000 7f6961951700 1 -- start start 2024-05-09T09:16:48.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.959+0000 7f6961951700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f695c100f70 0x7f695c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:48.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.959+0000 7f6961951700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f695c10e640 con 0x7f695c100f70 2024-05-09T09:16:48.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.960+0000 7f695affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f695c100f70 0x7f695c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:48.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.960+0000 7f695affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f695c100f70 0x7f695c19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.170:34886/0 (socket says 172.21.15.170:34886) 2024-05-09T09:16:48.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.960+0000 7f695affd700 1 -- 172.21.15.170:0/3451106337 learned_addr learned my addr 172.21.15.170:0/3451106337 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-09T09:16:48.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.960+0000 7f695affd700 1 -- 172.21.15.170:0/3451106337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695c19fd50 con 0x7f695c100f70 2024-05-09T09:16:48.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.960+0000 7f695affd700 1 --2- 172.21.15.170:0/3451106337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f695c100f70 0x7f695c19f810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f694c006010 tx=0x7f694c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:48.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.961+0000 7f696094f700 1 -- 172.21.15.170:0/3451106337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f694c027070 con 0x7f695c100f70 2024-05-09T09:16:48.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.961+0000 7f696094f700 1 -- 172.21.15.170:0/3451106337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f694c00be30 con 0x7f695c100f70 2024-05-09T09:16:48.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.961+0000 7f6961951700 1 -- 172.21.15.170:0/3451106337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695c1a0040 con 0x7f695c100f70 2024-05-09T09:16:48.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.961+0000 7f696094f700 1 -- 172.21.15.170:0/3451106337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f694c022d70 con 0x7f695c100f70 2024-05-09T09:16:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.961+0000 7f6961951700 1 -- 172.21.15.170:0/3451106337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695c1999d0 con 0x7f695c100f70 2024-05-09T09:16:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.961+0000 7f696094f700 1 -- 172.21.15.170:0/3451106337 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f694c02a070 con 0x7f695c100f70 2024-05-09T09:16:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.961+0000 7f6961951700 1 -- 172.21.15.170:0/3451106337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f695c102450 con 0x7f695c100f70 2024-05-09T09:16:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.962+0000 7f696094f700 1 --2- 172.21.15.170:0/3451106337 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6948039150 0x7f694803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.962+0000 7f696094f700 1 -- 172.21.15.170:0/3451106337 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f694c01e030 con 0x7f695c100f70 2024-05-09T09:16:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.962+0000 7f69527fc700 1 --2- 172.21.15.170:0/3451106337 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6948039150 0x7f694803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.962+0000 7f69527fc700 1 --2- 172.21.15.170:0/3451106337 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6948039150 0x7f694803b610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6944006fd0 tx=0x7f6944006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:48.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:48.967+0000 7f696094f700 1 -- 172.21.15.170:0/3451106337 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f694c013070 con 0x7f695c100f70 2024-05-09T09:16:49.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 podman[33598]: 2024-05-09 09:16:49.159072654 +0000 UTC m=+0.116368588 container create 84836b4b5320d46ffd4dbbb262bed3bd09d19771e3cc6bb17d859a137f865c82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-c, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-09T09:16:49.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:49.253+0000 7f6961951700 1 -- 172.21.15.170:0/3451106337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f695c19af00 con 0x7f695c100f70 2024-05-09T09:16:49.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:49.254+0000 7f696094f700 1 -- 172.21.15.170:0/3451106337 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f694c013c60 con 0x7f695c100f70 2024-05-09T09:16:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:16:49.255 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":1,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","modified":"2024-05-09T09:14:59.821401Z","created":"2024-05-09T09:14:59.821401Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T09:16:49.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:49.256+0000 7f6961951700 1 -- 172.21.15.170:0/3451106337 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6948039150 msgr2=0x7f694803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:49.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:49.256+0000 7f6961951700 1 --2- 172.21.15.170:0/3451106337 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6948039150 0x7f694803b610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6944006fd0 tx=0x7f6944006e40 comp rx=0 tx=0).stop 2024-05-09T09:16:49.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:49.256+0000 7f6961951700 1 -- 172.21.15.170:0/3451106337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f695c100f70 msgr2=0x7f695c19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:49.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:49.256+0000 7f6961951700 1 --2- 172.21.15.170:0/3451106337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f695c100f70 0x7f695c19f810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f694c006010 tx=0x7f694c005810 comp rx=0 tx=0).stop 2024-05-09T09:16:49.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:49.256+0000 7f6961951700 1 -- 172.21.15.170:0/3451106337 shutdown_connections 2024-05-09T09:16:49.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:49.256+0000 7f6961951700 1 --2- 172.21.15.170:0/3451106337 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6948039150 0x7f694803b610 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:49.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:49.256+0000 7f6961951700 1 --2- 172.21.15.170:0/3451106337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f695c100f70 0x7f695c19f810 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:49.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:49.256+0000 7f6961951700 1 -- 172.21.15.170:0/3451106337 >> 172.21.15.170:0/3451106337 conn(0x7f695c0fc800 msgr2=0x7f695c102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:49.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:49.256+0000 7f6961951700 1 -- 172.21.15.170:0/3451106337 shutdown_connections 2024-05-09T09:16:49.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:49.256+0000 7f6961951700 1 -- 172.21.15.170:0/3451106337 wait complete. 2024-05-09T09:16:49.258 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 1 2024-05-09T09:16:49.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 podman[33598]: 2024-05-09 09:16:49.362171348 +0000 UTC m=+0.319467315 container init 84836b4b5320d46ffd4dbbb262bed3bd09d19771e3cc6bb17d859a137f865c82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T09:16:49.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 podman[33598]: 2024-05-09 09:16:49.365900516 +0000 UTC m=+0.323196451 container start 84836b4b5320d46ffd4dbbb262bed3bd09d19771e3cc6bb17d859a137f865c82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T09:16:49.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T09:16:49.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev), process ceph-mon, pid 2 2024-05-09T09:16:49.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: pidfile_write: ignore empty --pid-file 2024-05-09T09:16:49.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: load: jerasure load: lrc 2024-05-09T09:16:49.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: RocksDB version: 7.9.2 2024-05-09T09:16:49.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Git sha 0 2024-05-09T09:16:49.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Compile date 2024-05-07 19:19:25 2024-05-09T09:16:49.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: DB SUMMARY 2024-05-09T09:16:49.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: DB Session ID: 45UHJCBNSSAJ23T3W3MM 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: CURRENT file: CURRENT 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.error_if_exists: 0 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.create_if_missing: 0 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.paranoid_checks: 1 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.env: 0x557719554c60 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T09:16:49.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.info_log: 0x55771ade7ac0 2024-05-09T09:16:49.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T09:16:49.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.statistics: (nil) 2024-05-09T09:16:49.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.use_fsync: 0 2024-05-09T09:16:49.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_log_file_size: 0 2024-05-09T09:16:49.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T09:16:49.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T09:16:49.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T09:16:49.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T09:16:49.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.allow_fallocate: 1 2024-05-09T09:16:49.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T09:16:49.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T09:16:49.747 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.use_direct_reads: 0 2024-05-09T09:16:49.747 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T09:16:49.747 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T09:16:49.747 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.db_log_dir: 2024-05-09T09:16:49.747 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.wal_dir: 2024-05-09T09:16:49.747 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T09:16:49.747 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T09:16:49.747 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T09:16:49.747 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T09:16:49.747 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T09:16:49.747 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T09:16:49.747 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T09:16:49.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T09:16:49.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.write_buffer_manager: 0x55771adba1e0 2024-05-09T09:16:49.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T09:16:49.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T09:16:49.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T09:16:49.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T09:16:49.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T09:16:49.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T09:16:49.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T09:16:49.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T09:16:49.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.unordered_write: 0 2024-05-09T09:16:49.749 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T09:16:49.749 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T09:16:49.749 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T09:16:49.749 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T09:16:49.749 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.row_cache: None 2024-05-09T09:16:49.749 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.wal_filter: None 2024-05-09T09:16:49.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T09:16:49.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T09:16:49.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.two_write_queues: 0 2024-05-09T09:16:49.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T09:16:49.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.wal_compression: 0 2024-05-09T09:16:49.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.atomic_flush: 0 2024-05-09T09:16:49.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T09:16:49.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T09:16:49.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.log_readahead_size: 0 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_background_jobs: 2 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_background_compactions: -1 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_subcompactions: 1 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T09:16:49.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_open_files: -1 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_background_flushes: -1 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Compression algorithms supported: 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T09:16:49.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: kZSTD supported: 0 2024-05-09T09:16:49.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: kXpressCompression supported: 0 2024-05-09T09:16:49.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T09:16:49.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: kZlibCompression supported: 1 2024-05-09T09:16:49.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: kSnappyCompression supported: 1 2024-05-09T09:16:49.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: kLZ4Compression supported: 1 2024-05-09T09:16:49.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: kBZip2Compression supported: 0 2024-05-09T09:16:49.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T09:16:49.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T09:16:49.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-09T09:16:49.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T09:16:49.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T09:16:49.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.merge_operator: 2024-05-09T09:16:49.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compaction_filter: None 2024-05-09T09:16:49.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T09:16:49.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T09:16:49.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T09:16:49.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T09:16:49.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55771ade7bc0) 2024-05-09T09:16:49.755 INFO:journalctl@ceph.mon.c.smithi170.stdout: cache_index_and_filter_blocks: 1 2024-05-09T09:16:49.756 INFO:journalctl@ceph.mon.c.smithi170.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T09:16:49.756 INFO:journalctl@ceph.mon.c.smithi170.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T09:16:49.756 INFO:journalctl@ceph.mon.c.smithi170.stdout: pin_top_level_index_and_filter: 1 2024-05-09T09:16:49.756 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_type: 0 2024-05-09T09:16:49.756 INFO:journalctl@ceph.mon.c.smithi170.stdout: data_block_index_type: 0 2024-05-09T09:16:49.756 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_shortening: 1 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: checksum: 4 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: no_block_cache: 0 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache: 0x55771adeb090 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_name: BinnedLRUCache 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_options: 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: capacity : 536870912 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: num_shard_bits : 4 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: strict_capacity_limit : 0 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: high_pri_pool_ratio: 0.000 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_compressed: (nil) 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: persistent_cache: (nil) 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_size: 4096 2024-05-09T09:16:49.757 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_size_deviation: 10 2024-05-09T09:16:49.758 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_restart_interval: 16 2024-05-09T09:16:49.758 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_block_restart_interval: 1 2024-05-09T09:16:49.758 INFO:journalctl@ceph.mon.c.smithi170.stdout: metadata_block_size: 4096 2024-05-09T09:16:49.758 INFO:journalctl@ceph.mon.c.smithi170.stdout: partition_filters: 0 2024-05-09T09:16:49.758 INFO:journalctl@ceph.mon.c.smithi170.stdout: use_delta_encoding: 1 2024-05-09T09:16:49.758 INFO:journalctl@ceph.mon.c.smithi170.stdout: filter_policy: bloomfilter 2024-05-09T09:16:49.758 INFO:journalctl@ceph.mon.c.smithi170.stdout: whole_key_filtering: 1 2024-05-09T09:16:49.758 INFO:journalctl@ceph.mon.c.smithi170.stdout: verify_compression: 0 2024-05-09T09:16:49.758 INFO:journalctl@ceph.mon.c.smithi170.stdout: read_amp_bytes_per_bit: 0 2024-05-09T09:16:49.759 INFO:journalctl@ceph.mon.c.smithi170.stdout: format_version: 5 2024-05-09T09:16:49.759 INFO:journalctl@ceph.mon.c.smithi170.stdout: enable_index_compression: 1 2024-05-09T09:16:49.759 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_align: 0 2024-05-09T09:16:49.759 INFO:journalctl@ceph.mon.c.smithi170.stdout: max_auto_readahead_size: 262144 2024-05-09T09:16:49.759 INFO:journalctl@ceph.mon.c.smithi170.stdout: prepopulate_block_cache: 0 2024-05-09T09:16:49.759 INFO:journalctl@ceph.mon.c.smithi170.stdout: initial_auto_readahead_size: 8192 2024-05-09T09:16:49.759 INFO:journalctl@ceph.mon.c.smithi170.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T09:16:49.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T09:16:49.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T09:16:49.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compression: NoCompression 2024-05-09T09:16:49.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T09:16:49.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T09:16:49.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.num_levels: 7 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T09:16:49.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T09:16:49.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T09:16:49.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T09:16:49.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T09:16:49.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T09:16:49.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T09:16:49.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T09:16:49.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T09:16:49.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T09:16:49.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T09:16:49.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T09:16:49.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T09:16:49.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T09:16:49.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T09:16:49.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T09:16:49.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T09:16:49.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T09:16:49.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T09:16:49.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T09:16:49.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T09:16:49.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T09:16:49.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T09:16:49.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T09:16:49.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T09:16:49.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T09:16:49.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T09:16:49.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T09:16:49.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.table_properties_collectors: 2024-05-09T09:16:49.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.inplace_update_support: 0 2024-05-09T09:16:49.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T09:16:49.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T09:16:49.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T09:16:49.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T09:16:49.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.bloom_locality: 0 2024-05-09T09:16:49.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.max_successive_merges: 0 2024-05-09T09:16:49.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T09:16:49.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T09:16:49.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T09:16:49.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T09:16:49.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.ttl: 2592000 2024-05-09T09:16:49.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T09:16:49.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T09:16:49.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T09:16:49.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.enable_blob_files: false 2024-05-09T09:16:49.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.min_blob_size: 0 2024-05-09T09:16:49.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T09:16:49.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T09:16:49.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T09:16:49.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T09:16:49.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T09:16:49.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T09:16:49.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T09:16:49.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T09:16:49.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-09T09:16:49.767 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-09T09:16:49.767 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0f896cd7-8868-4311-869e-bce763c54b4e 2024-05-09T09:16:49.767 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715246209406079, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-09T09:16:49.767 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-09T09:16:49.767 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715246209406493, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715246209, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0f896cd7-8868-4311-869e-bce763c54b4e", "db_session_id": "45UHJCBNSSAJ23T3W3MM", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-09T09:16:49.767 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715246209406589, "job": 1, "event": "recovery_finished"} 2024-05-09T09:16:49.767 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-09T09:16:49.767 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T09:16:49.767 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55771aedc000 2024-05-09T09:16:49.767 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: DB pointer 0x55771aec6000 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** DB Stats ** 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** Compaction Stats [default] ** 2024-05-09T09:16:49.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T09:16:49.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T09:16:49.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T09:16:49.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T09:16:49.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T09:16:49.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-05-09T09:16:49.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** Compaction Stats [default] ** 2024-05-09T09:16:49.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T09:16:49.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T09:16:49.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T09:16:49.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-05-09T09:16:49.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T09:16:49.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-05-09T09:16:49.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T09:16:49.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T09:16:49.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T09:16:49.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T09:16:49.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T09:16:49.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T09:16:49.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T09:16:49.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T09:16:49.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T09:16:49.771 INFO:journalctl@ceph.mon.c.smithi170.stdout: Block cache BinnedLRUCache@0x55771adeb090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-09T09:16:49.771 INFO:journalctl@ceph.mon.c.smithi170.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-09T09:16:49.771 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-05-09T09:16:49.771 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T09:16:49.771 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: using public_addr v2:172.21.15.170:0/0 -> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2024-05-09T09:16:49.771 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: starting mon.c rank -1 at public addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] at bind addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:16:49.771 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(???) e0 preinit fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:16:49.771 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(synchronizing).mds e1 new map 2024-05-09T09:16:49.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-09T09:16:49.772 INFO:journalctl@ceph.mon.c.smithi170.stdout: e1 2024-05-09T09:16:49.772 INFO:journalctl@ceph.mon.c.smithi170.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T09:16:49.772 INFO:journalctl@ceph.mon.c.smithi170.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T09:16:49.772 INFO:journalctl@ceph.mon.c.smithi170.stdout: legacy client fscid: -1 2024-05-09T09:16:49.772 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-05-09T09:16:49.772 INFO:journalctl@ceph.mon.c.smithi170.stdout: No filesystems configured 2024-05-09T09:16:49.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T09:16:49.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T09:16:49.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T09:16:49.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T09:16:49.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T09:16:49.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T09:16:49.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T09:16:49.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T09:16:49.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T09:16:49.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T09:16:49.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:16:49.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:49.774 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:49.774 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:16:49.774 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: Deploying cephadm binary to smithi170 2024-05-09T09:16:49.774 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.774 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.774 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.774 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.774 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: Updating smithi016:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: Updating smithi016:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T09:16:49.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: Added host smithi170 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3334774078' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3334774078' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:49.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:49.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:16:49.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: Updating smithi170:/etc/ceph/ceph.conf 2024-05-09T09:16:49.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:49.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: Updating smithi170:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:16:49.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:16:49.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: Updating smithi170:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T09:16:49.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:49.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi016:172.21.15.16=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:16:49.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi016:172.21.15.16=b;smithi170:172.21.15.170=c;count:3 2024-05-09T09:16:49.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:16:49.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:49.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:16:49.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:49.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:16:49.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:49.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: Deploying daemon mon.c on smithi170 2024-05-09T09:16:49.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:49.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 ceph-mon[33632]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T09:16:49.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 bash[33598]: 84836b4b5320d46ffd4dbbb262bed3bd09d19771e3cc6bb17d859a137f865c82 2024-05-09T09:16:49.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:49 smithi170 systemd[1]: Started Ceph mon.c for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:16:51.167 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T09:16:51.168 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-09T09:16:51.424 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:16:54.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:16:54.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: mon.a calling monitor election 2024-05-09T09:16:54.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:54.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:54.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:54.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:54.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: mon.c calling monitor election 2024-05-09T09:16:54.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:54.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:54.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:54.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:54.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T09:16:54.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: monmap epoch 2 2024-05-09T09:16:54.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: last_changed 2024-05-09T09:16:49.428964+0000 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: created 2024-05-09T09:14:59.821401+0000 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: min_mon_release 19 (squid) 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: election_strategy: 1 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: fsmap 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: mgrmap e13: x(active, since 49s) 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: overall HEALTH_OK 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:54.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:16:54.987 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:54.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:16:54.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: mon.a calling monitor election 2024-05-09T09:16:54.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:54.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:54.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:54.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:54.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: mon.c calling monitor election 2024-05-09T09:16:54.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:54.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:54.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:54.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:54.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: monmap epoch 2 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: last_changed 2024-05-09T09:16:49.428964+0000 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: created 2024-05-09T09:14:59.821401+0000 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: min_mon_release 19 (squid) 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: election_strategy: 1 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: fsmap 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: mgrmap e13: x(active, since 49s) 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: overall HEALTH_OK 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:54.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:54.995 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:16:54.995 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:55.733 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:55 smithi012 ceph-mon[30140]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:55.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:55 smithi012 ceph-mon[30140]: Deploying daemon mon.b on smithi016 2024-05-09T09:16:55.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:55.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.967+0000 7fa6c815d700 1 -- 172.21.15.170:0/445475401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6ac0035c0 msgr2=0x7fa6ac005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:55.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.967+0000 7fa6c815d700 1 --2- 172.21.15.170:0/445475401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6ac0035c0 0x7fa6ac005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0077460 tx=0x7fa6b0010ec0 comp rx=0 tx=0).stop 2024-05-09T09:16:55.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.967+0000 7fa6c815d700 1 -- 172.21.15.170:0/445475401 shutdown_connections 2024-05-09T09:16:55.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.967+0000 7fa6c815d700 1 --2- 172.21.15.170:0/445475401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6ac0035c0 0x7fa6ac005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:55.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.967+0000 7fa6c815d700 1 -- 172.21.15.170:0/445475401 >> 172.21.15.170:0/445475401 conn(0x7fa6c0076080 msgr2=0x7fa6c0076490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:55.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.967+0000 7fa6c815d700 1 -- 172.21.15.170:0/445475401 shutdown_connections 2024-05-09T09:16:55.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.967+0000 7fa6c815d700 1 -- 172.21.15.170:0/445475401 wait complete. 2024-05-09T09:16:55.968 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:55 smithi170 ceph-mon[33632]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:55.968 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:55 smithi170 ceph-mon[33632]: Deploying daemon mon.b on smithi016 2024-05-09T09:16:55.968 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:55.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.968+0000 7fa6c815d700 1 Processor -- start 2024-05-09T09:16:55.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.968+0000 7fa6c815d700 1 -- start start 2024-05-09T09:16:55.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c815d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6ac0035c0 0x7fa6c019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:55.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c815d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6c019dbb0 0x7fa6c01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:55.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c815d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0107d00 con 0x7fa6c019dbb0 2024-05-09T09:16:55.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c815d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0107b80 con 0x7fa6ac0035c0 2024-05-09T09:16:55.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c5ef9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6ac0035c0 0x7fa6c019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:55.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c5ef9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6ac0035c0 0x7fa6c019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48308/0 (socket says 172.21.15.170:48308) 2024-05-09T09:16:55.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c5ef9700 1 -- 172.21.15.170:0/2843466182 learned_addr learned my addr 172.21.15.170:0/2843466182 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-09T09:16:55.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c56f8700 1 --2- 172.21.15.170:0/2843466182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6c019dbb0 0x7fa6c01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:55.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c5ef9700 1 -- 172.21.15.170:0/2843466182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6ac0035c0 msgr2=0x7fa6c019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T09:16:55.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c5ef9700 1 -- 172.21.15.170:0/2843466182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6ac0035c0 msgr2=0x7fa6c019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T09:16:55.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c5ef9700 1 --2- 172.21.15.170:0/2843466182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6ac0035c0 0x7fa6c019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T09:16:55.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c5ef9700 1 --2- 172.21.15.170:0/2843466182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6ac0035c0 0x7fa6c019d670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T09:16:55.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c56f8700 1 -- 172.21.15.170:0/2843466182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6ac0035c0 msgr2=0x7fa6c019d670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:16:55.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c56f8700 1 --2- 172.21.15.170:0/2843466182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6ac0035c0 0x7fa6c019d670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:55.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c56f8700 1 -- 172.21.15.170:0/2843466182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c019e1f0 con 0x7fa6c019dbb0 2024-05-09T09:16:55.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.969+0000 7fa6c56f8700 1 --2- 172.21.15.170:0/2843466182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6c019dbb0 0x7fa6c01977f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00b4d0 tx=0x7fa6bc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:55.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.970+0000 7fa6b6ffd700 1 -- 172.21.15.170:0/2843466182 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa6bc011630 con 0x7fa6c019dbb0 2024-05-09T09:16:55.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.970+0000 7fa6b6ffd700 1 -- 172.21.15.170:0/2843466182 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa6bc011ca0 con 0x7fa6c019dbb0 2024-05-09T09:16:55.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.970+0000 7fa6c815d700 1 -- 172.21.15.170:0/2843466182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c0197d90 con 0x7fa6c019dbb0 2024-05-09T09:16:55.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.970+0000 7fa6b6ffd700 1 -- 172.21.15.170:0/2843466182 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa6bc010dd0 con 0x7fa6c019dbb0 2024-05-09T09:16:55.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.970+0000 7fa6c815d700 1 -- 172.21.15.170:0/2843466182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c0198370 con 0x7fa6c019dbb0 2024-05-09T09:16:55.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.971+0000 7fa6b6ffd700 1 -- 172.21.15.170:0/2843466182 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa6bc018020 con 0x7fa6c019dbb0 2024-05-09T09:16:55.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.971+0000 7fa6b6ffd700 1 --2- 172.21.15.170:0/2843466182 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa6a4038e90 0x7fa6a403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:55.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.971+0000 7fa6b6ffd700 1 -- 172.21.15.170:0/2843466182 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa6bc0117d0 con 0x7fa6c019dbb0 2024-05-09T09:16:55.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.971+0000 7fa6c815d700 1 -- 172.21.15.170:0/2843466182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c01986e0 con 0x7fa6c019dbb0 2024-05-09T09:16:55.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.971+0000 7fa6c5ef9700 1 --2- 172.21.15.170:0/2843466182 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa6a4038e90 0x7fa6a403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:55.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.972+0000 7fa6c5ef9700 1 --2- 172.21.15.170:0/2843466182 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa6a4038e90 0x7fa6a403b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0005d10 tx=0x7fa6b0005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:55.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:55.976+0000 7fa6b6ffd700 1 -- 172.21.15.170:0/2843466182 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa6bc01f020 con 0x7fa6c019dbb0 2024-05-09T09:16:56.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:56.260+0000 7fa6c815d700 1 -- 172.21.15.170:0/2843466182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa6c0198d20 con 0x7fa6c019dbb0 2024-05-09T09:16:56.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:56.261+0000 7fa6b6ffd700 1 -- 172.21.15.170:0/2843466182 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fa6bc018310 con 0x7fa6c019dbb0 2024-05-09T09:16:56.260 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:16:56.260 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":2,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","modified":"2024-05-09T09:16:49.428964Z","created":"2024-05-09T09:14:59.821401Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T09:16:56.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:56.263+0000 7fa6c815d700 1 -- 172.21.15.170:0/2843466182 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa6a4038e90 msgr2=0x7fa6a403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:56.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:56.263+0000 7fa6c815d700 1 --2- 172.21.15.170:0/2843466182 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa6a4038e90 0x7fa6a403b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0005d10 tx=0x7fa6b0005c40 comp rx=0 tx=0).stop 2024-05-09T09:16:56.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:56.263+0000 7fa6c815d700 1 -- 172.21.15.170:0/2843466182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6c019dbb0 msgr2=0x7fa6c01977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:56.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:56.263+0000 7fa6c815d700 1 --2- 172.21.15.170:0/2843466182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6c019dbb0 0x7fa6c01977f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00b4d0 tx=0x7fa6bc00b890 comp rx=0 tx=0).stop 2024-05-09T09:16:56.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:56.263+0000 7fa6c815d700 1 -- 172.21.15.170:0/2843466182 shutdown_connections 2024-05-09T09:16:56.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:56.263+0000 7fa6c815d700 1 --2- 172.21.15.170:0/2843466182 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa6a4038e90 0x7fa6a403b350 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:56.263+0000 7fa6c815d700 1 --2- 172.21.15.170:0/2843466182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6ac0035c0 0x7fa6c019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:56.263+0000 7fa6c815d700 1 --2- 172.21.15.170:0/2843466182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6c019dbb0 0x7fa6c01977f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:56.263+0000 7fa6c815d700 1 -- 172.21.15.170:0/2843466182 >> 172.21.15.170:0/2843466182 conn(0x7fa6c0076080 msgr2=0x7fa6c00ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:56.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:56.263+0000 7fa6c815d700 1 -- 172.21.15.170:0/2843466182 shutdown_connections 2024-05-09T09:16:56.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:16:56.263+0000 7fa6c815d700 1 -- 172.21.15.170:0/2843466182 wait complete. 2024-05-09T09:16:56.263 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 2 2024-05-09T09:16:56.526 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:56 smithi170 ceph-mon[33632]: from='client.? 172.21.15.170:0/2843466182' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T09:16:56.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:56 smithi012 ceph-mon[30140]: from='client.? 172.21.15.170:0/2843466182' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T09:16:57.838 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T09:16:57.839 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-09T09:16:57.868 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:16:57 smithi170 ceph-mon[33632]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:57.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:16:57 smithi012 ceph-mon[30140]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.085 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:16:58.372 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T09:16:58.372 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T09:16:58.372 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T09:16:58.372 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T09:16:58.372 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T09:16:58.372 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T09:16:58.373 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T09:16:58.373 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T09:16:58.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T09:16:58.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T09:16:58.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:16:58.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:16:58.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: Deploying cephadm binary to smithi170 2024-05-09T09:16:58.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:58.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:58.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:16:58.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-09T09:16:58.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: Updating smithi016:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:16:58.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:16:58.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:16:58.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: Updating smithi016:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T09:16:58.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: Added host smithi170 2024-05-09T09:16:58.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:16:58.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3334774078' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T09:16:58.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3334774078' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T09:16:58.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:16:58.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:58.757 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:58.757 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:16:58.757 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: Updating smithi170:/etc/ceph/ceph.conf 2024-05-09T09:16:58.757 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.757 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: Updating smithi170:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:16:58.757 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:16:58.757 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: Updating smithi170:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T09:16:58.757 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.757 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.757 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.757 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.757 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi016:172.21.15.16=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:16:58.758 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi016:172.21.15.16=b;smithi170:172.21.15.170=c;count:3 2024-05-09T09:16:58.758 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.758 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:16:58.758 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:58.758 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:16:58.758 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.758 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:16:58.758 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:58.758 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: Deploying daemon mon.c on smithi170 2024-05-09T09:16:58.758 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.758 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:16:58.758 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.a calling monitor election 2024-05-09T09:16:58.759 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:58.759 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:58.759 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.759 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:58.759 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.c calling monitor election 2024-05-09T09:16:58.759 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:58.759 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.759 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:58.760 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:58.760 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T09:16:58.760 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: monmap epoch 2 2024-05-09T09:16:58.760 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:16:58.760 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: last_changed 2024-05-09T09:16:49.428964+0000 2024-05-09T09:16:58.760 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: created 2024-05-09T09:14:59.821401+0000 2024-05-09T09:16:58.760 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: min_mon_release 19 (squid) 2024-05-09T09:16:58.760 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: election_strategy: 1 2024-05-09T09:16:58.760 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-09T09:16:58.760 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-05-09T09:16:58.760 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: fsmap 2024-05-09T09:16:58.760 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:16:58.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mgrmap e13: x(active, since 49s) 2024-05-09T09:16:58.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: overall HEALTH_OK 2024-05-09T09:16:58.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:16:58.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:16:58.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:58.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: Deploying daemon mon.b on smithi016 2024-05-09T09:16:58.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:16:58.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: from='client.? 172.21.15.170:0/2843466182' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T09:16:58.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:16:58.762 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 ceph-mon[33335]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T09:16:58.762 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 bash[33315]: f6666148f06493c709d485869a759f8ba89346c8036892f557136023e5cb327b 2024-05-09T09:16:58.762 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:16:58 smithi016 systemd[1]: Started Ceph mon.b for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:17:03.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:17:03.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: mon.c calling monitor election 2024-05-09T09:17:03.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: mon.a calling monitor election 2024-05-09T09:17:03.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:03.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: monmap epoch 3 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: last_changed 2024-05-09T09:16:58.378592+0000 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: created 2024-05-09T09:14:59.821401+0000 2024-05-09T09:17:03.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: min_mon_release 19 (squid) 2024-05-09T09:17:03.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: election_strategy: 1 2024-05-09T09:17:03.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-09T09:17:03.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-05-09T09:17:03.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: 2: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.b 2024-05-09T09:17:03.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: fsmap 2024-05-09T09:17:03.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:17:03.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: mgrmap e13: x(active, since 58s) 2024-05-09T09:17:03.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: overall HEALTH_OK 2024-05-09T09:17:03.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:03.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:03.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:03.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:03.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:17:03.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: mon.c calling monitor election 2024-05-09T09:17:03.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: mon.a calling monitor election 2024-05-09T09:17:03.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:03.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:17:03.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:03.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:03.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:03.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:03.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:03.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:03.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: monmap epoch 3 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: last_changed 2024-05-09T09:16:58.378592+0000 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: created 2024-05-09T09:14:59.821401+0000 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: min_mon_release 19 (squid) 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: election_strategy: 1 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: 2: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.b 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: fsmap 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: mgrmap e13: x(active, since 58s) 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: overall HEALTH_OK 2024-05-09T09:17:03.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:03.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:03.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:03.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.017+0000 7f47be0a6700 1 -- 172.21.15.170:0/3916214370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47a40035c0 msgr2=0x7f47a4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.017+0000 7f47be0a6700 1 --2- 172.21.15.170:0/3916214370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47a40035c0 0x7f47a4005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f47b8100c40 tx=0x7f47a800fe60 comp rx=0 tx=0).stop 2024-05-09T09:17:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.017+0000 7f47be0a6700 1 -- 172.21.15.170:0/3916214370 shutdown_connections 2024-05-09T09:17:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.017+0000 7f47be0a6700 1 --2- 172.21.15.170:0/3916214370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47a40035c0 0x7f47a4005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.017+0000 7f47be0a6700 1 -- 172.21.15.170:0/3916214370 >> 172.21.15.170:0/3916214370 conn(0x7f47b80fc800 msgr2=0x7f47b80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:04.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.017+0000 7f47be0a6700 1 -- 172.21.15.170:0/3916214370 shutdown_connections 2024-05-09T09:17:04.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.017+0000 7f47be0a6700 1 -- 172.21.15.170:0/3916214370 wait complete. 2024-05-09T09:17:04.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.018+0000 7f47be0a6700 1 Processor -- start 2024-05-09T09:17:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.018+0000 7f47be0a6700 1 -- start start 2024-05-09T09:17:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.018+0000 7f47be0a6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a40035c0 0x7f47b8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.018+0000 7f47be0a6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b8198f70 0x7f47b81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.018+0000 7f47be0a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b81a1aa0 0x7f47b81a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.018+0000 7f47be0a6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f47b8108d40 con 0x7f47b81a1aa0 2024-05-09T09:17:04.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.018+0000 7f47be0a6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47b8108bc0 con 0x7f47b8198f70 2024-05-09T09:17:04.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.018+0000 7f47be0a6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f47b8108ec0 con 0x7f47a40035c0 2024-05-09T09:17:04.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a40035c0 0x7f47b8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:04.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a40035c0 0x7f47b8198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50608/0 (socket says 172.21.15.170:50608) 2024-05-09T09:17:04.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b77fe700 1 -- 172.21.15.170:0/1158421003 learned_addr learned my addr 172.21.15.170:0/1158421003 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-09T09:17:04.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b7fff700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b81a1aa0 0x7f47b81a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:04.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47affff700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b8198f70 0x7f47b81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:04.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b77fe700 1 -- 172.21.15.170:0/1158421003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a40035c0 msgr2=0x7f47b8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T09:17:04.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b77fe700 1 -- 172.21.15.170:0/1158421003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a40035c0 msgr2=0x7f47b8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T09:17:04.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b77fe700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a40035c0 0x7f47b8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T09:17:04.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b77fe700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a40035c0 0x7f47b8198a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T09:17:04.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47affff700 1 -- 172.21.15.170:0/1158421003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b8198f70 msgr2=0x7f47b81993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-09T09:17:04.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47affff700 1 -- 172.21.15.170:0/1158421003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b8198f70 msgr2=0x7f47b81993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T09:17:04.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47affff700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b8198f70 0x7f47b81993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T09:17:04.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47affff700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b8198f70 0x7f47b81993f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T09:17:04.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b7fff700 1 -- 172.21.15.170:0/1158421003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a40035c0 msgr2=0x7f47b8198a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:17:04.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b7fff700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a40035c0 0x7f47b8198a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:04.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b7fff700 1 -- 172.21.15.170:0/1158421003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b8198f70 msgr2=0x7f47b81993f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:17:04.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b7fff700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b8198f70 0x7f47b81993f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b7fff700 1 -- 172.21.15.170:0/1158421003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b81a4590 con 0x7f47b81a1aa0 2024-05-09T09:17:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.019+0000 7f47b7fff700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b81a1aa0 0x7f47b81a3e90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f47b000bd50 tx=0x7f47b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.020+0000 7f47b57fa700 1 -- 172.21.15.170:0/1158421003 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0019070 con 0x7f47b81a1aa0 2024-05-09T09:17:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.020+0000 7f47be0a6700 1 -- 172.21.15.170:0/1158421003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b81a4880 con 0x7f47b81a1aa0 2024-05-09T09:17:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.020+0000 7f47b57fa700 1 -- 172.21.15.170:0/1158421003 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f47b000cd10 con 0x7f47b81a1aa0 2024-05-09T09:17:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.020+0000 7f47b57fa700 1 -- 172.21.15.170:0/1158421003 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b00127e0 con 0x7f47b81a1aa0 2024-05-09T09:17:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.020+0000 7f47be0a6700 1 -- 172.21.15.170:0/1158421003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b81a4dc0 con 0x7f47b81a1aa0 2024-05-09T09:17:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.021+0000 7f47be0a6700 1 -- 172.21.15.170:0/1158421003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b8063980 con 0x7f47b81a1aa0 2024-05-09T09:17:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.021+0000 7f47b57fa700 1 -- 172.21.15.170:0/1158421003 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f47b001c070 con 0x7f47b81a1aa0 2024-05-09T09:17:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.021+0000 7f47b57fa700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f47980395b0 0x7f479803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.021+0000 7f47b57fa700 1 -- 172.21.15.170:0/1158421003 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f47b004bbb0 con 0x7f47b81a1aa0 2024-05-09T09:17:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.021+0000 7f47b77fe700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f47980395b0 0x7f479803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.022+0000 7f47b77fe700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f47980395b0 0x7f479803ba70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f47a800fa10 tx=0x7f47a800f3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.027+0000 7f47b57fa700 1 -- 172.21.15.170:0/1158421003 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f47b0013c30 con 0x7f47b81a1aa0 2024-05-09T09:17:04.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.324+0000 7f47be0a6700 1 -- 172.21.15.170:0/1158421003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f47b81a5020 con 0x7f47b81a1aa0 2024-05-09T09:17:04.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.325+0000 7f47b57fa700 1 -- 172.21.15.170:0/1158421003 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f47b001d730 con 0x7f47b81a1aa0 2024-05-09T09:17:04.325 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-09T09:17:04.325 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":3,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","modified":"2024-05-09T09:16:58.378592Z","created":"2024-05-09T09:14:59.821401Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T09:17:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.327+0000 7f47be0a6700 1 -- 172.21.15.170:0/1158421003 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f47980395b0 msgr2=0x7f479803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.327+0000 7f47be0a6700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f47980395b0 0x7f479803ba70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f47a800fa10 tx=0x7f47a800f3c0 comp rx=0 tx=0).stop 2024-05-09T09:17:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.327+0000 7f47be0a6700 1 -- 172.21.15.170:0/1158421003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b81a1aa0 msgr2=0x7f47b81a3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.327+0000 7f47be0a6700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b81a1aa0 0x7f47b81a3e90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f47b000bd50 tx=0x7f47b000be30 comp rx=0 tx=0).stop 2024-05-09T09:17:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.327+0000 7f47be0a6700 1 -- 172.21.15.170:0/1158421003 shutdown_connections 2024-05-09T09:17:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.327+0000 7f47be0a6700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f47980395b0 0x7f479803ba70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.327+0000 7f47be0a6700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b81a1aa0 0x7f47b81a3e90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.327+0000 7f47be0a6700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a40035c0 0x7f47b8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.327+0000 7f47be0a6700 1 --2- 172.21.15.170:0/1158421003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b8198f70 0x7f47b81993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.327+0000 7f47be0a6700 1 -- 172.21.15.170:0/1158421003 >> 172.21.15.170:0/1158421003 conn(0x7f47b80fc800 msgr2=0x7f47b8104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:04.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.327+0000 7f47be0a6700 1 -- 172.21.15.170:0/1158421003 shutdown_connections 2024-05-09T09:17:04.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:04.327+0000 7f47be0a6700 1 -- 172.21.15.170:0/1158421003 wait complete. 2024-05-09T09:17:04.328 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 3 2024-05-09T09:17:04.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:04.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:04 smithi170 ceph-mon[33632]: from='client.? 172.21.15.170:0/1158421003' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T09:17:04.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:04.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:04.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:04 smithi012 ceph-mon[30140]: from='client.? 172.21.15.170:0/1158421003' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T09:17:04.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:05.043 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-09T09:17:05.043 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-09T09:17:05.304 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:17:05.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:17:05.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: mon.c calling monitor election 2024-05-09T09:17:05.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: mon.a calling monitor election 2024-05-09T09:17:05.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:05.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:17:05.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:05.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:05.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:05.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:05.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:05.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:05.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:05.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:05.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T09:17:05.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: monmap epoch 3 2024-05-09T09:17:05.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:17:05.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: last_changed 2024-05-09T09:16:58.378592+0000 2024-05-09T09:17:05.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: created 2024-05-09T09:14:59.821401+0000 2024-05-09T09:17:05.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: min_mon_release 19 (squid) 2024-05-09T09:17:05.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: election_strategy: 1 2024-05-09T09:17:05.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-09T09:17:05.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-05-09T09:17:05.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: 2: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.b 2024-05-09T09:17:05.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: fsmap 2024-05-09T09:17:05.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:17:05.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: mgrmap e13: x(active, since 58s) 2024-05-09T09:17:05.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: overall HEALTH_OK 2024-05-09T09:17:05.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:05.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:05.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:05.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:05.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:05.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='client.? 172.21.15.170:0/1158421003' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T09:17:05.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:06.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: mon.b calling monitor election 2024-05-09T09:17:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: mon.b calling monitor election 2024-05-09T09:17:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: mon.a calling monitor election 2024-05-09T09:17:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: mon.c calling monitor election 2024-05-09T09:17:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T09:17:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: monmap epoch 3 2024-05-09T09:17:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:17:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: last_changed 2024-05-09T09:16:58.378592+0000 2024-05-09T09:17:06.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: created 2024-05-09T09:14:59.821401+0000 2024-05-09T09:17:06.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: min_mon_release 19 (squid) 2024-05-09T09:17:06.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: election_strategy: 1 2024-05-09T09:17:06.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-09T09:17:06.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-05-09T09:17:06.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: 2: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.b 2024-05-09T09:17:06.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: fsmap 2024-05-09T09:17:06.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:17:06.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: mgrmap e13: x(active, since 60s) 2024-05-09T09:17:06.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: overall HEALTH_OK 2024-05-09T09:17:06.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:06.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:06.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: mon.b calling monitor election 2024-05-09T09:17:06.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: mon.b calling monitor election 2024-05-09T09:17:06.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: mon.a calling monitor election 2024-05-09T09:17:06.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: mon.c calling monitor election 2024-05-09T09:17:06.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T09:17:06.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: monmap epoch 3 2024-05-09T09:17:06.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:17:06.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: last_changed 2024-05-09T09:16:58.378592+0000 2024-05-09T09:17:06.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: created 2024-05-09T09:14:59.821401+0000 2024-05-09T09:17:06.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: min_mon_release 19 (squid) 2024-05-09T09:17:06.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: election_strategy: 1 2024-05-09T09:17:06.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-09T09:17:06.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-05-09T09:17:06.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: 2: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.b 2024-05-09T09:17:06.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: fsmap 2024-05-09T09:17:06.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:17:06.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: mgrmap e13: x(active, since 60s) 2024-05-09T09:17:06.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: overall HEALTH_OK 2024-05-09T09:17:06.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:06.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:06.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: mon.b calling monitor election 2024-05-09T09:17:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: mon.b calling monitor election 2024-05-09T09:17:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: mon.a calling monitor election 2024-05-09T09:17:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: mon.c calling monitor election 2024-05-09T09:17:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T09:17:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: monmap epoch 3 2024-05-09T09:17:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:17:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: last_changed 2024-05-09T09:16:58.378592+0000 2024-05-09T09:17:06.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: created 2024-05-09T09:14:59.821401+0000 2024-05-09T09:17:06.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: min_mon_release 19 (squid) 2024-05-09T09:17:06.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: election_strategy: 1 2024-05-09T09:17:06.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-09T09:17:06.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-05-09T09:17:06.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: 2: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.b 2024-05-09T09:17:06.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: fsmap 2024-05-09T09:17:06.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:17:06.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: mgrmap e13: x(active, since 60s) 2024-05-09T09:17:06.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: overall HEALTH_OK 2024-05-09T09:17:06.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:06.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:06.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:06.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.917+0000 7f03e8a73700 1 -- 172.21.15.12:0/498487669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e4102f30 msgr2=0x7f03e4103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:06.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.917+0000 7f03e8a73700 1 --2- 172.21.15.12:0/498487669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e4102f30 0x7f03e4103310 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009910 tx=0x7f03d4009c20 comp rx=0 tx=0).stop 2024-05-09T09:17:06.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.917+0000 7f03e8a73700 1 -- 172.21.15.12:0/498487669 shutdown_connections 2024-05-09T09:17:06.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.917+0000 7f03e8a73700 1 --2- 172.21.15.12:0/498487669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e4102f30 0x7f03e4103310 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:06.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.917+0000 7f03e8a73700 1 -- 172.21.15.12:0/498487669 >> 172.21.15.12:0/498487669 conn(0x7f03e40fc780 msgr2=0x7f03e40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:06.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.917+0000 7f03e8a73700 1 -- 172.21.15.12:0/498487669 shutdown_connections 2024-05-09T09:17:06.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.917+0000 7f03e8a73700 1 -- 172.21.15.12:0/498487669 wait complete. 2024-05-09T09:17:06.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.918+0000 7f03e8a73700 1 Processor -- start 2024-05-09T09:17:06.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.918+0000 7f03e8a73700 1 -- start start 2024-05-09T09:17:06.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.918+0000 7f03e8a73700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e4102f30 0x7f03e4100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:06.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.918+0000 7f03e8a73700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e41010d0 0x7f03e4101550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:06.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.918+0000 7f03e8a73700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03e41a3cf0 0x7f03e41a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:06.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.918+0000 7f03e8a73700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f03e410c3f0 con 0x7f03e41010d0 2024-05-09T09:17:06.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.918+0000 7f03e8a73700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03e410c270 con 0x7f03e4102f30 2024-05-09T09:17:06.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.918+0000 7f03e8a73700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f03e410c570 con 0x7f03e41a3cf0 2024-05-09T09:17:06.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.918+0000 7f03e1d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e41010d0 0x7f03e4101550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:06.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.918+0000 7f03e1d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e41010d0 0x7f03e4101550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35344/0 (socket says 172.21.15.12:35344) 2024-05-09T09:17:06.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.918+0000 7f03e1d9b700 1 -- 172.21.15.12:0/405751692 learned_addr learned my addr 172.21.15.12:0/405751692 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:17:06.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.918+0000 7f03e259c700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e4102f30 0x7f03e4100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:06.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.919+0000 7f03e2d9d700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03e41a3cf0 0x7f03e41a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:06.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.919+0000 7f03e1d9b700 1 -- 172.21.15.12:0/405751692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03e41a3cf0 msgr2=0x7f03e41a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:06.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.919+0000 7f03e1d9b700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03e41a3cf0 0x7f03e41a60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:06.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.919+0000 7f03e1d9b700 1 -- 172.21.15.12:0/405751692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e4102f30 msgr2=0x7f03e4100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:06.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.919+0000 7f03e1d9b700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e4102f30 0x7f03e4100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:06.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.919+0000 7f03e1d9b700 1 -- 172.21.15.12:0/405751692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e41a6810 con 0x7f03e41010d0 2024-05-09T09:17:06.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.919+0000 7f03e259c700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e4102f30 0x7f03e4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:17:06.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.919+0000 7f03e1d9b700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e41010d0 0x7f03e4101550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f03d800b780 tx=0x7f03d800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:06.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.919+0000 7f03d37fe700 1 -- 172.21.15.12:0/405751692 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8004230 con 0x7f03e41010d0 2024-05-09T09:17:06.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.919+0000 7f03e8a73700 1 -- 172.21.15.12:0/405751692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e41a6b00 con 0x7f03e41010d0 2024-05-09T09:17:06.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.919+0000 7f03d37fe700 1 -- 172.21.15.12:0/405751692 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f03d80043d0 con 0x7f03e41010d0 2024-05-09T09:17:06.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.919+0000 7f03d37fe700 1 -- 172.21.15.12:0/405751692 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8011c10 con 0x7f03e41010d0 2024-05-09T09:17:06.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.919+0000 7f03e8a73700 1 -- 172.21.15.12:0/405751692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e41a7040 con 0x7f03e41010d0 2024-05-09T09:17:06.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.920+0000 7f03d37fe700 1 -- 172.21.15.12:0/405751692 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f03d801a400 con 0x7f03e41010d0 2024-05-09T09:17:06.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.920+0000 7f03e8a73700 1 -- 172.21.15.12:0/405751692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e4063980 con 0x7f03e41010d0 2024-05-09T09:17:06.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.920+0000 7f03d37fe700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f03cc039580 0x7f03cc03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:06.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.920+0000 7f03d37fe700 1 -- 172.21.15.12:0/405751692 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f03d804bff0 con 0x7f03e41010d0 2024-05-09T09:17:06.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.920+0000 7f03e259c700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f03cc039580 0x7f03cc03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:06.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.921+0000 7f03e259c700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f03cc039580 0x7f03cc03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009750 tx=0x7f03d400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:06.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:06.925+0000 7f03d37fe700 1 -- 172.21.15.12:0/405751692 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f03d804b8c0 con 0x7f03e41010d0 2024-05-09T09:17:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.138+0000 7f03e8a73700 1 -- 172.21.15.12:0/405751692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f03e41a7340 con 0x7f03e41010d0 2024-05-09T09:17:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.138+0000 7f03d37fe700 1 -- 172.21.15.12:0/405751692 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f03d801a840 con 0x7f03e41010d0 2024-05-09T09:17:07.138 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: Updating smithi012:/etc/ceph/ceph.conf 2024-05-09T09:17:07.138 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-09T09:17:07.138 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: Updating smithi170:/etc/ceph/ceph.conf 2024-05-09T09:17:07.138 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: Updating smithi016:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: Updating smithi170:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: Updating smithi012:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:17:07.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:07.140 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: Reconfiguring daemon mon.a on smithi012 2024-05-09T09:17:07.140 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:07.140 INFO:teuthology.orchestra.run.smithi012.stdout:# minimal ceph.conf for 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:17:07.140 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-05-09T09:17:07.140 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:17:07.140 INFO:teuthology.orchestra.run.smithi012.stdout: mon_host = [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2024-05-09T09:17:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.141+0000 7f03e8a73700 1 -- 172.21.15.12:0/405751692 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f03cc039580 msgr2=0x7f03cc03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.141+0000 7f03e8a73700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f03cc039580 0x7f03cc03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009750 tx=0x7f03d400b540 comp rx=0 tx=0).stop 2024-05-09T09:17:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.141+0000 7f03e8a73700 1 -- 172.21.15.12:0/405751692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e41010d0 msgr2=0x7f03e4101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.141+0000 7f03e8a73700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e41010d0 0x7f03e4101550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f03d800b780 tx=0x7f03d800bb40 comp rx=0 tx=0).stop 2024-05-09T09:17:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.141+0000 7f03e8a73700 1 -- 172.21.15.12:0/405751692 shutdown_connections 2024-05-09T09:17:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.141+0000 7f03e8a73700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f03cc039580 0x7f03cc03ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.141+0000 7f03e8a73700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03e41a3cf0 0x7f03e41a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.141+0000 7f03e8a73700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e4102f30 0x7f03e4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.141+0000 7f03e8a73700 1 --2- 172.21.15.12:0/405751692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e41010d0 0x7f03e4101550 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.141+0000 7f03e8a73700 1 -- 172.21.15.12:0/405751692 >> 172.21.15.12:0/405751692 conn(0x7f03e40fc780 msgr2=0x7f03e4103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.142+0000 7f03e8a73700 1 -- 172.21.15.12:0/405751692 shutdown_connections 2024-05-09T09:17:07.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:07.142+0000 7f03e8a73700 1 -- 172.21.15.12:0/405751692 wait complete. 2024-05-09T09:17:07.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: Updating smithi012:/etc/ceph/ceph.conf 2024-05-09T09:17:07.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-09T09:17:07.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: Updating smithi170:/etc/ceph/ceph.conf 2024-05-09T09:17:07.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: Updating smithi016:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:17:07.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: Updating smithi170:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:17:07.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: Updating smithi012:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:17:07.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T09:17:07.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:17:07.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:17:07.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:07.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: Reconfiguring daemon mon.a on smithi012 2024-05-09T09:17:07.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: Updating smithi012:/etc/ceph/ceph.conf 2024-05-09T09:17:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-09T09:17:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: Updating smithi170:/etc/ceph/ceph.conf 2024-05-09T09:17:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: Updating smithi016:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:17:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: Updating smithi170:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:17:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: Updating smithi012:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:17:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:07.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T09:17:07.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:17:07.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:17:07.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:07.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: Reconfiguring daemon mon.a on smithi012 2024-05-09T09:17:07.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:17:08.145 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:08 smithi012 ceph-mon[30140]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:08.145 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:08 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/405751692' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:08.147 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-09T09:17:08.147 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:17:08.147 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T09:17:08.176 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:17:08.176 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:17:08.242 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-09T09:17:08.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T09:17:08.272 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:08 smithi016 ceph-mon[33335]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:08.272 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:08 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/405751692' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:08.279 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-09T09:17:08.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:17:08.349 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-09T09:17:08.349 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T09:17:08.377 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:08 smithi170 ceph-mon[33632]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:08.377 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:08 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/405751692' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:08.383 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-09T09:17:08.383 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:17:08.452 INFO:tasks.cephadm:Adding mgr.x on smithi012 2024-05-09T09:17:08.452 INFO:tasks.cephadm:Adding mgr.y on smithi016 2024-05-09T09:17:08.452 INFO:tasks.cephadm:Adding mgr.z on smithi170 2024-05-09T09:17:08.452 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi012=x;smithi016=y;smithi170=z' 2024-05-09T09:17:08.728 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:17:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:09 smithi016 ceph-mon[33335]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:09 smithi016 ceph-mon[33335]: Reconfiguring mon.b (monmap changed)... 2024-05-09T09:17:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:17:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:17:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:09 smithi016 ceph-mon[33335]: Reconfiguring daemon mon.b on smithi016 2024-05-09T09:17:09.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:09 smithi170 ceph-mon[33632]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:09.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:09.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:09.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:09 smithi170 ceph-mon[33632]: Reconfiguring mon.b (monmap changed)... 2024-05-09T09:17:09.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:17:09.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:17:09.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:09.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:09 smithi170 ceph-mon[33632]: Reconfiguring daemon mon.b on smithi016 2024-05-09T09:17:09.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:09 smithi012 ceph-mon[30140]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:09.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:09.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:09.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:09 smithi012 ceph-mon[30140]: Reconfiguring mon.b (monmap changed)... 2024-05-09T09:17:09.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:17:09.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:17:09.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:09.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:09 smithi012 ceph-mon[30140]: Reconfiguring daemon mon.b on smithi016 2024-05-09T09:17:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.224+0000 7f2041eaa700 1 -- 172.21.15.170:0/1141087816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c100dc0 msgr2=0x7f203c1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.224+0000 7f2041eaa700 1 --2- 172.21.15.170:0/1141087816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c100dc0 0x7f203c1011e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f202c009910 tx=0x7f202c009c20 comp rx=0 tx=0).stop 2024-05-09T09:17:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.224+0000 7f2041eaa700 1 -- 172.21.15.170:0/1141087816 shutdown_connections 2024-05-09T09:17:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.224+0000 7f2041eaa700 1 --2- 172.21.15.170:0/1141087816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c100dc0 0x7f203c1011e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.224+0000 7f2041eaa700 1 -- 172.21.15.170:0/1141087816 >> 172.21.15.170:0/1141087816 conn(0x7f203c0fc360 msgr2=0x7f203c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.224+0000 7f2041eaa700 1 -- 172.21.15.170:0/1141087816 shutdown_connections 2024-05-09T09:17:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.224+0000 7f2041eaa700 1 -- 172.21.15.170:0/1141087816 wait complete. 2024-05-09T09:17:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.225+0000 7f2041eaa700 1 Processor -- start 2024-05-09T09:17:10.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.225+0000 7f2041eaa700 1 -- start start 2024-05-09T09:17:10.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.225+0000 7f2041eaa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c100dc0 0x7f203c06b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:10.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.225+0000 7f2041eaa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f203c19a440 0x7f203c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:10.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.225+0000 7f2041eaa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c19ce10 0x7f203c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:10.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.225+0000 7f2041eaa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f203c106df0 con 0x7f203c19ce10 2024-05-09T09:17:10.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.225+0000 7f2041eaa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f203c106c70 con 0x7f203c100dc0 2024-05-09T09:17:10.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.225+0000 7f2041eaa700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f203c106f70 con 0x7f203c19a440 2024-05-09T09:17:10.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.225+0000 7f203affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f203c19a440 0x7f203c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:10.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.225+0000 7f203affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f203c19a440 0x7f203c19c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57684/0 (socket says 172.21.15.170:57684) 2024-05-09T09:17:10.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.225+0000 7f203affd700 1 -- 172.21.15.170:0/2278606628 learned_addr learned my addr 172.21.15.170:0/2278606628 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-09T09:17:10.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.225+0000 7f203bfff700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c19ce10 0x7f203c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:10.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.226+0000 7f203b7fe700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c100dc0 0x7f203c06b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:10.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.226+0000 7f203bfff700 1 -- 172.21.15.170:0/2278606628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f203c19a440 msgr2=0x7f203c19c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:10.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.226+0000 7f203bfff700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f203c19a440 0x7f203c19c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:10.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.226+0000 7f203bfff700 1 -- 172.21.15.170:0/2278606628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c100dc0 msgr2=0x7f203c06b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:10.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.226+0000 7f203bfff700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c100dc0 0x7f203c06b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:10.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.226+0000 7f203bfff700 1 -- 172.21.15.170:0/2278606628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203c19fb70 con 0x7f203c19ce10 2024-05-09T09:17:10.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.226+0000 7f203bfff700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c19ce10 0x7f203c19f2a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f203400bd50 tx=0x7f203400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:10.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.226+0000 7f2038ff9700 1 -- 172.21.15.170:0/2278606628 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034019070 con 0x7f203c19ce10 2024-05-09T09:17:10.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.226+0000 7f2041eaa700 1 -- 172.21.15.170:0/2278606628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203c19fe60 con 0x7f203c19ce10 2024-05-09T09:17:10.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.226+0000 7f2038ff9700 1 -- 172.21.15.170:0/2278606628 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f203400cc90 con 0x7f203c19ce10 2024-05-09T09:17:10.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.226+0000 7f2041eaa700 1 -- 172.21.15.170:0/2278606628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f203c1a0390 con 0x7f203c19ce10 2024-05-09T09:17:10.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.226+0000 7f2038ff9700 1 -- 172.21.15.170:0/2278606628 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034012740 con 0x7f203c19ce10 2024-05-09T09:17:10.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.227+0000 7f2041eaa700 1 -- 172.21.15.170:0/2278606628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203c063980 con 0x7f203c19ce10 2024-05-09T09:17:10.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.227+0000 7f2038ff9700 1 -- 172.21.15.170:0/2278606628 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f203401c070 con 0x7f203c19ce10 2024-05-09T09:17:10.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.228+0000 7f2038ff9700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2024039580 0x7f202403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:10.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.228+0000 7f2038ff9700 1 -- 172.21.15.170:0/2278606628 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f203404baa0 con 0x7f203c19ce10 2024-05-09T09:17:10.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.228+0000 7f203b7fe700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2024039580 0x7f202403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:10.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.228+0000 7f203b7fe700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2024039580 0x7f202403ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f202c00b5c0 tx=0x7f202c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:10.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.233+0000 7f2038ff9700 1 -- 172.21.15.170:0/2278606628 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2034007290 con 0x7f203c19ce10 2024-05-09T09:17:10.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.443+0000 7f2041eaa700 1 -- 172.21.15.170:0/2278606628 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi016=y;smithi170=z", "target": ["mon-mgr", ""]}) -- 0x7f203c0627f0 con 0x7f2024039580 2024-05-09T09:17:10.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.451+0000 7f2038ff9700 1 -- 172.21.15.170:0/2278606628 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f203c0627f0 con 0x7f2024039580 2024-05-09T09:17:10.451 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mgr update... 2024-05-09T09:17:10.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.454+0000 7f2041eaa700 1 -- 172.21.15.170:0/2278606628 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2024039580 msgr2=0x7f202403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:10.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.454+0000 7f2041eaa700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2024039580 0x7f202403ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f202c00b5c0 tx=0x7f202c01a040 comp rx=0 tx=0).stop 2024-05-09T09:17:10.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.454+0000 7f2041eaa700 1 -- 172.21.15.170:0/2278606628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c19ce10 msgr2=0x7f203c19f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:10.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.454+0000 7f2041eaa700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c19ce10 0x7f203c19f2a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f203400bd50 tx=0x7f203400bd80 comp rx=0 tx=0).stop 2024-05-09T09:17:10.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.454+0000 7f2041eaa700 1 -- 172.21.15.170:0/2278606628 shutdown_connections 2024-05-09T09:17:10.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.454+0000 7f2041eaa700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2024039580 0x7f202403ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:10.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.454+0000 7f2041eaa700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c19ce10 0x7f203c19f2a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:10.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.454+0000 7f2041eaa700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c100dc0 0x7f203c06b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:10.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.454+0000 7f2041eaa700 1 --2- 172.21.15.170:0/2278606628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f203c19a440 0x7f203c19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:10.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.454+0000 7f2041eaa700 1 -- 172.21.15.170:0/2278606628 >> 172.21.15.170:0/2278606628 conn(0x7f203c0fc360 msgr2=0x7f203c0fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:10.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.454+0000 7f2041eaa700 1 -- 172.21.15.170:0/2278606628 shutdown_connections 2024-05-09T09:17:10.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:17:10.454+0000 7f2041eaa700 1 -- 172.21.15.170:0/2278606628 wait complete. 2024-05-09T09:17:11.010 DEBUG:teuthology.orchestra.run.smithi016:mgr.y> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.y.service 2024-05-09T09:17:11.013 DEBUG:teuthology.orchestra.run.smithi170:mgr.z> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.z.service 2024-05-09T09:17:11.015 INFO:tasks.cephadm:Deploying OSDs... 2024-05-09T09:17:11.016 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:17:11.016 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T09:17:11.036 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T09:17:11.037 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-05-09T09:17:11.097 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T09:17:11.097 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:17:11.097 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T09:17:11.097 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:17:11.098 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:17:11.098 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-09 09:16:16.906980286 +0000 2024-05-09T09:17:11.098 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-09 09:11:49.180731121 +0000 2024-05-09T09:17:11.098 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-09 09:11:49.180731121 +0000 2024-05-09T09:17:11.098 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-05-09 09:11:49.180731121 +0000 2024-05-09T09:17:11.098 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T09:17:11.166 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-05-09T09:17:11.166 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-05-09T09:17:11.166 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000281767 s, 1.8 MB/s 2024-05-09T09:17:11.168 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T09:17:11.227 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-05-09T09:17:11.284 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T09:17:11.285 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:17:11.285 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T09:17:11.285 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:17:11.285 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:17:11.285 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-09 09:16:16.943979983 +0000 2024-05-09T09:17:11.285 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-09 09:11:49.539728508 +0000 2024-05-09T09:17:11.285 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-09 09:11:49.539728508 +0000 2024-05-09T09:17:11.285 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-05-09 09:11:49.539728508 +0000 2024-05-09T09:17:11.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T09:17:11.352 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-05-09T09:17:11.352 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-05-09T09:17:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000197403 s, 2.6 MB/s 2024-05-09T09:17:11.354 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T09:17:11.412 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-05-09T09:17:11.418 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:11.418 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:11.418 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:11 smithi170 ceph-mon[33632]: Reconfiguring mon.c (monmap changed)... 2024-05-09T09:17:11.418 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:17:11.418 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:17:11.418 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:11.418 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:11 smithi170 ceph-mon[33632]: Reconfiguring daemon mon.c on smithi170 2024-05-09T09:17:11.418 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:11.470 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T09:17:11.470 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:17:11.470 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-09T09:17:11.470 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:17:11.470 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:17:11.471 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-09 09:16:16.993979574 +0000 2024-05-09T09:17:11.471 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-09 09:11:49.895725916 +0000 2024-05-09T09:17:11.471 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-09 09:11:49.895725916 +0000 2024-05-09T09:17:11.471 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-05-09 09:11:49.895725916 +0000 2024-05-09T09:17:11.471 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T09:17:11.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:11.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:11.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:11 smithi012 ceph-mon[30140]: Reconfiguring mon.c (monmap changed)... 2024-05-09T09:17:11.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:17:11.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:17:11.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:11.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:11 smithi012 ceph-mon[30140]: Reconfiguring daemon mon.c on smithi170 2024-05-09T09:17:11.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:11.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:11.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:11.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:11 smithi016 ceph-mon[33335]: Reconfiguring mon.c (monmap changed)... 2024-05-09T09:17:11.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:17:11.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:17:11.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:11.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:11 smithi016 ceph-mon[33335]: Reconfiguring daemon mon.c on smithi170 2024-05-09T09:17:11.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:11.509 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-05-09T09:17:11.510 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-05-09T09:17:11.510 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000201954 s, 2.5 MB/s 2024-05-09T09:17:11.511 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T09:17:11.568 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-05-09T09:17:11.625 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T09:17:11.626 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:17:11.626 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-09T09:17:11.626 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:17:11.626 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:17:11.626 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-09 09:16:17.034979238 +0000 2024-05-09T09:17:11.626 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-09 09:11:50.265723222 +0000 2024-05-09T09:17:11.626 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-09 09:11:50.265723222 +0000 2024-05-09T09:17:11.626 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-05-09 09:11:50.265723222 +0000 2024-05-09T09:17:11.626 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T09:17:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-05-09T09:17:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-05-09T09:17:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.00024443 s, 2.1 MB/s 2024-05-09T09:17:11.693 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T09:17:11.750 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-09T09:17:11.750 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T09:17:11.771 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T09:17:11.772 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-05-09T09:17:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T09:17:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:17:11.832 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T09:17:11.832 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:17:11.832 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:17:11.832 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-09 09:16:57.805363801 +0000 2024-05-09T09:17:11.832 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-09 09:11:30.123681898 +0000 2024-05-09T09:17:11.832 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-09 09:11:30.123681898 +0000 2024-05-09T09:17:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-05-09 09:11:30.123681898 +0000 2024-05-09T09:17:11.833 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T09:17:11.901 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-05-09T09:17:11.901 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-05-09T09:17:11.901 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000128805 s, 4.0 MB/s 2024-05-09T09:17:11.903 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T09:17:11.963 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-05-09T09:17:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T09:17:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:17:12.022 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-09T09:17:12.022 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:17:12.023 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:17:12.023 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-09 09:16:57.805363801 +0000 2024-05-09T09:17:12.023 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-09 09:11:30.449676644 +0000 2024-05-09T09:17:12.023 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-09 09:11:30.449676644 +0000 2024-05-09T09:17:12.023 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-05-09 09:11:30.449676644 +0000 2024-05-09T09:17:12.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T09:17:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-05-09T09:17:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-05-09T09:17:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000119512 s, 4.3 MB/s 2024-05-09T09:17:12.093 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T09:17:12.154 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-05-09T09:17:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T09:17:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:17:12.214 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T09:17:12.214 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:17:12.214 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:17:12.214 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-09 09:16:57.806363785 +0000 2024-05-09T09:17:12.215 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-09 09:11:30.790671148 +0000 2024-05-09T09:17:12.215 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-09 09:11:30.790671148 +0000 2024-05-09T09:17:12.215 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-05-09 09:11:30.790671148 +0000 2024-05-09T09:17:12.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T09:17:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-05-09T09:17:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-05-09T09:17:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000121772 s, 4.2 MB/s 2024-05-09T09:17:12.283 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T09:17:12.342 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-05-09T09:17:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T09:17:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:17:12.400 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-09T09:17:12.401 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:17:12.401 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:17:12.401 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-09 09:16:57.806363785 +0000 2024-05-09T09:17:12.401 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-09 09:11:31.140665507 +0000 2024-05-09T09:17:12.401 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-09 09:11:31.140665507 +0000 2024-05-09T09:17:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-05-09 09:11:31.140665507 +0000 2024-05-09T09:17:12.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T09:17:12.465 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:12 smithi016 ceph-mon[33335]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi016=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:17:12.465 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:12 smithi016 ceph-mon[33335]: Saving service mgr spec with placement smithi012=x;smithi016=y;smithi170=z;count:3 2024-05-09T09:17:12.465 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:12 smithi016 ceph-mon[33335]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:12.465 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:12 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:12.465 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:12 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:12.465 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:12 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:12.467 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-05-09T09:17:12.468 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-05-09T09:17:12.468 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.00011902 s, 4.3 MB/s 2024-05-09T09:17:12.470 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T09:17:12.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:12 smithi012 ceph-mon[30140]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi016=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:17:12.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:12 smithi012 ceph-mon[30140]: Saving service mgr spec with placement smithi012=x;smithi016=y;smithi170=z;count:3 2024-05-09T09:17:12.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:12 smithi012 ceph-mon[30140]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:12.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:12 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:12.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:12 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:12.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:12 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:12.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:12 smithi170 ceph-mon[33632]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi016=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:17:12.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:12 smithi170 ceph-mon[33632]: Saving service mgr spec with placement smithi012=x;smithi016=y;smithi170=z;count:3 2024-05-09T09:17:12.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:12 smithi170 ceph-mon[33632]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:12.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:12 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:12.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:12 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:12.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:12 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:12.529 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-09T09:17:12.529 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T09:17:12.547 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T09:17:12.548 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2024-05-09T09:17:12.604 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T09:17:12.604 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:17:12.604 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-09T09:17:12.604 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:17:12.604 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:17:12.605 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-05-09 09:16:48.247287562 +0000 2024-05-09T09:17:12.605 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-05-09 09:11:22.564522879 +0000 2024-05-09T09:17:12.605 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-05-09 09:11:22.564522879 +0000 2024-05-09T09:17:12.605 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-05-09 09:11:22.564522879 +0000 2024-05-09T09:17:12.605 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T09:17:12.669 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-05-09T09:17:12.669 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-05-09T09:17:12.669 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 9.8983e-05 s, 5.2 MB/s 2024-05-09T09:17:12.670 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T09:17:12.727 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2024-05-09T09:17:12.782 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T09:17:12.783 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:17:12.783 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-09T09:17:12.783 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:17:12.783 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:17:12.783 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-05-09 09:16:48.248287539 +0000 2024-05-09T09:17:12.783 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-05-09 09:11:22.902515128 +0000 2024-05-09T09:17:12.783 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-05-09 09:11:22.902515128 +0000 2024-05-09T09:17:12.783 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-05-09 09:11:22.902515128 +0000 2024-05-09T09:17:12.783 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T09:17:12.848 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-05-09T09:17:12.848 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-05-09T09:17:12.848 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000114188 s, 4.5 MB/s 2024-05-09T09:17:12.849 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T09:17:12.907 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2024-05-09T09:17:12.964 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T09:17:12.964 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:17:12.964 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-09T09:17:12.964 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:17:12.964 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:17:12.964 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-05-09 09:16:48.248287539 +0000 2024-05-09T09:17:12.964 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-05-09 09:11:23.223507766 +0000 2024-05-09T09:17:12.964 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-05-09 09:11:23.223507766 +0000 2024-05-09T09:17:12.964 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-05-09 09:11:23.223507766 +0000 2024-05-09T09:17:12.965 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T09:17:13.028 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-05-09T09:17:13.028 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-05-09T09:17:13.028 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000113202 s, 4.5 MB/s 2024-05-09T09:17:13.029 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T09:17:13.086 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2024-05-09T09:17:13.142 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T09:17:13.143 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:17:13.143 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-09T09:17:13.143 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:17:13.143 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:17:13.143 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-05-09 09:16:48.249287516 +0000 2024-05-09T09:17:13.143 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-05-09 09:11:23.559500062 +0000 2024-05-09T09:17:13.143 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-05-09 09:11:23.559500062 +0000 2024-05-09T09:17:13.143 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-05-09 09:11:23.559500062 +0000 2024-05-09T09:17:13.143 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T09:17:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-05-09T09:17:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-05-09T09:17:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 9.4438e-05 s, 5.4 MB/s 2024-05-09T09:17:13.211 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T09:17:13.267 INFO:tasks.cephadm:Deploying osd.0 on smithi012 with /dev/vg_nvme/lv_4... 2024-05-09T09:17:13.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T09:17:13.332 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:13.333 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:13.333 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:13.333 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:17:13.333 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T09:17:13.333 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:17:13.333 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:13.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:13.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:13.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:13.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:17:13.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T09:17:13.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:17:13.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:13.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:13.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:13.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:13.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:17:13.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T09:17:13.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:17:13.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:13.520 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:17:14.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:14 smithi012 ceph-mon[30140]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:14.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:14 smithi012 ceph-mon[30140]: Deploying daemon mgr.z on smithi170 2024-05-09T09:17:14.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:14 smithi170 ceph-mon[33632]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:14.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:14 smithi170 ceph-mon[33632]: Deploying daemon mgr.z on smithi170 2024-05-09T09:17:14.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:14 smithi016 ceph-mon[33335]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:14.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:14 smithi016 ceph-mon[33335]: Deploying daemon mgr.z on smithi170 2024-05-09T09:17:14.916 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:14 smithi170 systemd[1]: Starting Ceph mgr.z for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:17:15.217 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:14 smithi170 podman[35474]: 2024-05-09 09:17:14.820394384 +0000 UTC m=+0.017965885 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:17:15.217 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:14 smithi170 podman[35474]: 2024-05-09 09:17:14.923422148 +0000 UTC m=+0.120993625 container create 8a125eab5f62942727f2831ef747bb99c92e8f35eeef73c48a3f4705004cddfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-96805dc, io.buildah.version=1.33.5) 2024-05-09T09:17:15.217 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:15 smithi170 podman[35474]: 2024-05-09 09:17:15.134783676 +0000 UTC m=+0.332355182 container init 8a125eab5f62942727f2831ef747bb99c92e8f35eeef73c48a3f4705004cddfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-09T09:17:15.217 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:15 smithi170 podman[35474]: 2024-05-09 09:17:15.138136442 +0000 UTC m=+0.335707931 container start 8a125eab5f62942727f2831ef747bb99c92e8f35eeef73c48a3f4705004cddfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-96805dc) 2024-05-09T09:17:15.217 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:15 smithi170 ceph-mgr[35494]: -- 172.21.15.170:0/2153879323 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55917476a5a0 con 0x55917473ec00 2024-05-09T09:17:15.217 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:15 smithi170 bash[35474]: 8a125eab5f62942727f2831ef747bb99c92e8f35eeef73c48a3f4705004cddfe 2024-05-09T09:17:15.409 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T09:17:15.409 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T09:17:15.409 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-05-09T09:17:15.409 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-05-09T09:17:15.409 INFO:teuthology.orchestra.run.smithi012.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131844 s, 795 MB/s 2024-05-09T09:17:15.409 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-05-09T09:17:15.492 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:15 smithi170 systemd[1]: Started Ceph mgr.z for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:17:15.493 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:15 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:15.295+0000 7f5e2109f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T09:17:15.922 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:15 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:15.495+0000 7f5e2109f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T09:17:15.922 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:15 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:15.602+0000 7f5e2109f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T09:17:15.922 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:15 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:15.922+0000 7f5e2109f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T09:17:16.006 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_4 2024-05-09T09:17:16.243 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:16 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:16.009+0000 7f5e2109f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T09:17:16.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:16 smithi170 ceph-mon[33632]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:16.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:16 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:16.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:16 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:16.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:16 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:16.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:16 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:17:16.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:16 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T09:17:16.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:16 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:17:16.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:16 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:16.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:16 smithi016 ceph-mon[33335]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:16.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:16 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:16.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:16 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:16.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:16 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:16.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:16 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:17:16.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:16 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T09:17:16.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:16 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:17:16.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:16 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:17:16.377 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:16 smithi012 ceph-mon[30140]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:16.377 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:16 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:16.377 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:16 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:16.377 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:16 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:16.378 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:16 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:17:16.378 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:16 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T09:17:16.378 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:16 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:17:16.378 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:16 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:17.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:17 smithi012 ceph-mon[30140]: Deploying daemon mgr.y on smithi016 2024-05-09T09:17:17.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:17 smithi170 ceph-mon[33632]: Deploying daemon mgr.y on smithi016 2024-05-09T09:17:17.492 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:17 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:17.204+0000 7f5e2109f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T09:17:17.493 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:17 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:17.371+0000 7f5e2109f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T09:17:17.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:17 smithi016 ceph-mon[33335]: Deploying daemon mgr.y on smithi016 2024-05-09T09:17:17.503 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:17 smithi016 systemd[1]: Starting Ceph mgr.y for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:17:17.832 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:17 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:17.545+0000 7f5e2109f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T09:17:17.832 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:17 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:17.832+0000 7f5e2109f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T09:17:17.874 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:17 smithi016 podman[34696]: 2024-05-09 09:17:17.548059786 +0000 UTC m=+0.098938513 container create b92a20322afa34af15a4dcdbd37a074fc22eafb48c1a029dbe26780d9553e16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T09:17:17.874 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:17 smithi016 podman[34696]: 2024-05-09 09:17:17.466787827 +0000 UTC m=+0.017666574 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:17:17.874 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:17 smithi016 podman[34696]: 2024-05-09 09:17:17.767040239 +0000 UTC m=+0.317918969 container init b92a20322afa34af15a4dcdbd37a074fc22eafb48c1a029dbe26780d9553e16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc) 2024-05-09T09:17:17.874 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:17 smithi016 podman[34696]: 2024-05-09 09:17:17.769876675 +0000 UTC m=+0.320755409 container start b92a20322afa34af15a4dcdbd37a074fc22eafb48c1a029dbe26780d9553e16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T09:17:17.875 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:17 smithi016 ceph-mgr[34715]: -- 172.21.15.16:0/994852057 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c0754485a0 con 0x55c07541cc00 2024-05-09T09:17:17.875 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:17 smithi016 bash[34696]: b92a20322afa34af15a4dcdbd37a074fc22eafb48c1a029dbe26780d9553e16a 2024-05-09T09:17:18.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.114+0000 7fa19ce5f700 1 -- 172.21.15.12:0/2731121371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa198102e10 msgr2=0x7fa198105200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:18.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.114+0000 7fa19ce5f700 1 --2- 172.21.15.12:0/2731121371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa198102e10 0x7fa198105200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa18800b0a0 tx=0x7fa18800b3b0 comp rx=0 tx=0).stop 2024-05-09T09:17:18.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.114+0000 7fa19ce5f700 1 -- 172.21.15.12:0/2731121371 shutdown_connections 2024-05-09T09:17:18.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.114+0000 7fa19ce5f700 1 --2- 172.21.15.12:0/2731121371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa198108070 0x7fa19810a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:18.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.114+0000 7fa19ce5f700 1 --2- 172.21.15.12:0/2731121371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa198102e10 0x7fa198105200 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:18.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.114+0000 7fa19ce5f700 1 --2- 172.21.15.12:0/2731121371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa198105740 0x7fa198107b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:18.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.114+0000 7fa19ce5f700 1 -- 172.21.15.12:0/2731121371 >> 172.21.15.12:0/2731121371 conn(0x7fa1980fc820 msgr2=0x7fa1980fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:18.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.114+0000 7fa19ce5f700 1 -- 172.21.15.12:0/2731121371 shutdown_connections 2024-05-09T09:17:18.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.114+0000 7fa19ce5f700 1 -- 172.21.15.12:0/2731121371 wait complete. 2024-05-09T09:17:18.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.115+0000 7fa19ce5f700 1 Processor -- start 2024-05-09T09:17:18.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.115+0000 7fa19ce5f700 1 -- start start 2024-05-09T09:17:18.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.115+0000 7fa19ce5f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa198102e10 0x7fa198198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:18.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.115+0000 7fa19ce5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa198105740 0x7fa198198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:18.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.115+0000 7fa19ce5f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa198108070 0x7fa19819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:18.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.115+0000 7fa19ce5f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa19810dfb0 con 0x7fa198108070 2024-05-09T09:17:18.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.115+0000 7fa19ce5f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa19810de30 con 0x7fa198105740 2024-05-09T09:17:18.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.115+0000 7fa19ce5f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa19810e130 con 0x7fa198102e10 2024-05-09T09:17:18.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.115+0000 7fa196d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa198108070 0x7fa19819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:18.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.115+0000 7fa196d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa198108070 0x7fa19819fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39908/0 (socket says 172.21.15.12:39908) 2024-05-09T09:17:18.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.115+0000 7fa19659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa198102e10 0x7fa198198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:18.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.115+0000 7fa196d9d700 1 -- 172.21.15.12:0/427260400 learned_addr learned my addr 172.21.15.12:0/427260400 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:17:18.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.115+0000 7fa195d9b700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa198105740 0x7fa198198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:18.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.116+0000 7fa19659c700 1 -- 172.21.15.12:0/427260400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa198105740 msgr2=0x7fa198198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:18.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.116+0000 7fa19659c700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa198105740 0x7fa198198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:18.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.116+0000 7fa19659c700 1 -- 172.21.15.12:0/427260400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa198108070 msgr2=0x7fa19819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:18.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.116+0000 7fa19659c700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa198108070 0x7fa19819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:18.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.116+0000 7fa19659c700 1 -- 172.21.15.12:0/427260400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1981a0280 con 0x7fa198102e10 2024-05-09T09:17:18.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.116+0000 7fa196d9d700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa198108070 0x7fa19819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:17:18.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.116+0000 7fa19659c700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa198102e10 0x7fa198198620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa188000f80 tx=0x7fa188007d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:18.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.116+0000 7fa18f7fe700 1 -- 172.21.15.12:0/427260400 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18800bd70 con 0x7fa198102e10 2024-05-09T09:17:18.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.116+0000 7fa19ce5f700 1 -- 172.21.15.12:0/427260400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1981a0510 con 0x7fa198102e10 2024-05-09T09:17:18.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.116+0000 7fa18f7fe700 1 -- 172.21.15.12:0/427260400 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa188003e10 con 0x7fa198102e10 2024-05-09T09:17:18.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.116+0000 7fa18f7fe700 1 -- 172.21.15.12:0/427260400 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188005530 con 0x7fa198102e10 2024-05-09T09:17:18.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.116+0000 7fa19ce5f700 1 -- 172.21.15.12:0/427260400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1981a09c0 con 0x7fa198102e10 2024-05-09T09:17:18.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.117+0000 7fa18f7fe700 1 -- 172.21.15.12:0/427260400 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa1880056d0 con 0x7fa198102e10 2024-05-09T09:17:18.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.117+0000 7fa19ce5f700 1 -- 172.21.15.12:0/427260400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa198069010 con 0x7fa198102e10 2024-05-09T09:17:18.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.117+0000 7fa18f7fe700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa184039330 0x7fa18403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:18.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.117+0000 7fa18f7fe700 1 -- 172.21.15.12:0/427260400 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa18800f020 con 0x7fa198102e10 2024-05-09T09:17:18.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.117+0000 7fa195d9b700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa184039330 0x7fa18403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:18.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.118+0000 7fa195d9b700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa184039330 0x7fa18403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa180005950 tx=0x7fa18000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:18.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.123+0000 7fa18f7fe700 1 -- 172.21.15.12:0/427260400 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa18802b020 con 0x7fa198102e10 2024-05-09T09:17:18.124 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:17 smithi016 systemd[1]: Started Ceph mgr.y for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:17:18.124 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:17 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:17.927+0000 7f25f601a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T09:17:18.124 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:18 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:18.125+0000 7f25f601a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T09:17:18.131 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:17 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:17.924+0000 7f5e2109f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T09:17:18.131 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:18 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:18.132+0000 7f5e2109f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T09:17:18.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:18.340+0000 7fa19ce5f700 1 -- 172.21.15.12:0/427260400 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa198062760 con 0x7fa184039330 2024-05-09T09:17:18.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:18 smithi012 ceph-mon[30140]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:18.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:18 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:18.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:18 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:18.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:18 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:18.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:18 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:18.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:18 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:18.491 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:18 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:18.236+0000 7f25f601a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T09:17:18.491 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:18 smithi016 ceph-mon[33335]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:18.491 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:18 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:18.491 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:18 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:18.491 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:18 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:18.491 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:18 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:18.491 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:18 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:18.492 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:18 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:18.333+0000 7f5e2109f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T09:17:18.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:18 smithi170 ceph-mon[33632]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:18.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:18 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:18 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:18 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:18 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:18 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:18.753 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:18 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:18.581+0000 7f25f601a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T09:17:18.753 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:18 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:18.680+0000 7f25f601a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T09:17:19.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:19 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:17:19.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:19 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:17:19.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:19 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:19.242 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:18 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:18.971+0000 7f5e2109f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T09:17:19.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:19 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:17:19.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:19 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:17:19.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:19 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:19.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:19 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:17:19.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:19 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:17:19.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:19 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:19.722 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:19 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:19.419+0000 7f5e2109f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T09:17:19.722 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:19 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:19.520+0000 7f5e2109f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T09:17:19.722 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:19 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:19.722+0000 7f5e2109f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T09:17:20.252 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:19 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:19.995+0000 7f25f601a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T09:17:20.253 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:20 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:20.171+0000 7f25f601a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T09:17:20.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:20 smithi016 ceph-mon[33335]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:17:20.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:20 smithi016 ceph-mon[33335]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:20.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:20 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:20 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:20 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:20 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:20 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:20.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:20 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:20.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:20 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:20 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:20 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:17:20.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:20 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:17:20.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:20 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:20 smithi012 ceph-mon[30140]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:17:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:20 smithi012 ceph-mon[30140]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:20 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:20 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:20 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:20 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:20 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:20 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:20.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:20 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:20 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:20 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:17:20.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:20 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:17:20.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:20 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:20.492 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:20 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:20.243+0000 7f5e2109f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T09:17:20.493 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:20 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:20.335+0000 7f5e2109f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T09:17:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:20 smithi170 ceph-mon[33632]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:17:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:20 smithi170 ceph-mon[33632]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:20 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:20 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:20 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:20 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:20 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:20 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:20.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:20 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:20 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:20.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:20 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:17:20.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:20 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:17:20.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:20 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:20.645 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:20 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:20.346+0000 7f25f601a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T09:17:20.645 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:20 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:20.645+0000 7f25f601a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T09:17:20.922 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:20 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:20.576+0000 7f5e2109f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T09:17:20.922 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:20 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:20.709+0000 7f5e2109f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T09:17:20.922 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:20 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:20.805+0000 7f5e2109f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T09:17:20.922 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:20 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:20.922+0000 7f5e2109f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T09:17:20.953 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:20 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:20.741+0000 7f25f601a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T09:17:20.953 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:20 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:20.951+0000 7f25f601a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T09:17:21.252 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:21.152+0000 7f25f601a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T09:17:21.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:21 smithi016 ceph-mon[33335]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T09:17:21.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:21 smithi016 ceph-mon[33335]: Reconfiguring daemon mgr.x on smithi012 2024-05-09T09:17:21.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:21 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3628441472' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3abc7d81-df98-4a60-8798-58a765426230"}]: dispatch 2024-05-09T09:17:21.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:21 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3628441472' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3abc7d81-df98-4a60-8798-58a765426230"}]': finished 2024-05-09T09:17:21.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:21 smithi016 ceph-mon[33335]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T09:17:21.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:21.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:21 smithi012 ceph-mon[30140]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T09:17:21.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:21 smithi012 ceph-mon[30140]: Reconfiguring daemon mgr.x on smithi012 2024-05-09T09:17:21.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:21 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3628441472' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3abc7d81-df98-4a60-8798-58a765426230"}]: dispatch 2024-05-09T09:17:21.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:21 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3628441472' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3abc7d81-df98-4a60-8798-58a765426230"}]': finished 2024-05-09T09:17:21.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:21 smithi012 ceph-mon[30140]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T09:17:21.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:21.492 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:21.238+0000 7f5e2109f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T09:17:21.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:21 smithi170 ceph-mon[33632]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T09:17:21.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:21 smithi170 ceph-mon[33632]: Reconfiguring daemon mgr.x on smithi012 2024-05-09T09:17:21.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:21 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3628441472' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3abc7d81-df98-4a60-8798-58a765426230"}]: dispatch 2024-05-09T09:17:21.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:21 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3628441472' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3abc7d81-df98-4a60-8798-58a765426230"}]': finished 2024-05-09T09:17:21.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:21 smithi170 ceph-mon[33632]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T09:17:21.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:22.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:22.182+0000 7fa18f7fe700 1 -- 172.21.15.12:0/427260400 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7fa18801b980 con 0x7fa198102e10 2024-05-09T09:17:22.184 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:21.782+0000 7f25f601a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T09:17:22.185 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:21.832+0000 7f5e2109f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T09:17:22.185 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:17:22 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z[35489]: 2024-05-09T09:17:22.043+0000 7f5e2109f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T09:17:22.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:22 smithi012 ceph-mon[30140]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:22 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2626950170' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:17:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:22 smithi012 ceph-mon[30140]: Standby manager daemon z started 2024-05-09T09:17:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:22 smithi012 ceph-mon[30140]: from='mgr.? 172.21.15.170:0/3200942943' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T09:17:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:22 smithi012 ceph-mon[30140]: from='mgr.? 172.21.15.170:0/3200942943' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T09:17:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:22 smithi012 ceph-mon[30140]: from='mgr.? 172.21.15.170:0/3200942943' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T09:17:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:22 smithi012 ceph-mon[30140]: from='mgr.? 172.21.15.170:0/3200942943' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T09:17:22.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:22 smithi170 ceph-mon[33632]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:22.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:22 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2626950170' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:17:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:22 smithi170 ceph-mon[33632]: Standby manager daemon z started 2024-05-09T09:17:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:22 smithi170 ceph-mon[33632]: from='mgr.? 172.21.15.170:0/3200942943' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T09:17:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:22 smithi170 ceph-mon[33632]: from='mgr.? 172.21.15.170:0/3200942943' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T09:17:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:22 smithi170 ceph-mon[33632]: from='mgr.? 172.21.15.170:0/3200942943' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T09:17:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:22 smithi170 ceph-mon[33632]: from='mgr.? 172.21.15.170:0/3200942943' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T09:17:22.502 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:22 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:22.221+0000 7f25f601a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T09:17:22.502 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:22 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:22.321+0000 7f25f601a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T09:17:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:22 smithi016 ceph-mon[33335]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:22 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2626950170' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:17:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:22 smithi016 ceph-mon[33335]: Standby manager daemon z started 2024-05-09T09:17:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:22 smithi016 ceph-mon[33335]: from='mgr.? 172.21.15.170:0/3200942943' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T09:17:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:22 smithi016 ceph-mon[33335]: from='mgr.? 172.21.15.170:0/3200942943' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T09:17:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:22 smithi016 ceph-mon[33335]: from='mgr.? 172.21.15.170:0/3200942943' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T09:17:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:22 smithi016 ceph-mon[33335]: from='mgr.? 172.21.15.170:0/3200942943' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T09:17:23.002 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:22 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:22.529+0000 7f25f601a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T09:17:23.424 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:23 smithi016 ceph-mon[33335]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-09T09:17:23.425 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:23 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T09:17:23.425 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:23 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:23.077+0000 7f25f601a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T09:17:23.425 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:23 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:23.173+0000 7f25f601a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T09:17:23.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:23 smithi012 ceph-mon[30140]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-09T09:17:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:23 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T09:17:23.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:23 smithi170 ceph-mon[33632]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-09T09:17:23.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:23 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T09:17:23.752 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:23 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:23.424+0000 7f25f601a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T09:17:23.752 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:23 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:23.563+0000 7f25f601a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T09:17:23.753 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:23 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:23.661+0000 7f25f601a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T09:17:24.092 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:23 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:23.781+0000 7f25f601a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T09:17:24.092 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:24 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:24.092+0000 7f25f601a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T09:17:24.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:24 smithi012 ceph-mon[30140]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:24.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:24 smithi170 ceph-mon[33632]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:24.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:24 smithi016 ceph-mon[33335]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:25.002 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:24 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:24.720+0000 7f25f601a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T09:17:25.003 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:17:24 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y[34711]: 2024-05-09T09:17:24.943+0000 7f25f601a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T09:17:25.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:25.202+0000 7fa18f7fe700 1 -- 172.21.15.12:0/427260400 <== mon.1 v2:172.21.15.170:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fa1880253b0 con 0x7fa198102e10 2024-05-09T09:17:25.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:25 smithi012 ceph-mon[30140]: Standby manager daemon y started 2024-05-09T09:17:25.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:25 smithi012 ceph-mon[30140]: from='mgr.? 172.21.15.16:0/570003716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T09:17:25.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:25 smithi012 ceph-mon[30140]: from='mgr.? 172.21.15.16:0/570003716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T09:17:25.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:25 smithi012 ceph-mon[30140]: from='mgr.? 172.21.15.16:0/570003716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T09:17:25.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:25 smithi012 ceph-mon[30140]: from='mgr.? 172.21.15.16:0/570003716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T09:17:25.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:25 smithi170 ceph-mon[33632]: Standby manager daemon y started 2024-05-09T09:17:25.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:25 smithi170 ceph-mon[33632]: from='mgr.? 172.21.15.16:0/570003716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T09:17:25.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:25 smithi170 ceph-mon[33632]: from='mgr.? 172.21.15.16:0/570003716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T09:17:25.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:25 smithi170 ceph-mon[33632]: from='mgr.? 172.21.15.16:0/570003716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T09:17:25.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:25 smithi170 ceph-mon[33632]: from='mgr.? 172.21.15.16:0/570003716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T09:17:25.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:25 smithi016 ceph-mon[33335]: Standby manager daemon y started 2024-05-09T09:17:25.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:25 smithi016 ceph-mon[33335]: from='mgr.? 172.21.15.16:0/570003716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T09:17:25.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:25 smithi016 ceph-mon[33335]: from='mgr.? 172.21.15.16:0/570003716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T09:17:25.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:25 smithi016 ceph-mon[33335]: from='mgr.? 172.21.15.16:0/570003716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T09:17:25.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:25 smithi016 ceph-mon[33335]: from='mgr.? 172.21.15.16:0/570003716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T09:17:26.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:26 smithi012 ceph-mon[30140]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:26.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:26 smithi012 ceph-mon[30140]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-09T09:17:26.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:26 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T09:17:26.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:26 smithi170 ceph-mon[33632]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:26.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:26 smithi170 ceph-mon[33632]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-09T09:17:26.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:26 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T09:17:26.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:26 smithi016 ceph-mon[33335]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:26.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:26 smithi016 ceph-mon[33335]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-09T09:17:26.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:26 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T09:17:27.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:27 smithi012 ceph-mon[30140]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:27.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:27 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:27.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:27 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:27.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:27 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:27.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:27 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:27.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:27 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:27.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:27 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:27.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:27 smithi170 ceph-mon[33632]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:27 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:27 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:27 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:27 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:27 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:27 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:28.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:27 smithi016 ceph-mon[33335]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:28.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:27 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:28.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:27 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:28.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:27 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:28.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:27 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:28.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:27 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:28.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:27 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:29 smithi016 ceph-mon[33335]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T09:17:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:29 smithi016 ceph-mon[33335]: Deploying daemon osd.0 on smithi012 2024-05-09T09:17:29.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:29 smithi012 ceph-mon[30140]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:29.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T09:17:29.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:29.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:29 smithi012 ceph-mon[30140]: Deploying daemon osd.0 on smithi012 2024-05-09T09:17:29.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:29 smithi170 ceph-mon[33632]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T09:17:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:29 smithi170 ceph-mon[33632]: Deploying daemon osd.0 on smithi012 2024-05-09T09:17:31.894 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:31 smithi012 ceph-mon[30140]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:31.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:31 smithi170 ceph-mon[33632]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:32.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:31 smithi016 ceph-mon[33335]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:33.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:33 smithi012 ceph-mon[30140]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:33.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:33 smithi170 ceph-mon[33632]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:34.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:33 smithi016 ceph-mon[33335]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:35.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:35 smithi012 ceph-mon[30140]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:35.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:35 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:35.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:35 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:35.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:35 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:35.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:35 smithi170 ceph-mon[33632]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:35.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:35 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:35.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:35 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:35.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:35 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:36.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:35 smithi016 ceph-mon[33335]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:36.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:35 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:36.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:35 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:36.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:35 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:37.757 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:37 smithi012 ceph-mon[30140]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:37.757 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:37.757 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:37.757 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:37.757 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:37.757 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:37.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:37 smithi170 ceph-mon[33632]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:37.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:37 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:37.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:37 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:37.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:37 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:37.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:37 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:37.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:37 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:38.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:37 smithi016 ceph-mon[33335]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:38.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:37 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:38.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:37 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:38.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:37 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:38.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:37 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:38.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:37 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:38.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:38.396+0000 7fa18f7fe700 1 -- 172.21.15.12:0/427260400 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa198062760 con 0x7fa184039330 2024-05-09T09:17:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 0 on host 'smithi012' 2024-05-09T09:17:38.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:38.399+0000 7fa19ce5f700 1 -- 172.21.15.12:0/427260400 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa184039330 msgr2=0x7fa18403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:38.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:38.399+0000 7fa19ce5f700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa184039330 0x7fa18403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa180005950 tx=0x7fa18000b410 comp rx=0 tx=0).stop 2024-05-09T09:17:38.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:38.399+0000 7fa19ce5f700 1 -- 172.21.15.12:0/427260400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa198102e10 msgr2=0x7fa198198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:38.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:38.399+0000 7fa19ce5f700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa198102e10 0x7fa198198620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa188000f80 tx=0x7fa188007d90 comp rx=0 tx=0).stop 2024-05-09T09:17:38.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:38.399+0000 7fa19ce5f700 1 -- 172.21.15.12:0/427260400 shutdown_connections 2024-05-09T09:17:38.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:38.399+0000 7fa19ce5f700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa184039330 0x7fa18403b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:38.399+0000 7fa19ce5f700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa198108070 0x7fa19819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:38.399+0000 7fa19ce5f700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa198102e10 0x7fa198198620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:38.399+0000 7fa19ce5f700 1 --2- 172.21.15.12:0/427260400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa198105740 0x7fa198198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:38.399+0000 7fa19ce5f700 1 -- 172.21.15.12:0/427260400 >> 172.21.15.12:0/427260400 conn(0x7fa1980fc820 msgr2=0x7fa198103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:38.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:38.399+0000 7fa19ce5f700 1 -- 172.21.15.12:0/427260400 shutdown_connections 2024-05-09T09:17:38.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:17:38.399+0000 7fa19ce5f700 1 -- 172.21.15.12:0/427260400 wait complete. 2024-05-09T09:17:39.057 DEBUG:teuthology.orchestra.run.smithi012:osd.0> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.0.service 2024-05-09T09:17:39.059 INFO:tasks.cephadm:Deploying osd.1 on smithi016 with /dev/vg_nvme/lv_4... 2024-05-09T09:17:39.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T09:17:39.315 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:17:39.688 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:39 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:39.688 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:39 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:39.688 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:39 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:39.688 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:39 smithi016 ceph-mon[33335]: from='osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T09:17:39.688 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:39 smithi016 ceph-mon[33335]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:39.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:39 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:39.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:39 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:39.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:39 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:39.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:39 smithi012 ceph-mon[30140]: from='osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T09:17:39.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:39 smithi012 ceph-mon[30140]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:39.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:39 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:39.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:39 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:39.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:39 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:39.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:39 smithi170 ceph-mon[33632]: from='osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T09:17:39.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:39 smithi170 ceph-mon[33632]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:40.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:40 smithi012 ceph-mon[30140]: from='osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T09:17:40.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:40 smithi012 ceph-mon[30140]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T09:17:40.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:40 smithi012 ceph-mon[30140]: from='osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-09T09:17:40.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:40 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:40.484 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:17:40 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0[45217]: 2024-05-09T09:17:40.397+0000 7fd59b992700 -1 osd.0 0 waiting for initial osdmap 2024-05-09T09:17:40.484 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:17:40 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0[45217]: 2024-05-09T09:17:40.400+0000 7fd597785700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:17:40.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:40 smithi016 ceph-mon[33335]: from='osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T09:17:40.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:40 smithi016 ceph-mon[33335]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T09:17:40.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:40 smithi016 ceph-mon[33335]: from='osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-09T09:17:40.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:40 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:40.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:40 smithi170 ceph-mon[33632]: from='osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T09:17:40.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:40 smithi170 ceph-mon[33632]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T09:17:40.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:40 smithi170 ceph-mon[33632]: from='osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-09T09:17:40.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:40 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T09:17:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T09:17:41.001 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-05-09T09:17:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-05-09T09:17:41.001 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00814687 s, 1.3 GB/s 2024-05-09T09:17:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-05-09T09:17:41.589 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:41 smithi016 ceph-mon[33335]: from='osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-09T09:17:41.590 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:41 smithi016 ceph-mon[33335]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T09:17:41.590 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:41.590 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:41.590 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:41 smithi016 ceph-mon[33335]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:41.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-05-09T09:17:41.707 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:41 smithi012 ceph-mon[30140]: from='osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-09T09:17:41.708 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:41 smithi012 ceph-mon[30140]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T09:17:41.708 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:41.708 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:41.708 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:41 smithi012 ceph-mon[30140]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:41.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:41 smithi170 ceph-mon[33632]: from='osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-09T09:17:41.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:41 smithi170 ceph-mon[33632]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T09:17:41.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:41.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:41.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:41 smithi170 ceph-mon[33632]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:17:41.870 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:17:42.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: purged_snaps scrub starts 2024-05-09T09:17:42.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: purged_snaps scrub ok 2024-05-09T09:17:42.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:42.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281] boot 2024-05-09T09:17:42.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T09:17:42.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:42.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: Detected new or changed devices on smithi012 2024-05-09T09:17:42.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:42.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:42.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:17:42.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: Adjusting osd_memory_target on smithi012 to 17093M 2024-05-09T09:17:42.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:42.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:42.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:42.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:42 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:42.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: purged_snaps scrub starts 2024-05-09T09:17:42.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: purged_snaps scrub ok 2024-05-09T09:17:42.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:42.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281] boot 2024-05-09T09:17:42.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T09:17:42.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:42.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: Detected new or changed devices on smithi012 2024-05-09T09:17:42.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:42.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:42.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:17:42.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: Adjusting osd_memory_target on smithi012 to 17093M 2024-05-09T09:17:42.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:42.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:42.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:42.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:42 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:42.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: purged_snaps scrub starts 2024-05-09T09:17:42.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: purged_snaps scrub ok 2024-05-09T09:17:42.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:42.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: osd.0 [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281] boot 2024-05-09T09:17:42.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T09:17:42.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:17:42.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: Detected new or changed devices on smithi012 2024-05-09T09:17:42.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:42.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:42.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:17:42.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: Adjusting osd_memory_target on smithi012 to 17093M 2024-05-09T09:17:42.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:42.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:42.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:42.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:42 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:43.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.405+0000 7f465f95e700 1 -- 172.21.15.16:0/1760987568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46581015c0 msgr2=0x7f4658101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.405+0000 7f465f95e700 1 --2- 172.21.15.16:0/1760987568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46581015c0 0x7f4658101a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4648009990 tx=0x7f4648009ca0 comp rx=0 tx=0).stop 2024-05-09T09:17:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.405+0000 7f465f95e700 1 -- 172.21.15.16:0/1760987568 shutdown_connections 2024-05-09T09:17:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.405+0000 7f465f95e700 1 --2- 172.21.15.16:0/1760987568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4658101f80 0x7f4658102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.405+0000 7f465f95e700 1 --2- 172.21.15.16:0/1760987568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4658100c60 0x7f4658101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.405+0000 7f465f95e700 1 --2- 172.21.15.16:0/1760987568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46581015c0 0x7f4658101a40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.405+0000 7f465f95e700 1 -- 172.21.15.16:0/1760987568 >> 172.21.15.16:0/1760987568 conn(0x7f46580fc820 msgr2=0x7f46580fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.405+0000 7f465f95e700 1 -- 172.21.15.16:0/1760987568 shutdown_connections 2024-05-09T09:17:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.405+0000 7f465f95e700 1 -- 172.21.15.16:0/1760987568 wait complete. 2024-05-09T09:17:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465f95e700 1 Processor -- start 2024-05-09T09:17:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465f95e700 1 -- start start 2024-05-09T09:17:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465f95e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4658100c60 0x7f465819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465f95e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46581015c0 0x7f465819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465f95e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4658101f80 0x7f46581a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465f95e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4658110170 con 0x7f4658100c60 2024-05-09T09:17:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465f95e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f465810fff0 con 0x7f4658101f80 2024-05-09T09:17:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465f95e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f46581102f0 con 0x7f46581015c0 2024-05-09T09:17:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465defb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4658101f80 0x7f46581a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465defb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4658101f80 0x7f46581a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52720/0 (socket says 172.21.15.16:52720) 2024-05-09T09:17:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465defb700 1 -- 172.21.15.16:0/309312302 learned_addr learned my addr 172.21.15.16:0/309312302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-09T09:17:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465d6fa700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4658100c60 0x7f465819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465cef9700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46581015c0 0x7f465819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465defb700 1 -- 172.21.15.16:0/309312302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46581015c0 msgr2=0x7f465819adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465defb700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46581015c0 0x7f465819adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465defb700 1 -- 172.21.15.16:0/309312302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4658100c60 msgr2=0x7f465819a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465defb700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4658100c60 0x7f465819a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465defb700 1 -- 172.21.15.16:0/309312302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46581a24d0 con 0x7f4658101f80 2024-05-09T09:17:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.406+0000 7f465cef9700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46581015c0 0x7f465819adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:17:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.407+0000 7f465defb700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4658101f80 0x7f46581a1e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4650007ee0 tx=0x7f465000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.407+0000 7f464e7fc700 1 -- 172.21.15.16:0/309312302 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465000ddd0 con 0x7f4658101f80 2024-05-09T09:17:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.407+0000 7f464e7fc700 1 -- 172.21.15.16:0/309312302 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4650004020 con 0x7f4658101f80 2024-05-09T09:17:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.407+0000 7f465f95e700 1 -- 172.21.15.16:0/309312302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46581a27c0 con 0x7f4658101f80 2024-05-09T09:17:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.407+0000 7f464e7fc700 1 -- 172.21.15.16:0/309312302 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650015660 con 0x7f4658101f80 2024-05-09T09:17:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.407+0000 7f465f95e700 1 -- 172.21.15.16:0/309312302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46581a2cb0 con 0x7f4658101f80 2024-05-09T09:17:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.408+0000 7f465f95e700 1 -- 172.21.15.16:0/309312302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4658069010 con 0x7f4658101f80 2024-05-09T09:17:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.409+0000 7f464e7fc700 1 -- 172.21.15.16:0/309312302 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f46500041d0 con 0x7f4658101f80 2024-05-09T09:17:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.409+0000 7f464e7fc700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46440a2b90 0x7f46440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.410+0000 7f464e7fc700 1 -- 172.21.15.16:0/309312302 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f46500ccf60 con 0x7f4658101f80 2024-05-09T09:17:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.410+0000 7f465d6fa700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46440a2b90 0x7f46440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.410+0000 7f465d6fa700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46440a2b90 0x7f46440a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4654007c60 tx=0x7f46540073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:43.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.414+0000 7f464e7fc700 1 -- 172.21.15.16:0/309312302 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4650010070 con 0x7f4658101f80 2024-05-09T09:17:43.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:17:43.629+0000 7f465f95e700 1 -- 172.21.15.16:0/309312302 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4658062760 con 0x7f46440a2b90 2024-05-09T09:17:44.384 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:44 smithi016 ceph-mon[33335]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:44.384 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:44 smithi016 ceph-mon[33335]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T09:17:44.384 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:44 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:17:44.384 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:44 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:17:44.384 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:44 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:44.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:44 smithi012 ceph-mon[30140]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:44.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:44 smithi012 ceph-mon[30140]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T09:17:44.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:44 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:17:44.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:44 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:17:44.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:44 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:44.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:44 smithi170 ceph-mon[33632]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:44.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:44 smithi170 ceph-mon[33632]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T09:17:44.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:44 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:17:44.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:44 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:17:44.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:44 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:45.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:45 smithi016 ceph-mon[33335]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:17:45.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:45 smithi012 ceph-mon[30140]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:17:45.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:45 smithi170 ceph-mon[33632]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:17:46.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:46 smithi012 ceph-mon[30140]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:46.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:46 smithi170 ceph-mon[33632]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:46.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:46 smithi016 ceph-mon[33335]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:47.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:47 smithi012 ceph-mon[30140]: from='client.? 172.21.15.16:0/3164734057' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4a1fa5a-7444-4090-a12d-c626ed8abdf6"}]: dispatch 2024-05-09T09:17:47.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:47 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4a1fa5a-7444-4090-a12d-c626ed8abdf6"}]: dispatch 2024-05-09T09:17:47.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:47 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4a1fa5a-7444-4090-a12d-c626ed8abdf6"}]': finished 2024-05-09T09:17:47.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:47 smithi012 ceph-mon[30140]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T09:17:47.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:17:47.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:47 smithi012 ceph-mon[30140]: from='client.? 172.21.15.16:0/3182796738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:17:47.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:47 smithi170 ceph-mon[33632]: from='client.? 172.21.15.16:0/3164734057' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4a1fa5a-7444-4090-a12d-c626ed8abdf6"}]: dispatch 2024-05-09T09:17:47.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:47 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4a1fa5a-7444-4090-a12d-c626ed8abdf6"}]: dispatch 2024-05-09T09:17:47.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:47 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4a1fa5a-7444-4090-a12d-c626ed8abdf6"}]': finished 2024-05-09T09:17:47.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:47 smithi170 ceph-mon[33632]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T09:17:47.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:17:47.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:47 smithi170 ceph-mon[33632]: from='client.? 172.21.15.16:0/3182796738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:17:47.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:47 smithi016 ceph-mon[33335]: from='client.? 172.21.15.16:0/3164734057' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4a1fa5a-7444-4090-a12d-c626ed8abdf6"}]: dispatch 2024-05-09T09:17:47.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:47 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4a1fa5a-7444-4090-a12d-c626ed8abdf6"}]: dispatch 2024-05-09T09:17:47.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:47 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4a1fa5a-7444-4090-a12d-c626ed8abdf6"}]': finished 2024-05-09T09:17:47.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:47 smithi016 ceph-mon[33335]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T09:17:47.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:17:47.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:47 smithi016 ceph-mon[33335]: from='client.? 172.21.15.16:0/3182796738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:17:48.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:48 smithi012 ceph-mon[30140]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:48.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:48 smithi170 ceph-mon[33632]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:48.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:48 smithi016 ceph-mon[33335]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:50.314 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:50 smithi016 ceph-mon[33335]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:50.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:50 smithi012 ceph-mon[30140]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:50.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:50 smithi170 ceph-mon[33632]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:52.137 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:52 smithi016 ceph-mon[33335]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:52.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:52 smithi012 ceph-mon[30140]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:52.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:52 smithi170 ceph-mon[33632]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:54.338 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:54 smithi016 ceph-mon[33335]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:54.339 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:54 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T09:17:54.339 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:54 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:54.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:54 smithi012 ceph-mon[30140]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T09:17:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:54.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:54 smithi170 ceph-mon[33632]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:54.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T09:17:54.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:55.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:55 smithi016 ceph-mon[33335]: Deploying daemon osd.1 on smithi016 2024-05-09T09:17:55.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:55 smithi012 ceph-mon[30140]: Deploying daemon osd.1 on smithi016 2024-05-09T09:17:55.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:55 smithi170 ceph-mon[33632]: Deploying daemon osd.1 on smithi016 2024-05-09T09:17:56.298 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:56 smithi016 ceph-mon[33335]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:56 smithi012 ceph-mon[30140]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:56.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:56 smithi170 ceph-mon[33632]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:58.342 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:58 smithi016 ceph-mon[33335]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:58.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:58 smithi012 ceph-mon[30140]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:58.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:58 smithi170 ceph-mon[33632]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:59.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:59 smithi012 ceph-mon[30140]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:59.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:59 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:59.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:59 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:59.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:17:59 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:59.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:59 smithi170 ceph-mon[33632]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:17:59.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:59 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:17:59.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:59 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:59.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:17:59 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:00.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:59 smithi016 ceph-mon[33335]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:18:00.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:59 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:00.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:59 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:00.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:17:59 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:01.736 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:01 smithi016 ceph-mon[33335]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:18:01.849 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 1 on host 'smithi016' 2024-05-09T09:18:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:01.850+0000 7f464e7fc700 1 -- 172.21.15.16:0/309312302 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4658062760 con 0x7f46440a2b90 2024-05-09T09:18:01.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:01.852+0000 7f465f95e700 1 -- 172.21.15.16:0/309312302 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46440a2b90 msgr2=0x7f46440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:01.852+0000 7f465f95e700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46440a2b90 0x7f46440a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4654007c60 tx=0x7f46540073d0 comp rx=0 tx=0).stop 2024-05-09T09:18:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:01.852+0000 7f465f95e700 1 -- 172.21.15.16:0/309312302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4658101f80 msgr2=0x7f46581a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:01.852+0000 7f465f95e700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4658101f80 0x7f46581a1e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4650007ee0 tx=0x7f465000d3b0 comp rx=0 tx=0).stop 2024-05-09T09:18:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:01.852+0000 7f465f95e700 1 -- 172.21.15.16:0/309312302 shutdown_connections 2024-05-09T09:18:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:01.852+0000 7f465f95e700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46440a2b90 0x7f46440a5050 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:01.852+0000 7f465f95e700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4658101f80 0x7f46581a1e30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:01.852+0000 7f465f95e700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4658100c60 0x7f465819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:01.852+0000 7f465f95e700 1 --2- 172.21.15.16:0/309312302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46581015c0 0x7f465819adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:01.852+0000 7f465f95e700 1 -- 172.21.15.16:0/309312302 >> 172.21.15.16:0/309312302 conn(0x7f46580fc820 msgr2=0x7f46580fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:01.852+0000 7f465f95e700 1 -- 172.21.15.16:0/309312302 shutdown_connections 2024-05-09T09:18:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:01.852+0000 7f465f95e700 1 -- 172.21.15.16:0/309312302 wait complete. 2024-05-09T09:18:01.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:01 smithi012 ceph-mon[30140]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:18:01.996 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:01 smithi170 ceph-mon[33632]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:18:02.391 DEBUG:teuthology.orchestra.run.smithi016:osd.1> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.1.service 2024-05-09T09:18:02.394 INFO:tasks.cephadm:Deploying osd.2 on smithi170 with /dev/vg_nvme/lv_4... 2024-05-09T09:18:02.394 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T09:18:02.644 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:18:02.688 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:18:02 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1[37878]: 2024-05-09T09:18:02.485+0000 7f0096e4c7c0 -1 osd.1 0 log_to_monitors true 2024-05-09T09:18:02.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:02.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:02.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:02.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:02.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:02 smithi170 ceph-mon[33632]: from='osd.1 [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T09:18:02.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:02 smithi170 ceph-mon[33632]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T09:18:03.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:03.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:03.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:03.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:03.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:02 smithi016 ceph-mon[33335]: from='osd.1 [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T09:18:03.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:02 smithi016 ceph-mon[33335]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T09:18:03.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:03.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:03.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:03.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:03.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:02 smithi012 ceph-mon[30140]: from='osd.1 [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T09:18:03.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:02 smithi012 ceph-mon[30140]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T09:18:03.897 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:03 smithi170 ceph-mon[33632]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:18:03.897 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:03 smithi170 ceph-mon[33632]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T09:18:03.897 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:03 smithi170 ceph-mon[33632]: from='osd.1 [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:18:03.897 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:03 smithi170 ceph-mon[33632]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T09:18:03.898 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:03.898 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:03 smithi170 ceph-mon[33632]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:18:04.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:03 smithi016 ceph-mon[33335]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:18:04.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:03 smithi016 ceph-mon[33335]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T09:18:04.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:03 smithi016 ceph-mon[33335]: from='osd.1 [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:18:04.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:03 smithi016 ceph-mon[33335]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T09:18:04.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:03 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:04.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:03 smithi016 ceph-mon[33335]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:18:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:03 smithi012 ceph-mon[30140]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:18:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:03 smithi012 ceph-mon[30140]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T09:18:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:03 smithi012 ceph-mon[30140]: from='osd.1 [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:18:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:03 smithi012 ceph-mon[30140]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T09:18:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:03 smithi012 ceph-mon[30140]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:18:04.446 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T09:18:04.446 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T09:18:04.446 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-05-09T09:18:04.446 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-05-09T09:18:04.446 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00841542 s, 1.2 GB/s 2024-05-09T09:18:04.446 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-05-09T09:18:04.752 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:18:04 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1[37878]: 2024-05-09T09:18:04.440+0000 7f008c4b0700 -1 osd.1 0 waiting for initial osdmap 2024-05-09T09:18:04.753 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:18:04 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1[37878]: 2024-05-09T09:18:04.444+0000 7f008429b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:18:04.998 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_4 2024-05-09T09:18:05.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:04 smithi012 ceph-mon[30140]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-09T09:18:05.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:04 smithi012 ceph-mon[30140]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T09:18:05.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:05.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:05.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:04 smithi012 ceph-mon[30140]: from='osd.1 ' entity='osd.1' 2024-05-09T09:18:05.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:05.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:04 smithi170 ceph-mon[33632]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-09T09:18:05.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:04 smithi170 ceph-mon[33632]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T09:18:05.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:05.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:05.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:04 smithi170 ceph-mon[33632]: from='osd.1 ' entity='osd.1' 2024-05-09T09:18:05.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:05.243 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:18:05.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:04 smithi016 ceph-mon[33335]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-09T09:18:05.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:04 smithi016 ceph-mon[33335]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T09:18:05.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:04 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:05.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:04 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:05.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:04 smithi016 ceph-mon[33335]: from='osd.1 ' entity='osd.1' 2024-05-09T09:18:05.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:04 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: purged_snaps scrub starts 2024-05-09T09:18:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: purged_snaps scrub ok 2024-05-09T09:18:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:18:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: osd.1 [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948] boot 2024-05-09T09:18:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T09:18:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:18:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:05.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: purged_snaps scrub starts 2024-05-09T09:18:05.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: purged_snaps scrub ok 2024-05-09T09:18:05.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:18:05.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: osd.1 [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948] boot 2024-05-09T09:18:05.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T09:18:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:18:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:05.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:05.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:05.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:05.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: purged_snaps scrub starts 2024-05-09T09:18:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: purged_snaps scrub ok 2024-05-09T09:18:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:18:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: osd.1 [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948] boot 2024-05-09T09:18:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T09:18:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:18:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:18:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:06.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:06.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:06.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:06.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:06.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:06.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.684+0000 7f7f95e30700 1 -- 172.21.15.170:0/285246937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f90102370 msgr2=0x7f7f90102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:06.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.684+0000 7f7f95e30700 1 --2- 172.21.15.170:0/285246937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f90102370 0x7f7f90102790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009990 tx=0x7f7f80009ca0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.684+0000 7f7f95e30700 1 -- 172.21.15.170:0/285246937 shutdown_connections 2024-05-09T09:18:06.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.684+0000 7f7f95e30700 1 --2- 172.21.15.170:0/285246937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f90103f10 0x7f7f9010a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.684+0000 7f7f95e30700 1 --2- 172.21.15.170:0/285246937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f90102370 0x7f7f90102790 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.684+0000 7f7f95e30700 1 --2- 172.21.15.170:0/285246937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f90103570 0x7f7f901039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.684+0000 7f7f95e30700 1 -- 172.21.15.170:0/285246937 >> 172.21.15.170:0/285246937 conn(0x7f7f900fd910 msgr2=0x7f7f900ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:06.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.684+0000 7f7f95e30700 1 -- 172.21.15.170:0/285246937 shutdown_connections 2024-05-09T09:18:06.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.685+0000 7f7f95e30700 1 -- 172.21.15.170:0/285246937 wait complete. 2024-05-09T09:18:06.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.685+0000 7f7f95e30700 1 Processor -- start 2024-05-09T09:18:06.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.685+0000 7f7f95e30700 1 -- start start 2024-05-09T09:18:06.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.685+0000 7f7f95e30700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f90102370 0x7f7f900760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.685+0000 7f7f95e30700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f90103570 0x7f7f90074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.685+0000 7f7f95e30700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f90103f10 0x7f7f90074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.685+0000 7f7f95e30700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9010df60 con 0x7f7f90103f10 2024-05-09T09:18:06.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.685+0000 7f7f95e30700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9010dde0 con 0x7f7f90102370 2024-05-09T09:18:06.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.685+0000 7f7f95e30700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9010e0e0 con 0x7f7f90103570 2024-05-09T09:18:06.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f90103570 0x7f7f90074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f90103570 0x7f7f90074710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56978/0 (socket says 172.21.15.170:56978) 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8effd700 1 -- 172.21.15.170:0/2487981857 learned_addr learned my addr 172.21.15.170:0/2487981857 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8ffff700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f90103f10 0x7f7f90074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8f7fe700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f90102370 0x7f7f900760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8effd700 1 -- 172.21.15.170:0/2487981857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f90102370 msgr2=0x7f7f900760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8effd700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f90102370 0x7f7f900760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8effd700 1 -- 172.21.15.170:0/2487981857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f90103f10 msgr2=0x7f7f90074c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8effd700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f90103f10 0x7f7f90074c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8effd700 1 -- 172.21.15.170:0/2487981857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f90075500 con 0x7f7f90103570 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8effd700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f90103570 0x7f7f90074710 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400e9c0 tx=0x7f7f8400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8cff9700 1 -- 172.21.15.170:0/2487981857 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400cbd0 con 0x7f7f90103570 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8cff9700 1 -- 172.21.15.170:0/2487981857 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7f8400ee60 con 0x7f7f90103570 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f95e30700 1 -- 172.21.15.170:0/2487981857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f901a2740 con 0x7f7f90103570 2024-05-09T09:18:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f8cff9700 1 -- 172.21.15.170:0/2487981857 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f840055f0 con 0x7f7f90103570 2024-05-09T09:18:06.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.686+0000 7f7f95e30700 1 -- 172.21.15.170:0/2487981857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f901a2c50 con 0x7f7f90103570 2024-05-09T09:18:06.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.687+0000 7f7f95e30700 1 -- 172.21.15.170:0/2487981857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f90069010 con 0x7f7f90103570 2024-05-09T09:18:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.688+0000 7f7f8cff9700 1 -- 172.21.15.170:0/2487981857 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f7f84004750 con 0x7f7f90103570 2024-05-09T09:18:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.689+0000 7f7f8cff9700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7f780a2c10 0x7f7f780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.689+0000 7f7f8cff9700 1 -- 172.21.15.170:0/2487981857 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f7f840cd9c0 con 0x7f7f90103570 2024-05-09T09:18:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.689+0000 7f7f8f7fe700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7f780a2c10 0x7f7f780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.690+0000 7f7f8f7fe700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7f780a2c10 0x7f7f780a50d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000b820 tx=0x7f7f8000b8f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:06.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.694+0000 7f7f8cff9700 1 -- 172.21.15.170:0/2487981857 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7f8409c170 con 0x7f7f90103570 2024-05-09T09:18:06.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:06.901+0000 7f7f95e30700 1 -- 172.21.15.170:0/2487981857 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7f90062760 con 0x7f7f780a2c10 2024-05-09T09:18:07.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:06 smithi012 ceph-mon[30140]: Detected new or changed devices on smithi016 2024-05-09T09:18:07.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:06 smithi012 ceph-mon[30140]: Adjusting osd_memory_target on smithi016 to 17093M 2024-05-09T09:18:07.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:06 smithi170 ceph-mon[33632]: Detected new or changed devices on smithi016 2024-05-09T09:18:07.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:06 smithi170 ceph-mon[33632]: Adjusting osd_memory_target on smithi016 to 17093M 2024-05-09T09:18:07.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:06 smithi016 ceph-mon[33335]: Detected new or changed devices on smithi016 2024-05-09T09:18:07.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:06 smithi016 ceph-mon[33335]: Adjusting osd_memory_target on smithi016 to 17093M 2024-05-09T09:18:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:07 smithi012 ceph-mon[30140]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:07 smithi012 ceph-mon[30140]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T09:18:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:07 smithi012 ceph-mon[30140]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:18:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:18:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:18:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:07 smithi170 ceph-mon[33632]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:07 smithi170 ceph-mon[33632]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T09:18:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:07 smithi170 ceph-mon[33632]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:18:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:18:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:18:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:08.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:07 smithi016 ceph-mon[33335]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:08.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:07 smithi016 ceph-mon[33335]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T09:18:08.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:07 smithi016 ceph-mon[33335]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:18:08.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:18:08.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:18:08.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:10.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:09 smithi012 ceph-mon[30140]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:10.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:09 smithi012 ceph-mon[30140]: from='client.? 172.21.15.170:0/3720832338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc35c021-d74c-4b11-8e03-c54002bf99e7"}]: dispatch 2024-05-09T09:18:10.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:09 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc35c021-d74c-4b11-8e03-c54002bf99e7"}]: dispatch 2024-05-09T09:18:10.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:09 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc35c021-d74c-4b11-8e03-c54002bf99e7"}]': finished 2024-05-09T09:18:10.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:09 smithi012 ceph-mon[30140]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T09:18:10.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:10.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:09 smithi170 ceph-mon[33632]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:10.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:09 smithi170 ceph-mon[33632]: from='client.? 172.21.15.170:0/3720832338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc35c021-d74c-4b11-8e03-c54002bf99e7"}]: dispatch 2024-05-09T09:18:10.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:09 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc35c021-d74c-4b11-8e03-c54002bf99e7"}]: dispatch 2024-05-09T09:18:10.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:09 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc35c021-d74c-4b11-8e03-c54002bf99e7"}]': finished 2024-05-09T09:18:10.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:09 smithi170 ceph-mon[33632]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T09:18:10.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:10.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:09 smithi016 ceph-mon[33335]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:10.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:09 smithi016 ceph-mon[33335]: from='client.? 172.21.15.170:0/3720832338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc35c021-d74c-4b11-8e03-c54002bf99e7"}]: dispatch 2024-05-09T09:18:10.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:09 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc35c021-d74c-4b11-8e03-c54002bf99e7"}]: dispatch 2024-05-09T09:18:10.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:09 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc35c021-d74c-4b11-8e03-c54002bf99e7"}]': finished 2024-05-09T09:18:10.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:09 smithi016 ceph-mon[33335]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T09:18:10.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:11.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:10 smithi012 ceph-mon[30140]: from='client.? 172.21.15.170:0/3677515565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:18:11.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:10 smithi170 ceph-mon[33632]: from='client.? 172.21.15.170:0/3677515565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:18:11.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:10 smithi016 ceph-mon[33335]: from='client.? 172.21.15.170:0/3677515565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:18:12.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:11 smithi012 ceph-mon[30140]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:12.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:11 smithi170 ceph-mon[33632]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:12.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:11 smithi016 ceph-mon[33335]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:14.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:13 smithi012 ceph-mon[30140]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:14.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:13 smithi170 ceph-mon[33632]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:14.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:13 smithi016 ceph-mon[33335]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:16.232 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:15 smithi170 ceph-mon[33632]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:16.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:15 smithi012 ceph-mon[30140]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:16.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:15 smithi016 ceph-mon[33335]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:17.184 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:16 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T09:18:17.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:16 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:17.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:16 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T09:18:17.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:16 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:17.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:16 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T09:18:17.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:16 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:18.132 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:17 smithi170 ceph-mon[33632]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:18.132 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:17 smithi170 ceph-mon[33632]: Deploying daemon osd.2 on smithi170 2024-05-09T09:18:18.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:17 smithi012 ceph-mon[30140]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:18.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:17 smithi012 ceph-mon[30140]: Deploying daemon osd.2 on smithi170 2024-05-09T09:18:18.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:17 smithi016 ceph-mon[33335]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:18.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:17 smithi016 ceph-mon[33335]: Deploying daemon osd.2 on smithi170 2024-05-09T09:18:19.987 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:19 smithi170 ceph-mon[33632]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:20.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:19 smithi012 ceph-mon[30140]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:20.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:19 smithi016 ceph-mon[33335]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:22.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:21 smithi012 ceph-mon[30140]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:22.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:22.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:22.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:22.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:21 smithi170 ceph-mon[33632]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:22.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:22.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:22.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:21 smithi016 ceph-mon[33335]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:24.065 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:23 smithi170 ceph-mon[33632]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:24.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:23 smithi012 ceph-mon[30140]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:24.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:23 smithi016 ceph-mon[33335]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:24.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:24.720+0000 7f7f8cff9700 1 -- 172.21.15.170:0/2487981857 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7f90062760 con 0x7f7f780a2c10 2024-05-09T09:18:24.719 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 2 on host 'smithi170' 2024-05-09T09:18:24.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:24.722+0000 7f7f95e30700 1 -- 172.21.15.170:0/2487981857 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7f780a2c10 msgr2=0x7f7f780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:24.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:24.722+0000 7f7f95e30700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7f780a2c10 0x7f7f780a50d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000b820 tx=0x7f7f8000b8f0 comp rx=0 tx=0).stop 2024-05-09T09:18:24.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:24.722+0000 7f7f95e30700 1 -- 172.21.15.170:0/2487981857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f90103570 msgr2=0x7f7f90074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:24.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:24.722+0000 7f7f95e30700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f90103570 0x7f7f90074710 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400e9c0 tx=0x7f7f8400ecd0 comp rx=0 tx=0).stop 2024-05-09T09:18:24.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:24.722+0000 7f7f95e30700 1 -- 172.21.15.170:0/2487981857 shutdown_connections 2024-05-09T09:18:24.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:24.722+0000 7f7f95e30700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7f780a2c10 0x7f7f780a50d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:24.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:24.722+0000 7f7f95e30700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f90103f10 0x7f7f90074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:24.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:24.722+0000 7f7f95e30700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f90102370 0x7f7f900760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:24.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:24.722+0000 7f7f95e30700 1 --2- 172.21.15.170:0/2487981857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f90103570 0x7f7f90074710 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:24.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:24.722+0000 7f7f95e30700 1 -- 172.21.15.170:0/2487981857 >> 172.21.15.170:0/2487981857 conn(0x7f7f900fd910 msgr2=0x7f7f90106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:24.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:24.722+0000 7f7f95e30700 1 -- 172.21.15.170:0/2487981857 shutdown_connections 2024-05-09T09:18:24.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:18:24.722+0000 7f7f95e30700 1 -- 172.21.15.170:0/2487981857 wait complete. 2024-05-09T09:18:25.215 DEBUG:teuthology.orchestra.run.smithi170:osd.2> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.2.service 2024-05-09T09:18:25.218 INFO:tasks.cephadm:Deploying osd.3 on smithi012 with /dev/vg_nvme/lv_3... 2024-05-09T09:18:25.218 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T09:18:25.489 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:18:25.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:25 smithi170 ceph-mon[33632]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:25.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:25 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:25.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:25 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:25.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:25 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:25.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:25 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:25.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:25 smithi170 ceph-mon[33632]: from='osd.2 [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T09:18:25.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:25 smithi170 ceph-mon[33632]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T09:18:25.938 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:18:25 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2[38770]: 2024-05-09T09:18:25.545+0000 7f0c9a36d7c0 -1 osd.2 0 log_to_monitors true 2024-05-09T09:18:25.939 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:25 smithi012 ceph-mon[30140]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:25.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:25 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:25.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:25 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:25.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:25 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:25.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:25 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:25.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:25 smithi012 ceph-mon[30140]: from='osd.2 [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T09:18:25.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:25 smithi012 ceph-mon[30140]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T09:18:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:25 smithi016 ceph-mon[33335]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:25 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:25 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:25 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:25 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:25 smithi016 ceph-mon[33335]: from='osd.2 [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T09:18:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:25 smithi016 ceph-mon[33335]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T09:18:27.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:27 smithi170 ceph-mon[33632]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T09:18:27.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:27 smithi170 ceph-mon[33632]: from='osd.2 [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:18:27.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:27 smithi170 ceph-mon[33632]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T09:18:27.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:27 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:27.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:27 smithi170 ceph-mon[33632]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:18:27.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:27 smithi170 ceph-mon[33632]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T09:18:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T09:18:27.518 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-05-09T09:18:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-05-09T09:18:27.518 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011276 s, 930 MB/s 2024-05-09T09:18:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-05-09T09:18:27.518 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:27 smithi012 ceph-mon[30140]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T09:18:27.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:27 smithi012 ceph-mon[30140]: from='osd.2 [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:18:27.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:27 smithi012 ceph-mon[30140]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T09:18:27.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:27 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:27.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:27 smithi012 ceph-mon[30140]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:18:27.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:27 smithi012 ceph-mon[30140]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:27.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:27 smithi016 ceph-mon[33335]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T09:18:27.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:27 smithi016 ceph-mon[33335]: from='osd.2 [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:18:27.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:27 smithi016 ceph-mon[33335]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T09:18:27.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:27 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:27.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:27 smithi016 ceph-mon[33335]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:18:27.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:27 smithi016 ceph-mon[33335]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:28.170 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_3 2024-05-09T09:18:28.246 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:18:27 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2[38770]: 2024-05-09T09:18:27.899+0000 7f0c8f9d1700 -1 osd.2 0 waiting for initial osdmap 2024-05-09T09:18:28.246 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:18:27 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2[38770]: 2024-05-09T09:18:27.904+0000 7f0c877bc700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:18:28.424 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:18:28.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:28 smithi012 ceph-mon[30140]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-05-09T09:18:28.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:28 smithi012 ceph-mon[30140]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T09:18:28.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:28 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:28.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:28 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:28.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:28 smithi012 ceph-mon[30140]: from='osd.2 ' entity='osd.2' 2024-05-09T09:18:28.511 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:28 smithi170 ceph-mon[33632]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-05-09T09:18:28.511 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:28 smithi170 ceph-mon[33632]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T09:18:28.512 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:28 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:28.512 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:28 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:28.512 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:28 smithi170 ceph-mon[33632]: from='osd.2 ' entity='osd.2' 2024-05-09T09:18:28.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:28 smithi016 ceph-mon[33335]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-05-09T09:18:28.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:28 smithi016 ceph-mon[33335]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T09:18:28.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:28 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:28.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:28 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:28.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:28 smithi016 ceph-mon[33335]: from='osd.2 ' entity='osd.2' 2024-05-09T09:18:29.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: purged_snaps scrub starts 2024-05-09T09:18:29.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: purged_snaps scrub ok 2024-05-09T09:18:29.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:29.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:29.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: Detected new or changed devices on smithi170 2024-05-09T09:18:29.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:18:29.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: Adjusting osd_memory_target on smithi170 to 17088M 2024-05-09T09:18:29.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:29.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:29.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:29.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:29.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:29.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: osd.2 [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859] boot 2024-05-09T09:18:29.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T09:18:29.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:29.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: purged_snaps scrub starts 2024-05-09T09:18:29.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: purged_snaps scrub ok 2024-05-09T09:18:29.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:29.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:29.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: Detected new or changed devices on smithi170 2024-05-09T09:18:29.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:18:29.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: Adjusting osd_memory_target on smithi170 to 17088M 2024-05-09T09:18:29.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:29.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:29.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:29.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:29.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:29.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: osd.2 [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859] boot 2024-05-09T09:18:29.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T09:18:29.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: purged_snaps scrub starts 2024-05-09T09:18:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: purged_snaps scrub ok 2024-05-09T09:18:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:18:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: Detected new or changed devices on smithi170 2024-05-09T09:18:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:18:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: Adjusting osd_memory_target on smithi170 to 17088M 2024-05-09T09:18:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: osd.2 [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859] boot 2024-05-09T09:18:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T09:18:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:18:30.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.067+0000 7f0a28871700 1 -- 172.21.15.12:0/3470341622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a241057c0 msgr2=0x7f0a24107bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:30.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.067+0000 7f0a28871700 1 --2- 172.21.15.12:0/3470341622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a241057c0 0x7f0a24107bb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009990 tx=0x7f0a18009ca0 comp rx=0 tx=0).stop 2024-05-09T09:18:30.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.067+0000 7f0a28871700 1 -- 172.21.15.12:0/3470341622 shutdown_connections 2024-05-09T09:18:30.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.067+0000 7f0a28871700 1 --2- 172.21.15.12:0/3470341622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a241080f0 0x7f0a2410a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:30.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.067+0000 7f0a28871700 1 --2- 172.21.15.12:0/3470341622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a24102e90 0x7f0a24105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:30.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.067+0000 7f0a28871700 1 --2- 172.21.15.12:0/3470341622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a241057c0 0x7f0a24107bb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:30.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.067+0000 7f0a28871700 1 -- 172.21.15.12:0/3470341622 >> 172.21.15.12:0/3470341622 conn(0x7f0a240fc840 msgr2=0x7f0a240fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:30.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.067+0000 7f0a28871700 1 -- 172.21.15.12:0/3470341622 shutdown_connections 2024-05-09T09:18:30.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.067+0000 7f0a28871700 1 -- 172.21.15.12:0/3470341622 wait complete. 2024-05-09T09:18:30.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a28871700 1 Processor -- start 2024-05-09T09:18:30.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a28871700 1 -- start start 2024-05-09T09:18:30.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a28871700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24102e90 0x7f0a241985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:30.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a28871700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a241057c0 0x7f0a24198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:30.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a28871700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a241080f0 0x7f0a2419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:30.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a28871700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2410dfb0 con 0x7f0a241057c0 2024-05-09T09:18:30.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a28871700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2410de30 con 0x7f0a24102e90 2024-05-09T09:18:30.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a28871700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2410e130 con 0x7f0a241080f0 2024-05-09T09:18:30.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a21d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a241057c0 0x7f0a24198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:30.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a2259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24102e90 0x7f0a241985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:30.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a21d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a241057c0 0x7f0a24198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58748/0 (socket says 172.21.15.12:58748) 2024-05-09T09:18:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a21d9b700 1 -- 172.21.15.12:0/753509810 learned_addr learned my addr 172.21.15.12:0/753509810 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:18:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a22d9d700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a241080f0 0x7f0a2419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a21d9b700 1 -- 172.21.15.12:0/753509810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a241080f0 msgr2=0x7f0a2419fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a21d9b700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a241080f0 0x7f0a2419fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a21d9b700 1 -- 172.21.15.12:0/753509810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24102e90 msgr2=0x7f0a241985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a21d9b700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24102e90 0x7f0a241985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a21d9b700 1 -- 172.21.15.12:0/753509810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a241a0230 con 0x7f0a241057c0 2024-05-09T09:18:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.068+0000 7f0a2259c700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24102e90 0x7f0a241985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:18:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.069+0000 7f0a21d9b700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a241057c0 0x7f0a24198b10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0a18005910 tx=0x7f0a18009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.069+0000 7f0a137fe700 1 -- 172.21.15.12:0/753509810 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18027070 con 0x7f0a241057c0 2024-05-09T09:18:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.069+0000 7f0a137fe700 1 -- 172.21.15.12:0/753509810 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0a18004140 con 0x7f0a241057c0 2024-05-09T09:18:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.069+0000 7f0a28871700 1 -- 172.21.15.12:0/753509810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a241a04c0 con 0x7f0a241057c0 2024-05-09T09:18:30.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.069+0000 7f0a137fe700 1 -- 172.21.15.12:0/753509810 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18022d60 con 0x7f0a241057c0 2024-05-09T09:18:30.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.069+0000 7f0a28871700 1 -- 172.21.15.12:0/753509810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a241a0910 con 0x7f0a241057c0 2024-05-09T09:18:30.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.070+0000 7f0a28871700 1 -- 172.21.15.12:0/753509810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a24069010 con 0x7f0a241057c0 2024-05-09T09:18:30.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.071+0000 7f0a137fe700 1 -- 172.21.15.12:0/753509810 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f0a1802a070 con 0x7f0a241057c0 2024-05-09T09:18:30.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.072+0000 7f0a137fe700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0a0c0a2b90 0x7f0a0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:30.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.072+0000 7f0a2259c700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0a0c0a2b90 0x7f0a0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:30.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.072+0000 7f0a137fe700 1 -- 172.21.15.12:0/753509810 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f0a180cf080 con 0x7f0a241057c0 2024-05-09T09:18:30.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.072+0000 7f0a2259c700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0a0c0a2b90 0x7f0a0c0a5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a14010420 tx=0x7f0a140073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:30.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.075+0000 7f0a137fe700 1 -- 172.21.15.12:0/753509810 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0a18031020 con 0x7f0a241057c0 2024-05-09T09:18:30.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:30.292+0000 7f0a28871700 1 -- 172.21.15.12:0/753509810 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0a24062760 con 0x7f0a0c0a2b90 2024-05-09T09:18:30.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:30 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:18:30.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:30 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:18:30.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:30 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:30.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:30 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:18:30.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:30 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:18:30.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:30 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:30.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:30 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:18:30.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:30 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:18:30.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:30 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:31.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:31 smithi012 ceph-mon[30140]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:18:31.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:31 smithi012 ceph-mon[30140]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T09:18:31.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:31 smithi012 ceph-mon[30140]: pgmap v82: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:31.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:31 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:18:31.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:31 smithi170 ceph-mon[33632]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:18:31.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:31 smithi170 ceph-mon[33632]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T09:18:31.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:31 smithi170 ceph-mon[33632]: pgmap v82: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:31.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:31 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:18:31.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:31 smithi016 ceph-mon[33335]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:18:31.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:31 smithi016 ceph-mon[33335]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T09:18:31.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:31 smithi016 ceph-mon[33335]: pgmap v82: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:31.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:31 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:18:32.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:32 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T09:18:32.739 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:32 smithi012 ceph-mon[30140]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T09:18:32.740 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:32 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:18:32.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:32 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T09:18:32.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:32 smithi170 ceph-mon[33632]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T09:18:32.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:32 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:18:32.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:32 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T09:18:32.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:32 smithi016 ceph-mon[33335]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T09:18:32.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:32 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:18:33.234 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:18:32 smithi012 sudo[50081]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T09:18:33.234 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:18:32 smithi012 sudo[50081]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:18:33.234 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:18:32 smithi012 sudo[50081]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:18:33.234 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:18:32 smithi012 sudo[50081]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:18:33.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:33.361+0000 7f0a137fe700 1 -- 172.21.15.12:0/753509810 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0a18099610 con 0x7f0a241057c0 2024-05-09T09:18:33.642 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:18:33 smithi012 sudo[50117]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T09:18:33.643 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:18:33 smithi012 sudo[50117]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:18:33.643 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:18:33 smithi012 sudo[50117]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:18:33.643 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:18:33 smithi012 sudo[50117]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:18:33.643 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:33 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T09:18:33.643 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:33 smithi012 ceph-mon[30140]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T09:18:33.643 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:33 smithi012 ceph-mon[30140]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:33.643 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:33 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1567360304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d122daa0-3729-41b6-b07d-3c2db6f9d514"}]: dispatch 2024-05-09T09:18:33.643 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:33 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1567360304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d122daa0-3729-41b6-b07d-3c2db6f9d514"}]': finished 2024-05-09T09:18:33.644 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:33 smithi012 ceph-mon[30140]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T09:18:33.644 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:33 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:33.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:33 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T09:18:33.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:33 smithi170 ceph-mon[33632]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T09:18:33.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:33 smithi170 ceph-mon[33632]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:33.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:33 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1567360304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d122daa0-3729-41b6-b07d-3c2db6f9d514"}]: dispatch 2024-05-09T09:18:33.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:33 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1567360304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d122daa0-3729-41b6-b07d-3c2db6f9d514"}]': finished 2024-05-09T09:18:33.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:33 smithi170 ceph-mon[33632]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T09:18:33.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:33 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:33.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:33 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T09:18:33.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:33 smithi016 ceph-mon[33335]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T09:18:33.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:33 smithi016 ceph-mon[33335]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:33.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:33 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1567360304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d122daa0-3729-41b6-b07d-3c2db6f9d514"}]: dispatch 2024-05-09T09:18:33.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:33 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1567360304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d122daa0-3729-41b6-b07d-3c2db6f9d514"}]': finished 2024-05-09T09:18:33.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:33 smithi016 ceph-mon[33335]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T09:18:33.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:33 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:34.252 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:18:33 smithi016 sudo[41955]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T09:18:34.252 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:18:33 smithi016 sudo[41955]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:18:34.252 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:18:33 smithi016 sudo[41955]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:18:34.253 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:18:33 smithi016 sudo[41955]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:18:34.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:34 smithi012 ceph-mon[30140]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T09:18:34.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:34.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:34 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/873046011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:18:34.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:34.742 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:18:34 smithi170 sudo[42385]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T09:18:34.742 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:18:34 smithi170 sudo[42385]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:18:34.742 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:18:34 smithi170 sudo[42385]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:18:34.743 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:18:34 smithi170 sudo[42385]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:18:34.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:34 smithi170 ceph-mon[33632]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T09:18:34.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:34 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:34.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:34 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/873046011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:18:34.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:34 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:34.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:34 smithi016 ceph-mon[33335]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T09:18:34.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:34 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:34.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:34 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/873046011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:18:34.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:34 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:34.753 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:18:34 smithi016 sudo[41958]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T09:18:34.753 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:18:34 smithi016 sudo[41958]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:18:34.753 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:18:34 smithi016 sudo[41958]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:18:34.753 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:18:34 smithi016 sudo[41958]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:18:35.242 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:18:34 smithi170 sudo[42388]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T09:18:35.243 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:18:34 smithi170 sudo[42388]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:18:35.243 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:18:34 smithi170 sudo[42388]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:18:35.243 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:18:34 smithi170 sudo[42388]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:18:35.727 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:35 smithi012 sudo[50563]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T09:18:35.727 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:35 smithi012 sudo[50563]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:18:35.727 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:35 smithi012 sudo[50563]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:18:35.727 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:35 smithi012 sudo[50563]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:18:35.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:35 smithi012 sudo[50935]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T09:18:35.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:35 smithi012 sudo[50935]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:18:35.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:35 smithi012 sudo[50935]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:18:35.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:35 smithi012 sudo[50935]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:18:36.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:35 smithi170 sudo[42391]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T09:18:36.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:35 smithi170 sudo[42391]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:18:36.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:35 smithi170 sudo[42391]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:18:36.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:35 smithi170 sudo[42391]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:18:36.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 sudo[42394]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T09:18:36.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 sudo[42394]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:18:36.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 sudo[42394]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:18:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:36 smithi012 ceph-mon[30140]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:36 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:36 smithi012 ceph-mon[30140]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:18:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:36 smithi012 ceph-mon[30140]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:18:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:36 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:18:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:36 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:18:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:36 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:18:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:36 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:18:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:36 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:18:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:36 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:18:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 sudo[42394]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:18:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 ceph-mon[33632]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 ceph-mon[33632]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:18:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 ceph-mon[33632]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:18:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:18:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:18:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:18:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:18:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:18:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:36 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:18:36.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 ceph-mon[33335]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:36.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:36.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 ceph-mon[33335]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:18:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 ceph-mon[33335]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:18:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:18:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:18:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:18:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:18:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:18:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:18:37.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 sudo[41961]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T09:18:37.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 sudo[41961]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:18:37.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 sudo[41961]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:18:37.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:36 smithi016 sudo[41961]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:18:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:37 smithi012 ceph-mon[30140]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:18:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:37 smithi012 ceph-mon[30140]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:18:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:37 smithi012 ceph-mon[30140]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:18:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:18:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:18:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:18:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:37 smithi012 ceph-mon[30140]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:18:37.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:37 smithi170 ceph-mon[33632]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:18:37.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:37 smithi170 ceph-mon[33632]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:18:37.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:37 smithi170 ceph-mon[33632]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:18:37.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:37 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:18:37.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:37 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:18:37.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:37 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:18:37.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:37 smithi170 ceph-mon[33632]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:18:37.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:37 smithi016 sudo[41964]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T09:18:37.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:37 smithi016 sudo[41964]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:18:37.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:37 smithi016 sudo[41964]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:18:37.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:37 smithi016 sudo[41964]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:18:37.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:37 smithi016 ceph-mon[33335]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:18:37.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:37 smithi016 ceph-mon[33335]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:18:37.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:37 smithi016 ceph-mon[33335]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:18:37.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:37 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:18:37.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:37 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:18:37.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:37 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:18:37.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:37 smithi016 ceph-mon[33335]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:18:38.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:38 smithi170 ceph-mon[33632]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:38.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:38 smithi016 ceph-mon[33335]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:38.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:38 smithi012 ceph-mon[30140]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:40.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:40 smithi012 ceph-mon[30140]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:40.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:40 smithi170 ceph-mon[33632]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:40.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:40 smithi016 ceph-mon[33335]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:41.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T09:18:41.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:41.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T09:18:41.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:41.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T09:18:41.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:42.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:42 smithi012 ceph-mon[30140]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:42 smithi012 ceph-mon[30140]: Deploying daemon osd.3 on smithi012 2024-05-09T09:18:42.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:42 smithi016 ceph-mon[33335]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:42.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:42 smithi016 ceph-mon[33335]: Deploying daemon osd.3 on smithi012 2024-05-09T09:18:42.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:42 smithi170 ceph-mon[33632]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:42.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:42 smithi170 ceph-mon[33632]: Deploying daemon osd.3 on smithi012 2024-05-09T09:18:44.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:44 smithi016 ceph-mon[33335]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-05-09T09:18:44.545 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:44 smithi170 ceph-mon[33632]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-05-09T09:18:44.557 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:44 smithi012 ceph-mon[30140]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-05-09T09:18:46.383 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:46 smithi012 ceph-mon[30140]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-05-09T09:18:46.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:46 smithi016 ceph-mon[33335]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-05-09T09:18:46.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:46 smithi170 ceph-mon[33632]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-05-09T09:18:48.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:48 smithi012 ceph-mon[30140]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-09T09:18:48.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:48 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:48.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:48 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:48.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:48 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:48.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:48 smithi016 ceph-mon[33335]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-09T09:18:48.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:48 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:48.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:48 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:48.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:48 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:48.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:48 smithi170 ceph-mon[33632]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-09T09:18:48.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:48 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:48.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:48 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:48.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:48 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:50.384 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:50 smithi012 ceph-mon[30140]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:18:50.384 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:50 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:50.385 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:50 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:50.385 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:50 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:50.385 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:50 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:50.385 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:50 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:50.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:50 smithi170 ceph-mon[33632]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:18:50.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:50 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:50.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:50 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:50.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:50 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:50.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:50 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:50.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:50 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:50.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:50 smithi016 ceph-mon[33335]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:18:50.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:50 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:50.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:50 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:50.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:50 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:50.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:50 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:50.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:50 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:51.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:51.070+0000 7f0a137fe700 1 -- 172.21.15.12:0/753509810 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0a24062760 con 0x7f0a0c0a2b90 2024-05-09T09:18:51.070 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 3 on host 'smithi012' 2024-05-09T09:18:51.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:51.072+0000 7f0a28871700 1 -- 172.21.15.12:0/753509810 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0a0c0a2b90 msgr2=0x7f0a0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:51.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:51.072+0000 7f0a28871700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0a0c0a2b90 0x7f0a0c0a5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a14010420 tx=0x7f0a140073d0 comp rx=0 tx=0).stop 2024-05-09T09:18:51.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:51.072+0000 7f0a28871700 1 -- 172.21.15.12:0/753509810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a241057c0 msgr2=0x7f0a24198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:51.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:51.072+0000 7f0a28871700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a241057c0 0x7f0a24198b10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0a18005910 tx=0x7f0a18009580 comp rx=0 tx=0).stop 2024-05-09T09:18:51.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:51.072+0000 7f0a28871700 1 -- 172.21.15.12:0/753509810 shutdown_connections 2024-05-09T09:18:51.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:51.072+0000 7f0a28871700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0a0c0a2b90 0x7f0a0c0a5050 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:51.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:51.072+0000 7f0a28871700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a241080f0 0x7f0a2419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:51.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:51.072+0000 7f0a28871700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24102e90 0x7f0a241985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:51.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:51.072+0000 7f0a28871700 1 --2- 172.21.15.12:0/753509810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a241057c0 0x7f0a24198b10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:51.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:51.072+0000 7f0a28871700 1 -- 172.21.15.12:0/753509810 >> 172.21.15.12:0/753509810 conn(0x7f0a240fc840 msgr2=0x7f0a241037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:51.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:51.073+0000 7f0a28871700 1 -- 172.21.15.12:0/753509810 shutdown_connections 2024-05-09T09:18:51.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:18:51.073+0000 7f0a28871700 1 -- 172.21.15.12:0/753509810 wait complete. 2024-05-09T09:18:51.719 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:51 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:51.719 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:51 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:51.719 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:51 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:51.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:51 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:51.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:51 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:51.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:51 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:51.746 DEBUG:teuthology.orchestra.run.smithi012:osd.3> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.3.service 2024-05-09T09:18:51.752 INFO:tasks.cephadm:Deploying osd.4 on smithi016 with /dev/vg_nvme/lv_3... 2024-05-09T09:18:51.752 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T09:18:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:51 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:51 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:51 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:52.009 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:18:52.406 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:52 smithi016 ceph-mon[33335]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:18:52.407 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:52 smithi016 ceph-mon[33335]: from='osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T09:18:52.702 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:52 smithi012 ceph-mon[30140]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:18:52.702 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:52 smithi012 ceph-mon[30140]: from='osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T09:18:52.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:52 smithi170 ceph-mon[33632]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:18:52.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:52 smithi170 ceph-mon[33632]: from='osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T09:18:53.281 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:53 smithi012 ceph-mon[30140]: from='osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T09:18:53.281 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:53 smithi012 ceph-mon[30140]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T09:18:53.281 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:53 smithi012 ceph-mon[30140]: from='osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-09T09:18:53.281 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:53 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:53.426 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:53 smithi016 ceph-mon[33335]: from='osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T09:18:53.427 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:53 smithi016 ceph-mon[33335]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T09:18:53.427 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:53 smithi016 ceph-mon[33335]: from='osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-09T09:18:53.427 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:53 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:53.542 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:18:53 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3[52176]: 2024-05-09T09:18:53.280+0000 7f1272b7b700 -1 osd.3 0 waiting for initial osdmap 2024-05-09T09:18:53.542 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:18:53 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3[52176]: 2024-05-09T09:18:53.285+0000 7f126e96e700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:18:53.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:53 smithi170 ceph-mon[33632]: from='osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T09:18:53.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:53 smithi170 ceph-mon[33632]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T09:18:53.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:53 smithi170 ceph-mon[33632]: from='osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-09T09:18:53.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:53 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:54.008 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T09:18:54.008 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T09:18:54.008 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-05-09T09:18:54.008 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-05-09T09:18:54.008 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00837276 s, 1.3 GB/s 2024-05-09T09:18:54.009 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 2024-05-09T09:18:54.009 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-05-09T09:18:54.481 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:54 smithi012 ceph-mon[30140]: purged_snaps scrub starts 2024-05-09T09:18:54.481 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:54 smithi012 ceph-mon[30140]: purged_snaps scrub ok 2024-05-09T09:18:54.481 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:54 smithi012 ceph-mon[30140]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:54.481 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:54 smithi012 ceph-mon[30140]: from='osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-09T09:18:54.481 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:54 smithi012 ceph-mon[30140]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T09:18:54.481 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:54.481 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:54.481 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:54 smithi012 ceph-mon[30140]: osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762] boot 2024-05-09T09:18:54.482 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:54 smithi012 ceph-mon[30140]: osdmap e25: 4 total, 4 up, 4 in 2024-05-09T09:18:54.482 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:54.667 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:54 smithi016 ceph-mon[33335]: purged_snaps scrub starts 2024-05-09T09:18:54.667 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:54 smithi016 ceph-mon[33335]: purged_snaps scrub ok 2024-05-09T09:18:54.667 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:54 smithi016 ceph-mon[33335]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:54.668 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:54 smithi016 ceph-mon[33335]: from='osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-09T09:18:54.668 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:54 smithi016 ceph-mon[33335]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T09:18:54.668 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:54 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:54.668 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:54 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:54.668 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:54 smithi016 ceph-mon[33335]: osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762] boot 2024-05-09T09:18:54.668 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:54 smithi016 ceph-mon[33335]: osdmap e25: 4 total, 4 up, 4 in 2024-05-09T09:18:54.668 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:54 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:54.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-05-09T09:18:54.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:54 smithi170 ceph-mon[33632]: purged_snaps scrub starts 2024-05-09T09:18:54.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:54 smithi170 ceph-mon[33632]: purged_snaps scrub ok 2024-05-09T09:18:54.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:54 smithi170 ceph-mon[33632]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:18:54.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:54 smithi170 ceph-mon[33632]: from='osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-09T09:18:54.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:54 smithi170 ceph-mon[33632]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T09:18:54.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:54.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:54.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:54 smithi170 ceph-mon[33632]: osd.3 [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762] boot 2024-05-09T09:18:54.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:54 smithi170 ceph-mon[33632]: osdmap e25: 4 total, 4 up, 4 in 2024-05-09T09:18:54.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:18:54.996 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:18:55.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:55 smithi016 ceph-mon[33335]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:18:55.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:55 smithi016 ceph-mon[33335]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T09:18:55.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:55 smithi016 ceph-mon[33335]: Detected new or changed devices on smithi012 2024-05-09T09:18:55.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:55.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:55.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:18:55.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:18:55.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:55 smithi016 ceph-mon[33335]: Adjusting osd_memory_target on smithi012 to 8546M 2024-05-09T09:18:55.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:55.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:55.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:55.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:55.824 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:55 smithi170 ceph-mon[33632]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:18:55.824 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:55 smithi170 ceph-mon[33632]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T09:18:55.824 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:55 smithi170 ceph-mon[33632]: Detected new or changed devices on smithi012 2024-05-09T09:18:55.824 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:55.824 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:55.824 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:18:55.824 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:18:55.824 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:55 smithi170 ceph-mon[33632]: Adjusting osd_memory_target on smithi012 to 8546M 2024-05-09T09:18:55.824 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:55.824 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:55.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:55.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:55.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:55 smithi012 ceph-mon[30140]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:18:55.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:55 smithi012 ceph-mon[30140]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T09:18:55.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:55 smithi012 ceph-mon[30140]: Detected new or changed devices on smithi012 2024-05-09T09:18:55.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:55.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:55.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:18:55.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:18:55.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:55 smithi012 ceph-mon[30140]: Adjusting osd_memory_target on smithi012 to 8546M 2024-05-09T09:18:55.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:55.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:55.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:55.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:18:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.612+0000 7fde3d7de700 1 -- 172.21.15.16:0/1948609908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde381023a0 msgr2=0x7fde381027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.612+0000 7fde3d7de700 1 --2- 172.21.15.16:0/1948609908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde381023a0 0x7fde381027c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fde280099c0 tx=0x7fde28009cd0 comp rx=0 tx=0).stop 2024-05-09T09:18:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.612+0000 7fde3d7de700 1 -- 172.21.15.16:0/1948609908 shutdown_connections 2024-05-09T09:18:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.612+0000 7fde3d7de700 1 --2- 172.21.15.16:0/1948609908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde38103f40 0x7fde3810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.612+0000 7fde3d7de700 1 --2- 172.21.15.16:0/1948609908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde381023a0 0x7fde381027c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.612+0000 7fde3d7de700 1 --2- 172.21.15.16:0/1948609908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde381035a0 0x7fde38103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.612+0000 7fde3d7de700 1 -- 172.21.15.16:0/1948609908 >> 172.21.15.16:0/1948609908 conn(0x7fde380fd940 msgr2=0x7fde380ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.612+0000 7fde3d7de700 1 -- 172.21.15.16:0/1948609908 shutdown_connections 2024-05-09T09:18:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.612+0000 7fde3d7de700 1 -- 172.21.15.16:0/1948609908 wait complete. 2024-05-09T09:18:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.612+0000 7fde3d7de700 1 Processor -- start 2024-05-09T09:18:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.612+0000 7fde3d7de700 1 -- start start 2024-05-09T09:18:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde3d7de700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde381023a0 0x7fde380760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde3d7de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde381035a0 0x7fde38074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde3d7de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde38103f40 0x7fde38074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde3d7de700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fde3810df90 con 0x7fde381035a0 2024-05-09T09:18:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde3d7de700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde3810de10 con 0x7fde38103f40 2024-05-09T09:18:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde3d7de700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fde3810e110 con 0x7fde381023a0 2024-05-09T09:18:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde377fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde38103f40 0x7fde38074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde377fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde38103f40 0x7fde38074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53354/0 (socket says 172.21.15.16:53354) 2024-05-09T09:18:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde377fe700 1 -- 172.21.15.16:0/3888493386 learned_addr learned my addr 172.21.15.16:0/3888493386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-09T09:18:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde36ffd700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde381023a0 0x7fde380760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde367fc700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde381035a0 0x7fde38074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde377fe700 1 -- 172.21.15.16:0/3888493386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde381023a0 msgr2=0x7fde380760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde377fe700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde381023a0 0x7fde380760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde377fe700 1 -- 172.21.15.16:0/3888493386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde381035a0 msgr2=0x7fde38074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde377fe700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde381035a0 0x7fde38074710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde377fe700 1 -- 172.21.15.16:0/3888493386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde38075500 con 0x7fde38103f40 2024-05-09T09:18:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.613+0000 7fde377fe700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde38103f40 0x7fde38074c50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde30007ee0 tx=0x7fde3000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.614+0000 7fde1ffff700 1 -- 172.21.15.16:0/3888493386 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30004bb0 con 0x7fde38103f40 2024-05-09T09:18:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.614+0000 7fde3d7de700 1 -- 172.21.15.16:0/3888493386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde381a2790 con 0x7fde38103f40 2024-05-09T09:18:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.614+0000 7fde1ffff700 1 -- 172.21.15.16:0/3888493386 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde30004d50 con 0x7fde38103f40 2024-05-09T09:18:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.614+0000 7fde1ffff700 1 -- 172.21.15.16:0/3888493386 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30005970 con 0x7fde38103f40 2024-05-09T09:18:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.614+0000 7fde3d7de700 1 -- 172.21.15.16:0/3888493386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde381a2c50 con 0x7fde38103f40 2024-05-09T09:18:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.615+0000 7fde3d7de700 1 -- 172.21.15.16:0/3888493386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde38069010 con 0x7fde38103f40 2024-05-09T09:18:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.616+0000 7fde1ffff700 1 -- 172.21.15.16:0/3888493386 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fde3001a020 con 0x7fde38103f40 2024-05-09T09:18:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.617+0000 7fde1ffff700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fde200a2c20 0x7fde200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.617+0000 7fde1ffff700 1 -- 172.21.15.16:0/3888493386 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7fde300cdc40 con 0x7fde38103f40 2024-05-09T09:18:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.617+0000 7fde36ffd700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fde200a2c20 0x7fde200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.618+0000 7fde36ffd700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fde200a2c20 0x7fde200a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fde2800b580 tx=0x7fde28005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:56.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.620+0000 7fde1ffff700 1 -- 172.21.15.16:0/3888493386 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fde30010070 con 0x7fde38103f40 2024-05-09T09:18:56.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:56 smithi016 ceph-mon[33335]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T09:18:56.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:18:56.835+0000 7fde3d7de700 1 -- 172.21.15.16:0/3888493386 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fde3802dfd0 con 0x7fde200a2c20 2024-05-09T09:18:56.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:56 smithi012 ceph-mon[30140]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T09:18:56.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:56 smithi170 ceph-mon[33632]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T09:18:57.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:57 smithi016 ceph-mon[33335]: pgmap v103: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:18:57.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:57 smithi016 ceph-mon[33335]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-09T09:18:57.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:57 smithi016 ceph-mon[33335]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T09:18:57.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:57 smithi016 ceph-mon[33335]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:18:57.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:57 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:18:57.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:57 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:18:57.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:57 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:57.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:57 smithi012 ceph-mon[30140]: pgmap v103: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:18:57.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:57 smithi012 ceph-mon[30140]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-09T09:18:57.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:57 smithi012 ceph-mon[30140]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T09:18:57.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:57 smithi012 ceph-mon[30140]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:18:57.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:57 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:18:57.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:57 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:18:57.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:57 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:57.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:57 smithi170 ceph-mon[33632]: pgmap v103: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:18:57.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:57 smithi170 ceph-mon[33632]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-09T09:18:57.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:57 smithi170 ceph-mon[33632]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T09:18:57.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:57 smithi170 ceph-mon[33632]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:18:57.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:57 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:18:57.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:57 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:18:57.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:57 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:58.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:58 smithi016 ceph-mon[33335]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T09:18:58.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:58 smithi012 ceph-mon[30140]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T09:18:58.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:58 smithi170 ceph-mon[33632]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T09:18:59.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:18:59 smithi012 ceph-mon[30140]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:18:59.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:18:59 smithi170 ceph-mon[33632]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:19:00.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:18:59 smithi016 ceph-mon[33335]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:19:00.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:00 smithi012 ceph-mon[30140]: from='client.? 172.21.15.16:0/512190731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d05507ed-2594-4791-93e2-0bf32b70942a"}]: dispatch 2024-05-09T09:19:00.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:00 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d05507ed-2594-4791-93e2-0bf32b70942a"}]: dispatch 2024-05-09T09:19:00.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:00 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d05507ed-2594-4791-93e2-0bf32b70942a"}]': finished 2024-05-09T09:19:00.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:00 smithi012 ceph-mon[30140]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T09:19:00.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:00 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:00.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:00 smithi012 ceph-mon[30140]: from='client.? 172.21.15.16:0/2393817797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:19:00.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:00 smithi170 ceph-mon[33632]: from='client.? 172.21.15.16:0/512190731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d05507ed-2594-4791-93e2-0bf32b70942a"}]: dispatch 2024-05-09T09:19:00.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:00 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d05507ed-2594-4791-93e2-0bf32b70942a"}]: dispatch 2024-05-09T09:19:00.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:00 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d05507ed-2594-4791-93e2-0bf32b70942a"}]': finished 2024-05-09T09:19:00.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:00 smithi170 ceph-mon[33632]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T09:19:00.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:00 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:00.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:00 smithi170 ceph-mon[33632]: from='client.? 172.21.15.16:0/2393817797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:19:01.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:00 smithi016 ceph-mon[33335]: from='client.? 172.21.15.16:0/512190731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d05507ed-2594-4791-93e2-0bf32b70942a"}]: dispatch 2024-05-09T09:19:01.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:00 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d05507ed-2594-4791-93e2-0bf32b70942a"}]: dispatch 2024-05-09T09:19:01.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:00 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d05507ed-2594-4791-93e2-0bf32b70942a"}]': finished 2024-05-09T09:19:01.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:00 smithi016 ceph-mon[33335]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T09:19:01.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:00 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:01.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:00 smithi016 ceph-mon[33335]: from='client.? 172.21.15.16:0/2393817797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:19:01.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:01 smithi012 ceph-mon[30140]: pgmap v108: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:19:01.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:01 smithi170 ceph-mon[33632]: pgmap v108: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:19:02.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:01 smithi016 ceph-mon[33335]: pgmap v108: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:19:02.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:02 smithi012 ceph-mon[30140]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-09T09:19:02.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:02 smithi012 ceph-mon[30140]: Cluster is now healthy 2024-05-09T09:19:02.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:02 smithi170 ceph-mon[33632]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-09T09:19:02.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:02 smithi170 ceph-mon[33632]: Cluster is now healthy 2024-05-09T09:19:03.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:02 smithi016 ceph-mon[33335]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-09T09:19:03.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:02 smithi016 ceph-mon[33335]: Cluster is now healthy 2024-05-09T09:19:03.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:03 smithi012 ceph-mon[30140]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T09:19:03.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:03 smithi170 ceph-mon[33632]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T09:19:04.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:03 smithi016 ceph-mon[33335]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T09:19:05.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:05 smithi016 ceph-mon[33335]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:19:05.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:05 smithi012 ceph-mon[30140]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:19:05.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:05 smithi170 ceph-mon[33632]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:19:07.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:07 smithi016 ceph-mon[33335]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-09T09:19:07.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T09:19:07.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:07.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:07 smithi016 ceph-mon[33335]: Deploying daemon osd.4 on smithi016 2024-05-09T09:19:07.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:07 smithi012 ceph-mon[30140]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-09T09:19:07.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T09:19:07.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:07.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:07 smithi012 ceph-mon[30140]: Deploying daemon osd.4 on smithi016 2024-05-09T09:19:07.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:07 smithi170 ceph-mon[33632]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-09T09:19:07.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T09:19:07.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:07.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:07 smithi170 ceph-mon[33632]: Deploying daemon osd.4 on smithi016 2024-05-09T09:19:09.940 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:09 smithi016 ceph-mon[33335]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:19:09.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:09 smithi012 ceph-mon[30140]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:19:09.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:09 smithi170 ceph-mon[33632]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:19:11.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:11 smithi012 ceph-mon[30140]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-09T09:19:11.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:11 smithi170 ceph-mon[33632]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-09T09:19:12.006 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:11 smithi016 ceph-mon[33335]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-09T09:19:13.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:13 smithi012 ceph-mon[30140]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T09:19:13.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:13.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:13.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:13.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:13 smithi170 ceph-mon[33632]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T09:19:13.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:13.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:13.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:14.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:13 smithi016 ceph-mon[33335]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T09:19:14.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:14.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:14.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:15.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:15 smithi016 ceph-mon[33335]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:19:15.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:15 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:15.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:15 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:15.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:15 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:15.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:15 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:15.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:15 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:15.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:15 smithi012 ceph-mon[30140]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:19:15.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:15 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:15.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:15 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:15.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:15 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:15.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:15 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:15.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:15 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:15.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:15 smithi170 ceph-mon[33632]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:19:15.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:15 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:15.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:15 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:15.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:15 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:15.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:15 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:15.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:15 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:16.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:19:16.553+0000 7fde1ffff700 1 -- 172.21.15.16:0/3888493386 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fde3802dfd0 con 0x7fde200a2c20 2024-05-09T09:19:16.553 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 4 on host 'smithi016' 2024-05-09T09:19:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:19:16.555+0000 7fde3d7de700 1 -- 172.21.15.16:0/3888493386 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fde200a2c20 msgr2=0x7fde200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:19:16.555+0000 7fde3d7de700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fde200a2c20 0x7fde200a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fde2800b580 tx=0x7fde28005fb0 comp rx=0 tx=0).stop 2024-05-09T09:19:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:19:16.555+0000 7fde3d7de700 1 -- 172.21.15.16:0/3888493386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde38103f40 msgr2=0x7fde38074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:16.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:19:16.555+0000 7fde3d7de700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde38103f40 0x7fde38074c50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde30007ee0 tx=0x7fde3000d3b0 comp rx=0 tx=0).stop 2024-05-09T09:19:16.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:19:16.555+0000 7fde3d7de700 1 -- 172.21.15.16:0/3888493386 shutdown_connections 2024-05-09T09:19:16.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:19:16.555+0000 7fde3d7de700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fde200a2c20 0x7fde200a50e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:16.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:19:16.555+0000 7fde3d7de700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde38103f40 0x7fde38074c50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:16.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:19:16.555+0000 7fde3d7de700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde381023a0 0x7fde380760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:16.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:19:16.555+0000 7fde3d7de700 1 --2- 172.21.15.16:0/3888493386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde381035a0 0x7fde38074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:16.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:19:16.555+0000 7fde3d7de700 1 -- 172.21.15.16:0/3888493386 >> 172.21.15.16:0/3888493386 conn(0x7fde380fd940 msgr2=0x7fde38106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:16.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:19:16.555+0000 7fde3d7de700 1 -- 172.21.15.16:0/3888493386 shutdown_connections 2024-05-09T09:19:16.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:19:16.555+0000 7fde3d7de700 1 -- 172.21.15.16:0/3888493386 wait complete. 2024-05-09T09:19:16.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:16 smithi016 ceph-mon[33335]: from='osd.4 [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T09:19:16.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:16 smithi016 ceph-mon[33335]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T09:19:16.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:16 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:16.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:16 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:16.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:16 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:16.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:16 smithi012 ceph-mon[30140]: from='osd.4 [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T09:19:16.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:16 smithi012 ceph-mon[30140]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T09:19:16.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:16 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:16.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:16 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:16.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:16 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:16.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:16 smithi170 ceph-mon[33632]: from='osd.4 [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T09:19:16.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:16 smithi170 ceph-mon[33632]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T09:19:16.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:16 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:16.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:16 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:16.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:16 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:17.190 DEBUG:teuthology.orchestra.run.smithi016:osd.4> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.4.service 2024-05-09T09:19:17.193 INFO:tasks.cephadm:Deploying osd.5 on smithi170 with /dev/vg_nvme/lv_3... 2024-05-09T09:19:17.193 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T09:19:17.441 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:19:17.820 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:17 smithi170 ceph-mon[33632]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:19:17.820 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:17 smithi170 ceph-mon[33632]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T09:19:17.820 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:17 smithi170 ceph-mon[33632]: from='osd.4 [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:19:17.820 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:17 smithi170 ceph-mon[33632]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T09:19:17.820 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:17 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:17.820 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:17 smithi170 ceph-mon[33632]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:19:17.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:17 smithi012 ceph-mon[30140]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:19:17.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:17 smithi012 ceph-mon[30140]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T09:19:17.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:17 smithi012 ceph-mon[30140]: from='osd.4 [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:19:17.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:17 smithi012 ceph-mon[30140]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T09:19:17.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:17 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:17.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:17 smithi012 ceph-mon[30140]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:19:18.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:17 smithi016 ceph-mon[33335]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:19:18.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:17 smithi016 ceph-mon[33335]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T09:19:18.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:17 smithi016 ceph-mon[33335]: from='osd.4 [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:19:18.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:17 smithi016 ceph-mon[33335]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T09:19:18.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:17 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:18.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:17 smithi016 ceph-mon[33335]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:19:18.502 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:19:18 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4[44768]: 2024-05-09T09:19:18.294+0000 7f2d6a072700 -1 osd.4 0 waiting for initial osdmap 2024-05-09T09:19:18.503 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:19:18 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4[44768]: 2024-05-09T09:19:18.299+0000 7f2d61e5d700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:19:18.832 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:18 smithi016 ceph-mon[33335]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-09T09:19:18.832 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:18 smithi016 ceph-mon[33335]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T09:19:18.832 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:18 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:18.833 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:18 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:18.833 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:18 smithi016 ceph-mon[33335]: from='osd.4 ' entity='osd.4' 2024-05-09T09:19:18.868 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:18 smithi170 ceph-mon[33632]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-09T09:19:18.869 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:18 smithi170 ceph-mon[33632]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T09:19:18.869 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:18 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:18.869 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:18 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:18.869 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:18 smithi170 ceph-mon[33632]: from='osd.4 ' entity='osd.4' 2024-05-09T09:19:18.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:18 smithi012 ceph-mon[30140]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-09T09:19:18.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:18 smithi012 ceph-mon[30140]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T09:19:18.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:18 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:18.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:18 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:18.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:18 smithi012 ceph-mon[30140]: from='osd.4 ' entity='osd.4' 2024-05-09T09:19:19.439 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T09:19:19.439 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T09:19:19.439 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-05-09T09:19:19.439 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-05-09T09:19:19.439 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00891292 s, 1.2 GB/s 2024-05-09T09:19:19.439 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-05-09T09:19:19.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:19 smithi012 ceph-mon[30140]: purged_snaps scrub starts 2024-05-09T09:19:19.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:19 smithi012 ceph-mon[30140]: purged_snaps scrub ok 2024-05-09T09:19:19.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:19 smithi012 ceph-mon[30140]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:19:19.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:19 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:19.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:19 smithi012 ceph-mon[30140]: osd.4 [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120] boot 2024-05-09T09:19:19.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:19 smithi012 ceph-mon[30140]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T09:19:19.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:19 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:19.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:19 smithi170 ceph-mon[33632]: purged_snaps scrub starts 2024-05-09T09:19:19.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:19 smithi170 ceph-mon[33632]: purged_snaps scrub ok 2024-05-09T09:19:19.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:19 smithi170 ceph-mon[33632]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:19:19.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:19 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:19.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:19 smithi170 ceph-mon[33632]: osd.4 [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120] boot 2024-05-09T09:19:19.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:19 smithi170 ceph-mon[33632]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T09:19:19.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:19 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:20.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:19 smithi016 ceph-mon[33335]: purged_snaps scrub starts 2024-05-09T09:19:20.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:19 smithi016 ceph-mon[33335]: purged_snaps scrub ok 2024-05-09T09:19:20.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:19 smithi016 ceph-mon[33335]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:19:20.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:19 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:20.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:19 smithi016 ceph-mon[33335]: osd.4 [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120] boot 2024-05-09T09:19:20.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:19 smithi016 ceph-mon[33335]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T09:19:20.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:19 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:19:20.075 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_3 2024-05-09T09:19:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:19:21.437 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:21 smithi170 ceph-mon[33632]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T09:19:21.437 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:21 smithi170 ceph-mon[33632]: Detected new or changed devices on smithi016 2024-05-09T09:19:21.437 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:21 smithi170 ceph-mon[33632]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:21.437 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:21.438 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:21.438 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:19:21.438 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:19:21.438 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:21 smithi170 ceph-mon[33632]: Adjusting osd_memory_target on smithi016 to 8546M 2024-05-09T09:19:21.438 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:21.438 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:21.438 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:21.438 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:21.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:21 smithi012 ceph-mon[30140]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T09:19:21.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:21 smithi012 ceph-mon[30140]: Detected new or changed devices on smithi016 2024-05-09T09:19:21.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:21 smithi012 ceph-mon[30140]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:21.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:21.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:21.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:19:21.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:19:21.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:21 smithi012 ceph-mon[30140]: Adjusting osd_memory_target on smithi016 to 8546M 2024-05-09T09:19:21.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:21.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:21.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:21.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:21.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:21 smithi016 ceph-mon[33335]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T09:19:21.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:21 smithi016 ceph-mon[33335]: Detected new or changed devices on smithi016 2024-05-09T09:19:21.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:21 smithi016 ceph-mon[33335]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:21.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:21.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:21.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:19:21.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:19:21.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:21 smithi016 ceph-mon[33335]: Adjusting osd_memory_target on smithi016 to 8546M 2024-05-09T09:19:21.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:21.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:21.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:21.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:21.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.961+0000 7ff57035e700 1 -- 172.21.15.170:0/1533577713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff568103f20 msgr2=0x7ff56810a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:21.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.961+0000 7ff57035e700 1 --2- 172.21.15.170:0/1533577713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff568103f20 0x7ff56810a7d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff56000b0a0 tx=0x7ff56000b3b0 comp rx=0 tx=0).stop 2024-05-09T09:19:21.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.962+0000 7ff57035e700 1 -- 172.21.15.170:0/1533577713 shutdown_connections 2024-05-09T09:19:21.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.962+0000 7ff57035e700 1 --2- 172.21.15.170:0/1533577713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff568103f20 0x7ff56810a7d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:21.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.962+0000 7ff57035e700 1 --2- 172.21.15.170:0/1533577713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff568102380 0x7ff5681027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:21.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.962+0000 7ff57035e700 1 --2- 172.21.15.170:0/1533577713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff568103580 0x7ff5681039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:21.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.962+0000 7ff57035e700 1 -- 172.21.15.170:0/1533577713 >> 172.21.15.170:0/1533577713 conn(0x7ff5680fd920 msgr2=0x7ff5680ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.962+0000 7ff57035e700 1 -- 172.21.15.170:0/1533577713 shutdown_connections 2024-05-09T09:19:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.962+0000 7ff57035e700 1 -- 172.21.15.170:0/1533577713 wait complete. 2024-05-09T09:19:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.962+0000 7ff57035e700 1 Processor -- start 2024-05-09T09:19:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.962+0000 7ff57035e700 1 -- start start 2024-05-09T09:19:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.962+0000 7ff57035e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff568102380 0x7ff568198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.962+0000 7ff57035e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff568103580 0x7ff568198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff57035e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff568103f20 0x7ff56819fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff57035e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff56810dfd0 con 0x7ff568103f20 2024-05-09T09:19:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff57035e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff56810de50 con 0x7ff568102380 2024-05-09T09:19:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff57035e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff56810e150 con 0x7ff568103580 2024-05-09T09:19:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff56d8f9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff568103580 0x7ff568198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:21.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff56d8f9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff568103580 0x7ff568198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59974/0 (socket says 172.21.15.170:59974) 2024-05-09T09:19:21.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff56d8f9700 1 -- 172.21.15.170:0/3454028522 learned_addr learned my addr 172.21.15.170:0/3454028522 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-09T09:19:21.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff56e8fb700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff568103f20 0x7ff56819fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:21.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff56d8f9700 1 -- 172.21.15.170:0/3454028522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff568102380 msgr2=0x7ff568198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:21.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff56e0fa700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff568102380 0x7ff568198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff56d8f9700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff568102380 0x7ff568198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff56d8f9700 1 -- 172.21.15.170:0/3454028522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff568103f20 msgr2=0x7ff56819fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff56d8f9700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff568103f20 0x7ff56819fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff56d8f9700 1 -- 172.21.15.170:0/3454028522 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5681a01e0 con 0x7ff568103580 2024-05-09T09:19:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff56e0fa700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff568102380 0x7ff568198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:19:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff56d8f9700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff568103580 0x7ff568198ac0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff56400c950 tx=0x7ff56400cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff55b7fe700 1 -- 172.21.15.170:0/3454028522 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564007ac0 con 0x7ff568103580 2024-05-09T09:19:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff57035e700 1 -- 172.21.15.170:0/3454028522 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5681a04d0 con 0x7ff568103580 2024-05-09T09:19:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff55b7fe700 1 -- 172.21.15.170:0/3454028522 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff564004d00 con 0x7ff568103580 2024-05-09T09:19:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff55b7fe700 1 -- 172.21.15.170:0/3454028522 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564005680 con 0x7ff568103580 2024-05-09T09:19:21.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.963+0000 7ff57035e700 1 -- 172.21.15.170:0/3454028522 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5681a0a60 con 0x7ff568103580 2024-05-09T09:19:21.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.964+0000 7ff57035e700 1 -- 172.21.15.170:0/3454028522 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff568069010 con 0x7ff568103580 2024-05-09T09:19:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.965+0000 7ff55b7fe700 1 -- 172.21.15.170:0/3454028522 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff564004750 con 0x7ff568103580 2024-05-09T09:19:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.966+0000 7ff55b7fe700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff5540a2c20 0x7ff5540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.966+0000 7ff55b7fe700 1 -- 172.21.15.170:0/3454028522 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7ff5640cdff0 con 0x7ff568103580 2024-05-09T09:19:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.966+0000 7ff56e0fa700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff5540a2c20 0x7ff5540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.967+0000 7ff56e0fa700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff5540a2c20 0x7ff5540a50e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff5681033e0 tx=0x7ff55c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:21.970+0000 7ff55b7fe700 1 -- 172.21.15.170:0/3454028522 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff564014030 con 0x7ff568103580 2024-05-09T09:19:22.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:22.176+0000 7ff57035e700 1 -- 172.21.15.170:0/3454028522 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff568062760 con 0x7ff5540a2c20 2024-05-09T09:19:22.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:22 smithi170 ceph-mon[33632]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:19:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:22 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:19:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:22 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:19:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:22 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:22.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:22 smithi012 ceph-mon[30140]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:19:22.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:22 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:19:22.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:22 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:19:22.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:22 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:22.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:22 smithi016 ceph-mon[33335]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:19:22.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:22 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:19:22.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:22 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:19:22.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:22 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:23.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:23 smithi170 ceph-mon[33632]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:23.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:23 smithi012 ceph-mon[30140]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:23.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:23 smithi016 ceph-mon[33335]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:25.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:25 smithi012 ceph-mon[30140]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:25.987 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:25 smithi012 ceph-mon[30140]: from='client.? 172.21.15.170:0/3222071123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f06cd7-6010-43cb-9dab-71ea07e02745"}]: dispatch 2024-05-09T09:19:25.987 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:25 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f06cd7-6010-43cb-9dab-71ea07e02745"}]: dispatch 2024-05-09T09:19:25.987 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:25 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40f06cd7-6010-43cb-9dab-71ea07e02745"}]': finished 2024-05-09T09:19:25.987 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:25 smithi012 ceph-mon[30140]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T09:19:25.987 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:25 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:19:25.987 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:25 smithi012 ceph-mon[30140]: from='client.? 172.21.15.170:0/2394229509' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:19:25.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:25 smithi170 ceph-mon[33632]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:25.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:25 smithi170 ceph-mon[33632]: from='client.? 172.21.15.170:0/3222071123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f06cd7-6010-43cb-9dab-71ea07e02745"}]: dispatch 2024-05-09T09:19:25.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:25 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f06cd7-6010-43cb-9dab-71ea07e02745"}]: dispatch 2024-05-09T09:19:25.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:25 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40f06cd7-6010-43cb-9dab-71ea07e02745"}]': finished 2024-05-09T09:19:25.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:25 smithi170 ceph-mon[33632]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T09:19:25.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:25 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:19:25.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:25 smithi170 ceph-mon[33632]: from='client.? 172.21.15.170:0/2394229509' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:19:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:25 smithi016 ceph-mon[33335]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:25 smithi016 ceph-mon[33335]: from='client.? 172.21.15.170:0/3222071123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f06cd7-6010-43cb-9dab-71ea07e02745"}]: dispatch 2024-05-09T09:19:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:25 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f06cd7-6010-43cb-9dab-71ea07e02745"}]: dispatch 2024-05-09T09:19:26.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:25 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40f06cd7-6010-43cb-9dab-71ea07e02745"}]': finished 2024-05-09T09:19:26.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:25 smithi016 ceph-mon[33335]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T09:19:26.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:25 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:19:26.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:25 smithi016 ceph-mon[33335]: from='client.? 172.21.15.170:0/2394229509' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:19:27.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:27 smithi012 ceph-mon[30140]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:27 smithi170 ceph-mon[33632]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:27.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:27 smithi016 ceph-mon[33335]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:29.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:29 smithi170 ceph-mon[33632]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:29.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:29 smithi012 ceph-mon[30140]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:30.005 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:29 smithi016 ceph-mon[33335]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:31 smithi012 ceph-mon[30140]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:31.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:31 smithi170 ceph-mon[33632]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:32.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:31 smithi016 ceph-mon[33335]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:32.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:32 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T09:19:32.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:32 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:32.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:32 smithi170 ceph-mon[33632]: Deploying daemon osd.5 on smithi170 2024-05-09T09:19:32.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:32 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T09:19:32.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:32 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:32.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:32 smithi012 ceph-mon[30140]: Deploying daemon osd.5 on smithi170 2024-05-09T09:19:33.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:32 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T09:19:33.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:32 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:33.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:32 smithi016 ceph-mon[33335]: Deploying daemon osd.5 on smithi170 2024-05-09T09:19:33.876 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:33 smithi170 ceph-mon[33632]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:33.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:33 smithi012 ceph-mon[30140]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:34.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:33 smithi016 ceph-mon[33335]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:35.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:35 smithi170 ceph-mon[33632]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:35.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:35 smithi012 ceph-mon[30140]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:36.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:35 smithi016 ceph-mon[33335]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:37.733 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:37 smithi012 ceph-mon[30140]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:37.958 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:37 smithi170 ceph-mon[33632]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:38.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:37 smithi016 ceph-mon[33335]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:38.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:38.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:38.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:38.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:38.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:38.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:39.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:39.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:39.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:39.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:39 smithi170 ceph-mon[33632]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:39.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:39 smithi012 ceph-mon[30140]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:40.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:39 smithi016 ceph-mon[33335]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:41.393 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 5 on host 'smithi170' 2024-05-09T09:19:41.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:41.393+0000 7ff55b7fe700 1 -- 172.21.15.170:0/3454028522 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff568062760 con 0x7ff5540a2c20 2024-05-09T09:19:41.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:41.396+0000 7ff57035e700 1 -- 172.21.15.170:0/3454028522 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff5540a2c20 msgr2=0x7ff5540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:41.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:41.396+0000 7ff57035e700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff5540a2c20 0x7ff5540a50e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff5681033e0 tx=0x7ff55c008040 comp rx=0 tx=0).stop 2024-05-09T09:19:41.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:41.396+0000 7ff57035e700 1 -- 172.21.15.170:0/3454028522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff568103580 msgr2=0x7ff568198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:41.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:41.396+0000 7ff57035e700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff568103580 0x7ff568198ac0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff56400c950 tx=0x7ff56400cc60 comp rx=0 tx=0).stop 2024-05-09T09:19:41.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:41.396+0000 7ff57035e700 1 -- 172.21.15.170:0/3454028522 shutdown_connections 2024-05-09T09:19:41.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:41.396+0000 7ff57035e700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff5540a2c20 0x7ff5540a50e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:41.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:41.396+0000 7ff57035e700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff568103f20 0x7ff56819fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:41.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:41.396+0000 7ff57035e700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff568102380 0x7ff568198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:41.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:41.396+0000 7ff57035e700 1 --2- 172.21.15.170:0/3454028522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff568103580 0x7ff568198ac0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:41.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:41.396+0000 7ff57035e700 1 -- 172.21.15.170:0/3454028522 >> 172.21.15.170:0/3454028522 conn(0x7ff5680fd920 msgr2=0x7ff568106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:41.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:41.396+0000 7ff57035e700 1 -- 172.21.15.170:0/3454028522 shutdown_connections 2024-05-09T09:19:41.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:19:41.396+0000 7ff57035e700 1 -- 172.21.15.170:0/3454028522 wait complete. 2024-05-09T09:19:41.719 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.719 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.720 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:41.720 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:41.720 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.720 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:41 smithi170 ceph-mon[33632]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:41.720 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.720 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.720 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:41.720 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:41 smithi170 ceph-mon[33632]: from='osd.5 [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T09:19:41.720 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:41 smithi170 ceph-mon[33632]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T09:19:41.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:41.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:41.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:41 smithi012 ceph-mon[30140]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:41.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:41.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:41 smithi012 ceph-mon[30140]: from='osd.5 [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T09:19:41.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:41 smithi012 ceph-mon[30140]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T09:19:41.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:41.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:41.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:41 smithi016 ceph-mon[33335]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:41.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:41.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:41.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:41 smithi016 ceph-mon[33335]: from='osd.5 [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T09:19:41.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:41 smithi016 ceph-mon[33335]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T09:19:42.048 DEBUG:teuthology.orchestra.run.smithi170:osd.5> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.5.service 2024-05-09T09:19:42.050 INFO:tasks.cephadm:Deploying osd.6 on smithi012 with /dev/vg_nvme/lv_2... 2024-05-09T09:19:42.050 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T09:19:42.309 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:19:43.450 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:43 smithi170 ceph-mon[33632]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T09:19:43.451 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:43 smithi170 ceph-mon[33632]: from='osd.5 [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:19:43.451 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:43 smithi170 ceph-mon[33632]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T09:19:43.451 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:43 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:19:43.451 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:43 smithi170 ceph-mon[33632]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:19:43.451 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:43 smithi170 ceph-mon[33632]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:43.451 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:19:43 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:19:43.381+0000 7fc944f3e700 -1 osd.5 0 waiting for initial osdmap 2024-05-09T09:19:43.451 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:19:43 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:19:43.386+0000 7fc940d31700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:19:43.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:43 smithi012 ceph-mon[30140]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T09:19:43.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:43 smithi012 ceph-mon[30140]: from='osd.5 [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:19:43.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:43 smithi012 ceph-mon[30140]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T09:19:43.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:43 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:19:43.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:43 smithi012 ceph-mon[30140]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:19:43.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:43 smithi012 ceph-mon[30140]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:43.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:43 smithi016 ceph-mon[33335]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T09:19:43.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:43 smithi016 ceph-mon[33335]: from='osd.5 [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:19:43.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:43 smithi016 ceph-mon[33335]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T09:19:43.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:43 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:19:43.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:43 smithi016 ceph-mon[33335]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:19:43.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:43 smithi016 ceph-mon[33335]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:19:44.248 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T09:19:44.248 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T09:19:44.248 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-05-09T09:19:44.248 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-05-09T09:19:44.248 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011337 s, 925 MB/s 2024-05-09T09:19:44.248 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-05-09T09:19:44.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:44 smithi170 ceph-mon[33632]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-05-09T09:19:44.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:44 smithi170 ceph-mon[33632]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T09:19:44.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:44 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:19:44.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:44 smithi170 ceph-mon[33632]: osd.5 [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273] boot 2024-05-09T09:19:44.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:44 smithi170 ceph-mon[33632]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T09:19:44.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:44 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:19:44.568 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:44 smithi012 ceph-mon[30140]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-05-09T09:19:44.568 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:44 smithi012 ceph-mon[30140]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T09:19:44.568 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:44 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:19:44.568 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:44 smithi012 ceph-mon[30140]: osd.5 [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273] boot 2024-05-09T09:19:44.568 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:44 smithi012 ceph-mon[30140]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T09:19:44.568 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:44 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:19:44.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:44 smithi016 ceph-mon[33335]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-05-09T09:19:44.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:44 smithi016 ceph-mon[33335]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T09:19:44.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:44 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:19:44.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:44 smithi016 ceph-mon[33335]: osd.5 [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273] boot 2024-05-09T09:19:44.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:44 smithi016 ceph-mon[33335]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T09:19:44.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:44 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:19:44.925 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_2 2024-05-09T09:19:45.177 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:19:45.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: purged_snaps scrub starts 2024-05-09T09:19:45.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: purged_snaps scrub ok 2024-05-09T09:19:45.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:19:45.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T09:19:45.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: Detected new or changed devices on smithi170 2024-05-09T09:19:45.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:45.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:45.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:19:45.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:19:45.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: Adjusting osd_memory_target on smithi170 to 8544M 2024-05-09T09:19:45.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:45.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:45.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:45.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:45.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: purged_snaps scrub starts 2024-05-09T09:19:45.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: purged_snaps scrub ok 2024-05-09T09:19:45.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:19:45.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T09:19:45.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: Detected new or changed devices on smithi170 2024-05-09T09:19:45.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:45.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:45.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:19:45.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:19:45.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: Adjusting osd_memory_target on smithi170 to 8544M 2024-05-09T09:19:45.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:45.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:45.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:45.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:45.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: purged_snaps scrub starts 2024-05-09T09:19:45.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: purged_snaps scrub ok 2024-05-09T09:19:45.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:19:45.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T09:19:45.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: Detected new or changed devices on smithi170 2024-05-09T09:19:45.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:45.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:45.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:19:45.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:19:45.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: Adjusting osd_memory_target on smithi170 to 8544M 2024-05-09T09:19:45.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:45.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:45.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:45.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:19:46.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:46 smithi012 ceph-mon[30140]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T09:19:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.891+0000 7f48b2f70700 1 -- 172.21.15.12:0/1323215189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48ac101280 msgr2=0x7f48ac1016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.891+0000 7f48b2f70700 1 --2- 172.21.15.12:0/1323215189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48ac101280 0x7f48ac1016a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f48a00099c0 tx=0x7f48a0009cd0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.891+0000 7f48b2f70700 1 -- 172.21.15.12:0/1323215189 shutdown_connections 2024-05-09T09:19:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.891+0000 7f48b2f70700 1 --2- 172.21.15.12:0/1323215189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48ac102e20 0x7f48ac1096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.891+0000 7f48b2f70700 1 --2- 172.21.15.12:0/1323215189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48ac101280 0x7f48ac1016a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.891+0000 7f48b2f70700 1 --2- 172.21.15.12:0/1323215189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac102480 0x7f48ac1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.891+0000 7f48b2f70700 1 -- 172.21.15.12:0/1323215189 >> 172.21.15.12:0/1323215189 conn(0x7f48ac0fc820 msgr2=0x7f48ac0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.892+0000 7f48b2f70700 1 -- 172.21.15.12:0/1323215189 shutdown_connections 2024-05-09T09:19:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.892+0000 7f48b2f70700 1 -- 172.21.15.12:0/1323215189 wait complete. 2024-05-09T09:19:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.892+0000 7f48b2f70700 1 Processor -- start 2024-05-09T09:19:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.892+0000 7f48b2f70700 1 -- start start 2024-05-09T09:19:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.892+0000 7f48b2f70700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48ac101280 0x7f48ac06b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.892+0000 7f48b2f70700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48ac102480 0x7f48ac19d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.892+0000 7f48b2f70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac102e20 0x7f48ac19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.892+0000 7f48b2f70700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac10ce20 con 0x7f48ac102480 2024-05-09T09:19:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.892+0000 7f48b2f70700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac10cca0 con 0x7f48ac102e20 2024-05-09T09:19:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.892+0000 7f48b2f70700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac10cfa0 con 0x7f48ac101280 2024-05-09T09:19:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48abfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48ac102480 0x7f48ac19d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48b0d0c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48ac101280 0x7f48ac06b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48b0d0c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48ac101280 0x7f48ac06b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:59490/0 (socket says 172.21.15.12:59490) 2024-05-09T09:19:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48abfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48ac102480 0x7f48ac19d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45022/0 (socket says 172.21.15.12:45022) 2024-05-09T09:19:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48b0d0c700 1 -- 172.21.15.12:0/4141356790 learned_addr learned my addr 172.21.15.12:0/4141356790 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:19:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48b150d700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac102e20 0x7f48ac19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48b0d0c700 1 -- 172.21.15.12:0/4141356790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac102e20 msgr2=0x7f48ac19f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48b0d0c700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac102e20 0x7f48ac19f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48b0d0c700 1 -- 172.21.15.12:0/4141356790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48ac102480 msgr2=0x7f48ac19d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48b0d0c700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48ac102480 0x7f48ac19d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48b0d0c700 1 -- 172.21.15.12:0/4141356790 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ac1a01d0 con 0x7f48ac101280 2024-05-09T09:19:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48abfff700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48ac102480 0x7f48ac19d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:19:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48b0d0c700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48ac101280 0x7f48ac06b5b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f48a0005840 tx=0x7f48a0009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48a9ffb700 1 -- 172.21.15.12:0/4141356790 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a0027070 con 0x7f48ac101280 2024-05-09T09:19:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48b2f70700 1 -- 172.21.15.12:0/4141356790 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ac1a0460 con 0x7f48ac101280 2024-05-09T09:19:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48a9ffb700 1 -- 172.21.15.12:0/4141356790 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f48a0003fa0 con 0x7f48ac101280 2024-05-09T09:19:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.893+0000 7f48a9ffb700 1 -- 172.21.15.12:0/4141356790 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a0022bf0 con 0x7f48ac101280 2024-05-09T09:19:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.894+0000 7f48b2f70700 1 -- 172.21.15.12:0/4141356790 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ac1a0990 con 0x7f48ac101280 2024-05-09T09:19:46.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.894+0000 7f48b2f70700 1 -- 172.21.15.12:0/4141356790 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ac069010 con 0x7f48ac101280 2024-05-09T09:19:46.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.896+0000 7f48a9ffb700 1 -- 172.21.15.12:0/4141356790 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f48a002a070 con 0x7f48ac101280 2024-05-09T09:19:46.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.897+0000 7f48a9ffb700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f489c0a2c20 0x7f489c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.897+0000 7f48abfff700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f489c0a2c20 0x7f489c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.897+0000 7f48a9ffb700 1 -- 172.21.15.12:0/4141356790 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4533+0+0 (secure 0 0 0) 0x7f48a00cf4e0 con 0x7f48ac101280 2024-05-09T09:19:46.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.897+0000 7f48abfff700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f489c0a2c20 0x7f489c0a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48ac06bb30 tx=0x7f489800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:46.903+0000 7f48a9ffb700 1 -- 172.21.15.12:0/4141356790 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f48a0013070 con 0x7f48ac101280 2024-05-09T09:19:46.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:46 smithi170 ceph-mon[33632]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T09:19:47.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:46 smithi016 ceph-mon[33335]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T09:19:47.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:19:47.119+0000 7f48b2f70700 1 -- 172.21.15.12:0/4141356790 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f48ac062760 con 0x7f489c0a2c20 2024-05-09T09:19:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:47 smithi170 ceph-mon[33632]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T09:19:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:47 smithi170 ceph-mon[33632]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T09:19:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:47 smithi170 ceph-mon[33632]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:19:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:19:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:19:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:47.888 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:47 smithi012 ceph-mon[30140]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T09:19:47.889 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:47 smithi012 ceph-mon[30140]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T09:19:47.889 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:47 smithi012 ceph-mon[30140]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:19:47.889 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:19:47.889 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:19:47.889 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:47.949 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:47 smithi016 ceph-mon[33335]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T09:19:47.949 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:47 smithi016 ceph-mon[33335]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T09:19:47.949 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:47 smithi016 ceph-mon[33335]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:19:47.949 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:19:47.949 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:19:47.949 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:48.875 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:48 smithi012 ceph-mon[30140]: osdmap e42: 6 total, 6 up, 6 in 2024-05-09T09:19:48.924 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:48 smithi016 ceph-mon[33335]: osdmap e42: 6 total, 6 up, 6 in 2024-05-09T09:19:48.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:48 smithi170 ceph-mon[33632]: osdmap e42: 6 total, 6 up, 6 in 2024-05-09T09:19:49.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:49 smithi012 ceph-mon[30140]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T09:19:49.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:49 smithi170 ceph-mon[33632]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T09:19:50.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:49 smithi016 ceph-mon[33335]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T09:19:50.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:50 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2225689795' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "580dac7a-d03d-4830-9996-84ed2c9e9bf5"}]: dispatch 2024-05-09T09:19:50.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:50 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2225689795' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "580dac7a-d03d-4830-9996-84ed2c9e9bf5"}]': finished 2024-05-09T09:19:50.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:50 smithi012 ceph-mon[30140]: osdmap e43: 7 total, 6 up, 7 in 2024-05-09T09:19:50.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:50 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:19:50.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:50 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2645154703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:19:50.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:50 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2225689795' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "580dac7a-d03d-4830-9996-84ed2c9e9bf5"}]: dispatch 2024-05-09T09:19:50.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:50 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2225689795' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "580dac7a-d03d-4830-9996-84ed2c9e9bf5"}]': finished 2024-05-09T09:19:50.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:50 smithi170 ceph-mon[33632]: osdmap e43: 7 total, 6 up, 7 in 2024-05-09T09:19:50.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:50 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:19:50.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:50 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2645154703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:19:51.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:50 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2225689795' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "580dac7a-d03d-4830-9996-84ed2c9e9bf5"}]: dispatch 2024-05-09T09:19:51.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:50 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2225689795' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "580dac7a-d03d-4830-9996-84ed2c9e9bf5"}]': finished 2024-05-09T09:19:51.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:50 smithi016 ceph-mon[33335]: osdmap e43: 7 total, 6 up, 7 in 2024-05-09T09:19:51.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:50 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:19:51.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:50 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2645154703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:19:52.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:51 smithi012 ceph-mon[30140]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-09T09:19:52.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:51 smithi170 ceph-mon[33632]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-09T09:19:52.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:51 smithi016 ceph-mon[33335]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-09T09:19:54.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:53 smithi012 ceph-mon[30140]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:19:54.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:53 smithi170 ceph-mon[33632]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:19:54.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:53 smithi016 ceph-mon[33335]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:19:55.960 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:55 smithi012 ceph-mon[30140]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:19:56.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:55 smithi170 ceph-mon[33632]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:19:56.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:55 smithi016 ceph-mon[33335]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:19:58.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:57 smithi012 ceph-mon[30140]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:19:58.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:57 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T09:19:58.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:57 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:58.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:57 smithi170 ceph-mon[33632]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:19:58.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:57 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T09:19:58.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:57 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:58.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:57 smithi016 ceph-mon[33335]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:19:58.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:57 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T09:19:58.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:57 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:59.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:58 smithi012 ceph-mon[30140]: Deploying daemon osd.6 on smithi012 2024-05-09T09:19:59.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:58 smithi170 ceph-mon[33632]: Deploying daemon osd.6 on smithi012 2024-05-09T09:19:59.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:58 smithi016 ceph-mon[33335]: Deploying daemon osd.6 on smithi012 2024-05-09T09:20:00.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:19:59 smithi012 ceph-mon[30140]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:00.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:19:59 smithi170 ceph-mon[33632]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:00.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:19:59 smithi016 ceph-mon[33335]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:01.178 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:00 smithi012 ceph-mon[30140]: overall HEALTH_OK 2024-05-09T09:20:01.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:00 smithi170 ceph-mon[33632]: overall HEALTH_OK 2024-05-09T09:20:01.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:00 smithi016 ceph-mon[33335]: overall HEALTH_OK 2024-05-09T09:20:02.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:01 smithi012 ceph-mon[30140]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:02.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:01 smithi170 ceph-mon[33632]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:02.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:01 smithi016 ceph-mon[33335]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:03 smithi012 ceph-mon[30140]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:03 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:04.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:03 smithi170 ceph-mon[33632]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:04.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:04.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:04.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:03 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:04.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:03 smithi016 ceph-mon[33335]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:04.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:03 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:04.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:03 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:04.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:03 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:06.162 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:05 smithi012 ceph-mon[30140]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:06.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:05 smithi170 ceph-mon[33632]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:06.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:05 smithi016 ceph-mon[33335]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:07.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:20:07.878+0000 7f48a9ffb700 1 -- 172.21.15.12:0/4141356790 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f48ac062760 con 0x7f489c0a2c20 2024-05-09T09:20:07.878 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 6 on host 'smithi012' 2024-05-09T09:20:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:20:07.881+0000 7f48b2f70700 1 -- 172.21.15.12:0/4141356790 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f489c0a2c20 msgr2=0x7f489c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:20:07.881+0000 7f48b2f70700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f489c0a2c20 0x7f489c0a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48ac06bb30 tx=0x7f489800a300 comp rx=0 tx=0).stop 2024-05-09T09:20:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:20:07.881+0000 7f48b2f70700 1 -- 172.21.15.12:0/4141356790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48ac101280 msgr2=0x7f48ac06b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:20:07.881+0000 7f48b2f70700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48ac101280 0x7f48ac06b5b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f48a0005840 tx=0x7f48a0009510 comp rx=0 tx=0).stop 2024-05-09T09:20:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:20:07.881+0000 7f48b2f70700 1 -- 172.21.15.12:0/4141356790 shutdown_connections 2024-05-09T09:20:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:20:07.881+0000 7f48b2f70700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f489c0a2c20 0x7f489c0a50e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:07.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:20:07.881+0000 7f48b2f70700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac102e20 0x7f48ac19f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:07.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:20:07.881+0000 7f48b2f70700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48ac101280 0x7f48ac06b5b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:07.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:20:07.881+0000 7f48b2f70700 1 --2- 172.21.15.12:0/4141356790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48ac102480 0x7f48ac19d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:07.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:20:07.881+0000 7f48b2f70700 1 -- 172.21.15.12:0/4141356790 >> 172.21.15.12:0/4141356790 conn(0x7f48ac0fc820 msgr2=0x7f48ac105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:07.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:20:07.881+0000 7f48b2f70700 1 -- 172.21.15.12:0/4141356790 shutdown_connections 2024-05-09T09:20:07.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:20:07.881+0000 7f48b2f70700 1 -- 172.21.15.12:0/4141356790 wait complete. 2024-05-09T09:20:08.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:07 smithi012 ceph-mon[30140]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:08.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:07 smithi012 ceph-mon[30140]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T09:20:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:07 smithi170 ceph-mon[33632]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:07 smithi170 ceph-mon[33632]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T09:20:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:08.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:07 smithi016 ceph-mon[33335]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:08.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:08.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:08.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:07 smithi016 ceph-mon[33335]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T09:20:08.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:08.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:08.596 DEBUG:teuthology.orchestra.run.smithi012:osd.6> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.6.service 2024-05-09T09:20:08.598 INFO:tasks.cephadm:Deploying osd.7 on smithi016 with /dev/vg_nvme/lv_2... 2024-05-09T09:20:08.599 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T09:20:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:20:09.214 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:08 smithi016 ceph-mon[33335]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T09:20:09.214 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:08 smithi016 ceph-mon[33335]: osdmap e44: 7 total, 6 up, 7 in 2024-05-09T09:20:09.214 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:08 smithi016 ceph-mon[33335]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-09T09:20:09.215 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:08 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:09.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:08 smithi012 ceph-mon[30140]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T09:20:09.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:08 smithi012 ceph-mon[30140]: osdmap e44: 7 total, 6 up, 7 in 2024-05-09T09:20:09.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:08 smithi012 ceph-mon[30140]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-09T09:20:09.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:08 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:09.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:08 smithi170 ceph-mon[33632]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T09:20:09.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:08 smithi170 ceph-mon[33632]: osdmap e44: 7 total, 6 up, 7 in 2024-05-09T09:20:09.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:08 smithi170 ceph-mon[33632]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-09T09:20:09.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:08 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:09.896 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:20:09 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:20:09.630+0000 7f9a79e33700 -1 osd.6 0 waiting for initial osdmap 2024-05-09T09:20:09.896 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:20:09 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:20:09.639+0000 7f9a71c1e700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:20:10.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:09 smithi012 ceph-mon[30140]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:10.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:09 smithi012 ceph-mon[30140]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-09T09:20:10.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:09 smithi012 ceph-mon[30140]: osdmap e45: 7 total, 6 up, 7 in 2024-05-09T09:20:10.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:10.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:10.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:09 smithi012 ceph-mon[30140]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' 2024-05-09T09:20:10.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:09 smithi170 ceph-mon[33632]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:10.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:09 smithi170 ceph-mon[33632]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-09T09:20:10.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:09 smithi170 ceph-mon[33632]: osdmap e45: 7 total, 6 up, 7 in 2024-05-09T09:20:10.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:10.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:10.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:09 smithi170 ceph-mon[33632]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' 2024-05-09T09:20:10.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:09 smithi016 ceph-mon[33335]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:10.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:09 smithi016 ceph-mon[33335]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-09T09:20:10.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:09 smithi016 ceph-mon[33335]: osdmap e45: 7 total, 6 up, 7 in 2024-05-09T09:20:10.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:10.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:10.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:09 smithi016 ceph-mon[33335]: from='osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494]' entity='osd.6' 2024-05-09T09:20:11.162 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T09:20:11.162 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T09:20:11.163 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-05-09T09:20:11.163 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-05-09T09:20:11.163 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00843752 s, 1.2 GB/s 2024-05-09T09:20:11.163 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-05-09T09:20:11.187 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:10 smithi016 ceph-mon[33335]: purged_snaps scrub starts 2024-05-09T09:20:11.187 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:10 smithi016 ceph-mon[33335]: purged_snaps scrub ok 2024-05-09T09:20:11.187 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:10 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:11.188 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:10 smithi016 ceph-mon[33335]: osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494] boot 2024-05-09T09:20:11.188 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:10 smithi016 ceph-mon[33335]: osdmap e46: 7 total, 7 up, 7 in 2024-05-09T09:20:11.188 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:10 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:11.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:10 smithi012 ceph-mon[30140]: purged_snaps scrub starts 2024-05-09T09:20:11.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:10 smithi012 ceph-mon[30140]: purged_snaps scrub ok 2024-05-09T09:20:11.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:10 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:11.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:10 smithi012 ceph-mon[30140]: osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494] boot 2024-05-09T09:20:11.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:10 smithi012 ceph-mon[30140]: osdmap e46: 7 total, 7 up, 7 in 2024-05-09T09:20:11.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:10 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:11.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:10 smithi170 ceph-mon[33632]: purged_snaps scrub starts 2024-05-09T09:20:11.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:10 smithi170 ceph-mon[33632]: purged_snaps scrub ok 2024-05-09T09:20:11.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:10 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:11.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:10 smithi170 ceph-mon[33632]: osd.6 [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494] boot 2024-05-09T09:20:11.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:10 smithi170 ceph-mon[33632]: osdmap e46: 7 total, 7 up, 7 in 2024-05-09T09:20:11.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:10 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:20:11.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-05-09T09:20:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:20:12.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:11 smithi012 ceph-mon[30140]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:12.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:12.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:12.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:12.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:12.236 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:12.236 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:12.236 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:12.236 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:12.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:11 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:12.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:11 smithi170 ceph-mon[33632]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:12.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:12.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:12.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:12.245 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:12.245 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:12.245 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:12.245 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:12.245 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:12.245 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:11 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:12.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:11 smithi016 ceph-mon[33335]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:20:12.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:12.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:12.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:12.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:12.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:12.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:12.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:12.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:12.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:11 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:13.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:12 smithi012 ceph-mon[30140]: Detected new or changed devices on smithi012 2024-05-09T09:20:13.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:12 smithi012 ceph-mon[30140]: Adjusting osd_memory_target on smithi012 to 5697M 2024-05-09T09:20:13.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:12 smithi012 ceph-mon[30140]: osdmap e47: 7 total, 7 up, 7 in 2024-05-09T09:20:13.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:12 smithi170 ceph-mon[33632]: Detected new or changed devices on smithi012 2024-05-09T09:20:13.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:12 smithi170 ceph-mon[33632]: Adjusting osd_memory_target on smithi012 to 5697M 2024-05-09T09:20:13.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:12 smithi170 ceph-mon[33632]: osdmap e47: 7 total, 7 up, 7 in 2024-05-09T09:20:13.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:12 smithi016 ceph-mon[33335]: Detected new or changed devices on smithi012 2024-05-09T09:20:13.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:12 smithi016 ceph-mon[33335]: Adjusting osd_memory_target on smithi012 to 5697M 2024-05-09T09:20:13.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:12 smithi016 ceph-mon[33335]: osdmap e47: 7 total, 7 up, 7 in 2024-05-09T09:20:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.694+0000 7f372c584700 1 -- 172.21.15.16:0/1065478860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3724102e10 msgr2=0x7f3724105200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.694+0000 7f372c584700 1 --2- 172.21.15.16:0/1065478860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3724102e10 0x7f3724105200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3720009990 tx=0x7f3720009ca0 comp rx=0 tx=0).stop 2024-05-09T09:20:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.694+0000 7f372c584700 1 -- 172.21.15.16:0/1065478860 shutdown_connections 2024-05-09T09:20:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.694+0000 7f372c584700 1 --2- 172.21.15.16:0/1065478860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3724108070 0x7f372410a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.694+0000 7f372c584700 1 --2- 172.21.15.16:0/1065478860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3724102e10 0x7f3724105200 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.694+0000 7f372c584700 1 --2- 172.21.15.16:0/1065478860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3724105740 0x7f3724107b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.694+0000 7f372c584700 1 -- 172.21.15.16:0/1065478860 >> 172.21.15.16:0/1065478860 conn(0x7f37240fc840 msgr2=0x7f37240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.694+0000 7f372c584700 1 -- 172.21.15.16:0/1065478860 shutdown_connections 2024-05-09T09:20:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.694+0000 7f372c584700 1 -- 172.21.15.16:0/1065478860 wait complete. 2024-05-09T09:20:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372c584700 1 Processor -- start 2024-05-09T09:20:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372c584700 1 -- start start 2024-05-09T09:20:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372c584700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3724102e10 0x7f3724102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372c584700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3724105740 0x7f3724100dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372c584700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3724108070 0x7f3724101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372c584700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f372410e010 con 0x7f3724108070 2024-05-09T09:20:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372c584700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f372410de90 con 0x7f3724102e10 2024-05-09T09:20:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372c584700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f372410e190 con 0x7f3724105740 2024-05-09T09:20:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372a320700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3724102e10 0x7f3724102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372ab21700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3724108070 0x7f3724101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372a320700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3724102e10 0x7f3724102770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58908/0 (socket says 172.21.15.16:58908) 2024-05-09T09:20:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372a320700 1 -- 172.21.15.16:0/3798178667 learned_addr learned my addr 172.21.15.16:0/3798178667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-09T09:20:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f3729b1f700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3724105740 0x7f3724100dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372a320700 1 -- 172.21.15.16:0/3798178667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3724105740 msgr2=0x7f3724100dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.695+0000 7f372a320700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3724105740 0x7f3724100dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.696+0000 7f372a320700 1 -- 172.21.15.16:0/3798178667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3724108070 msgr2=0x7f3724101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.696+0000 7f372a320700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3724108070 0x7f3724101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.696+0000 7f372a320700 1 -- 172.21.15.16:0/3798178667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3724101840 con 0x7f3724102e10 2024-05-09T09:20:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.696+0000 7f372ab21700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3724108070 0x7f3724101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.696+0000 7f372a320700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3724102e10 0x7f3724102770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3720005910 tx=0x7f37200057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.696+0000 7f371b7fe700 1 -- 172.21.15.16:0/3798178667 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720027070 con 0x7f3724102e10 2024-05-09T09:20:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.696+0000 7f371b7fe700 1 -- 172.21.15.16:0/3798178667 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3720004140 con 0x7f3724102e10 2024-05-09T09:20:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.696+0000 7f372c584700 1 -- 172.21.15.16:0/3798178667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3724101ad0 con 0x7f3724102e10 2024-05-09T09:20:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.696+0000 7f371b7fe700 1 -- 172.21.15.16:0/3798178667 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372000f460 con 0x7f3724102e10 2024-05-09T09:20:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.696+0000 7f372c584700 1 -- 172.21.15.16:0/3798178667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37241a70b0 con 0x7f3724102e10 2024-05-09T09:20:13.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.697+0000 7f372c584700 1 -- 172.21.15.16:0/3798178667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3724069010 con 0x7f3724102e10 2024-05-09T09:20:13.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.698+0000 7f371b7fe700 1 -- 172.21.15.16:0/3798178667 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f372002a070 con 0x7f3724102e10 2024-05-09T09:20:13.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.699+0000 7f371b7fe700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37100a2ca0 0x7f37100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:13.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.699+0000 7f371b7fe700 1 -- 172.21.15.16:0/3798178667 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f37200cf6b0 con 0x7f3724102e10 2024-05-09T09:20:13.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.699+0000 7f3729b1f700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37100a2ca0 0x7f37100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:13.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.700+0000 7f3729b1f700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37100a2ca0 0x7f37100a5160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3714009a10 tx=0x7f3714009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.703+0000 7f371b7fe700 1 -- 172.21.15.16:0/3798178667 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3720013070 con 0x7f3724102e10 2024-05-09T09:20:13.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:13.916+0000 7f372c584700 1 -- 172.21.15.16:0/3798178667 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3724062760 con 0x7f37100a2ca0 2024-05-09T09:20:14.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:13 smithi012 ceph-mon[30140]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:14.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:20:14.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:20:14.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:14.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:13 smithi170 ceph-mon[33632]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:14.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:20:14.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:20:14.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:14.256 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:13 smithi016 ceph-mon[33335]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:14.256 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:20:14.256 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:20:14.256 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:15.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:14 smithi012 ceph-mon[30140]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:20:15.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:14 smithi170 ceph-mon[33632]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:20:15.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:14 smithi016 ceph-mon[33335]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:20:16.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:15 smithi012 ceph-mon[30140]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:16.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:15 smithi170 ceph-mon[33632]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:16.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:15 smithi016 ceph-mon[33335]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:17.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:16 smithi012 ceph-mon[30140]: from='client.? 172.21.15.16:0/3858348029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c7f5786-3123-417c-b8eb-9c44e5284858"}]: dispatch 2024-05-09T09:20:17.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:16 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c7f5786-3123-417c-b8eb-9c44e5284858"}]: dispatch 2024-05-09T09:20:17.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:16 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c7f5786-3123-417c-b8eb-9c44e5284858"}]': finished 2024-05-09T09:20:17.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:16 smithi012 ceph-mon[30140]: osdmap e48: 8 total, 7 up, 8 in 2024-05-09T09:20:17.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:16 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:17.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:16 smithi170 ceph-mon[33632]: from='client.? 172.21.15.16:0/3858348029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c7f5786-3123-417c-b8eb-9c44e5284858"}]: dispatch 2024-05-09T09:20:17.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:16 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c7f5786-3123-417c-b8eb-9c44e5284858"}]: dispatch 2024-05-09T09:20:17.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:16 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c7f5786-3123-417c-b8eb-9c44e5284858"}]': finished 2024-05-09T09:20:17.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:16 smithi170 ceph-mon[33632]: osdmap e48: 8 total, 7 up, 8 in 2024-05-09T09:20:17.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:16 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:17.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:16 smithi016 ceph-mon[33335]: from='client.? 172.21.15.16:0/3858348029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c7f5786-3123-417c-b8eb-9c44e5284858"}]: dispatch 2024-05-09T09:20:17.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:16 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c7f5786-3123-417c-b8eb-9c44e5284858"}]: dispatch 2024-05-09T09:20:17.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:16 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c7f5786-3123-417c-b8eb-9c44e5284858"}]': finished 2024-05-09T09:20:17.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:16 smithi016 ceph-mon[33335]: osdmap e48: 8 total, 7 up, 8 in 2024-05-09T09:20:17.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:16 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:18.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:17 smithi012 ceph-mon[30140]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:18.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:17 smithi012 ceph-mon[30140]: from='client.? 172.21.15.16:0/1671874078' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:20:18.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:17 smithi170 ceph-mon[33632]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:18.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:17 smithi170 ceph-mon[33632]: from='client.? 172.21.15.16:0/1671874078' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:20:18.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:17 smithi016 ceph-mon[33335]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:18.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:17 smithi016 ceph-mon[33335]: from='client.? 172.21.15.16:0/1671874078' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:20:20.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:19 smithi012 ceph-mon[30140]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:20.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:19 smithi170 ceph-mon[33632]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:20.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:19 smithi016 ceph-mon[33335]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:22.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:21 smithi012 ceph-mon[30140]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:22.245 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:21 smithi170 ceph-mon[33632]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:21 smithi016 ceph-mon[33335]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:24.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:23 smithi016 ceph-mon[33335]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:24.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:23 smithi012 ceph-mon[30140]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:24.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:23 smithi170 ceph-mon[33632]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:25.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:24 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T09:20:25.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:24 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:25.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:24 smithi012 ceph-mon[30140]: Deploying daemon osd.7 on smithi016 2024-05-09T09:20:25.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:24 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T09:20:25.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:24 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:25.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:24 smithi170 ceph-mon[33632]: Deploying daemon osd.7 on smithi016 2024-05-09T09:20:25.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:24 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T09:20:25.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:24 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:25.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:24 smithi016 ceph-mon[33335]: Deploying daemon osd.7 on smithi016 2024-05-09T09:20:26.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:25 smithi016 ceph-mon[33335]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:26.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:25 smithi012 ceph-mon[30140]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:26.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:25 smithi170 ceph-mon[33632]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:28.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:27 smithi012 ceph-mon[30140]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:28.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:27 smithi170 ceph-mon[33632]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:28.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:27 smithi016 ceph-mon[33335]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:30.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:29 smithi012 ceph-mon[30140]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:30.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:30.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:30.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:30.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:29 smithi170 ceph-mon[33632]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:30.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:30.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:30.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:30.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:29 smithi016 ceph-mon[33335]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:30.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:30.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:30.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:32.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:31 smithi012 ceph-mon[30140]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:32.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:31 smithi170 ceph-mon[33632]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:32.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:31 smithi016 ceph-mon[33335]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:33.183+0000 7f371b7fe700 1 -- 172.21.15.16:0/3798178667 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3724062760 con 0x7f37100a2ca0 2024-05-09T09:20:33.184 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 7 on host 'smithi016' 2024-05-09T09:20:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:33.185+0000 7f372c584700 1 -- 172.21.15.16:0/3798178667 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37100a2ca0 msgr2=0x7f37100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:33.185+0000 7f372c584700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37100a2ca0 0x7f37100a5160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3714009a10 tx=0x7f3714009190 comp rx=0 tx=0).stop 2024-05-09T09:20:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:33.185+0000 7f372c584700 1 -- 172.21.15.16:0/3798178667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3724102e10 msgr2=0x7f3724102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:33.185+0000 7f372c584700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3724102e10 0x7f3724102770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3720005910 tx=0x7f37200057a0 comp rx=0 tx=0).stop 2024-05-09T09:20:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:33.185+0000 7f372c584700 1 -- 172.21.15.16:0/3798178667 shutdown_connections 2024-05-09T09:20:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:33.185+0000 7f372c584700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37100a2ca0 0x7f37100a5160 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:33.185+0000 7f372c584700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3724108070 0x7f3724101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:33.185+0000 7f372c584700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3724102e10 0x7f3724102770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:33.185+0000 7f372c584700 1 --2- 172.21.15.16:0/3798178667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3724105740 0x7f3724100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:33.185+0000 7f372c584700 1 -- 172.21.15.16:0/3798178667 >> 172.21.15.16:0/3798178667 conn(0x7f37240fc840 msgr2=0x7f3724103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:33.185+0000 7f372c584700 1 -- 172.21.15.16:0/3798178667 shutdown_connections 2024-05-09T09:20:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:20:33.185+0000 7f372c584700 1 -- 172.21.15.16:0/3798178667 wait complete. 2024-05-09T09:20:33.979 DEBUG:teuthology.orchestra.run.smithi016:osd.7> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.7.service 2024-05-09T09:20:33.986 INFO:tasks.cephadm:Deploying osd.8 on smithi170 with /dev/vg_nvme/lv_2... 2024-05-09T09:20:33.986 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T09:20:34.239 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:20:34.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:34 smithi016 ceph-mon[33335]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:34.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:34 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:34.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:34 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:34.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:34 smithi016 ceph-mon[33335]: from='osd.7 [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T09:20:34.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:34 smithi016 ceph-mon[33335]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T09:20:34.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:34 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:34.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:34 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:34.361 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:34 smithi170 ceph-mon[33632]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:34.361 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:34 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:34.361 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:34 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:34.362 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:34 smithi170 ceph-mon[33632]: from='osd.7 [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T09:20:34.362 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:34 smithi170 ceph-mon[33632]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T09:20:34.362 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:34 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:34.362 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:34 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:34 smithi012 ceph-mon[30140]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:34 smithi012 ceph-mon[30140]: from='osd.7 [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T09:20:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:34 smithi012 ceph-mon[30140]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T09:20:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:34.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:35.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:35 smithi016 ceph-mon[33335]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T09:20:35.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:35 smithi016 ceph-mon[33335]: osdmap e49: 8 total, 7 up, 8 in 2024-05-09T09:20:35.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:35 smithi016 ceph-mon[33335]: from='osd.7 [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:20:35.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:35 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:35.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:35 smithi016 ceph-mon[33335]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:20:35.253 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:20:35 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:20:35.161+0000 7f94a6b99700 -1 osd.7 0 waiting for initial osdmap 2024-05-09T09:20:35.254 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:20:35 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:20:35.169+0000 7f94a298c700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:20:35.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:35 smithi012 ceph-mon[30140]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T09:20:35.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:35 smithi012 ceph-mon[30140]: osdmap e49: 8 total, 7 up, 8 in 2024-05-09T09:20:35.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:35 smithi012 ceph-mon[30140]: from='osd.7 [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:20:35.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:35 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:35.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:35 smithi012 ceph-mon[30140]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:20:35.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:35 smithi170 ceph-mon[33632]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T09:20:35.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:35 smithi170 ceph-mon[33632]: osdmap e49: 8 total, 7 up, 8 in 2024-05-09T09:20:35.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:35 smithi170 ceph-mon[33632]: from='osd.7 [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:20:35.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:35 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:35.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:35 smithi170 ceph-mon[33632]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:20:36.203 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T09:20:36.203 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T09:20:36.203 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-05-09T09:20:36.203 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-05-09T09:20:36.203 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00846685 s, 1.2 GB/s 2024-05-09T09:20:36.203 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-05-09T09:20:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:36 smithi012 ceph-mon[30140]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:36 smithi012 ceph-mon[30140]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-09T09:20:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:36 smithi012 ceph-mon[30140]: osdmap e50: 8 total, 7 up, 8 in 2024-05-09T09:20:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:36 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:36 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:36.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:36 smithi170 ceph-mon[33632]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:36 smithi170 ceph-mon[33632]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-09T09:20:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:36 smithi170 ceph-mon[33632]: osdmap e50: 8 total, 7 up, 8 in 2024-05-09T09:20:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:36 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:36 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:36.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:36 smithi016 ceph-mon[33335]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:20:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:36 smithi016 ceph-mon[33335]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-09T09:20:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:36 smithi016 ceph-mon[33335]: osdmap e50: 8 total, 7 up, 8 in 2024-05-09T09:20:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:36 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:36 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:36.851 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_2 2024-05-09T09:20:37.094 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:20:37.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:37 smithi016 ceph-mon[33335]: purged_snaps scrub starts 2024-05-09T09:20:37.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:37 smithi016 ceph-mon[33335]: purged_snaps scrub ok 2024-05-09T09:20:37.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:37 smithi016 ceph-mon[33335]: osd.7 [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888] boot 2024-05-09T09:20:37.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:37 smithi016 ceph-mon[33335]: osdmap e51: 8 total, 8 up, 8 in 2024-05-09T09:20:37.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:37 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:37.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:37 smithi170 ceph-mon[33632]: purged_snaps scrub starts 2024-05-09T09:20:37.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:37 smithi170 ceph-mon[33632]: purged_snaps scrub ok 2024-05-09T09:20:37.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:37 smithi170 ceph-mon[33632]: osd.7 [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888] boot 2024-05-09T09:20:37.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:37 smithi170 ceph-mon[33632]: osdmap e51: 8 total, 8 up, 8 in 2024-05-09T09:20:37.415 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:37 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:37 smithi012 ceph-mon[30140]: purged_snaps scrub starts 2024-05-09T09:20:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:37 smithi012 ceph-mon[30140]: purged_snaps scrub ok 2024-05-09T09:20:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:37 smithi012 ceph-mon[30140]: osd.7 [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888] boot 2024-05-09T09:20:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:37 smithi012 ceph-mon[30140]: osdmap e51: 8 total, 8 up, 8 in 2024-05-09T09:20:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:37 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:20:38.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:38.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: osdmap e52: 8 total, 8 up, 8 in 2024-05-09T09:20:38.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:38.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:38.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:38.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:38.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:38.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:38.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:38.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:38.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:38 smithi170 ceph-mon[33632]: osdmap e53: 8 total, 8 up, 8 in 2024-05-09T09:20:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: osdmap e52: 8 total, 8 up, 8 in 2024-05-09T09:20:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:38.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:38.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:38.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:38.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:38 smithi012 ceph-mon[30140]: osdmap e53: 8 total, 8 up, 8 in 2024-05-09T09:20:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: osdmap e52: 8 total, 8 up, 8 in 2024-05-09T09:20:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:20:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:38.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:38.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:38.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:38.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:38.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:38.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:38 smithi016 ceph-mon[33335]: osdmap e53: 8 total, 8 up, 8 in 2024-05-09T09:20:38.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.713+0000 7f40843f4700 1 -- 172.21.15.170:0/3700297894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f407c102e10 msgr2=0x7f407c105200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:38.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.713+0000 7f40843f4700 1 --2- 172.21.15.170:0/3700297894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f407c102e10 0x7f407c105200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f407800b0a0 tx=0x7f407800b3b0 comp rx=0 tx=0).stop 2024-05-09T09:20:38.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.713+0000 7f40843f4700 1 -- 172.21.15.170:0/3700297894 shutdown_connections 2024-05-09T09:20:38.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.713+0000 7f40843f4700 1 --2- 172.21.15.170:0/3700297894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f407c108070 0x7f407c10a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.713+0000 7f40843f4700 1 --2- 172.21.15.170:0/3700297894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f407c102e10 0x7f407c105200 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.713+0000 7f40843f4700 1 --2- 172.21.15.170:0/3700297894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f407c105740 0x7f407c107b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.713+0000 7f40843f4700 1 -- 172.21.15.170:0/3700297894 >> 172.21.15.170:0/3700297894 conn(0x7f407c0fc820 msgr2=0x7f407c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.713+0000 7f40843f4700 1 -- 172.21.15.170:0/3700297894 shutdown_connections 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.713+0000 7f40843f4700 1 -- 172.21.15.170:0/3700297894 wait complete. 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f40843f4700 1 Processor -- start 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f40843f4700 1 -- start start 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f40843f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f407c102e10 0x7f407c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f40843f4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f407c105740 0x7f407c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f40843f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f407c108070 0x7f407c19fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f40843f4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f407c10df60 con 0x7f407c108070 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f40843f4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f407c10dde0 con 0x7f407c102e10 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f40843f4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f407c10e0e0 con 0x7f407c105740 2024-05-09T09:20:38.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f408198f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f407c105740 0x7f407c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f408198f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f407c105740 0x7f407c198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38888/0 (socket says 172.21.15.170:38888) 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f408198f700 1 -- 172.21.15.170:0/218644687 learned_addr learned my addr 172.21.15.170:0/218644687 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f4082190700 1 --2- 172.21.15.170:0/218644687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f407c102e10 0x7f407c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f408198f700 1 -- 172.21.15.170:0/218644687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f407c102e10 msgr2=0x7f407c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f408198f700 1 --2- 172.21.15.170:0/218644687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f407c102e10 0x7f407c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f408198f700 1 -- 172.21.15.170:0/218644687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f407c108070 msgr2=0x7f407c19fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f408198f700 1 --2- 172.21.15.170:0/218644687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f407c108070 0x7f407c19fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.714+0000 7f408198f700 1 -- 172.21.15.170:0/218644687 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407c1a0200 con 0x7f407c105740 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.715+0000 7f408198f700 1 --2- 172.21.15.170:0/218644687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f407c105740 0x7f407c198ae0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f406c00d980 tx=0x7f406c00dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.715+0000 7f40737fe700 1 -- 172.21.15.170:0/218644687 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c016070 con 0x7f407c105740 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.715+0000 7f40737fe700 1 -- 172.21.15.170:0/218644687 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f406c009b40 con 0x7f407c105740 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.715+0000 7f40737fe700 1 -- 172.21.15.170:0/218644687 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c00f3c0 con 0x7f407c105740 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.715+0000 7f40843f4700 1 -- 172.21.15.170:0/218644687 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407c1a04f0 con 0x7f407c105740 2024-05-09T09:20:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.715+0000 7f40843f4700 1 -- 172.21.15.170:0/218644687 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f407c1a09e0 con 0x7f407c105740 2024-05-09T09:20:38.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.716+0000 7f40843f4700 1 -- 172.21.15.170:0/218644687 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f407c069010 con 0x7f407c105740 2024-05-09T09:20:38.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.717+0000 7f40737fe700 1 -- 172.21.15.170:0/218644687 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f406c019070 con 0x7f407c105740 2024-05-09T09:20:38.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.718+0000 7f40737fe700 1 --2- 172.21.15.170:0/218644687 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f40680a2c20 0x7f40680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.718+0000 7f40737fe700 1 -- 172.21.15.170:0/218644687 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7f406c0ce0c0 con 0x7f407c105740 2024-05-09T09:20:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.718+0000 7f4082190700 1 --2- 172.21.15.170:0/218644687 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f40680a2c20 0x7f40680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.718+0000 7f4082190700 1 --2- 172.21.15.170:0/218644687 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f40680a2c20 0x7f40680a50e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40780062a0 tx=0x7f40780061f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:38.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.722+0000 7f40737fe700 1 -- 172.21.15.170:0/218644687 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f406c09bb80 con 0x7f407c105740 2024-05-09T09:20:38.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:38.926+0000 7f40843f4700 1 -- 172.21.15.170:0/218644687 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f407c062760 con 0x7f40680a2c20 2024-05-09T09:20:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:39 smithi170 ceph-mon[33632]: Detected new or changed devices on smithi016 2024-05-09T09:20:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:39 smithi170 ceph-mon[33632]: Adjusting osd_memory_target on smithi016 to 5697M 2024-05-09T09:20:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:39 smithi170 ceph-mon[33632]: osdmap e54: 8 total, 8 up, 8 in 2024-05-09T09:20:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:39 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:20:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:39 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:20:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:39 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:39.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:39 smithi012 ceph-mon[30140]: Detected new or changed devices on smithi016 2024-05-09T09:20:39.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:39 smithi012 ceph-mon[30140]: Adjusting osd_memory_target on smithi016 to 5697M 2024-05-09T09:20:39.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:39 smithi012 ceph-mon[30140]: osdmap e54: 8 total, 8 up, 8 in 2024-05-09T09:20:39.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:39 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:20:39.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:39 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:20:39.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:39 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:39.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:39 smithi016 ceph-mon[33335]: Detected new or changed devices on smithi016 2024-05-09T09:20:39.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:39 smithi016 ceph-mon[33335]: Adjusting osd_memory_target on smithi016 to 5697M 2024-05-09T09:20:39.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:39 smithi016 ceph-mon[33335]: osdmap e54: 8 total, 8 up, 8 in 2024-05-09T09:20:39.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:39 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:20:39.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:39 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:20:39.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:39 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:40.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:40 smithi012 ceph-mon[30140]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:40.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:40 smithi012 ceph-mon[30140]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:20:40.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:40 smithi012 ceph-mon[30140]: osdmap e55: 8 total, 8 up, 8 in 2024-05-09T09:20:40.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:40 smithi170 ceph-mon[33632]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:40.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:40 smithi170 ceph-mon[33632]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:20:40.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:40 smithi170 ceph-mon[33632]: osdmap e55: 8 total, 8 up, 8 in 2024-05-09T09:20:40.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:40 smithi016 ceph-mon[33335]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:40.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:40 smithi016 ceph-mon[33335]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:20:40.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:40 smithi016 ceph-mon[33335]: osdmap e55: 8 total, 8 up, 8 in 2024-05-09T09:20:42.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:42 smithi170 ceph-mon[33632]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:42.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:42 smithi170 ceph-mon[33632]: from='client.? 172.21.15.170:0/3597053255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d78f2036-c1f3-4efc-a4bc-d173ed8cd846"}]: dispatch 2024-05-09T09:20:42.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:42 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d78f2036-c1f3-4efc-a4bc-d173ed8cd846"}]: dispatch 2024-05-09T09:20:42.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:42 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d78f2036-c1f3-4efc-a4bc-d173ed8cd846"}]': finished 2024-05-09T09:20:42.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:42 smithi170 ceph-mon[33632]: osdmap e56: 9 total, 8 up, 9 in 2024-05-09T09:20:42.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:42 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:20:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:42 smithi012 ceph-mon[30140]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:42 smithi012 ceph-mon[30140]: from='client.? 172.21.15.170:0/3597053255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d78f2036-c1f3-4efc-a4bc-d173ed8cd846"}]: dispatch 2024-05-09T09:20:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:42 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d78f2036-c1f3-4efc-a4bc-d173ed8cd846"}]: dispatch 2024-05-09T09:20:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:42 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d78f2036-c1f3-4efc-a4bc-d173ed8cd846"}]': finished 2024-05-09T09:20:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:42 smithi012 ceph-mon[30140]: osdmap e56: 9 total, 8 up, 9 in 2024-05-09T09:20:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:42 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:20:42.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:42 smithi016 ceph-mon[33335]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:42.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:42 smithi016 ceph-mon[33335]: from='client.? 172.21.15.170:0/3597053255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d78f2036-c1f3-4efc-a4bc-d173ed8cd846"}]: dispatch 2024-05-09T09:20:42.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:42 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d78f2036-c1f3-4efc-a4bc-d173ed8cd846"}]: dispatch 2024-05-09T09:20:42.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:42 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d78f2036-c1f3-4efc-a4bc-d173ed8cd846"}]': finished 2024-05-09T09:20:42.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:42 smithi016 ceph-mon[33335]: osdmap e56: 9 total, 8 up, 9 in 2024-05-09T09:20:42.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:42 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:20:43.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:43 smithi012 ceph-mon[30140]: from='client.? 172.21.15.170:0/2366422592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:20:43.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:43 smithi170 ceph-mon[33632]: from='client.? 172.21.15.170:0/2366422592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:20:43.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:43 smithi016 ceph-mon[33335]: from='client.? 172.21.15.170:0/2366422592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:20:44.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:44 smithi012 ceph-mon[30140]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:44.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:44 smithi170 ceph-mon[33632]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:44.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:44 smithi016 ceph-mon[33335]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:46.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:46 smithi012 ceph-mon[30140]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:46.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:46 smithi170 ceph-mon[33632]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:46.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:46 smithi016 ceph-mon[33335]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:48.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:48 smithi012 ceph-mon[30140]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:48.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:48 smithi170 ceph-mon[33632]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:48.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:48 smithi016 ceph-mon[33335]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:49.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:49 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T09:20:49.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:49 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:49.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T09:20:49.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:49.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:49 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T09:20:49.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:49 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:50.306 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:50 smithi170 ceph-mon[33632]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:50.306 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:50 smithi170 ceph-mon[33632]: Deploying daemon osd.8 on smithi170 2024-05-09T09:20:50.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:50 smithi012 ceph-mon[30140]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:50.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:50 smithi012 ceph-mon[30140]: Deploying daemon osd.8 on smithi170 2024-05-09T09:20:50.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:50 smithi016 ceph-mon[33335]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:50.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:50 smithi016 ceph-mon[33335]: Deploying daemon osd.8 on smithi170 2024-05-09T09:20:52.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:52 smithi012 ceph-mon[30140]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:52.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:52 smithi170 ceph-mon[33632]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:52.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:52 smithi016 ceph-mon[33335]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:54 smithi012 ceph-mon[30140]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:54.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:54 smithi170 ceph-mon[33632]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:54.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:54 smithi016 ceph-mon[33335]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:55.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:55.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:55.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:55.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:55.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:55.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:55.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:55.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:55.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:56.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:56 smithi170 ceph-mon[33632]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:56.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:56 smithi012 ceph-mon[30140]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:56.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:56 smithi016 ceph-mon[33335]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:58.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:58.149+0000 7f40737fe700 1 -- 172.21.15.170:0/218644687 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f407c062760 con 0x7f40680a2c20 2024-05-09T09:20:58.150 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 8 on host 'smithi170' 2024-05-09T09:20:58.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:58.152+0000 7f40843f4700 1 -- 172.21.15.170:0/218644687 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f40680a2c20 msgr2=0x7f40680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:58.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:58.152+0000 7f40843f4700 1 --2- 172.21.15.170:0/218644687 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f40680a2c20 0x7f40680a50e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40780062a0 tx=0x7f40780061f0 comp rx=0 tx=0).stop 2024-05-09T09:20:58.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:58.152+0000 7f40843f4700 1 -- 172.21.15.170:0/218644687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f407c105740 msgr2=0x7f407c198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:58.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:58.152+0000 7f40843f4700 1 --2- 172.21.15.170:0/218644687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f407c105740 0x7f407c198ae0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f406c00d980 tx=0x7f406c00dd40 comp rx=0 tx=0).stop 2024-05-09T09:20:58.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:58.152+0000 7f40843f4700 1 -- 172.21.15.170:0/218644687 shutdown_connections 2024-05-09T09:20:58.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:58.152+0000 7f40843f4700 1 --2- 172.21.15.170:0/218644687 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f40680a2c20 0x7f40680a50e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:58.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:58.152+0000 7f40843f4700 1 --2- 172.21.15.170:0/218644687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f407c108070 0x7f407c19fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:58.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:58.152+0000 7f40843f4700 1 --2- 172.21.15.170:0/218644687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f407c102e10 0x7f407c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:58.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:58.152+0000 7f40843f4700 1 --2- 172.21.15.170:0/218644687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f407c105740 0x7f407c198ae0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:58.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:58.152+0000 7f40843f4700 1 -- 172.21.15.170:0/218644687 >> 172.21.15.170:0/218644687 conn(0x7f407c0fc820 msgr2=0x7f407c1036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:58.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:58.152+0000 7f40843f4700 1 -- 172.21.15.170:0/218644687 shutdown_connections 2024-05-09T09:20:58.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:20:58.152+0000 7f40843f4700 1 -- 172.21.15.170:0/218644687 wait complete. 2024-05-09T09:20:58.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:58 smithi170 ceph-mon[33632]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:58.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:58 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:58 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:58 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:58.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:58 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:58.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:58 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:58 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:58 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:58 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:58.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:58 smithi170 ceph-mon[33632]: from='osd.8 [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T09:20:58.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:20:58 smithi170 ceph-mon[33632]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T09:20:58.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:58 smithi016 ceph-mon[33335]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:58.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:58.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:58.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:58 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:58.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:58 smithi016 ceph-mon[33335]: from='osd.8 [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T09:20:58.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:20:58 smithi016 ceph-mon[33335]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T09:20:58.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:58 smithi012 ceph-mon[30140]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:20:58.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:58 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:58 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:58 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:58.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:58 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:58.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:58 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:58 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:58 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:20:58.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:58 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:58.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:58 smithi012 ceph-mon[30140]: from='osd.8 [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T09:20:58.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:20:58 smithi012 ceph-mon[30140]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T09:20:58.810 DEBUG:teuthology.orchestra.run.smithi170:osd.8> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.8.service 2024-05-09T09:20:58.812 INFO:tasks.cephadm:Deploying osd.9 on smithi012 with /dev/vg_nvme/lv_1... 2024-05-09T09:20:58.812 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T09:20:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:21:00.330 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:00 smithi170 ceph-mon[33632]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:21:00.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:00 smithi170 ceph-mon[33632]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T09:21:00.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:00 smithi170 ceph-mon[33632]: from='osd.8 [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:21:00.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:00 smithi170 ceph-mon[33632]: osdmap e57: 9 total, 8 up, 9 in 2024-05-09T09:21:00.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:00 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:00.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:00 smithi170 ceph-mon[33632]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:21:00.431 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:00 smithi012 ceph-mon[30140]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:21:00.431 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:00 smithi012 ceph-mon[30140]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T09:21:00.431 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:00 smithi012 ceph-mon[30140]: from='osd.8 [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:21:00.431 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:00 smithi012 ceph-mon[30140]: osdmap e57: 9 total, 8 up, 9 in 2024-05-09T09:21:00.432 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:00 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:00.432 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:00 smithi012 ceph-mon[30140]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:21:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:00 smithi016 ceph-mon[33335]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:21:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:00 smithi016 ceph-mon[33335]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T09:21:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:00 smithi016 ceph-mon[33335]: from='osd.8 [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:21:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:00 smithi016 ceph-mon[33335]: osdmap e57: 9 total, 8 up, 9 in 2024-05-09T09:21:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:00 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:00 smithi016 ceph-mon[33335]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:21:00.993 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:21:00 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:21:00.647+0000 7f39cbea6700 -1 osd.8 0 waiting for initial osdmap 2024-05-09T09:21:00.994 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:21:00 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:21:00.659+0000 7f39c3c91700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:21:01.203 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T09:21:01.203 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T09:21:01.203 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-05-09T09:21:01.203 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-05-09T09:21:01.203 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111603 s, 940 MB/s 2024-05-09T09:21:01.203 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-05-09T09:21:01.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:01 smithi012 ceph-mon[30140]: purged_snaps scrub starts 2024-05-09T09:21:01.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:01 smithi012 ceph-mon[30140]: purged_snaps scrub ok 2024-05-09T09:21:01.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:01 smithi012 ceph-mon[30140]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-05-09T09:21:01.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:01 smithi012 ceph-mon[30140]: osdmap e58: 9 total, 8 up, 9 in 2024-05-09T09:21:01.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:01 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:01.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:01 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:01.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:01 smithi012 ceph-mon[30140]: from='osd.8 ' entity='osd.8' 2024-05-09T09:21:01.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:01 smithi170 ceph-mon[33632]: purged_snaps scrub starts 2024-05-09T09:21:01.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:01 smithi170 ceph-mon[33632]: purged_snaps scrub ok 2024-05-09T09:21:01.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:01 smithi170 ceph-mon[33632]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-05-09T09:21:01.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:01 smithi170 ceph-mon[33632]: osdmap e58: 9 total, 8 up, 9 in 2024-05-09T09:21:01.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:01 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:01.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:01 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:01.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:01 smithi170 ceph-mon[33632]: from='osd.8 ' entity='osd.8' 2024-05-09T09:21:01.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:01 smithi016 ceph-mon[33335]: purged_snaps scrub starts 2024-05-09T09:21:01.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:01 smithi016 ceph-mon[33335]: purged_snaps scrub ok 2024-05-09T09:21:01.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:01 smithi016 ceph-mon[33335]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-05-09T09:21:01.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:01 smithi016 ceph-mon[33335]: osdmap e58: 9 total, 8 up, 9 in 2024-05-09T09:21:01.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:01 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:01.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:01 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:01.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:01 smithi016 ceph-mon[33335]: from='osd.8 ' entity='osd.8' 2024-05-09T09:21:01.930 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_1 2024-05-09T09:21:02.190 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:21:02.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:21:02.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:02.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: osd.8 [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847] boot 2024-05-09T09:21:02.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: osdmap e59: 9 total, 9 up, 9 in 2024-05-09T09:21:02.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:02.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:02.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:02.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:02.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:02.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:02.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:02.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:02.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:02.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:02.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:21:02.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:02.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: osd.8 [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847] boot 2024-05-09T09:21:02.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: osdmap e59: 9 total, 9 up, 9 in 2024-05-09T09:21:02.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:02.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:02.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:02.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:02.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:02.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:02.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:02.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:02.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:02.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:02.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:21:02.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:02.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: osd.8 [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847] boot 2024-05-09T09:21:02.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: osdmap e59: 9 total, 9 up, 9 in 2024-05-09T09:21:02.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:21:02.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:02.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:03.302 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:03 smithi012 ceph-mon[30140]: Detected new or changed devices on smithi170 2024-05-09T09:21:03.303 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:03 smithi012 ceph-mon[30140]: Adjusting osd_memory_target on smithi170 to 5696M 2024-05-09T09:21:03.303 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:03 smithi012 ceph-mon[30140]: osdmap e60: 9 total, 9 up, 9 in 2024-05-09T09:21:03.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:03 smithi170 ceph-mon[33632]: Detected new or changed devices on smithi170 2024-05-09T09:21:03.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:03 smithi170 ceph-mon[33632]: Adjusting osd_memory_target on smithi170 to 5696M 2024-05-09T09:21:03.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:03 smithi170 ceph-mon[33632]: osdmap e60: 9 total, 9 up, 9 in 2024-05-09T09:21:03.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:03 smithi016 ceph-mon[33335]: Detected new or changed devices on smithi170 2024-05-09T09:21:03.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:03 smithi016 ceph-mon[33335]: Adjusting osd_memory_target on smithi170 to 5696M 2024-05-09T09:21:03.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:03 smithi016 ceph-mon[33335]: osdmap e60: 9 total, 9 up, 9 in 2024-05-09T09:21:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.961+0000 7f62b794f700 1 -- 172.21.15.12:0/1192883685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b0102e20 msgr2=0x7f62b01096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.961+0000 7f62b794f700 1 --2- 172.21.15.12:0/1192883685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b0102e20 0x7f62b01096d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f62a800b0d0 tx=0x7f62a800b3e0 comp rx=0 tx=0).stop 2024-05-09T09:21:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.961+0000 7f62b794f700 1 -- 172.21.15.12:0/1192883685 shutdown_connections 2024-05-09T09:21:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.961+0000 7f62b794f700 1 --2- 172.21.15.12:0/1192883685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b0102e20 0x7f62b01096d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.961+0000 7f62b794f700 1 --2- 172.21.15.12:0/1192883685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b0101280 0x7f62b01016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.961+0000 7f62b794f700 1 --2- 172.21.15.12:0/1192883685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b0102480 0x7f62b01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.961+0000 7f62b794f700 1 -- 172.21.15.12:0/1192883685 >> 172.21.15.12:0/1192883685 conn(0x7f62b00fc820 msgr2=0x7f62b00fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.961+0000 7f62b794f700 1 -- 172.21.15.12:0/1192883685 shutdown_connections 2024-05-09T09:21:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.961+0000 7f62b794f700 1 -- 172.21.15.12:0/1192883685 wait complete. 2024-05-09T09:21:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.962+0000 7f62b794f700 1 Processor -- start 2024-05-09T09:21:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.962+0000 7f62b794f700 1 -- start start 2024-05-09T09:21:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.962+0000 7f62b794f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b0101280 0x7f62b0105210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.962+0000 7f62b794f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b0102480 0x7f62b0103860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.962+0000 7f62b794f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b0102e20 0x7f62b0103da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.962+0000 7f62b794f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f62b010ced0 con 0x7f62b0102e20 2024-05-09T09:21:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.962+0000 7f62b794f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62b010cd50 con 0x7f62b0101280 2024-05-09T09:21:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.962+0000 7f62b794f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f62b010d050 con 0x7f62b0102480 2024-05-09T09:21:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.962+0000 7f62b5eec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b0102e20 0x7f62b0103da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62b56eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b0101280 0x7f62b0105210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62b5eec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b0102e20 0x7f62b0103da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48048/0 (socket says 172.21.15.12:48048) 2024-05-09T09:21:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62b5eec700 1 -- 172.21.15.12:0/741259012 learned_addr learned my addr 172.21.15.12:0/741259012 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:21:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62b4eea700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b0102480 0x7f62b0103860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62b5eec700 1 -- 172.21.15.12:0/741259012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b0102480 msgr2=0x7f62b0103860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62b5eec700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b0102480 0x7f62b0103860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62b5eec700 1 -- 172.21.15.12:0/741259012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b0101280 msgr2=0x7f62b0105210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62b5eec700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b0101280 0x7f62b0105210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62b5eec700 1 -- 172.21.15.12:0/741259012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b0104650 con 0x7f62b0102e20 2024-05-09T09:21:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62b4eea700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b0102480 0x7f62b0103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:21:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62b5eec700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b0102e20 0x7f62b0103da0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f62a8007a50 tx=0x7f62a8007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62a27fc700 1 -- 172.21.15.12:0/741259012 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a801f070 con 0x7f62b0102e20 2024-05-09T09:21:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62a27fc700 1 -- 172.21.15.12:0/741259012 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f62a8003c00 con 0x7f62b0102e20 2024-05-09T09:21:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62a27fc700 1 -- 172.21.15.12:0/741259012 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a801b6a0 con 0x7f62b0102e20 2024-05-09T09:21:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62b794f700 1 -- 172.21.15.12:0/741259012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b01a2710 con 0x7f62b0102e20 2024-05-09T09:21:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.963+0000 7f62b794f700 1 -- 172.21.15.12:0/741259012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b01a2be0 con 0x7f62b0102e20 2024-05-09T09:21:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.964+0000 7f62b794f700 1 -- 172.21.15.12:0/741259012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b0069010 con 0x7f62b0102e20 2024-05-09T09:21:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.966+0000 7f62a27fc700 1 -- 172.21.15.12:0/741259012 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f62a8022070 con 0x7f62b0102e20 2024-05-09T09:21:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.966+0000 7f62a27fc700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f629c0a2ca0 0x7f629c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.966+0000 7f62b56eb700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f629c0a2ca0 0x7f629c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.966+0000 7f62a27fc700 1 -- 172.21.15.12:0/741259012 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5829+0+0 (secure 0 0 0) 0x7f62a800f020 con 0x7f62b0102e20 2024-05-09T09:21:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.967+0000 7f62b56eb700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f629c0a2ca0 0x7f629c0a5160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62b01022e0 tx=0x7f62ac009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:03.970+0000 7f62a27fc700 1 -- 172.21.15.12:0/741259012 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f62a8029020 con 0x7f62b0102e20 2024-05-09T09:21:04.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:04.182+0000 7f62b794f700 1 -- 172.21.15.12:0/741259012 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f62b00fe9c0 con 0x7f629c0a2ca0 2024-05-09T09:21:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:04 smithi012 ceph-mon[30140]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:04 smithi012 ceph-mon[30140]: osdmap e61: 9 total, 9 up, 9 in 2024-05-09T09:21:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:04 smithi012 ceph-mon[30140]: osdmap e62: 9 total, 9 up, 9 in 2024-05-09T09:21:04.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:04 smithi170 ceph-mon[33632]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:04.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:04 smithi170 ceph-mon[33632]: osdmap e61: 9 total, 9 up, 9 in 2024-05-09T09:21:04.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:04 smithi170 ceph-mon[33632]: osdmap e62: 9 total, 9 up, 9 in 2024-05-09T09:21:04.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:04 smithi016 ceph-mon[33335]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:04.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:04 smithi016 ceph-mon[33335]: osdmap e61: 9 total, 9 up, 9 in 2024-05-09T09:21:04.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:04 smithi016 ceph-mon[33335]: osdmap e62: 9 total, 9 up, 9 in 2024-05-09T09:21:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:05 smithi012 ceph-mon[30140]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:21:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:21:05.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:21:05.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:05.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:05 smithi012 ceph-mon[30140]: osdmap e63: 9 total, 9 up, 9 in 2024-05-09T09:21:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:05 smithi170 ceph-mon[33632]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:21:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:21:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:21:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:05.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:05 smithi170 ceph-mon[33632]: osdmap e63: 9 total, 9 up, 9 in 2024-05-09T09:21:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:05 smithi016 ceph-mon[33335]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:21:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:21:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:21:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:05.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:05 smithi016 ceph-mon[33335]: osdmap e63: 9 total, 9 up, 9 in 2024-05-09T09:21:06.447 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:06 smithi012 ceph-mon[30140]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:06 smithi170 ceph-mon[33632]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:06.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:06 smithi016 ceph-mon[33335]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:08.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:08 smithi012 ceph-mon[30140]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-09T09:21:08.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:08 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/408350176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be6068d1-eaf5-4694-a49f-e7baa8613fe3"}]: dispatch 2024-05-09T09:21:08.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:08 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/408350176' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be6068d1-eaf5-4694-a49f-e7baa8613fe3"}]': finished 2024-05-09T09:21:08.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:08 smithi012 ceph-mon[30140]: osdmap e64: 10 total, 9 up, 10 in 2024-05-09T09:21:08.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:08 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:08.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:08 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2296889053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:21:08.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:08 smithi170 ceph-mon[33632]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-09T09:21:08.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:08 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/408350176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be6068d1-eaf5-4694-a49f-e7baa8613fe3"}]: dispatch 2024-05-09T09:21:08.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:08 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/408350176' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be6068d1-eaf5-4694-a49f-e7baa8613fe3"}]': finished 2024-05-09T09:21:08.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:08 smithi170 ceph-mon[33632]: osdmap e64: 10 total, 9 up, 10 in 2024-05-09T09:21:08.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:08 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:08.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:08 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2296889053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:21:08.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:08 smithi016 ceph-mon[33335]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-09T09:21:08.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:08 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/408350176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be6068d1-eaf5-4694-a49f-e7baa8613fe3"}]: dispatch 2024-05-09T09:21:08.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:08 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/408350176' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be6068d1-eaf5-4694-a49f-e7baa8613fe3"}]': finished 2024-05-09T09:21:08.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:08 smithi016 ceph-mon[33335]: osdmap e64: 10 total, 9 up, 10 in 2024-05-09T09:21:08.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:08 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:08.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:08 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2296889053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:21:10.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:10 smithi012 ceph-mon[30140]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-09T09:21:10.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:10 smithi170 ceph-mon[33632]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-09T09:21:10.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:10 smithi016 ceph-mon[33335]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-09T09:21:12.349 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:12 smithi012 ceph-mon[30140]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T09:21:12.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:12 smithi170 ceph-mon[33632]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T09:21:12.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:12 smithi016 ceph-mon[33335]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T09:21:14.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:14 smithi012 ceph-mon[30140]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:21:14.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:14 smithi170 ceph-mon[33632]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:21:14.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:14 smithi016 ceph-mon[33335]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:21:15.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:15 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T09:21:15.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:15 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:15.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:15 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T09:21:15.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:15 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:15.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:15 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T09:21:15.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:15 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:16.404 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:16 smithi016 ceph-mon[33335]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:21:16.404 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:16 smithi016 ceph-mon[33335]: Deploying daemon osd.9 on smithi012 2024-05-09T09:21:16.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:16 smithi012 ceph-mon[30140]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:21:16.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:16 smithi012 ceph-mon[30140]: Deploying daemon osd.9 on smithi012 2024-05-09T09:21:16.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:16 smithi170 ceph-mon[33632]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:21:16.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:16 smithi170 ceph-mon[33632]: Deploying daemon osd.9 on smithi012 2024-05-09T09:21:18.400 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:18 smithi012 ceph-mon[30140]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:18 smithi170 ceph-mon[33632]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:18.506 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:18 smithi016 ceph-mon[33335]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:20.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:20 smithi012 ceph-mon[30140]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:20.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:20 smithi170 ceph-mon[33632]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:20.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:20 smithi016 ceph-mon[33335]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:22 smithi012 ceph-mon[30140]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:22 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:22 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:22 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:22.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:22 smithi170 ceph-mon[33632]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:22.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:22 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:22.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:22 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:22.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:22 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:22.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:22 smithi016 ceph-mon[33335]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:22.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:22 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:22.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:22 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:22.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:22 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:24.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:24 smithi012 ceph-mon[30140]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:24.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:24 smithi170 ceph-mon[33632]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:24.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:24 smithi016 ceph-mon[33335]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:25.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:25.346+0000 7f62a27fc700 1 -- 172.21.15.12:0/741259012 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f62b00fe9c0 con 0x7f629c0a2ca0 2024-05-09T09:21:25.423 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 9 on host 'smithi012' 2024-05-09T09:21:25.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:25.348+0000 7f62b794f700 1 -- 172.21.15.12:0/741259012 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f629c0a2ca0 msgr2=0x7f629c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:25.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:25.348+0000 7f62b794f700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f629c0a2ca0 0x7f629c0a5160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62b01022e0 tx=0x7f62ac009190 comp rx=0 tx=0).stop 2024-05-09T09:21:25.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:25.348+0000 7f62b794f700 1 -- 172.21.15.12:0/741259012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b0102e20 msgr2=0x7f62b0103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:25.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:25.348+0000 7f62b794f700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b0102e20 0x7f62b0103da0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f62a8007a50 tx=0x7f62a8007a80 comp rx=0 tx=0).stop 2024-05-09T09:21:25.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:25.348+0000 7f62b794f700 1 -- 172.21.15.12:0/741259012 shutdown_connections 2024-05-09T09:21:25.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:25.348+0000 7f62b794f700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f629c0a2ca0 0x7f629c0a5160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:25.348+0000 7f62b794f700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b0102e20 0x7f62b0103da0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:25.348+0000 7f62b794f700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b0101280 0x7f62b0105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:25.348+0000 7f62b794f700 1 --2- 172.21.15.12:0/741259012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b0102480 0x7f62b0103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:25.349+0000 7f62b794f700 1 -- 172.21.15.12:0/741259012 >> 172.21.15.12:0/741259012 conn(0x7f62b00fc820 msgr2=0x7f62b00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:25.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:25.349+0000 7f62b794f700 1 -- 172.21.15.12:0/741259012 shutdown_connections 2024-05-09T09:21:25.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:21:25.349+0000 7f62b794f700 1 -- 172.21.15.12:0/741259012 wait complete. 2024-05-09T09:21:25.684 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:25 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.684 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:25 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.684 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:25 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:25.685 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:25 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:25.685 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:25 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.685 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:25 smithi012 ceph-mon[30140]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:25.685 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:25 smithi012 ceph-mon[30140]: from='osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T09:21:25.685 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:25 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.685 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:25 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.685 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:25 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:25.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:25 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:25 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:25 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:25.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:25 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:25.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:25 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:25 smithi170 ceph-mon[33632]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:25.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:25 smithi170 ceph-mon[33632]: from='osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T09:21:25.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:25 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:25 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:25 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:25.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:25 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:25 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:25 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:25.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:25 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:25.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:25 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:25 smithi016 ceph-mon[33335]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:25.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:25 smithi016 ceph-mon[33335]: from='osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T09:21:25.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:25 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:25 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:25.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:25 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:26.014 DEBUG:teuthology.orchestra.run.smithi012:osd.9> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.9.service 2024-05-09T09:21:26.017 INFO:tasks.cephadm:Deploying osd.10 on smithi016 with /dev/vg_nvme/lv_1... 2024-05-09T09:21:26.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T09:21:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:21:26.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:26 smithi012 ceph-mon[30140]: from='osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T09:21:26.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:26 smithi012 ceph-mon[30140]: osdmap e65: 10 total, 9 up, 10 in 2024-05-09T09:21:26.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:26 smithi012 ceph-mon[30140]: from='osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-09T09:21:26.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:26 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:26.589 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:21:26 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:21:26.502+0000 7f7ddf63e700 -1 osd.9 0 waiting for initial osdmap 2024-05-09T09:21:26.589 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:21:26 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:21:26.522+0000 7f7ddb431700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:21:26.652 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:26 smithi016 ceph-mon[33335]: from='osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T09:21:26.653 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:26 smithi016 ceph-mon[33335]: osdmap e65: 10 total, 9 up, 10 in 2024-05-09T09:21:26.653 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:26 smithi016 ceph-mon[33335]: from='osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-09T09:21:26.653 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:26 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:26.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:26 smithi170 ceph-mon[33632]: from='osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T09:21:26.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:26 smithi170 ceph-mon[33632]: osdmap e65: 10 total, 9 up, 10 in 2024-05-09T09:21:26.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:26 smithi170 ceph-mon[33632]: from='osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-09T09:21:26.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:26 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:27.630 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:27 smithi012 ceph-mon[30140]: from='osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-09T09:21:27.630 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:27 smithi012 ceph-mon[30140]: osdmap e66: 10 total, 9 up, 10 in 2024-05-09T09:21:27.630 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:27 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:27.630 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:27 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:27.630 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:27 smithi012 ceph-mon[30140]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:27.729 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:27 smithi016 ceph-mon[33335]: from='osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-09T09:21:27.729 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:27 smithi016 ceph-mon[33335]: osdmap e66: 10 total, 9 up, 10 in 2024-05-09T09:21:27.729 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:27 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:27.729 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:27 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:27.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:27 smithi016 ceph-mon[33335]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:27 smithi170 ceph-mon[33632]: from='osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-09T09:21:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:27 smithi170 ceph-mon[33632]: osdmap e66: 10 total, 9 up, 10 in 2024-05-09T09:21:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:27 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:27 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:27 smithi170 ceph-mon[33632]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:21:28.465 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T09:21:28.465 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T09:21:28.465 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-05-09T09:21:28.465 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-05-09T09:21:28.465 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0082582 s, 1.3 GB/s 2024-05-09T09:21:28.465 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 2024-05-09T09:21:28.466 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-05-09T09:21:28.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:28 smithi016 ceph-mon[33335]: purged_snaps scrub starts 2024-05-09T09:21:28.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:28 smithi016 ceph-mon[33335]: purged_snaps scrub ok 2024-05-09T09:21:28.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:28 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:28.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:28 smithi016 ceph-mon[33335]: osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834] boot 2024-05-09T09:21:28.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:28 smithi016 ceph-mon[33335]: osdmap e67: 10 total, 10 up, 10 in 2024-05-09T09:21:28.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:28 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:28.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:28 smithi012 ceph-mon[30140]: purged_snaps scrub starts 2024-05-09T09:21:28.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:28 smithi012 ceph-mon[30140]: purged_snaps scrub ok 2024-05-09T09:21:28.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:28 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:28.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:28 smithi012 ceph-mon[30140]: osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834] boot 2024-05-09T09:21:28.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:28 smithi012 ceph-mon[30140]: osdmap e67: 10 total, 10 up, 10 in 2024-05-09T09:21:28.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:28 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:28.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:28 smithi170 ceph-mon[33632]: purged_snaps scrub starts 2024-05-09T09:21:28.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:28 smithi170 ceph-mon[33632]: purged_snaps scrub ok 2024-05-09T09:21:28.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:28 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:28.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:28 smithi170 ceph-mon[33632]: osd.9 [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834] boot 2024-05-09T09:21:28.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:28 smithi170 ceph-mon[33632]: osdmap e67: 10 total, 10 up, 10 in 2024-05-09T09:21:28.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:28 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:21:29.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-05-09T09:21:29.407 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:21:29.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:29 smithi012 ceph-mon[30140]: osdmap e68: 10 total, 10 up, 10 in 2024-05-09T09:21:29.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:29 smithi012 ceph-mon[30140]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:29.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:29.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:29.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:29.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:29.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:29.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:29.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:29.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:29.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:29.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:29.785 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:29 smithi016 ceph-mon[33335]: osdmap e68: 10 total, 10 up, 10 in 2024-05-09T09:21:29.785 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:29 smithi016 ceph-mon[33335]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:29.785 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:29.785 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:29.785 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:29.785 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:29.785 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:29.785 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:29.785 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:29.786 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:29.786 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:29.786 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:29 smithi170 ceph-mon[33632]: osdmap e68: 10 total, 10 up, 10 in 2024-05-09T09:21:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:29 smithi170 ceph-mon[33632]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:29.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:29.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:29.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:30.833 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:30 smithi016 ceph-mon[33335]: Detected new or changed devices on smithi012 2024-05-09T09:21:30.834 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:30 smithi016 ceph-mon[33335]: Adjusting osd_memory_target on smithi012 to 4273M 2024-05-09T09:21:30.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:30 smithi012 ceph-mon[30140]: Detected new or changed devices on smithi012 2024-05-09T09:21:30.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:30 smithi012 ceph-mon[30140]: Adjusting osd_memory_target on smithi012 to 4273M 2024-05-09T09:21:30.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:30 smithi170 ceph-mon[33632]: Detected new or changed devices on smithi012 2024-05-09T09:21:30.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:30 smithi170 ceph-mon[33632]: Adjusting osd_memory_target on smithi012 to 4273M 2024-05-09T09:21:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.071+0000 7f11b39c2700 1 -- 172.21.15.16:0/2635718747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac100c20 msgr2=0x7f11ac101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.071+0000 7f11b39c2700 1 --2- 172.21.15.16:0/2635718747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac100c20 0x7f11ac101040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009990 tx=0x7f11a8009ca0 comp rx=0 tx=0).stop 2024-05-09T09:21:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.071+0000 7f11b39c2700 1 -- 172.21.15.16:0/2635718747 shutdown_connections 2024-05-09T09:21:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.071+0000 7f11b39c2700 1 --2- 172.21.15.16:0/2635718747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11ac101f40 0x7f11ac1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.071+0000 7f11b39c2700 1 --2- 172.21.15.16:0/2635718747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac100c20 0x7f11ac101040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.071+0000 7f11b39c2700 1 --2- 172.21.15.16:0/2635718747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11ac101580 0x7f11ac101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.071+0000 7f11b39c2700 1 -- 172.21.15.16:0/2635718747 >> 172.21.15.16:0/2635718747 conn(0x7f11ac0fc840 msgr2=0x7f11ac0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.071+0000 7f11b39c2700 1 -- 172.21.15.16:0/2635718747 shutdown_connections 2024-05-09T09:21:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.071+0000 7f11b39c2700 1 -- 172.21.15.16:0/2635718747 wait complete. 2024-05-09T09:21:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.071+0000 7f11b39c2700 1 Processor -- start 2024-05-09T09:21:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.071+0000 7f11b39c2700 1 -- start start 2024-05-09T09:21:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b39c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11ac100c20 0x7f11ac19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b39c2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11ac101580 0x7f11ac19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b39c2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac101f40 0x7f11ac1a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b39c2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac110150 con 0x7f11ac101580 2024-05-09T09:21:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b39c2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac10ffd0 con 0x7f11ac101f40 2024-05-09T09:21:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b39c2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1102d0 con 0x7f11ac100c20 2024-05-09T09:21:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b1f5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac101f40 0x7f11ac1a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b1f5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac101f40 0x7f11ac1a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50266/0 (socket says 172.21.15.16:50266) 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b1f5f700 1 -- 172.21.15.16:0/1521158091 learned_addr learned my addr 172.21.15.16:0/1521158091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b0f5d700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11ac101580 0x7f11ac19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b175e700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11ac100c20 0x7f11ac19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b1f5f700 1 -- 172.21.15.16:0/1521158091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11ac100c20 msgr2=0x7f11ac19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b1f5f700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11ac100c20 0x7f11ac19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b1f5f700 1 -- 172.21.15.16:0/1521158091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11ac101580 msgr2=0x7f11ac19acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b1f5f700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11ac101580 0x7f11ac19acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b1f5f700 1 -- 172.21.15.16:0/1521158091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac1a23d0 con 0x7f11ac101f40 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11b1f5f700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac101f40 0x7f11ac1a1d30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f11a400ada0 tx=0x7f11a400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.072+0000 7f11a27fc700 1 -- 172.21.15.16:0/1521158091 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4017020 con 0x7f11ac101f40 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.073+0000 7f11a27fc700 1 -- 172.21.15.16:0/1521158091 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f11a400ce50 con 0x7f11ac101f40 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.073+0000 7f11b39c2700 1 -- 172.21.15.16:0/1521158091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ac1a26c0 con 0x7f11ac101f40 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.073+0000 7f11a27fc700 1 -- 172.21.15.16:0/1521158091 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4005520 con 0x7f11ac101f40 2024-05-09T09:21:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.073+0000 7f11b39c2700 1 -- 172.21.15.16:0/1521158091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ac1a2c00 con 0x7f11ac101f40 2024-05-09T09:21:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.073+0000 7f11b39c2700 1 -- 172.21.15.16:0/1521158091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ac069010 con 0x7f11ac101f40 2024-05-09T09:21:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.075+0000 7f11a27fc700 1 -- 172.21.15.16:0/1521158091 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f11a400c980 con 0x7f11ac101f40 2024-05-09T09:21:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.076+0000 7f11a27fc700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f11980a2ca0 0x7f11980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.076+0000 7f11a27fc700 1 -- 172.21.15.16:0/1521158091 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f11a40ce660 con 0x7f11ac101f40 2024-05-09T09:21:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.076+0000 7f11b175e700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f11980a2ca0 0x7f11980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.077+0000 7f11b175e700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f11980a2ca0 0x7f11980a5160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f11a800b2e0 tx=0x7f11a801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.080+0000 7f11a27fc700 1 -- 172.21.15.16:0/1521158091 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f11a4021020 con 0x7f11ac101f40 2024-05-09T09:21:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:31.295+0000 7f11b39c2700 1 -- 172.21.15.16:0/1521158091 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f11ac062820 con 0x7f11980a2ca0 2024-05-09T09:21:31.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:31 smithi016 ceph-mon[33335]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:31.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:31 smithi016 ceph-mon[33335]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:21:31.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:31 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:21:31.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:31 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:21:31.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:31 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:31 smithi012 ceph-mon[30140]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:31 smithi012 ceph-mon[30140]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:21:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:31 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:21:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:31 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:21:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:31 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:31.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:31 smithi170 ceph-mon[33632]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:31.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:31 smithi170 ceph-mon[33632]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:21:31.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:31 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:21:31.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:31 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:21:31.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:31 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:33.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:33 smithi012 ceph-mon[30140]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:33.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:33 smithi170 ceph-mon[33632]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:34.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:33 smithi016 ceph-mon[33335]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:34.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:34 smithi012 ceph-mon[30140]: from='client.? 172.21.15.16:0/1009666901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa08b1c-2f8b-4853-accc-f84809ba98f1"}]: dispatch 2024-05-09T09:21:34.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:34 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa08b1c-2f8b-4853-accc-f84809ba98f1"}]: dispatch 2024-05-09T09:21:34.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:34 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caa08b1c-2f8b-4853-accc-f84809ba98f1"}]': finished 2024-05-09T09:21:34.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:34 smithi012 ceph-mon[30140]: osdmap e69: 11 total, 10 up, 11 in 2024-05-09T09:21:34.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:34 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:34.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:34 smithi170 ceph-mon[33632]: from='client.? 172.21.15.16:0/1009666901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa08b1c-2f8b-4853-accc-f84809ba98f1"}]: dispatch 2024-05-09T09:21:34.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:34 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa08b1c-2f8b-4853-accc-f84809ba98f1"}]: dispatch 2024-05-09T09:21:34.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:34 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caa08b1c-2f8b-4853-accc-f84809ba98f1"}]': finished 2024-05-09T09:21:34.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:34 smithi170 ceph-mon[33632]: osdmap e69: 11 total, 10 up, 11 in 2024-05-09T09:21:34.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:34 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:35.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:34 smithi016 ceph-mon[33335]: from='client.? 172.21.15.16:0/1009666901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa08b1c-2f8b-4853-accc-f84809ba98f1"}]: dispatch 2024-05-09T09:21:35.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:34 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa08b1c-2f8b-4853-accc-f84809ba98f1"}]: dispatch 2024-05-09T09:21:35.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:34 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caa08b1c-2f8b-4853-accc-f84809ba98f1"}]': finished 2024-05-09T09:21:35.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:34 smithi016 ceph-mon[33335]: osdmap e69: 11 total, 10 up, 11 in 2024-05-09T09:21:35.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:34 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:35.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:35 smithi012 ceph-mon[30140]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:35.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:35 smithi012 ceph-mon[30140]: from='client.? 172.21.15.16:0/3258650890' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:21:35.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:35 smithi170 ceph-mon[33632]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:35.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:35 smithi170 ceph-mon[33632]: from='client.? 172.21.15.16:0/3258650890' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:21:36.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:35 smithi016 ceph-mon[33335]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:36.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:35 smithi016 ceph-mon[33335]: from='client.? 172.21.15.16:0/3258650890' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:21:37.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:37 smithi012 ceph-mon[30140]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:37.859 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:37 smithi016 ceph-mon[33335]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:37.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:37 smithi170 ceph-mon[33632]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:39.907 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:39 smithi016 ceph-mon[33335]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:39.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:39 smithi012 ceph-mon[30140]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:39.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:39 smithi170 ceph-mon[33632]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:41.684 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:41 smithi016 ceph-mon[33335]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:41.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:41 smithi012 ceph-mon[30140]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:41.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:41 smithi170 ceph-mon[33632]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:42.628 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:42 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T09:21:42.628 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:42 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:42.628 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:42 smithi016 ceph-mon[33335]: Deploying daemon osd.10 on smithi016 2024-05-09T09:21:42.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:42 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T09:21:42.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:42 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:42.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:42 smithi012 ceph-mon[30140]: Deploying daemon osd.10 on smithi016 2024-05-09T09:21:42.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:42 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T09:21:42.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:42 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:42.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:42 smithi170 ceph-mon[33632]: Deploying daemon osd.10 on smithi016 2024-05-09T09:21:43.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:43 smithi016 ceph-mon[33335]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:43.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:43 smithi012 ceph-mon[30140]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:43.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:43 smithi170 ceph-mon[33632]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:45.789 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:45 smithi016 ceph-mon[33335]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:45.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:45 smithi012 ceph-mon[30140]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:45.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:45 smithi170 ceph-mon[33632]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:47.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:47 smithi016 ceph-mon[33335]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:47.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:47.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:47.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:47.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:47 smithi012 ceph-mon[30140]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:47.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:47.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:47.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:47.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:47 smithi170 ceph-mon[33632]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:47.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:47.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:47.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:49.819 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:49 smithi016 ceph-mon[33335]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:49.986 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:49 smithi012 ceph-mon[30140]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:49.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:49 smithi170 ceph-mon[33632]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:51.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:51.022+0000 7f11a27fc700 1 -- 172.21.15.16:0/1521158091 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f11ac062820 con 0x7f11980a2ca0 2024-05-09T09:21:51.023 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 10 on host 'smithi016' 2024-05-09T09:21:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:51.024+0000 7f11b39c2700 1 -- 172.21.15.16:0/1521158091 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f11980a2ca0 msgr2=0x7f11980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:51.024+0000 7f11b39c2700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f11980a2ca0 0x7f11980a5160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f11a800b2e0 tx=0x7f11a801a040 comp rx=0 tx=0).stop 2024-05-09T09:21:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:51.024+0000 7f11b39c2700 1 -- 172.21.15.16:0/1521158091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac101f40 msgr2=0x7f11ac1a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:51.024+0000 7f11b39c2700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac101f40 0x7f11ac1a1d30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f11a400ada0 tx=0x7f11a400c330 comp rx=0 tx=0).stop 2024-05-09T09:21:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:51.025+0000 7f11b39c2700 1 -- 172.21.15.16:0/1521158091 shutdown_connections 2024-05-09T09:21:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:51.025+0000 7f11b39c2700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f11980a2ca0 0x7f11980a5160 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:51.025+0000 7f11b39c2700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac101f40 0x7f11ac1a1d30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:51.025+0000 7f11b39c2700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11ac100c20 0x7f11ac19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:51.025+0000 7f11b39c2700 1 --2- 172.21.15.16:0/1521158091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11ac101580 0x7f11ac19acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:51.025+0000 7f11b39c2700 1 -- 172.21.15.16:0/1521158091 >> 172.21.15.16:0/1521158091 conn(0x7f11ac0fc840 msgr2=0x7f11ac0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:51.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:51.025+0000 7f11b39c2700 1 -- 172.21.15.16:0/1521158091 shutdown_connections 2024-05-09T09:21:51.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-09T09:21:51.025+0000 7f11b39c2700 1 -- 172.21.15.16:0/1521158091 wait complete. 2024-05-09T09:21:51.668 DEBUG:teuthology.orchestra.run.smithi016:osd.10> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.10.service 2024-05-09T09:21:51.670 INFO:tasks.cephadm:Deploying osd.11 on smithi170 with /dev/vg_nvme/lv_1... 2024-05-09T09:21:51.671 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T09:21:51.920 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:21:52.241 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:51 smithi170 ceph-mon[33632]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:52.241 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:51 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:52.241 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:51 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:52.241 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:51 smithi170 ceph-mon[33632]: from='osd.10 [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T09:21:52.241 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:51 smithi170 ceph-mon[33632]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T09:21:52.241 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:51 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:52.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:51 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:52.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:51 smithi016 ceph-mon[33335]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:52.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:51 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:52.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:51 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:52.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:51 smithi016 ceph-mon[33335]: from='osd.10 [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T09:21:52.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:51 smithi016 ceph-mon[33335]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T09:21:52.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:51 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:52.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:51 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:52.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:51 smithi012 ceph-mon[30140]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:52.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:51 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:52.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:51 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:52.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:51 smithi012 ceph-mon[30140]: from='osd.10 [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T09:21:52.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:51 smithi012 ceph-mon[30140]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T09:21:52.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:51 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:52.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:51 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:53.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:53 smithi016 ceph-mon[33335]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T09:21:53.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:53 smithi016 ceph-mon[33335]: from='osd.10 [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:21:53.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:53 smithi016 ceph-mon[33335]: osdmap e70: 11 total, 10 up, 11 in 2024-05-09T09:21:53.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:53 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:53.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:53 smithi016 ceph-mon[33335]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:21:53.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:21:53 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:21:53.004+0000 7f1eb4747700 -1 osd.10 0 waiting for initial osdmap 2024-05-09T09:21:53.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:21:53 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:21:53.018+0000 7f1eb053a700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:21:53.280 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:53 smithi170 ceph-mon[33632]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T09:21:53.280 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:53 smithi170 ceph-mon[33632]: from='osd.10 [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:21:53.281 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:53 smithi170 ceph-mon[33632]: osdmap e70: 11 total, 10 up, 11 in 2024-05-09T09:21:53.281 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:53 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:53.281 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:53 smithi170 ceph-mon[33632]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:21:53.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:53 smithi012 ceph-mon[30140]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T09:21:53.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:53 smithi012 ceph-mon[30140]: from='osd.10 [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:21:53.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:53 smithi012 ceph-mon[30140]: osdmap e70: 11 total, 10 up, 11 in 2024-05-09T09:21:53.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:53 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:53.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:53 smithi012 ceph-mon[30140]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-09T09:21:54.035 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:54 smithi016 ceph-mon[33335]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:54.036 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:54 smithi016 ceph-mon[33335]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-09T09:21:54.036 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:54 smithi016 ceph-mon[33335]: osdmap e71: 11 total, 10 up, 11 in 2024-05-09T09:21:54.036 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:54 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:54.036 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:54 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:54.036 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:54 smithi016 ceph-mon[33335]: osd.10 [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423] boot 2024-05-09T09:21:54.036 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:54 smithi016 ceph-mon[33335]: osdmap e72: 11 total, 11 up, 11 in 2024-05-09T09:21:54.036 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:54 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:54.064 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T09:21:54.064 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T09:21:54.065 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-05-09T09:21:54.065 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-05-09T09:21:54.065 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00859153 s, 1.2 GB/s 2024-05-09T09:21:54.065 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-05-09T09:21:54.362 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:54 smithi170 ceph-mon[33632]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:54.362 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:54 smithi170 ceph-mon[33632]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-09T09:21:54.362 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:54 smithi170 ceph-mon[33632]: osdmap e71: 11 total, 10 up, 11 in 2024-05-09T09:21:54.363 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:54.363 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:54.363 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:54 smithi170 ceph-mon[33632]: osd.10 [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423] boot 2024-05-09T09:21:54.363 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:54 smithi170 ceph-mon[33632]: osdmap e72: 11 total, 11 up, 11 in 2024-05-09T09:21:54.363 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:54 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:54 smithi012 ceph-mon[30140]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:21:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:54 smithi012 ceph-mon[30140]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-09T09:21:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:54 smithi012 ceph-mon[30140]: osdmap e71: 11 total, 10 up, 11 in 2024-05-09T09:21:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:54 smithi012 ceph-mon[30140]: osd.10 [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423] boot 2024-05-09T09:21:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:54 smithi012 ceph-mon[30140]: osdmap e72: 11 total, 11 up, 11 in 2024-05-09T09:21:54.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:54 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:21:54.666 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_1 2024-05-09T09:21:54.907 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:21:55.359 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:55 smithi170 ceph-mon[33632]: purged_snaps scrub starts 2024-05-09T09:21:55.359 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:55 smithi170 ceph-mon[33632]: purged_snaps scrub ok 2024-05-09T09:21:55.359 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:55 smithi170 ceph-mon[33632]: osdmap e73: 11 total, 11 up, 11 in 2024-05-09T09:21:55.385 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:55 smithi012 ceph-mon[30140]: purged_snaps scrub starts 2024-05-09T09:21:55.385 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:55 smithi012 ceph-mon[30140]: purged_snaps scrub ok 2024-05-09T09:21:55.385 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:55 smithi012 ceph-mon[30140]: osdmap e73: 11 total, 11 up, 11 in 2024-05-09T09:21:55.399 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:55 smithi016 ceph-mon[33335]: purged_snaps scrub starts 2024-05-09T09:21:55.399 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:55 smithi016 ceph-mon[33335]: purged_snaps scrub ok 2024-05-09T09:21:55.399 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:55 smithi016 ceph-mon[33335]: osdmap e73: 11 total, 11 up, 11 in 2024-05-09T09:21:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:56 smithi170 ceph-mon[33632]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:21:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:56 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:56 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:56 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:56 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:56 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:56 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:56 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:56 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:56.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:56 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:56.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:56 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:56.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:56 smithi170 ceph-mon[33632]: osdmap e74: 11 total, 11 up, 11 in 2024-05-09T09:21:56.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:56 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:56.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.474+0000 7fbaf09f2700 1 -- 172.21.15.170:0/806860168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec102480 msgr2=0x7fbaec1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:56.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.474+0000 7fbaf09f2700 1 --2- 172.21.15.170:0/806860168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec102480 0x7fbaec1028e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009990 tx=0x7fbae0009ca0 comp rx=0 tx=0).stop 2024-05-09T09:21:56.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.474+0000 7fbaf09f2700 1 -- 172.21.15.170:0/806860168 shutdown_connections 2024-05-09T09:21:56.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.474+0000 7fbaf09f2700 1 --2- 172.21.15.170:0/806860168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaec102e20 0x7fbaec1096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:56.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.474+0000 7fbaf09f2700 1 --2- 172.21.15.170:0/806860168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaec101280 0x7fbaec1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:56.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.474+0000 7fbaf09f2700 1 --2- 172.21.15.170:0/806860168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec102480 0x7fbaec1028e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:56.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.474+0000 7fbaf09f2700 1 -- 172.21.15.170:0/806860168 >> 172.21.15.170:0/806860168 conn(0x7fbaec0fc820 msgr2=0x7fbaec0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:56.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.474+0000 7fbaf09f2700 1 -- 172.21.15.170:0/806860168 shutdown_connections 2024-05-09T09:21:56.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.474+0000 7fbaf09f2700 1 -- 172.21.15.170:0/806860168 wait complete. 2024-05-09T09:21:56.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaf09f2700 1 Processor -- start 2024-05-09T09:21:56.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaf09f2700 1 -- start start 2024-05-09T09:21:56.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaf09f2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec101280 0x7fbaec198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:56.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaf09f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaec102480 0x7fbaec198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:56.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaf09f2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaec102e20 0x7fbaec19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:56.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaf09f2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec10ceb0 con 0x7fbaec102480 2024-05-09T09:21:56.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaf09f2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec10cd30 con 0x7fbaec102e20 2024-05-09T09:21:56.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaf09f2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec10d030 con 0x7fbaec101280 2024-05-09T09:21:56.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaea59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec101280 0x7fbaec198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:56.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaea59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec101280 0x7fbaec198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52128/0 (socket says 172.21.15.170:52128) 2024-05-09T09:21:56.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaea59c700 1 -- 172.21.15.170:0/3022649260 learned_addr learned my addr 172.21.15.170:0/3022649260 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-09T09:21:56.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbae9d9b700 1 --2- 172.21.15.170:0/3022649260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaec102480 0x7fbaec198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:56.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaea59c700 1 -- 172.21.15.170:0/3022649260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaec102e20 msgr2=0x7fbaec19fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:56.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaea59c700 1 --2- 172.21.15.170:0/3022649260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaec102e20 0x7fbaec19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:56.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaea59c700 1 -- 172.21.15.170:0/3022649260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaec102480 msgr2=0x7fbaec198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:56.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaea59c700 1 --2- 172.21.15.170:0/3022649260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaec102480 0x7fbaec198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:56.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaea59c700 1 -- 172.21.15.170:0/3022649260 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec1a0270 con 0x7fbaec101280 2024-05-09T09:21:56.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.475+0000 7fbaea59c700 1 --2- 172.21.15.170:0/3022649260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec101280 0x7fbaec198660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00c920 tx=0x7fbadc00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:56.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.476+0000 7fbadb7fe700 1 -- 172.21.15.170:0/3022649260 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc004220 con 0x7fbaec101280 2024-05-09T09:21:56.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.476+0000 7fbaf09f2700 1 -- 172.21.15.170:0/3022649260 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec1a0560 con 0x7fbaec101280 2024-05-09T09:21:56.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.476+0000 7fbadb7fe700 1 -- 172.21.15.170:0/3022649260 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbadc007d10 con 0x7fbaec101280 2024-05-09T09:21:56.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.476+0000 7fbadb7fe700 1 -- 172.21.15.170:0/3022649260 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc003c50 con 0x7fbaec101280 2024-05-09T09:21:56.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.476+0000 7fbaf09f2700 1 -- 172.21.15.170:0/3022649260 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec1a0a50 con 0x7fbaec101280 2024-05-09T09:21:56.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.477+0000 7fbaf09f2700 1 -- 172.21.15.170:0/3022649260 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaec069040 con 0x7fbaec101280 2024-05-09T09:21:56.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.478+0000 7fbadb7fe700 1 -- 172.21.15.170:0/3022649260 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbadc0043c0 con 0x7fbaec101280 2024-05-09T09:21:56.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.478+0000 7fbadb7fe700 1 --2- 172.21.15.170:0/3022649260 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbad40a2c20 0x7fbad40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:56.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.479+0000 7fbae9d9b700 1 --2- 172.21.15.170:0/3022649260 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbad40a2c20 0x7fbad40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:56.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.479+0000 7fbadb7fe700 1 -- 172.21.15.170:0/3022649260 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7fbadc0ce9e0 con 0x7fbaec101280 2024-05-09T09:21:56.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.479+0000 7fbae9d9b700 1 --2- 172.21.15.170:0/3022649260 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbad40a2c20 0x7fbad40a50e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbae000b2e0 tx=0x7fbae000b8f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:56.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.482+0000 7fbadb7fe700 1 -- 172.21.15.170:0/3022649260 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbaec069040 con 0x7fbaec101280 2024-05-09T09:21:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:56 smithi012 ceph-mon[30140]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:21:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:56 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:56 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:56 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:56 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:56 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:56 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:56 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:56 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:56 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:56 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:56 smithi012 ceph-mon[30140]: osdmap e74: 11 total, 11 up, 11 in 2024-05-09T09:21:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:56 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:56.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:56 smithi016 ceph-mon[33335]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:21:56.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:56 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:56.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:56 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:56.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:56 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:56.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:56 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:56.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:56 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:56.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:56 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:21:56.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:56 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:56.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:56 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:56.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:56 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:56.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:56 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:56.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:56 smithi016 ceph-mon[33335]: osdmap e74: 11 total, 11 up, 11 in 2024-05-09T09:21:56.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:56 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:56.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:21:56.688+0000 7fbaf09f2700 1 -- 172.21.15.170:0/3022649260 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbaec0fea30 con 0x7fbad40a2c20 2024-05-09T09:21:57.300 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:57 smithi170 ceph-mon[33632]: Detected new or changed devices on smithi016 2024-05-09T09:21:57.300 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:57 smithi170 ceph-mon[33632]: Adjusting osd_memory_target on smithi016 to 4273M 2024-05-09T09:21:57.300 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:57 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:57.300 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:57 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:57.300 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:57 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:57.300 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:57 smithi170 ceph-mon[33632]: osdmap e75: 11 total, 11 up, 11 in 2024-05-09T09:21:57.300 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:57 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:21:57.300 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:57 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:21:57.300 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:57 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:57.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:57 smithi012 ceph-mon[30140]: Detected new or changed devices on smithi016 2024-05-09T09:21:57.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:57 smithi012 ceph-mon[30140]: Adjusting osd_memory_target on smithi016 to 4273M 2024-05-09T09:21:57.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:57 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:57.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:57 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:57.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:57 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:57.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:57 smithi012 ceph-mon[30140]: osdmap e75: 11 total, 11 up, 11 in 2024-05-09T09:21:57.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:57 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:21:57.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:57 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:21:57.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:57 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:57.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:57 smithi016 ceph-mon[33335]: Detected new or changed devices on smithi016 2024-05-09T09:21:57.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:57 smithi016 ceph-mon[33335]: Adjusting osd_memory_target on smithi016 to 4273M 2024-05-09T09:21:57.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:57 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:57.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:57 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:57.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:57 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:21:57.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:57 smithi016 ceph-mon[33335]: osdmap e75: 11 total, 11 up, 11 in 2024-05-09T09:21:57.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:57 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:21:57.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:57 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:21:57.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:57 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:58.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:58 smithi016 ceph-mon[33335]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T09:21:58.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:58 smithi016 ceph-mon[33335]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:21:58.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:21:58 smithi016 ceph-mon[33335]: osdmap e76: 11 total, 11 up, 11 in 2024-05-09T09:21:58.305 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:58 smithi170 ceph-mon[33632]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T09:21:58.305 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:58 smithi170 ceph-mon[33632]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:21:58.305 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:21:58 smithi170 ceph-mon[33632]: osdmap e76: 11 total, 11 up, 11 in 2024-05-09T09:21:58.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:58 smithi012 ceph-mon[30140]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T09:21:58.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:58 smithi012 ceph-mon[30140]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:21:58.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:21:58 smithi012 ceph-mon[30140]: osdmap e76: 11 total, 11 up, 11 in 2024-05-09T09:22:00.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:00 smithi170 ceph-mon[33632]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T09:22:00.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:00 smithi170 ceph-mon[33632]: from='client.? 172.21.15.170:0/84722835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7bab10-d9ad-4917-9a11-a4b4bd363e70"}]: dispatch 2024-05-09T09:22:00.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:00 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7bab10-d9ad-4917-9a11-a4b4bd363e70"}]: dispatch 2024-05-09T09:22:00.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:00 smithi170 ceph-mon[33632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d7bab10-d9ad-4917-9a11-a4b4bd363e70"}]': finished 2024-05-09T09:22:00.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:00 smithi170 ceph-mon[33632]: osdmap e77: 12 total, 11 up, 12 in 2024-05-09T09:22:00.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:00 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:00.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:00 smithi012 ceph-mon[30140]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T09:22:00.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:00 smithi012 ceph-mon[30140]: from='client.? 172.21.15.170:0/84722835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7bab10-d9ad-4917-9a11-a4b4bd363e70"}]: dispatch 2024-05-09T09:22:00.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:00 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7bab10-d9ad-4917-9a11-a4b4bd363e70"}]: dispatch 2024-05-09T09:22:00.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:00 smithi012 ceph-mon[30140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d7bab10-d9ad-4917-9a11-a4b4bd363e70"}]': finished 2024-05-09T09:22:00.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:00 smithi012 ceph-mon[30140]: osdmap e77: 12 total, 11 up, 12 in 2024-05-09T09:22:00.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:00 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:00 smithi016 ceph-mon[33335]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T09:22:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:00 smithi016 ceph-mon[33335]: from='client.? 172.21.15.170:0/84722835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7bab10-d9ad-4917-9a11-a4b4bd363e70"}]: dispatch 2024-05-09T09:22:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:00 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7bab10-d9ad-4917-9a11-a4b4bd363e70"}]: dispatch 2024-05-09T09:22:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:00 smithi016 ceph-mon[33335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d7bab10-d9ad-4917-9a11-a4b4bd363e70"}]': finished 2024-05-09T09:22:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:00 smithi016 ceph-mon[33335]: osdmap e77: 12 total, 11 up, 12 in 2024-05-09T09:22:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:00 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:01.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:01 smithi012 ceph-mon[30140]: from='client.? 172.21.15.170:0/3024151701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:22:01.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:01 smithi170 ceph-mon[33632]: from='client.? 172.21.15.170:0/3024151701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:22:01.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:01 smithi016 ceph-mon[33335]: from='client.? 172.21.15.170:0/3024151701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:22:02.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:02 smithi012 ceph-mon[30140]: pgmap v245: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-09T09:22:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:02 smithi170 ceph-mon[33632]: pgmap v245: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-09T09:22:02.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:02 smithi016 ceph-mon[33335]: pgmap v245: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-09T09:22:04.138 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:04 smithi170 ceph-mon[33632]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:04.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:04 smithi012 ceph-mon[30140]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:04.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:04 smithi016 ceph-mon[33335]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:06.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:06 smithi012 ceph-mon[30140]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:06 smithi170 ceph-mon[33632]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:06.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:06 smithi016 ceph-mon[33335]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:07.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T09:22:07.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:07.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T09:22:07.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:07.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T09:22:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:08 smithi170 ceph-mon[33632]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:08 smithi170 ceph-mon[33632]: Deploying daemon osd.11 on smithi170 2024-05-09T09:22:08.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:08 smithi012 ceph-mon[30140]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:08.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:08 smithi012 ceph-mon[30140]: Deploying daemon osd.11 on smithi170 2024-05-09T09:22:08.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:08 smithi016 ceph-mon[33335]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:08.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:08 smithi016 ceph-mon[33335]: Deploying daemon osd.11 on smithi170 2024-05-09T09:22:10.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:10 smithi170 ceph-mon[33632]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:10.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:10 smithi012 ceph-mon[30140]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:10.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:10 smithi016 ceph-mon[33335]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:12.408 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:12 smithi170 ceph-mon[33632]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:12.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:12 smithi012 ceph-mon[30140]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:12.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:12 smithi016 ceph-mon[33335]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:13.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:13 smithi012 ceph-mon[30140]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:13.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:13.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:22:13.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:13 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:13.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:13 smithi170 ceph-mon[33632]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:13.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:13.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:22:13.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:13 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:14.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:13 smithi016 ceph-mon[33335]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:14.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:14.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:22:14.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:13 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:15.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:15 smithi170 ceph-mon[33632]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:15.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:15 smithi012 ceph-mon[30140]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:16.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:15 smithi016 ceph-mon[33335]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:16.758 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 11 on host 'smithi170' 2024-05-09T09:22:16.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:16.757+0000 7fbadb7fe700 1 -- 172.21.15.170:0/3022649260 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbaec0fea30 con 0x7fbad40a2c20 2024-05-09T09:22:16.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:16.760+0000 7fbaf09f2700 1 -- 172.21.15.170:0/3022649260 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbad40a2c20 msgr2=0x7fbad40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:16.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:16.760+0000 7fbaf09f2700 1 --2- 172.21.15.170:0/3022649260 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbad40a2c20 0x7fbad40a50e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbae000b2e0 tx=0x7fbae000b8f0 comp rx=0 tx=0).stop 2024-05-09T09:22:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:16.760+0000 7fbaf09f2700 1 -- 172.21.15.170:0/3022649260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec101280 msgr2=0x7fbaec198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:16.760+0000 7fbaf09f2700 1 --2- 172.21.15.170:0/3022649260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec101280 0x7fbaec198660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00c920 tx=0x7fbadc00cc30 comp rx=0 tx=0).stop 2024-05-09T09:22:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:16.760+0000 7fbaf09f2700 1 -- 172.21.15.170:0/3022649260 shutdown_connections 2024-05-09T09:22:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:16.760+0000 7fbaf09f2700 1 --2- 172.21.15.170:0/3022649260 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbad40a2c20 0x7fbad40a50e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:16.760+0000 7fbaf09f2700 1 --2- 172.21.15.170:0/3022649260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaec102e20 0x7fbaec19fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:16.760+0000 7fbaf09f2700 1 --2- 172.21.15.170:0/3022649260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec101280 0x7fbaec198660 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:16.760+0000 7fbaf09f2700 1 --2- 172.21.15.170:0/3022649260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaec102480 0x7fbaec198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:16.760+0000 7fbaf09f2700 1 -- 172.21.15.170:0/3022649260 >> 172.21.15.170:0/3022649260 conn(0x7fbaec0fc820 msgr2=0x7fbaec0fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:16.760+0000 7fbaf09f2700 1 -- 172.21.15.170:0/3022649260 shutdown_connections 2024-05-09T09:22:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:16.760+0000 7fbaf09f2700 1 -- 172.21.15.170:0/3022649260 wait complete. 2024-05-09T09:22:17.397 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:17 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:17 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:17 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:17.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:17 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:22:17.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:17 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:17 smithi170 ceph-mon[33632]: from='osd.11 [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T09:22:17.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:17 smithi170 ceph-mon[33632]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T09:22:17.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:17 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:17 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:17 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:22:17.431 DEBUG:teuthology.orchestra.run.smithi170:osd.11> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.11.service 2024-05-09T09:22:17.434 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-09T09:22:17.435 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-09T09:22:17.465 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:17 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.465 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:17 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.465 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:17 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:17.465 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:17 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:22:17.465 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:17 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.465 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:17 smithi012 ceph-mon[30140]: from='osd.11 [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T09:22:17.465 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:17 smithi012 ceph-mon[30140]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T09:22:17.465 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:17 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.466 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:17 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.466 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:17 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:22:17.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:17 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:17 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:17 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:17.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:17 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:22:17.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:17 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:17 smithi016 ceph-mon[33335]: from='osd.11 [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T09:22:17.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:17 smithi016 ceph-mon[33335]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T09:22:17.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:17 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:17 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:17.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:17 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:22:17.697 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:22:18.072 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:18 smithi012 ceph-mon[30140]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:18.072 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:18 smithi012 ceph-mon[30140]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T09:22:18.072 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:18 smithi012 ceph-mon[30140]: osdmap e78: 12 total, 11 up, 12 in 2024-05-09T09:22:18.072 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:18 smithi012 ceph-mon[30140]: from='osd.11 [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:22:18.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:18 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:18.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:18 smithi012 ceph-mon[30140]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:22:18.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:18 smithi016 ceph-mon[33335]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:18.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:18 smithi016 ceph-mon[33335]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T09:22:18.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:18 smithi016 ceph-mon[33335]: osdmap e78: 12 total, 11 up, 12 in 2024-05-09T09:22:18.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:18 smithi016 ceph-mon[33335]: from='osd.11 [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:22:18.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:18 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:18.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:18 smithi016 ceph-mon[33335]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:22:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:18 smithi170 ceph-mon[33632]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:18 smithi170 ceph-mon[33632]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T09:22:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:18 smithi170 ceph-mon[33632]: osdmap e78: 12 total, 11 up, 12 in 2024-05-09T09:22:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:18 smithi170 ceph-mon[33632]: from='osd.11 [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:22:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:18 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:18 smithi170 ceph-mon[33632]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-05-09T09:22:18.864 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:22:18 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:22:18.556+0000 7f23edf63700 -1 osd.11 0 waiting for initial osdmap 2024-05-09T09:22:18.864 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:22:18 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:22:18.571+0000 7f23e5d4e700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:22:19.156 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:19 smithi012 ceph-mon[30140]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-05-09T09:22:19.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:19 smithi012 ceph-mon[30140]: osdmap e79: 12 total, 11 up, 12 in 2024-05-09T09:22:19.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:19 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:19.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:19 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:19.157 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:19 smithi012 ceph-mon[30140]: from='osd.11 ' entity='osd.11' 2024-05-09T09:22:19.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:19 smithi170 ceph-mon[33632]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-05-09T09:22:19.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:19 smithi170 ceph-mon[33632]: osdmap e79: 12 total, 11 up, 12 in 2024-05-09T09:22:19.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:19 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:19.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:19 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:19.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:19 smithi170 ceph-mon[33632]: from='osd.11 ' entity='osd.11' 2024-05-09T09:22:19.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.403+0000 7f39da895700 1 -- 172.21.15.12:0/981594344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d4103590 msgr2=0x7f39d41039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:19.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.403+0000 7f39da895700 1 --2- 172.21.15.12:0/981594344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d4103590 0x7f39d41039f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0099c0 tx=0x7f39bc009cd0 comp rx=0 tx=0).stop 2024-05-09T09:22:19.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.403+0000 7f39da895700 1 -- 172.21.15.12:0/981594344 shutdown_connections 2024-05-09T09:22:19.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.403+0000 7f39da895700 1 --2- 172.21.15.12:0/981594344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d4103f30 0x7f39d410a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:19.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.403+0000 7f39da895700 1 --2- 172.21.15.12:0/981594344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d4102390 0x7f39d41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:19.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.403+0000 7f39da895700 1 --2- 172.21.15.12:0/981594344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d4103590 0x7f39d41039f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:19.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.403+0000 7f39da895700 1 -- 172.21.15.12:0/981594344 >> 172.21.15.12:0/981594344 conn(0x7f39d40fd930 msgr2=0x7f39d40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:19.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.403+0000 7f39da895700 1 -- 172.21.15.12:0/981594344 shutdown_connections 2024-05-09T09:22:19.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.403+0000 7f39da895700 1 -- 172.21.15.12:0/981594344 wait complete. 2024-05-09T09:22:19.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.404+0000 7f39da895700 1 Processor -- start 2024-05-09T09:22:19.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.404+0000 7f39da895700 1 -- start start 2024-05-09T09:22:19.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.404+0000 7f39da895700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d4102390 0x7f39d40760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:19.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.404+0000 7f39da895700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d4103590 0x7f39d4074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:19.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.404+0000 7f39da895700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d4103f30 0x7f39d4074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:19.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.404+0000 7f39da895700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f39d410df10 con 0x7f39d4103f30 2024-05-09T09:22:19.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.404+0000 7f39da895700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39d410dd90 con 0x7f39d4103590 2024-05-09T09:22:19.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.404+0000 7f39da895700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39d410e090 con 0x7f39d4102390 2024-05-09T09:22:19.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.404+0000 7f39d8e32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d4103f30 0x7f39d4074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:19.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.404+0000 7f39d8e32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d4103f30 0x7f39d4074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58418/0 (socket says 172.21.15.12:58418) 2024-05-09T09:22:19.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.404+0000 7f39d8e32700 1 -- 172.21.15.12:0/1589294878 learned_addr learned my addr 172.21.15.12:0/1589294878 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.404+0000 7f39cbfff700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d4103590 0x7f39d4074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.405+0000 7f39d3fff700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d4102390 0x7f39d40760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.405+0000 7f39d8e32700 1 -- 172.21.15.12:0/1589294878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d4102390 msgr2=0x7f39d40760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.405+0000 7f39d8e32700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d4102390 0x7f39d40760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.405+0000 7f39d8e32700 1 -- 172.21.15.12:0/1589294878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d4103590 msgr2=0x7f39d4074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.405+0000 7f39d8e32700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d4103590 0x7f39d4074710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.405+0000 7f39d8e32700 1 -- 172.21.15.12:0/1589294878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d4075500 con 0x7f39d4103f30 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.405+0000 7f39d3fff700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d4102390 0x7f39d40760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.405+0000 7f39d8e32700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d4103f30 0x7f39d4074c50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39cc007ee0 tx=0x7f39cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.405+0000 7f39d1ffb700 1 -- 172.21.15.12:0/1589294878 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc00dbd0 con 0x7f39d4103f30 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.405+0000 7f39d1ffb700 1 -- 172.21.15.12:0/1589294878 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f39cc00dd70 con 0x7f39d4103f30 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.405+0000 7f39da895700 1 -- 172.21.15.12:0/1589294878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d41a26c0 con 0x7f39d4103f30 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.405+0000 7f39d1ffb700 1 -- 172.21.15.12:0/1589294878 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0059b0 con 0x7f39d4103f30 2024-05-09T09:22:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.405+0000 7f39da895700 1 -- 172.21.15.12:0/1589294878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d41a2bd0 con 0x7f39d4103f30 2024-05-09T09:22:19.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.406+0000 7f39da895700 1 -- 172.21.15.12:0/1589294878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d4069010 con 0x7f39d4103f30 2024-05-09T09:22:19.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.407+0000 7f39d1ffb700 1 -- 172.21.15.12:0/1589294878 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f39cc01a020 con 0x7f39d4103f30 2024-05-09T09:22:19.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.408+0000 7f39d1ffb700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f39c00a2ca0 0x7f39c00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:19.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.408+0000 7f39d3fff700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f39c00a2ca0 0x7f39c00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:19.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.409+0000 7f39d1ffb700 1 -- 172.21.15.12:0/1589294878 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7f39cc0ce1f0 con 0x7f39d4103f30 2024-05-09T09:22:19.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.409+0000 7f39d3fff700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f39c00a2ca0 0x7f39c00a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f39d41033f0 tx=0x7f39c400a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.413+0000 7f39d1ffb700 1 -- 172.21.15.12:0/1589294878 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f39cc010070 con 0x7f39d4103f30 2024-05-09T09:22:19.506 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:19 smithi016 ceph-mon[33335]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-05-09T09:22:19.506 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:19 smithi016 ceph-mon[33335]: osdmap e79: 12 total, 11 up, 12 in 2024-05-09T09:22:19.506 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:19 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:19.506 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:19 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:19.506 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:19 smithi016 ceph-mon[33335]: from='osd.11 ' entity='osd.11' 2024-05-09T09:22:19.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.621+0000 7f39da895700 1 -- 172.21.15.12:0/1589294878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f39d4062f30 con 0x7f39d4103f30 2024-05-09T09:22:19.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.622+0000 7f39d1ffb700 1 -- 172.21.15.12:0/1589294878 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f39cc09b670 con 0x7f39d4103f30 2024-05-09T09:22:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.624+0000 7f39da895700 1 -- 172.21.15.12:0/1589294878 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f39c00a2ca0 msgr2=0x7f39c00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.624+0000 7f39da895700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f39c00a2ca0 0x7f39c00a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f39d41033f0 tx=0x7f39c400a680 comp rx=0 tx=0).stop 2024-05-09T09:22:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.624+0000 7f39da895700 1 -- 172.21.15.12:0/1589294878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d4103f30 msgr2=0x7f39d4074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.624+0000 7f39da895700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d4103f30 0x7f39d4074c50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39cc007ee0 tx=0x7f39cc00d3b0 comp rx=0 tx=0).stop 2024-05-09T09:22:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.624+0000 7f39da895700 1 -- 172.21.15.12:0/1589294878 shutdown_connections 2024-05-09T09:22:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.624+0000 7f39da895700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f39c00a2ca0 0x7f39c00a5160 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.624+0000 7f39da895700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d4103f30 0x7f39d4074c50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.624+0000 7f39da895700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d4102390 0x7f39d40760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.624+0000 7f39da895700 1 --2- 172.21.15.12:0/1589294878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d4103590 0x7f39d4074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.624+0000 7f39da895700 1 -- 172.21.15.12:0/1589294878 >> 172.21.15.12:0/1589294878 conn(0x7f39d40fd930 msgr2=0x7f39d41067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.624+0000 7f39da895700 1 -- 172.21.15.12:0/1589294878 shutdown_connections 2024-05-09T09:22:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:19.624+0000 7f39da895700 1 -- 172.21.15.12:0/1589294878 wait complete. 2024-05-09T09:22:20.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:20 smithi012 ceph-mon[30140]: purged_snaps scrub starts 2024-05-09T09:22:20.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:20 smithi012 ceph-mon[30140]: purged_snaps scrub ok 2024-05-09T09:22:20.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:20 smithi012 ceph-mon[30140]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:20.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:20 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:20.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:20 smithi012 ceph-mon[30140]: osd.11 [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659] boot 2024-05-09T09:22:20.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:20 smithi012 ceph-mon[30140]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T09:22:20.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:20 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:20.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:20 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1589294878' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T09:22:20.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:20 smithi170 ceph-mon[33632]: purged_snaps scrub starts 2024-05-09T09:22:20.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:20 smithi170 ceph-mon[33632]: purged_snaps scrub ok 2024-05-09T09:22:20.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:20 smithi170 ceph-mon[33632]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:20.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:20 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:20.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:20 smithi170 ceph-mon[33632]: osd.11 [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659] boot 2024-05-09T09:22:20.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:20 smithi170 ceph-mon[33632]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T09:22:20.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:20 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:20.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:20 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1589294878' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T09:22:20.421 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1715246539,"num_in_osds":12,"osd_in_since":1715246519,"num_remapped_pgs":0} 2024-05-09T09:22:20.421 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-09T09:22:20.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:20 smithi016 ceph-mon[33335]: purged_snaps scrub starts 2024-05-09T09:22:20.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:20 smithi016 ceph-mon[33335]: purged_snaps scrub ok 2024-05-09T09:22:20.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:20 smithi016 ceph-mon[33335]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:22:20.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:20 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:20.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:20 smithi016 ceph-mon[33335]: osd.11 [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659] boot 2024-05-09T09:22:20.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:20 smithi016 ceph-mon[33335]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T09:22:20.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:20 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:22:20.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:20 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1589294878' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T09:22:20.724 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:22:21.962 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:21.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: Detected new or changed devices on smithi170 2024-05-09T09:22:21.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:21.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:21.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:22:21.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:22:21.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:22:21.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:22:21.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: Adjusting osd_memory_target on smithi170 to 4272M 2024-05-09T09:22:21.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:21.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:21.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:22:21.964 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:21.964 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:21 smithi012 ceph-mon[30140]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T09:22:22.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:22.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: Detected new or changed devices on smithi170 2024-05-09T09:22:22.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:22.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:22.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:22:22.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:22:22.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:22:22.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:22:22.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: Adjusting osd_memory_target on smithi170 to 4272M 2024-05-09T09:22:22.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:22.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:22.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:22:22.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:22.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:21 smithi170 ceph-mon[33632]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T09:22:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: Detected new or changed devices on smithi170 2024-05-09T09:22:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:22:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:22:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:22:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:22:22.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: Adjusting osd_memory_target on smithi170 to 4272M 2024-05-09T09:22:22.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:22.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:22.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:22:22.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:22.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:21 smithi016 ceph-mon[33335]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T09:22:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.498+0000 7fab07675700 1 -- 172.21.15.12:0/3283037824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab00107680 msgr2=0x7fab0010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.498+0000 7fab07675700 1 --2- 172.21.15.12:0/3283037824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab00107680 0x7fab0010f250 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faaf800b0a0 tx=0x7faaf800b3b0 comp rx=0 tx=0).stop 2024-05-09T09:22:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.498+0000 7fab07675700 1 -- 172.21.15.12:0/3283037824 shutdown_connections 2024-05-09T09:22:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.498+0000 7fab07675700 1 --2- 172.21.15.12:0/3283037824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab00107680 0x7fab0010f250 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.498+0000 7fab07675700 1 --2- 172.21.15.12:0/3283037824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab00102600 0x7fab001029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.498+0000 7fab07675700 1 --2- 172.21.15.12:0/3283037824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00102fb0 0x7fab00106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.498+0000 7fab07675700 1 -- 172.21.15.12:0/3283037824 >> 172.21.15.12:0/3283037824 conn(0x7fab00079490 msgr2=0x7fab000798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.499+0000 7fab07675700 1 -- 172.21.15.12:0/3283037824 shutdown_connections 2024-05-09T09:22:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.499+0000 7fab07675700 1 -- 172.21.15.12:0/3283037824 wait complete. 2024-05-09T09:22:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.499+0000 7fab07675700 1 Processor -- start 2024-05-09T09:22:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.499+0000 7fab07675700 1 -- start start 2024-05-09T09:22:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.499+0000 7fab07675700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab00102600 0x7fab0019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.499+0000 7fab07675700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00102fb0 0x7fab0019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.499+0000 7fab07675700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab00107680 0x7fab00198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.499+0000 7fab07675700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fab00112670 con 0x7fab00102600 2024-05-09T09:22:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab07675700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab001124f0 con 0x7fab00102fb0 2024-05-09T09:22:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab07675700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab001127f0 con 0x7fab00107680 2024-05-09T09:22:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab05411700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab00102600 0x7fab0019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab05411700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab00102600 0x7fab0019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44812/0 (socket says 172.21.15.12:44812) 2024-05-09T09:22:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab05411700 1 -- 172.21.15.12:0/758916667 learned_addr learned my addr 172.21.15.12:0/758916667 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab05c12700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab00107680 0x7fab00198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab04c10700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00102fb0 0x7fab0019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab05411700 1 -- 172.21.15.12:0/758916667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab00107680 msgr2=0x7fab00198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab05411700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab00107680 0x7fab00198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab05411700 1 -- 172.21.15.12:0/758916667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00102fb0 msgr2=0x7fab0019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab05411700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00102fb0 0x7fab0019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab05411700 1 -- 172.21.15.12:0/758916667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab00199070 con 0x7fab00102600 2024-05-09T09:22:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab05c12700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab00107680 0x7fab00198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab04c10700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00102fb0 0x7fab0019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab05411700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab00102600 0x7fab0019e700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faafc00d980 tx=0x7faafc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7faaf67fc700 1 -- 172.21.15.12:0/758916667 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc016070 con 0x7fab00102600 2024-05-09T09:22:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab07675700 1 -- 172.21.15.12:0/758916667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab00199360 con 0x7fab00102600 2024-05-09T09:22:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7faaf67fc700 1 -- 172.21.15.12:0/758916667 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7faafc0044f0 con 0x7fab00102600 2024-05-09T09:22:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7faaf67fc700 1 -- 172.21.15.12:0/758916667 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc004cd0 con 0x7fab00102600 2024-05-09T09:22:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.500+0000 7fab07675700 1 -- 172.21.15.12:0/758916667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab001a55d0 con 0x7fab00102600 2024-05-09T09:22:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.501+0000 7fab07675700 1 -- 172.21.15.12:0/758916667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab00069010 con 0x7fab00102600 2024-05-09T09:22:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.503+0000 7faaf67fc700 1 -- 172.21.15.12:0/758916667 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faafc00bc80 con 0x7fab00102600 2024-05-09T09:22:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.504+0000 7faaf67fc700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7faaec0a2c20 0x7faaec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.504+0000 7fab04c10700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7faaec0a2c20 0x7faaec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.504+0000 7faaf67fc700 1 -- 172.21.15.12:0/758916667 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7faafc0cef50 con 0x7fab00102600 2024-05-09T09:22:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.504+0000 7fab04c10700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7faaec0a2c20 0x7faaec0a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab00108a90 tx=0x7faaf0009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:22.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.507+0000 7faaf67fc700 1 -- 172.21.15.12:0/758916667 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faafc01f020 con 0x7fab00102600 2024-05-09T09:22:22.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.713+0000 7fab07675700 1 -- 172.21.15.12:0/758916667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab00199df0 con 0x7fab00102600 2024-05-09T09:22:22.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.714+0000 7faaf67fc700 1 -- 172.21.15.12:0/758916667 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18433 (secure 0 0 0) 0x7faafc09c350 con 0x7fab00102600 2024-05-09T09:22:22.715 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:22.716 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:22:21.034614+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:22:22.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.717+0000 7fab07675700 1 -- 172.21.15.12:0/758916667 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7faaec0a2c20 msgr2=0x7faaec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.717+0000 7fab07675700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7faaec0a2c20 0x7faaec0a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab00108a90 tx=0x7faaf0009550 comp rx=0 tx=0).stop 2024-05-09T09:22:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.717+0000 7fab07675700 1 -- 172.21.15.12:0/758916667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab00102600 msgr2=0x7fab0019e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.717+0000 7fab07675700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab00102600 0x7fab0019e700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faafc00d980 tx=0x7faafc00dd40 comp rx=0 tx=0).stop 2024-05-09T09:22:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.717+0000 7fab07675700 1 -- 172.21.15.12:0/758916667 shutdown_connections 2024-05-09T09:22:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.717+0000 7fab07675700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7faaec0a2c20 0x7faaec0a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.717+0000 7fab07675700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab00107680 0x7fab00198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.717+0000 7fab07675700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab00102600 0x7fab0019e700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.717+0000 7fab07675700 1 --2- 172.21.15.12:0/758916667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00102fb0 0x7fab0019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.717+0000 7fab07675700 1 -- 172.21.15.12:0/758916667 >> 172.21.15.12:0/758916667 conn(0x7fab00079490 msgr2=0x7fab00100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.717+0000 7fab07675700 1 -- 172.21.15.12:0/758916667 shutdown_connections 2024-05-09T09:22:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:22.717+0000 7fab07675700 1 -- 172.21.15.12:0/758916667 wait complete. 2024-05-09T09:22:22.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:22 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/758916667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:22:23.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:22 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/758916667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:22:23.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:22 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/758916667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:22:23.440 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T09:22:23.440 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-09T09:22:23.698 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:22:24.032 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:23 smithi012 ceph-mon[30140]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:24.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:23 smithi170 ceph-mon[33632]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:24.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:23 smithi016 ceph-mon[33335]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:25.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.503+0000 7f7a56e5f700 1 -- 172.21.15.12:0/1047993418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a50102f80 msgr2=0x7f7a50106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:25.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.503+0000 7f7a56e5f700 1 --2- 172.21.15.12:0/1047993418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a50102f80 0x7f7a50106f80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009990 tx=0x7f7a44009ca0 comp rx=0 tx=0).stop 2024-05-09T09:22:25.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.503+0000 7f7a56e5f700 1 -- 172.21.15.12:0/1047993418 shutdown_connections 2024-05-09T09:22:25.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.503+0000 7f7a56e5f700 1 --2- 172.21.15.12:0/1047993418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a50107660 0x7f7a5010f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:25.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.503+0000 7f7a56e5f700 1 --2- 172.21.15.12:0/1047993418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a501025d0 0x7f7a501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:25.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.503+0000 7f7a56e5f700 1 --2- 172.21.15.12:0/1047993418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a50102f80 0x7f7a50106f80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:25.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.503+0000 7f7a56e5f700 1 -- 172.21.15.12:0/1047993418 >> 172.21.15.12:0/1047993418 conn(0x7f7a50079480 msgr2=0x7f7a50079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:25.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.503+0000 7f7a56e5f700 1 -- 172.21.15.12:0/1047993418 shutdown_connections 2024-05-09T09:22:25.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.503+0000 7f7a56e5f700 1 -- 172.21.15.12:0/1047993418 wait complete. 2024-05-09T09:22:25.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a56e5f700 1 Processor -- start 2024-05-09T09:22:25.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a56e5f700 1 -- start start 2024-05-09T09:22:25.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a56e5f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a501025d0 0x7f7a5019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:25.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a56e5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a50102f80 0x7f7a5019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:25.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a56e5f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a50107660 0x7f7a50198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:25.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a56e5f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7a50112670 con 0x7f7a50107660 2024-05-09T09:22:25.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a56e5f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501124f0 con 0x7f7a50102f80 2024-05-09T09:22:25.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a56e5f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501127f0 con 0x7f7a501025d0 2024-05-09T09:22:25.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a553fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a50107660 0x7f7a50198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:25.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a553fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a50107660 0x7f7a50198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44836/0 (socket says 172.21.15.12:44836) 2024-05-09T09:22:25.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a553fc700 1 -- 172.21.15.12:0/3624285879 learned_addr learned my addr 172.21.15.12:0/3624285879 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:25.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a54bfb700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a501025d0 0x7f7a5019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:25.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a4ffff700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a50102f80 0x7f7a5019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:25.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a553fc700 1 -- 172.21.15.12:0/3624285879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a501025d0 msgr2=0x7f7a5019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:25.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a553fc700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a501025d0 0x7f7a5019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:25.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a553fc700 1 -- 172.21.15.12:0/3624285879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a50102f80 msgr2=0x7f7a5019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:25.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a553fc700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a50102f80 0x7f7a5019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:25.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.504+0000 7f7a553fc700 1 -- 172.21.15.12:0/3624285879 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a501990f0 con 0x7f7a50107660 2024-05-09T09:22:25.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.505+0000 7f7a553fc700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a50107660 0x7f7a50198990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800ada0 tx=0x7f7a4800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:25.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.505+0000 7f7a4dffb700 1 -- 172.21.15.12:0/3624285879 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48017020 con 0x7f7a50107660 2024-05-09T09:22:25.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.505+0000 7f7a4dffb700 1 -- 172.21.15.12:0/3624285879 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7a4800ce50 con 0x7f7a50107660 2024-05-09T09:22:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.505+0000 7f7a56e5f700 1 -- 172.21.15.12:0/3624285879 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a501993e0 con 0x7f7a50107660 2024-05-09T09:22:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.505+0000 7f7a4dffb700 1 -- 172.21.15.12:0/3624285879 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48005380 con 0x7f7a50107660 2024-05-09T09:22:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.505+0000 7f7a56e5f700 1 -- 172.21.15.12:0/3624285879 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a501a56a0 con 0x7f7a50107660 2024-05-09T09:22:25.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.506+0000 7f7a56e5f700 1 -- 172.21.15.12:0/3624285879 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a50103a60 con 0x7f7a50107660 2024-05-09T09:22:25.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.507+0000 7f7a4dffb700 1 -- 172.21.15.12:0/3624285879 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7a4800c980 con 0x7f7a50107660 2024-05-09T09:22:25.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.508+0000 7f7a4dffb700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7a380a2ca0 0x7f7a380a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:25.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.508+0000 7f7a54bfb700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7a380a2ca0 0x7f7a380a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:25.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.508+0000 7f7a4dffb700 1 -- 172.21.15.12:0/3624285879 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7a480ce2e0 con 0x7f7a50107660 2024-05-09T09:22:25.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.508+0000 7f7a54bfb700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7a380a2ca0 0x7f7a380a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7a50199ad0 tx=0x7f7a40009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:25.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.512+0000 7f7a4dffb700 1 -- 172.21.15.12:0/3624285879 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7a48021020 con 0x7f7a50107660 2024-05-09T09:22:25.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.716+0000 7f7a56e5f700 1 -- 172.21.15.12:0/3624285879 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7a5002db80 con 0x7f7a50107660 2024-05-09T09:22:25.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.717+0000 7f7a4dffb700 1 -- 172.21.15.12:0/3624285879 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f7a4800cc70 con 0x7f7a50107660 2024-05-09T09:22:25.717 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:22:25.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.719+0000 7f7a56e5f700 1 -- 172.21.15.12:0/3624285879 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7a380a2ca0 msgr2=0x7f7a380a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:25.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.719+0000 7f7a56e5f700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7a380a2ca0 0x7f7a380a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7a50199ad0 tx=0x7f7a40009190 comp rx=0 tx=0).stop 2024-05-09T09:22:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.719+0000 7f7a56e5f700 1 -- 172.21.15.12:0/3624285879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a50107660 msgr2=0x7f7a50198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.719+0000 7f7a56e5f700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a50107660 0x7f7a50198990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800ada0 tx=0x7f7a4800c330 comp rx=0 tx=0).stop 2024-05-09T09:22:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.719+0000 7f7a56e5f700 1 -- 172.21.15.12:0/3624285879 shutdown_connections 2024-05-09T09:22:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.719+0000 7f7a56e5f700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7a380a2ca0 0x7f7a380a5160 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.719+0000 7f7a56e5f700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a50107660 0x7f7a50198990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.719+0000 7f7a56e5f700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a501025d0 0x7f7a5019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.719+0000 7f7a56e5f700 1 --2- 172.21.15.12:0/3624285879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a50102f80 0x7f7a5019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.719+0000 7f7a56e5f700 1 -- 172.21.15.12:0/3624285879 >> 172.21.15.12:0/3624285879 conn(0x7f7a50079480 msgr2=0x7f7a50100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.719+0000 7f7a56e5f700 1 -- 172.21.15.12:0/3624285879 shutdown_connections 2024-05-09T09:22:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:25.719+0000 7f7a56e5f700 1 -- 172.21.15.12:0/3624285879 wait complete. 2024-05-09T09:22:25.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:25 smithi012 ceph-mon[30140]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:25.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:25 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3624285879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:22:26.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:25 smithi170 ceph-mon[33632]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:26.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:25 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3624285879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:22:26.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:25 smithi016 ceph-mon[33335]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:26.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:25 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3624285879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:22:26.516 INFO:tasks.cephadm:Adding mds.a on smithi012 2024-05-09T09:22:26.516 INFO:tasks.cephadm:Adding mds.d on smithi012 2024-05-09T09:22:26.516 INFO:tasks.cephadm:Adding mds.g on smithi012 2024-05-09T09:22:26.516 INFO:tasks.cephadm:Adding mds.j on smithi012 2024-05-09T09:22:26.516 INFO:tasks.cephadm:Adding mds.b on smithi016 2024-05-09T09:22:26.516 INFO:tasks.cephadm:Adding mds.e on smithi016 2024-05-09T09:22:26.516 INFO:tasks.cephadm:Adding mds.h on smithi016 2024-05-09T09:22:26.517 INFO:tasks.cephadm:Adding mds.k on smithi016 2024-05-09T09:22:26.517 INFO:tasks.cephadm:Adding mds.c on smithi170 2024-05-09T09:22:26.517 INFO:tasks.cephadm:Adding mds.f on smithi170 2024-05-09T09:22:26.517 INFO:tasks.cephadm:Adding mds.i on smithi170 2024-05-09T09:22:26.517 INFO:tasks.cephadm:Adding mds.l on smithi170 2024-05-09T09:22:26.517 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l' 2024-05-09T09:22:26.761 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:22:28.052 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:27 smithi170 ceph-mon[33632]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:28.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:27 smithi012 ceph-mon[30140]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:28.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:27 smithi016 ceph-mon[33335]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:28.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.406+0000 7f674b303700 1 -- 172.21.15.170:0/2474614348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744102e20 msgr2=0x7f67441096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:28.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.406+0000 7f674b303700 1 --2- 172.21.15.170:0/2474614348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744102e20 0x7f67441096d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f673c00b0a0 tx=0x7f673c00b3b0 comp rx=0 tx=0).stop 2024-05-09T09:22:28.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.406+0000 7f674b303700 1 -- 172.21.15.170:0/2474614348 shutdown_connections 2024-05-09T09:22:28.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.406+0000 7f674b303700 1 --2- 172.21.15.170:0/2474614348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744102e20 0x7f67441096d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:28.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.406+0000 7f674b303700 1 --2- 172.21.15.170:0/2474614348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6744101280 0x7f67441016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.406+0000 7f674b303700 1 --2- 172.21.15.170:0/2474614348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6744102480 0x7f67441028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.406+0000 7f674b303700 1 -- 172.21.15.170:0/2474614348 >> 172.21.15.170:0/2474614348 conn(0x7f67440fc820 msgr2=0x7f67440fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.406+0000 7f674b303700 1 -- 172.21.15.170:0/2474614348 shutdown_connections 2024-05-09T09:22:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.406+0000 7f674b303700 1 -- 172.21.15.170:0/2474614348 wait complete. 2024-05-09T09:22:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f674b303700 1 Processor -- start 2024-05-09T09:22:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f674b303700 1 -- start start 2024-05-09T09:22:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f674b303700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6744101280 0x7f674419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f674b303700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6744102480 0x7f674419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f674b303700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744102e20 0x7f67441a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f674b303700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f674410cf20 con 0x7f6744101280 2024-05-09T09:22:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f674b303700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f674410cda0 con 0x7f6744102480 2024-05-09T09:22:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f674b303700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f674410d0a0 con 0x7f6744102e20 2024-05-09T09:22:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f67498a0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744102e20 0x7f67441a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f67498a0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744102e20 0x7f67441a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44784/0 (socket says 172.21.15.170:44784) 2024-05-09T09:22:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f67498a0700 1 -- 172.21.15.170:0/848067600 learned_addr learned my addr 172.21.15.170:0/848067600 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-09T09:22:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f674889e700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6744102480 0x7f674419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f674909f700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6744101280 0x7f674419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f67498a0700 1 -- 172.21.15.170:0/848067600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6744102480 msgr2=0x7f674419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f67498a0700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6744102480 0x7f674419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f67498a0700 1 -- 172.21.15.170:0/848067600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6744101280 msgr2=0x7f674419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f67498a0700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6744101280 0x7f674419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f67498a0700 1 -- 172.21.15.170:0/848067600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67441a2460 con 0x7f6744102e20 2024-05-09T09:22:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.407+0000 7f674909f700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6744101280 0x7f674419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.408+0000 7f67498a0700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744102e20 0x7f67441a1dc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f673c00b070 tx=0x7f673c007d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.408+0000 7f673a7fc700 1 -- 172.21.15.170:0/848067600 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c003e10 con 0x7f6744102e20 2024-05-09T09:22:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.408+0000 7f673a7fc700 1 -- 172.21.15.170:0/848067600 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f673c01d940 con 0x7f6744102e20 2024-05-09T09:22:28.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.408+0000 7f674b303700 1 -- 172.21.15.170:0/848067600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67441a26f0 con 0x7f6744102e20 2024-05-09T09:22:28.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.408+0000 7f673a7fc700 1 -- 172.21.15.170:0/848067600 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c005680 con 0x7f6744102e20 2024-05-09T09:22:28.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.408+0000 7f674b303700 1 -- 172.21.15.170:0/848067600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67441a2ba0 con 0x7f6744102e20 2024-05-09T09:22:28.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.409+0000 7f674b303700 1 -- 172.21.15.170:0/848067600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6744069010 con 0x7f6744102e20 2024-05-09T09:22:28.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.410+0000 7f673a7fc700 1 -- 172.21.15.170:0/848067600 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f673c005820 con 0x7f6744102e20 2024-05-09T09:22:28.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.411+0000 7f673a7fc700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67300a2ca0 0x7f67300a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:28.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.411+0000 7f673a7fc700 1 -- 172.21.15.170:0/848067600 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f673c00f020 con 0x7f6744102e20 2024-05-09T09:22:28.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.411+0000 7f674909f700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67300a2ca0 0x7f67300a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:28.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.411+0000 7f674909f700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67300a2ca0 0x7f67300a5160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f67441022e0 tx=0x7f6740008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:28.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.414+0000 7f673a7fc700 1 -- 172.21.15.170:0/848067600 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f673c02b020 con 0x7f6744102e20 2024-05-09T09:22:28.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.618+0000 7f674b303700 1 -- 172.21.15.170:0/848067600 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}) -- 0x7f67440fea30 con 0x7f67300a2ca0 2024-05-09T09:22:28.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.627+0000 7f673a7fc700 1 -- 172.21.15.170:0/848067600 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f67440fea30 con 0x7f67300a2ca0 2024-05-09T09:22:28.626 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mds.all update... 2024-05-09T09:22:28.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.629+0000 7f674b303700 1 -- 172.21.15.170:0/848067600 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67300a2ca0 msgr2=0x7f67300a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:28.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.629+0000 7f674b303700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67300a2ca0 0x7f67300a5160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f67441022e0 tx=0x7f6740008040 comp rx=0 tx=0).stop 2024-05-09T09:22:28.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.629+0000 7f674b303700 1 -- 172.21.15.170:0/848067600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744102e20 msgr2=0x7f67441a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:28.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.629+0000 7f674b303700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744102e20 0x7f67441a1dc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f673c00b070 tx=0x7f673c007d10 comp rx=0 tx=0).stop 2024-05-09T09:22:28.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.629+0000 7f674b303700 1 -- 172.21.15.170:0/848067600 shutdown_connections 2024-05-09T09:22:28.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.629+0000 7f674b303700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67300a2ca0 0x7f67300a5160 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:28.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.629+0000 7f674b303700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744102e20 0x7f67441a1dc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:28.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.629+0000 7f674b303700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6744101280 0x7f674419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:28.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.629+0000 7f674b303700 1 --2- 172.21.15.170:0/848067600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6744102480 0x7f674419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:28.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.629+0000 7f674b303700 1 -- 172.21.15.170:0/848067600 >> 172.21.15.170:0/848067600 conn(0x7f67440fc820 msgr2=0x7f67440fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:28.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.629+0000 7f674b303700 1 -- 172.21.15.170:0/848067600 shutdown_connections 2024-05-09T09:22:28.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-09T09:22:28.629+0000 7f674b303700 1 -- 172.21.15.170:0/848067600 wait complete. 2024-05-09T09:22:29.234 DEBUG:teuthology.orchestra.run.smithi012:mds.a> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.a.service 2024-05-09T09:22:29.237 DEBUG:teuthology.orchestra.run.smithi012:mds.d> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.d.service 2024-05-09T09:22:29.282 DEBUG:teuthology.orchestra.run.smithi012:mds.g> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.g.service 2024-05-09T09:22:29.324 DEBUG:teuthology.orchestra.run.smithi012:mds.j> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.j.service 2024-05-09T09:22:29.367 DEBUG:teuthology.orchestra.run.smithi016:mds.b> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.b.service 2024-05-09T09:22:29.370 DEBUG:teuthology.orchestra.run.smithi016:mds.e> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.e.service 2024-05-09T09:22:29.413 DEBUG:teuthology.orchestra.run.smithi016:mds.h> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.h.service 2024-05-09T09:22:29.457 DEBUG:teuthology.orchestra.run.smithi016:mds.k> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.k.service 2024-05-09T09:22:29.500 DEBUG:teuthology.orchestra.run.smithi170:mds.c> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.c.service 2024-05-09T09:22:29.502 DEBUG:teuthology.orchestra.run.smithi170:mds.f> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.f.service 2024-05-09T09:22:29.545 DEBUG:teuthology.orchestra.run.smithi170:mds.i> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.i.service 2024-05-09T09:22:29.589 DEBUG:teuthology.orchestra.run.smithi170:mds.l> sudo journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.l.service 2024-05-09T09:22:29.632 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-09T09:22:29.633 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:22:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:29 smithi016 ceph-mon[33335]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:29 smithi016 ceph-mon[33335]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:22:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:29 smithi016 ceph-mon[33335]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-05-09T09:22:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:22:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:22:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:29.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:29 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:29.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:29 smithi016 ceph-mon[33335]: Deploying daemon mds.l on smithi170 2024-05-09T09:22:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.754+0000 7f4c21cd4640 1 -- 172.21.15.12:0/182050404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c1c100770 msgr2=0x7f4c1c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.754+0000 7f4c21cd4640 1 --2- 172.21.15.12:0/182050404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c1c100770 0x7f4c1c100b90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009900 tx=0x7f4c0c02a410 comp rx=0 tx=0).stop 2024-05-09T09:22:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.754+0000 7f4c21cd4640 1 -- 172.21.15.12:0/182050404 shutdown_connections 2024-05-09T09:22:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.754+0000 7f4c21cd4640 1 --2- 172.21.15.12:0/182050404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c1c102320 0x7f4c1c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.754+0000 7f4c21cd4640 1 --2- 172.21.15.12:0/182050404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c101960 0x7f4c1c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.754+0000 7f4c21cd4640 1 --2- 172.21.15.12:0/182050404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c1c100770 0x7f4c1c100b90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.754+0000 7f4c21cd4640 1 -- 172.21.15.12:0/182050404 >> 172.21.15.12:0/182050404 conn(0x7f4c1c0fbce0 msgr2=0x7f4c1c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.754+0000 7f4c21cd4640 1 -- 172.21.15.12:0/182050404 shutdown_connections 2024-05-09T09:22:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.754+0000 7f4c21cd4640 1 -- 172.21.15.12:0/182050404 wait complete. 2024-05-09T09:22:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c21cd4640 1 Processor -- start 2024-05-09T09:22:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c21cd4640 1 -- start start 2024-05-09T09:22:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c21cd4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c1c100770 0x7f4c1c10b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c21cd4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c1c101960 0x7f4c1c109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c21cd4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c102320 0x7f4c1c109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c21cd4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c10d4e0 con 0x7f4c1c101960 2024-05-09T09:22:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c21cd4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c10d360 con 0x7f4c1c100770 2024-05-09T09:22:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c21cd4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c10d660 con 0x7f4c1c102320 2024-05-09T09:22:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c1affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c1c101960 0x7f4c1c109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c1affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c1c101960 0x7f4c1c109730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44846/0 (socket says 172.21.15.12:44846) 2024-05-09T09:22:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c1affd640 1 -- 172.21.15.12:0/3418109899 learned_addr learned my addr 172.21.15.12:0/3418109899 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c1b7fe640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c1c100770 0x7f4c1c10b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c1bfff640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c102320 0x7f4c1c109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c1affd640 1 -- 172.21.15.12:0/3418109899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c102320 msgr2=0x7f4c1c109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c1affd640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c102320 0x7f4c1c109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c1affd640 1 -- 172.21.15.12:0/3418109899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c1c100770 msgr2=0x7f4c1c10b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c1affd640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c1c100770 0x7f4c1c10b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c1affd640 1 -- 172.21.15.12:0/3418109899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1c10a500 con 0x7f4c1c101960 2024-05-09T09:22:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.755+0000 7f4c1bfff640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c102320 0x7f4c1c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.756+0000 7f4c1affd640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c1c101960 0x7f4c1c109730 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000d770 tx=0x7f4c1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.756+0000 7f4c18ff9640 1 -- 172.21.15.12:0/3418109899 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10017070 con 0x7f4c1c101960 2024-05-09T09:22:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.756+0000 7f4c18ff9640 1 -- 172.21.15.12:0/3418109899 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4c100054f0 con 0x7f4c1c101960 2024-05-09T09:22:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.756+0000 7f4c21cd4640 1 -- 172.21.15.12:0/3418109899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c1c1a1d40 con 0x7f4c1c101960 2024-05-09T09:22:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.756+0000 7f4c18ff9640 1 -- 172.21.15.12:0/3418109899 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10004d60 con 0x7f4c1c101960 2024-05-09T09:22:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.756+0000 7f4c21cd4640 1 -- 172.21.15.12:0/3418109899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c1c1a2180 con 0x7f4c1c101960 2024-05-09T09:22:29.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.757+0000 7f4c21cd4640 1 -- 172.21.15.12:0/3418109899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c1c107b00 con 0x7f4c1c101960 2024-05-09T09:22:29.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.758+0000 7f4c18ff9640 1 -- 172.21.15.12:0/3418109899 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4c10004f00 con 0x7f4c1c101960 2024-05-09T09:22:29.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.759+0000 7f4c18ff9640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4c040a2b90 0x7f4c040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:29.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.759+0000 7f4c1b7fe640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4c040a2b90 0x7f4c040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:29.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.759+0000 7f4c18ff9640 1 -- 172.21.15.12:0/3418109899 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4c100ce650 con 0x7f4c1c101960 2024-05-09T09:22:29.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.759+0000 7f4c1b7fe640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4c040a2b90 0x7f4c040a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c006010 tx=0x7f4c0c031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.763+0000 7f4c18ff9640 1 -- 172.21.15.12:0/3418109899 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4c1009ba50 con 0x7f4c1c101960 2024-05-09T09:22:29.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:29 smithi170 ceph-mon[33632]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:29.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:29 smithi170 ceph-mon[33632]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:22:29.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:29 smithi170 ceph-mon[33632]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-05-09T09:22:29.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:29.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:22:29.937 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:29.937 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:22:29.937 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:29.937 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:29.937 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:29.937 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:29 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:29.937 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:29 smithi170 ceph-mon[33632]: Deploying daemon mds.l on smithi170 2024-05-09T09:22:29.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.949+0000 7f4c21cd4640 1 -- 172.21.15.12:0/3418109899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c1c02d3e0 con 0x7f4c1c101960 2024-05-09T09:22:29.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.950+0000 7f4c18ff9640 1 -- 172.21.15.12:0/3418109899 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18433 (secure 0 0 0) 0x7f4c1009b350 con 0x7f4c1c101960 2024-05-09T09:22:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:29.951 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:22:21.034614+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:22:29.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.952+0000 7f4c21cd4640 1 -- 172.21.15.12:0/3418109899 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4c040a2b90 msgr2=0x7f4c040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.952+0000 7f4c21cd4640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4c040a2b90 0x7f4c040a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c006010 tx=0x7f4c0c031510 comp rx=0 tx=0).stop 2024-05-09T09:22:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.952+0000 7f4c21cd4640 1 -- 172.21.15.12:0/3418109899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c1c101960 msgr2=0x7f4c1c109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.952+0000 7f4c21cd4640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c1c101960 0x7f4c1c109730 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000d770 tx=0x7f4c1000dc30 comp rx=0 tx=0).stop 2024-05-09T09:22:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.952+0000 7f4c21cd4640 1 -- 172.21.15.12:0/3418109899 shutdown_connections 2024-05-09T09:22:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.952+0000 7f4c21cd4640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4c040a2b90 0x7f4c040a5050 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.952+0000 7f4c21cd4640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c102320 0x7f4c1c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.952+0000 7f4c21cd4640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c1c101960 0x7f4c1c109730 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.952+0000 7f4c21cd4640 1 --2- 172.21.15.12:0/3418109899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c1c100770 0x7f4c1c10b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.952+0000 7f4c21cd4640 1 -- 172.21.15.12:0/3418109899 >> 172.21.15.12:0/3418109899 conn(0x7f4c1c0fbce0 msgr2=0x7f4c1c104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.952+0000 7f4c21cd4640 1 -- 172.21.15.12:0/3418109899 shutdown_connections 2024-05-09T09:22:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:29.952+0000 7f4c21cd4640 1 -- 172.21.15.12:0/3418109899 wait complete. 2024-05-09T09:22:29.962 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:29 smithi012 ceph-mon[30140]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:29.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:29 smithi012 ceph-mon[30140]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:22:29.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:29 smithi012 ceph-mon[30140]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-05-09T09:22:29.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:29.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:22:29.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:29.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:22:29.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:29.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:29.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:29.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:29 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:29.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:29 smithi012 ceph-mon[30140]: Deploying daemon mds.l on smithi170 2024-05-09T09:22:29.965 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T09:22:29.965 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:22:30.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.073+0000 7f898bd42640 1 -- 172.21.15.12:0/2481532339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8984101400 msgr2=0x7f8984101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.073+0000 7f898bd42640 1 --2- 172.21.15.12:0/2481532339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8984101400 0x7f8984101880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f897c009ca0 tx=0x7f897c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.073+0000 7f898bd42640 1 -- 172.21.15.12:0/2481532339 shutdown_connections 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.073+0000 7f898bd42640 1 --2- 172.21.15.12:0/2481532339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8984101400 0x7f8984101880 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.073+0000 7f898bd42640 1 --2- 172.21.15.12:0/2481532339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8984100a60 0x7f8984100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.073+0000 7f898bd42640 1 --2- 172.21.15.12:0/2481532339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8984100100 0x7f8984100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.073+0000 7f898bd42640 1 -- 172.21.15.12:0/2481532339 >> 172.21.15.12:0/2481532339 conn(0x7f89840fbd20 msgr2=0x7f89840fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.073+0000 7f898bd42640 1 -- 172.21.15.12:0/2481532339 shutdown_connections 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.073+0000 7f898bd42640 1 -- 172.21.15.12:0/2481532339 wait complete. 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898bd42640 1 Processor -- start 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898bd42640 1 -- start start 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898bd42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8984100100 0x7f8984199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898bd42640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8984100a60 0x7f898419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898bd42640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8984101400 0x7f89841a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898bd42640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f898410f700 con 0x7f8984101400 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898bd42640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f898410f580 con 0x7f8984100100 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898bd42640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f898410f880 con 0x7f8984100a60 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898a2df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8984101400 0x7f89841a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898a2df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8984101400 0x7f89841a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44868/0 (socket says 172.21.15.12:44868) 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898a2df640 1 -- 172.21.15.12:0/2653926599 learned_addr learned my addr 172.21.15.12:0/2653926599 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f89892dd640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8984100a60 0x7f898419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f8989ade640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8984100100 0x7f8984199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898a2df640 1 -- 172.21.15.12:0/2653926599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8984100a60 msgr2=0x7f898419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898a2df640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8984100a60 0x7f898419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898a2df640 1 -- 172.21.15.12:0/2653926599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8984100100 msgr2=0x7f8984199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898a2df640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8984100100 0x7f8984199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898a2df640 1 -- 172.21.15.12:0/2653926599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89841a1960 con 0x7f8984101400 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.074+0000 7f898a2df640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8984101400 0x7f89841a12c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f897c004af0 tx=0x7f897c0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.075+0000 7f897affd640 1 -- 172.21.15.12:0/2653926599 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c003b70 con 0x7f8984101400 2024-05-09T09:22:30.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.075+0000 7f897affd640 1 -- 172.21.15.12:0/2653926599 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f897c003d10 con 0x7f8984101400 2024-05-09T09:22:30.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.075+0000 7f897affd640 1 -- 172.21.15.12:0/2653926599 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c02fb90 con 0x7f8984101400 2024-05-09T09:22:30.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.075+0000 7f898bd42640 1 -- 172.21.15.12:0/2653926599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89841a1bf0 con 0x7f8984101400 2024-05-09T09:22:30.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.075+0000 7f898bd42640 1 -- 172.21.15.12:0/2653926599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89841a2140 con 0x7f8984101400 2024-05-09T09:22:30.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.076+0000 7f898bd42640 1 -- 172.21.15.12:0/2653926599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898410ae50 con 0x7f8984101400 2024-05-09T09:22:30.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.077+0000 7f897affd640 1 -- 172.21.15.12:0/2653926599 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f897c02fd30 con 0x7f8984101400 2024-05-09T09:22:30.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.078+0000 7f897affd640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f89700a2b90 0x7f89700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.078+0000 7f8989ade640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f89700a2b90 0x7f89700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.078+0000 7f897affd640 1 -- 172.21.15.12:0/2653926599 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f897c0f0000 con 0x7f8984101400 2024-05-09T09:22:30.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.078+0000 7f8989ade640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f89700a2b90 0x7f89700a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f898419b220 tx=0x7f8980005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:30.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.082+0000 7f897affd640 1 -- 172.21.15.12:0/2653926599 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f897c0bd400 con 0x7f8984101400 2024-05-09T09:22:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.266+0000 7f898bd42640 1 -- 172.21.15.12:0/2653926599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f898402d410 con 0x7f8984101400 2024-05-09T09:22:30.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.266+0000 7f897affd640 1 -- 172.21.15.12:0/2653926599 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f897c0bcd00 con 0x7f8984101400 2024-05-09T09:22:30.267 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:22:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.268+0000 7f898bd42640 1 -- 172.21.15.12:0/2653926599 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f89700a2b90 msgr2=0x7f89700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.268+0000 7f898bd42640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f89700a2b90 0x7f89700a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f898419b220 tx=0x7f8980005f90 comp rx=0 tx=0).stop 2024-05-09T09:22:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.268+0000 7f898bd42640 1 -- 172.21.15.12:0/2653926599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8984101400 msgr2=0x7f89841a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.268+0000 7f898bd42640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8984101400 0x7f89841a12c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f897c004af0 tx=0x7f897c0317e0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.268+0000 7f898bd42640 1 -- 172.21.15.12:0/2653926599 shutdown_connections 2024-05-09T09:22:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.268+0000 7f898bd42640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f89700a2b90 0x7f89700a5050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.268+0000 7f898bd42640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8984101400 0x7f89841a12c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.268+0000 7f898bd42640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8984100a60 0x7f898419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.268+0000 7f898bd42640 1 --2- 172.21.15.12:0/2653926599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8984100100 0x7f8984199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.268+0000 7f898bd42640 1 -- 172.21.15.12:0/2653926599 >> 172.21.15.12:0/2653926599 conn(0x7f89840fbd20 msgr2=0x7f89840fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:30.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.268+0000 7f898bd42640 1 -- 172.21.15.12:0/2653926599 shutdown_connections 2024-05-09T09:22:30.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.268+0000 7f898bd42640 1 -- 172.21.15.12:0/2653926599 wait complete. 2024-05-09T09:22:30.280 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.384+0000 7f2742f89640 1 -- 172.21.15.12:0/2504855917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f273c100790 msgr2=0x7f273c100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.384+0000 7f2742f89640 1 --2- 172.21.15.12:0/2504855917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f273c100790 0x7f273c100bb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2724009900 tx=0x7f272402a410 comp rx=0 tx=0).stop 2024-05-09T09:22:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.384+0000 7f2742f89640 1 -- 172.21.15.12:0/2504855917 shutdown_connections 2024-05-09T09:22:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.384+0000 7f2742f89640 1 --2- 172.21.15.12:0/2504855917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f273c102340 0x7f273c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.384+0000 7f2742f89640 1 --2- 172.21.15.12:0/2504855917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c101980 0x7f273c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.384+0000 7f2742f89640 1 --2- 172.21.15.12:0/2504855917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f273c100790 0x7f273c100bb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.384+0000 7f2742f89640 1 -- 172.21.15.12:0/2504855917 >> 172.21.15.12:0/2504855917 conn(0x7f273c0fbd20 msgr2=0x7f273c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.384+0000 7f2742f89640 1 -- 172.21.15.12:0/2504855917 shutdown_connections 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.384+0000 7f2742f89640 1 -- 172.21.15.12:0/2504855917 wait complete. 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f2742f89640 1 Processor -- start 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f2742f89640 1 -- start start 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f2742f89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c100790 0x7f273c199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f2742f89640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f273c101980 0x7f273c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f2742f89640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f273c102340 0x7f273c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f2742f89640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f273c10c450 con 0x7f273c101980 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f2742f89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f273c10c2d0 con 0x7f273c100790 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f2742f89640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f273c10c5d0 con 0x7f273c102340 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f273bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f273c101980 0x7f273c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f273bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f273c101980 0x7f273c19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44890/0 (socket says 172.21.15.12:44890) 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f273bfff640 1 -- 172.21.15.12:0/1467350831 learned_addr learned my addr 172.21.15.12:0/1467350831 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f2741526640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f273c102340 0x7f273c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f2740d25640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c100790 0x7f273c199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f273bfff640 1 -- 172.21.15.12:0/1467350831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f273c102340 msgr2=0x7f273c1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f273bfff640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f273c102340 0x7f273c1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f273bfff640 1 -- 172.21.15.12:0/1467350831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c100790 msgr2=0x7f273c199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f273bfff640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c100790 0x7f273c199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f273bfff640 1 -- 172.21.15.12:0/1467350831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c1a1a30 con 0x7f273c101980 2024-05-09T09:22:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.385+0000 7f273bfff640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f273c101980 0x7f273c19a350 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f272c00eab0 tx=0x7f272c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.386+0000 7f2739ffb640 1 -- 172.21.15.12:0/1467350831 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c00ce40 con 0x7f273c101980 2024-05-09T09:22:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.386+0000 7f2739ffb640 1 -- 172.21.15.12:0/1467350831 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f272c011940 con 0x7f273c101980 2024-05-09T09:22:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.386+0000 7f2742f89640 1 -- 172.21.15.12:0/1467350831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273c1a1d20 con 0x7f273c101980 2024-05-09T09:22:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.386+0000 7f2739ffb640 1 -- 172.21.15.12:0/1467350831 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c010a80 con 0x7f273c101980 2024-05-09T09:22:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.386+0000 7f2742f89640 1 -- 172.21.15.12:0/1467350831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273c1a2290 con 0x7f273c101980 2024-05-09T09:22:30.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.387+0000 7f2742f89640 1 -- 172.21.15.12:0/1467350831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f273c1a2520 con 0x7f273c101980 2024-05-09T09:22:30.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.388+0000 7f2739ffb640 1 -- 172.21.15.12:0/1467350831 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f272c011af0 con 0x7f273c101980 2024-05-09T09:22:30.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.388+0000 7f2739ffb640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f27280a2b90 0x7f27280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.389+0000 7f2740d25640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f27280a2b90 0x7f27280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.389+0000 7f2739ffb640 1 -- 172.21.15.12:0/1467350831 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f272c0cf240 con 0x7f273c101980 2024-05-09T09:22:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.389+0000 7f2740d25640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f27280a2b90 0x7f27280a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2724005c40 tx=0x7f2724000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.393+0000 7f2739ffb640 1 -- 172.21.15.12:0/1467350831 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f272c016030 con 0x7f273c101980 2024-05-09T09:22:30.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.619+0000 7f2742f89640 1 -- 172.21.15.12:0/1467350831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f273c1a26f0 con 0x7f273c101980 2024-05-09T09:22:30.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.620+0000 7f2739ffb640 1 -- 172.21.15.12:0/1467350831 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f272c09c5a0 con 0x7f273c101980 2024-05-09T09:22:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:30.622 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-09T09:22:30.622 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 1 2024-05-09T09:22:30.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.622+0000 7f2742f89640 1 -- 172.21.15.12:0/1467350831 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f27280a2b90 msgr2=0x7f27280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.622+0000 7f2742f89640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f27280a2b90 0x7f27280a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2724005c40 tx=0x7f2724000bc0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.622+0000 7f2742f89640 1 -- 172.21.15.12:0/1467350831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f273c101980 msgr2=0x7f273c19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.622+0000 7f2742f89640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f273c101980 0x7f273c19a350 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f272c00eab0 tx=0x7f272c00ef70 comp rx=0 tx=0).stop 2024-05-09T09:22:30.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.622+0000 7f2742f89640 1 -- 172.21.15.12:0/1467350831 shutdown_connections 2024-05-09T09:22:30.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.622+0000 7f2742f89640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f27280a2b90 0x7f27280a5050 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.622+0000 7f2742f89640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f273c102340 0x7f273c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.622+0000 7f2742f89640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f273c101980 0x7f273c19a350 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.622+0000 7f2742f89640 1 --2- 172.21.15.12:0/1467350831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c100790 0x7f273c199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.622+0000 7f2742f89640 1 -- 172.21.15.12:0/1467350831 >> 172.21.15.12:0/1467350831 conn(0x7f273c0fbd20 msgr2=0x7f273c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:30.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.622+0000 7f2742f89640 1 -- 172.21.15.12:0/1467350831 shutdown_connections 2024-05-09T09:22:30.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:30.622+0000 7f2742f89640 1 -- 172.21.15.12:0/1467350831 wait complete. 2024-05-09T09:22:30.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:30 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3418109899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:22:30.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:30 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2653926599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:22:30.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:30 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1467350831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:30.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:30 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3418109899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:22:30.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:30 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2653926599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:22:30.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:30 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1467350831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:31.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:30 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3418109899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:22:31.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:30 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2653926599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:22:31.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:30 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1467350831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:31.631 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:22:31 smithi170 systemd[1]: Starting Ceph mds.l for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:22:31.884 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:31 smithi170 ceph-mon[33632]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:31.884 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:22:31 smithi170 podman[63923]: 2024-05-09 09:22:31.550619473 +0000 UTC m=+0.018030977 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:22:31.884 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:22:31 smithi170 podman[63923]: 2024-05-09 09:22:31.657247376 +0000 UTC m=+0.124658909 container create 415b7f396a939f0327e29254712785229eeec599a22211a6eac42d426341905a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-96805dc) 2024-05-09T09:22:31.884 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:22:31 smithi170 podman[63923]: 2024-05-09 09:22:31.884970386 +0000 UTC m=+0.352381877 container init 415b7f396a939f0327e29254712785229eeec599a22211a6eac42d426341905a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-l, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:22:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:31 smithi012 ceph-mon[30140]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:32.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:31 smithi016 ceph-mon[33335]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:32.243 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:22:31 smithi170 podman[63923]: 2024-05-09 09:22:31.888129911 +0000 UTC m=+0.355541408 container start 415b7f396a939f0327e29254712785229eeec599a22211a6eac42d426341905a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-l, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-09T09:22:32.243 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:22:31 smithi170 ceph-mds[63942]: -- 172.21.15.170:0/516107116 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55747187da40 con 0x557471850800 2024-05-09T09:22:32.243 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:22:31 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-l[63938]: starting mds.l at 2024-05-09T09:22:32.243 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:22:31 smithi170 bash[63923]: 415b7f396a939f0327e29254712785229eeec599a22211a6eac42d426341905a 2024-05-09T09:22:32.243 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:22:31 smithi170 systemd[1]: Started Ceph mds.l for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:22:32.635 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:32.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.753+0000 7f87ff61b640 1 -- 172.21.15.12:0/3285610772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f8101980 msgr2=0x7f87f8101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:32.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.753+0000 7f87ff61b640 1 --2- 172.21.15.12:0/3285610772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f8101980 0x7f87f8101e00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009900 tx=0x7f87e802a410 comp rx=0 tx=0).stop 2024-05-09T09:22:32.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.753+0000 7f87ff61b640 1 -- 172.21.15.12:0/3285610772 shutdown_connections 2024-05-09T09:22:32.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.753+0000 7f87ff61b640 1 --2- 172.21.15.12:0/3285610772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f8102340 0x7f87f8108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.753+0000 7f87ff61b640 1 --2- 172.21.15.12:0/3285610772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f8101980 0x7f87f8101e00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.753+0000 7f87ff61b640 1 --2- 172.21.15.12:0/3285610772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8100790 0x7f87f8100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.753+0000 7f87ff61b640 1 -- 172.21.15.12:0/3285610772 >> 172.21.15.12:0/3285610772 conn(0x7f87f80fbd20 msgr2=0x7f87f80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.753+0000 7f87ff61b640 1 -- 172.21.15.12:0/3285610772 shutdown_connections 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.753+0000 7f87ff61b640 1 -- 172.21.15.12:0/3285610772 wait complete. 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.754+0000 7f87ff61b640 1 Processor -- start 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.754+0000 7f87ff61b640 1 -- start start 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.754+0000 7f87ff61b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8100790 0x7f87f8196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.754+0000 7f87ff61b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f8101980 0x7f87f8197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.754+0000 7f87ff61b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f8102340 0x7f87f819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.754+0000 7f87ff61b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f87f810c3f0 con 0x7f87f8101980 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.754+0000 7f87ff61b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87f810c270 con 0x7f87f8102340 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.754+0000 7f87ff61b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f87f810c570 con 0x7f87f8100790 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.754+0000 7f87fcbb6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f8101980 0x7f87f8197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.754+0000 7f87fcbb6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f8101980 0x7f87f8197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59214/0 (socket says 172.21.15.12:59214) 2024-05-09T09:22:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.754+0000 7f87fcbb6640 1 -- 172.21.15.12:0/3688766794 learned_addr learned my addr 172.21.15.12:0/3688766794 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.754+0000 7f87fdbb8640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f8102340 0x7f87f819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.754+0000 7f87fd3b7640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8100790 0x7f87f8196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.755+0000 7f87fcbb6640 1 -- 172.21.15.12:0/3688766794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8100790 msgr2=0x7f87f8196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.755+0000 7f87fcbb6640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8100790 0x7f87f8196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.755+0000 7f87fcbb6640 1 -- 172.21.15.12:0/3688766794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f8102340 msgr2=0x7f87f819e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.755+0000 7f87fcbb6640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f8102340 0x7f87f819e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.755+0000 7f87fcbb6640 1 -- 172.21.15.12:0/3688766794 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f819e6e0 con 0x7f87f8101980 2024-05-09T09:22:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.755+0000 7f87fcbb6640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f8101980 0x7f87f8197000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009230 tx=0x7f87e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.755+0000 7f87ee7fc640 1 -- 172.21.15.12:0/3688766794 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e803f030 con 0x7f87f8101980 2024-05-09T09:22:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.755+0000 7f87ff61b640 1 -- 172.21.15.12:0/3688766794 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f819e970 con 0x7f87f8101980 2024-05-09T09:22:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.755+0000 7f87ee7fc640 1 -- 172.21.15.12:0/3688766794 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f87e8031980 con 0x7f87f8101980 2024-05-09T09:22:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.755+0000 7f87ee7fc640 1 -- 172.21.15.12:0/3688766794 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8035070 con 0x7f87f8101980 2024-05-09T09:22:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.755+0000 7f87ff61b640 1 -- 172.21.15.12:0/3688766794 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f819eeb0 con 0x7f87f8101980 2024-05-09T09:22:32.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.756+0000 7f87ff61b640 1 -- 172.21.15.12:0/3688766794 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f8107ab0 con 0x7f87f8101980 2024-05-09T09:22:32.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.757+0000 7f87ee7fc640 1 -- 172.21.15.12:0/3688766794 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f87e8042030 con 0x7f87f8101980 2024-05-09T09:22:32.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.758+0000 7f87ee7fc640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f87e40a2b90 0x7f87e40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.758+0000 7f87ee7fc640 1 -- 172.21.15.12:0/3688766794 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f87e80ef050 con 0x7f87f8101980 2024-05-09T09:22:32.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.758+0000 7f87fd3b7640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f87e40a2b90 0x7f87e40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.759+0000 7f87fd3b7640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f87e40a2b90 0x7f87e40a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f87f81017e0 tx=0x7f87f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:32.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.762+0000 7f87ee7fc640 1 -- 172.21.15.12:0/3688766794 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f87e80bc450 con 0x7f87f8101980 2024-05-09T09:22:32.986 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:32 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.170:6832/4010027838,v1:172.21.15.170:6833/4010027838] up:boot 2024-05-09T09:22:32.986 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:32 smithi016 ceph-mon[33335]: fsmap 1 up:standby 2024-05-09T09:22:32.986 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:32 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T09:22:32.986 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:32 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:32.986 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:32 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:32.987 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:32 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:32.987 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:32 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:32.987 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:32 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:32.987 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:32 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:32.987 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:32 smithi016 ceph-mon[33335]: Deploying daemon mds.k on smithi016 2024-05-09T09:22:32.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.989+0000 7f87ff61b640 1 -- 172.21.15.12:0/3688766794 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87f802d410 con 0x7f87f8101980 2024-05-09T09:22:32.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.989+0000 7f87ee7fc640 1 -- 172.21.15.12:0/3688766794 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f87e80bbd50 con 0x7f87f8101980 2024-05-09T09:22:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:32.990 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T09:22:32.990 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 2 2024-05-09T09:22:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.991+0000 7f87ff61b640 1 -- 172.21.15.12:0/3688766794 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f87e40a2b90 msgr2=0x7f87e40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.991+0000 7f87ff61b640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f87e40a2b90 0x7f87e40a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f87f81017e0 tx=0x7f87f4005f90 comp rx=0 tx=0).stop 2024-05-09T09:22:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.991+0000 7f87ff61b640 1 -- 172.21.15.12:0/3688766794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f8101980 msgr2=0x7f87f8197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.991+0000 7f87ff61b640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f8101980 0x7f87f8197000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009230 tx=0x7f87e80042f0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.991+0000 7f87ff61b640 1 -- 172.21.15.12:0/3688766794 shutdown_connections 2024-05-09T09:22:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.991+0000 7f87ff61b640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f87e40a2b90 0x7f87e40a5050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.991+0000 7f87ff61b640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f8102340 0x7f87f819e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.991+0000 7f87ff61b640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f8101980 0x7f87f8197000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.991+0000 7f87ff61b640 1 --2- 172.21.15.12:0/3688766794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8100790 0x7f87f8196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.991+0000 7f87ff61b640 1 -- 172.21.15.12:0/3688766794 >> 172.21.15.12:0/3688766794 conn(0x7f87f80fbd20 msgr2=0x7f87f8104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.992+0000 7f87ff61b640 1 -- 172.21.15.12:0/3688766794 shutdown_connections 2024-05-09T09:22:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:32.992+0000 7f87ff61b640 1 -- 172.21.15.12:0/3688766794 wait complete. 2024-05-09T09:22:33.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:32 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.170:6832/4010027838,v1:172.21.15.170:6833/4010027838] up:boot 2024-05-09T09:22:33.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:32 smithi012 ceph-mon[30140]: fsmap 1 up:standby 2024-05-09T09:22:33.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:32 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T09:22:33.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:32 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:33.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:32 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:33.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:32 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:33.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:32 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:33.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:32 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:33.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:32 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:33.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:32 smithi012 ceph-mon[30140]: Deploying daemon mds.k on smithi016 2024-05-09T09:22:33.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:32 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.170:6832/4010027838,v1:172.21.15.170:6833/4010027838] up:boot 2024-05-09T09:22:33.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:32 smithi170 ceph-mon[33632]: fsmap 1 up:standby 2024-05-09T09:22:33.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:32 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T09:22:33.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:32 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:33.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:32 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:33.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:32 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:33.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:32 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:33.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:32 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:33.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:32 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:33.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:32 smithi170 ceph-mon[33632]: Deploying daemon mds.k on smithi016 2024-05-09T09:22:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:33 smithi016 ceph-mon[33335]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:33 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3688766794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:34.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:33 smithi012 ceph-mon[30140]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:34.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:33 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3688766794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:34.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:33 smithi170 ceph-mon[33632]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:34.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:33 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3688766794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:34.503 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:22:34 smithi016 systemd[1]: Starting Ceph mds.k for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:22:34.804 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:22:34 smithi016 podman[63065]: 2024-05-09 09:22:34.455060071 +0000 UTC m=+0.018665265 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:22:34.804 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:22:34 smithi016 podman[63065]: 2024-05-09 09:22:34.569675769 +0000 UTC m=+0.133280945 container create e88be52650b2ec17e3b89ee7191d71da0171385df2031999ab40e20642c10476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-09T09:22:35.004 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:35.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.121+0000 7f7ba35a2640 1 -- 172.21.15.12:0/2183640555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c102320 msgr2=0x7f7b9c108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:35.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.121+0000 7f7ba35a2640 1 --2- 172.21.15.12:0/2183640555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c102320 0x7f7b9c108bb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009ca0 tx=0x7f7b9402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.121+0000 7f7ba35a2640 1 -- 172.21.15.12:0/2183640555 shutdown_connections 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.121+0000 7f7ba35a2640 1 --2- 172.21.15.12:0/2183640555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c102320 0x7f7b9c108bb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.121+0000 7f7ba35a2640 1 --2- 172.21.15.12:0/2183640555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b9c101960 0x7f7b9c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.121+0000 7f7ba35a2640 1 --2- 172.21.15.12:0/2183640555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b9c100770 0x7f7b9c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.121+0000 7f7ba35a2640 1 -- 172.21.15.12:0/2183640555 >> 172.21.15.12:0/2183640555 conn(0x7f7b9c0fbce0 msgr2=0x7f7b9c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.121+0000 7f7ba35a2640 1 -- 172.21.15.12:0/2183640555 shutdown_connections 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.121+0000 7f7ba35a2640 1 -- 172.21.15.12:0/2183640555 wait complete. 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.121+0000 7f7ba35a2640 1 Processor -- start 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.121+0000 7f7ba35a2640 1 -- start start 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba35a2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b9c100770 0x7f7b9c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba35a2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c101960 0x7f7b9c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba35a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b9c102320 0x7f7b9c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba35a2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10c3c0 con 0x7f7b9c101960 2024-05-09T09:22:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba35a2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10c240 con 0x7f7b9c102320 2024-05-09T09:22:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba35a2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10c540 con 0x7f7b9c100770 2024-05-09T09:22:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba0b3d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c101960 0x7f7b9c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba0b3d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c101960 0x7f7b9c198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59230/0 (socket says 172.21.15.12:59230) 2024-05-09T09:22:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba0b3d640 1 -- 172.21.15.12:0/1781721914 learned_addr learned my addr 172.21.15.12:0/1781721914 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba133e640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b9c100770 0x7f7b9c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba1b3f640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b9c102320 0x7f7b9c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba0b3d640 1 -- 172.21.15.12:0/1781721914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b9c100770 msgr2=0x7f7b9c197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba0b3d640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b9c100770 0x7f7b9c197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba0b3d640 1 -- 172.21.15.12:0/1781721914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b9c102320 msgr2=0x7f7b9c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba0b3d640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b9c102320 0x7f7b9c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba0b3d640 1 -- 172.21.15.12:0/1781721914 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c19f770 con 0x7f7b9c101960 2024-05-09T09:22:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba133e640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b9c100770 0x7f7b9c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:22:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.122+0000 7f7ba0b3d640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c101960 0x7f7b9c198090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00d950 tx=0x7f7b8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:35.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.123+0000 7f7b927fc640 1 -- 172.21.15.12:0/1781721914 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c017070 con 0x7f7b9c101960 2024-05-09T09:22:35.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.123+0000 7f7b927fc640 1 -- 172.21.15.12:0/1781721914 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7b8c009c30 con 0x7f7b9c101960 2024-05-09T09:22:35.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.123+0000 7f7ba35a2640 1 -- 172.21.15.12:0/1781721914 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c19fa60 con 0x7f7b9c101960 2024-05-09T09:22:35.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.123+0000 7f7b927fc640 1 -- 172.21.15.12:0/1781721914 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00f6b0 con 0x7f7b9c101960 2024-05-09T09:22:35.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.123+0000 7f7ba35a2640 1 -- 172.21.15.12:0/1781721914 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c19ffb0 con 0x7f7b9c101960 2024-05-09T09:22:35.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.124+0000 7f7ba35a2640 1 -- 172.21.15.12:0/1781721914 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9c1a01e0 con 0x7f7b9c101960 2024-05-09T09:22:35.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.125+0000 7f7b927fc640 1 -- 172.21.15.12:0/1781721914 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7b8c010a50 con 0x7f7b9c101960 2024-05-09T09:22:35.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.126+0000 7f7b927fc640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7b880a2b90 0x7f7b880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.126+0000 7f7b927fc640 1 -- 172.21.15.12:0/1781721914 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7b8c0ce760 con 0x7f7b9c101960 2024-05-09T09:22:35.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.126+0000 7f7ba133e640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7b880a2b90 0x7f7b880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:35.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.126+0000 7f7ba133e640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7b880a2b90 0x7f7b880a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c1017c0 tx=0x7f7b98005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:35.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.129+0000 7f7b927fc640 1 -- 172.21.15.12:0/1781721914 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7b8c009de0 con 0x7f7b9c101960 2024-05-09T09:22:35.253 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:22:34 smithi016 podman[63065]: 2024-05-09 09:22:34.805623384 +0000 UTC m=+0.369228567 container init e88be52650b2ec17e3b89ee7191d71da0171385df2031999ab40e20642c10476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-96805dc, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:22:35.253 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:22:34 smithi016 podman[63065]: 2024-05-09 09:22:34.808660041 +0000 UTC m=+0.372265222 container start e88be52650b2ec17e3b89ee7191d71da0171385df2031999ab40e20642c10476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-k, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T09:22:35.253 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:22:34 smithi016 ceph-mds[63084]: -- 172.21.15.16:0/3590707910 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55725bd83a40 con 0x55725bd57000 2024-05-09T09:22:35.253 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:22:34 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-k[63080]: starting mds.k at 2024-05-09T09:22:35.253 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:22:34 smithi016 bash[63065]: e88be52650b2ec17e3b89ee7191d71da0171385df2031999ab40e20642c10476 2024-05-09T09:22:35.253 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:22:34 smithi016 systemd[1]: Started Ceph mds.k for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:22:35.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.381+0000 7f7ba35a2640 1 -- 172.21.15.12:0/1781721914 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b9c1a03b0 con 0x7f7b9c101960 2024-05-09T09:22:35.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.381+0000 7f7b927fc640 1 -- 172.21.15.12:0/1781721914 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f7b8c09b510 con 0x7f7b9c101960 2024-05-09T09:22:35.381 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:35.381 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T09:22:35.381 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 3 2024-05-09T09:22:35.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.383+0000 7f7ba35a2640 1 -- 172.21.15.12:0/1781721914 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7b880a2b90 msgr2=0x7f7b880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:35.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.383+0000 7f7ba35a2640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7b880a2b90 0x7f7b880a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c1017c0 tx=0x7f7b98005f90 comp rx=0 tx=0).stop 2024-05-09T09:22:35.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.383+0000 7f7ba35a2640 1 -- 172.21.15.12:0/1781721914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c101960 msgr2=0x7f7b9c198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:35.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.383+0000 7f7ba35a2640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c101960 0x7f7b9c198090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00d950 tx=0x7f7b8c00de10 comp rx=0 tx=0).stop 2024-05-09T09:22:35.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.383+0000 7f7ba35a2640 1 -- 172.21.15.12:0/1781721914 shutdown_connections 2024-05-09T09:22:35.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.383+0000 7f7ba35a2640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7b880a2b90 0x7f7b880a5050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.383+0000 7f7ba35a2640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b9c102320 0x7f7b9c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.383+0000 7f7ba35a2640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c101960 0x7f7b9c198090 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.383+0000 7f7ba35a2640 1 --2- 172.21.15.12:0/1781721914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b9c100770 0x7f7b9c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.383+0000 7f7ba35a2640 1 -- 172.21.15.12:0/1781721914 >> 172.21.15.12:0/1781721914 conn(0x7f7b9c0fbce0 msgr2=0x7f7b9c0fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:35.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.383+0000 7f7ba35a2640 1 -- 172.21.15.12:0/1781721914 shutdown_connections 2024-05-09T09:22:35.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:35.383+0000 7f7ba35a2640 1 -- 172.21.15.12:0/1781721914 wait complete. 2024-05-09T09:22:35.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:35 smithi012 ceph-mon[30140]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:35.942 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:35 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.16:6832/3659922754,v1:172.21.15.16:6833/3659922754] up:boot 2024-05-09T09:22:35.942 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:35 smithi012 ceph-mon[30140]: fsmap 2 up:standby 2024-05-09T09:22:35.942 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:35 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T09:22:35.942 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:35 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:35.942 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:35 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:35.942 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:35 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:35.942 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:35 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:35.942 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:35 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:35.942 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:35 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:35.942 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:35 smithi012 ceph-mon[30140]: Deploying daemon mds.j on smithi012 2024-05-09T09:22:35.942 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:35 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1781721914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:36.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:35 smithi170 ceph-mon[33632]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:36.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:35 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.16:6832/3659922754,v1:172.21.15.16:6833/3659922754] up:boot 2024-05-09T09:22:36.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:35 smithi170 ceph-mon[33632]: fsmap 2 up:standby 2024-05-09T09:22:36.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:35 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T09:22:36.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:35 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:36.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:35 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:36.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:35 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:36.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:35 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:36.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:35 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:36.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:35 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:36.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:35 smithi170 ceph-mon[33632]: Deploying daemon mds.j on smithi012 2024-05-09T09:22:36.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:35 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1781721914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:36.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:35 smithi016 ceph-mon[33335]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:36.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:35 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.16:6832/3659922754,v1:172.21.15.16:6833/3659922754] up:boot 2024-05-09T09:22:36.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:35 smithi016 ceph-mon[33335]: fsmap 2 up:standby 2024-05-09T09:22:36.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:35 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T09:22:36.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:35 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:36.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:35 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:36.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:35 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:36.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:35 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:36.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:35 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:36.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:35 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:36.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:35 smithi016 ceph-mon[33335]: Deploying daemon mds.j on smithi012 2024-05-09T09:22:36.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:35 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1781721914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:37.395 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.507+0000 7f6569f7e640 1 -- 172.21.15.12:0/3748241146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6564104ce0 msgr2=0x7f65641070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.507+0000 7f6569f7e640 1 --2- 172.21.15.12:0/3748241146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6564104ce0 0x7f65641070d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6558009900 tx=0x7f655802a410 comp rx=0 tx=0).stop 2024-05-09T09:22:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.507+0000 7f6569f7e640 1 -- 172.21.15.12:0/3748241146 shutdown_connections 2024-05-09T09:22:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.507+0000 7f6569f7e640 1 --2- 172.21.15.12:0/3748241146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6564107610 0x7f6564109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.507+0000 7f6569f7e640 1 --2- 172.21.15.12:0/3748241146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6564104ce0 0x7f65641070d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.507+0000 7f6569f7e640 1 --2- 172.21.15.12:0/3748241146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65641023b0 0x7f65641047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.507+0000 7f6569f7e640 1 -- 172.21.15.12:0/3748241146 >> 172.21.15.12:0/3748241146 conn(0x7f65640fbd20 msgr2=0x7f65640fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.507+0000 7f6569f7e640 1 -- 172.21.15.12:0/3748241146 shutdown_connections 2024-05-09T09:22:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.507+0000 7f6569f7e640 1 -- 172.21.15.12:0/3748241146 wait complete. 2024-05-09T09:22:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.507+0000 7f6569f7e640 1 Processor -- start 2024-05-09T09:22:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f6569f7e640 1 -- start start 2024-05-09T09:22:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f6569f7e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65641023b0 0x7f6564101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f6569f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564104ce0 0x7f6564100360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f6569f7e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6564107610 0x7f65641008a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:37.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f6569f7e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f656410d510 con 0x7f65641023b0 2024-05-09T09:22:37.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f6569f7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f656410d390 con 0x7f6564104ce0 2024-05-09T09:22:37.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f6569f7e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f656410d690 con 0x7f6564107610 2024-05-09T09:22:37.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f65637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65641023b0 0x7f6564101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:37.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f65637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65641023b0 0x7f6564101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59238/0 (socket says 172.21.15.12:59238) 2024-05-09T09:22:37.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f65637fe640 1 -- 172.21.15.12:0/3647126090 learned_addr learned my addr 172.21.15.12:0/3647126090 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:37.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f6562ffd640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564104ce0 0x7f6564100360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:37.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f6563fff640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6564107610 0x7f65641008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f65637fe640 1 -- 172.21.15.12:0/3647126090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6564107610 msgr2=0x7f65641008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f65637fe640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6564107610 0x7f65641008a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f65637fe640 1 -- 172.21.15.12:0/3647126090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564104ce0 msgr2=0x7f6564100360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f65637fe640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564104ce0 0x7f6564100360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f65637fe640 1 -- 172.21.15.12:0/3647126090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6564101130 con 0x7f65641023b0 2024-05-09T09:22:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f65637fe640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65641023b0 0x7f6564101d20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f655400e9f0 tx=0x7f655400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f6560ff9640 1 -- 172.21.15.12:0/3647126090 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655400cd80 con 0x7f65641023b0 2024-05-09T09:22:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f6560ff9640 1 -- 172.21.15.12:0/3647126090 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6554010900 con 0x7f65641023b0 2024-05-09T09:22:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f6569f7e640 1 -- 172.21.15.12:0/3647126090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65641a1cc0 con 0x7f65641023b0 2024-05-09T09:22:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.508+0000 7f6560ff9640 1 -- 172.21.15.12:0/3647126090 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540189e0 con 0x7f65641023b0 2024-05-09T09:22:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.509+0000 7f6569f7e640 1 -- 172.21.15.12:0/3647126090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65641a2190 con 0x7f65641023b0 2024-05-09T09:22:37.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.509+0000 7f6569f7e640 1 -- 172.21.15.12:0/3647126090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65640fd810 con 0x7f65641023b0 2024-05-09T09:22:37.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.511+0000 7f6560ff9640 1 -- 172.21.15.12:0/3647126090 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6554010ab0 con 0x7f65641023b0 2024-05-09T09:22:37.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.512+0000 7f6560ff9640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f654c0a2b90 0x7f654c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:37.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.512+0000 7f6562ffd640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f654c0a2b90 0x7f654c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:37.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.512+0000 7f6560ff9640 1 -- 172.21.15.12:0/3647126090 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f65540cefa0 con 0x7f65641023b0 2024-05-09T09:22:37.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.512+0000 7f6562ffd640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f654c0a2b90 0x7f654c0a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6558006010 tx=0x7f6558031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:37.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.516+0000 7f6560ff9640 1 -- 172.21.15.12:0/3647126090 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f655409c3a0 con 0x7f65641023b0 2024-05-09T09:22:37.541 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 09:22:37 smithi012 podman[71162]: 2024-05-09 09:22:37.54250382 +0000 UTC m=+0.144043653 container create b9e335ae8c04a148b475ee394944130de70cad5d9cbe809bb83e04069bf6c840 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-j, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-09T09:22:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.750+0000 7f6569f7e640 1 -- 172.21.15.12:0/3647126090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f656402d1c0 con 0x7f65641023b0 2024-05-09T09:22:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.750+0000 7f6560ff9640 1 -- 172.21.15.12:0/3647126090 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f656402d1c0 con 0x7f65641023b0 2024-05-09T09:22:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:37.750 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T09:22:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 3 2024-05-09T09:22:37.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.752+0000 7f6569f7e640 1 -- 172.21.15.12:0/3647126090 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f654c0a2b90 msgr2=0x7f654c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:37.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.752+0000 7f6569f7e640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f654c0a2b90 0x7f654c0a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6558006010 tx=0x7f6558031510 comp rx=0 tx=0).stop 2024-05-09T09:22:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.753+0000 7f6569f7e640 1 -- 172.21.15.12:0/3647126090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65641023b0 msgr2=0x7f6564101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.753+0000 7f6569f7e640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65641023b0 0x7f6564101d20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f655400e9f0 tx=0x7f655400eeb0 comp rx=0 tx=0).stop 2024-05-09T09:22:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.753+0000 7f6569f7e640 1 -- 172.21.15.12:0/3647126090 shutdown_connections 2024-05-09T09:22:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.753+0000 7f6569f7e640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f654c0a2b90 0x7f654c0a5050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.753+0000 7f6569f7e640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6564107610 0x7f65641008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.753+0000 7f6569f7e640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564104ce0 0x7f6564100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.753+0000 7f6569f7e640 1 --2- 172.21.15.12:0/3647126090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65641023b0 0x7f6564101d20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.753+0000 7f6569f7e640 1 -- 172.21.15.12:0/3647126090 >> 172.21.15.12:0/3647126090 conn(0x7f65640fbd20 msgr2=0x7f6564102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.753+0000 7f6569f7e640 1 -- 172.21.15.12:0/3647126090 shutdown_connections 2024-05-09T09:22:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:37.753+0000 7f6569f7e640 1 -- 172.21.15.12:0/3647126090 wait complete. 2024-05-09T09:22:38.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:37 smithi012 ceph-mon[30140]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:38.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:37 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3647126090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:38.234 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 09:22:37 smithi012 podman[71162]: 2024-05-09 09:22:37.812281462 +0000 UTC m=+0.413821305 container init b9e335ae8c04a148b475ee394944130de70cad5d9cbe809bb83e04069bf6c840 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507) 2024-05-09T09:22:38.234 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 09:22:37 smithi012 podman[71162]: 2024-05-09 09:22:37.815279263 +0000 UTC m=+0.416819101 container start b9e335ae8c04a148b475ee394944130de70cad5d9cbe809bb83e04069bf6c840 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-j, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-09T09:22:38.234 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 09:22:37 smithi012 ceph-mds[71222]: -- 172.21.15.12:0/3074029416 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5647d20c9a40 con 0x5647d209d000 2024-05-09T09:22:38.234 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 09:22:37 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-j[71216]: starting mds.j at 2024-05-09T09:22:38.235 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 09:22:37 smithi012 bash[71162]: b9e335ae8c04a148b475ee394944130de70cad5d9cbe809bb83e04069bf6c840 2024-05-09T09:22:38.235 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 09:22:37 smithi012 systemd[1]: Started Ceph mds.j for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:22:38.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:37 smithi016 ceph-mon[33335]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:38.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:37 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3647126090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:38.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:37 smithi170 ceph-mon[33632]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:38.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:37 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3647126090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:39.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:38 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.12:6834/2151845805,v1:172.21.15.12:6835/2151845805] up:boot 2024-05-09T09:22:39.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:38 smithi012 ceph-mon[30140]: fsmap 3 up:standby 2024-05-09T09:22:39.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T09:22:39.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:39.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:39.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:39.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:39.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:39.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:38 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:39.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:38 smithi012 ceph-mon[30140]: Deploying daemon mds.d on smithi012 2024-05-09T09:22:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:38 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.12:6834/2151845805,v1:172.21.15.12:6835/2151845805] up:boot 2024-05-09T09:22:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:38 smithi170 ceph-mon[33632]: fsmap 3 up:standby 2024-05-09T09:22:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T09:22:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:38 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:39.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:38 smithi170 ceph-mon[33632]: Deploying daemon mds.d on smithi012 2024-05-09T09:22:39.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:38 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.12:6834/2151845805,v1:172.21.15.12:6835/2151845805] up:boot 2024-05-09T09:22:39.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:38 smithi016 ceph-mon[33335]: fsmap 3 up:standby 2024-05-09T09:22:39.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T09:22:39.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:39.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:39.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:39.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:39.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:39.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:38 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:39.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:38 smithi016 ceph-mon[33335]: Deploying daemon mds.d on smithi012 2024-05-09T09:22:39.764 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:39.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.895+0000 7ff37b39e640 1 -- 172.21.15.12:0/4059693040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3741034e0 msgr2=0x7ff374109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:39.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.895+0000 7ff37b39e640 1 --2- 172.21.15.12:0/4059693040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3741034e0 0x7ff374109d70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff36c009cd0 tx=0x7ff36c02a8e0 comp rx=0 tx=0).stop 2024-05-09T09:22:39.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.895+0000 7ff37b39e640 1 -- 172.21.15.12:0/4059693040 shutdown_connections 2024-05-09T09:22:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.895+0000 7ff37b39e640 1 --2- 172.21.15.12:0/4059693040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3741034e0 0x7ff374109d70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.895+0000 7ff37b39e640 1 --2- 172.21.15.12:0/4059693040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff374102b20 0x7ff374102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.895+0000 7ff37b39e640 1 --2- 172.21.15.12:0/4059693040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3740ffb50 0x7ff3740fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.895+0000 7ff37b39e640 1 -- 172.21.15.12:0/4059693040 >> 172.21.15.12:0/4059693040 conn(0x7ff3740fb460 msgr2=0x7ff3740fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.895+0000 7ff37b39e640 1 -- 172.21.15.12:0/4059693040 shutdown_connections 2024-05-09T09:22:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.895+0000 7ff37b39e640 1 -- 172.21.15.12:0/4059693040 wait complete. 2024-05-09T09:22:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.896+0000 7ff37b39e640 1 Processor -- start 2024-05-09T09:22:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.896+0000 7ff37b39e640 1 -- start start 2024-05-09T09:22:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.896+0000 7ff37b39e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3740ffb50 0x7ff374197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.896+0000 7ff37b39e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff374102b20 0x7ff374198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.896+0000 7ff37b39e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3741034e0 0x7ff37419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.896+0000 7ff37b39e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff37410d560 con 0x7ff374102b20 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.896+0000 7ff37b39e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff37410d3e0 con 0x7ff3741034e0 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.896+0000 7ff37b39e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff37410d6e0 con 0x7ff3740ffb50 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.896+0000 7ff378939640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff374102b20 0x7ff374198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff378939640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff374102b20 0x7ff374198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59280/0 (socket says 172.21.15.12:59280) 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff378939640 1 -- 172.21.15.12:0/409042749 learned_addr learned my addr 172.21.15.12:0/409042749 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff37913a640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3740ffb50 0x7ff374197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff37993b640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3741034e0 0x7ff37419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff378939640 1 -- 172.21.15.12:0/409042749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3740ffb50 msgr2=0x7ff374197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff378939640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3740ffb50 0x7ff374197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff378939640 1 -- 172.21.15.12:0/409042749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3741034e0 msgr2=0x7ff37419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff378939640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3741034e0 0x7ff37419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff378939640 1 -- 172.21.15.12:0/409042749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37419f710 con 0x7ff374102b20 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff37993b640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3741034e0 0x7ff37419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff378939640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff374102b20 0x7ff374198030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c920 tx=0x7ff36400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:39.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff36a7fc640 1 -- 172.21.15.12:0/409042749 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364007c70 con 0x7ff374102b20 2024-05-09T09:22:39.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff36a7fc640 1 -- 172.21.15.12:0/409042749 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff364007e10 con 0x7ff374102b20 2024-05-09T09:22:39.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff37b39e640 1 -- 172.21.15.12:0/409042749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37419fa00 con 0x7ff374102b20 2024-05-09T09:22:39.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff36a7fc640 1 -- 172.21.15.12:0/409042749 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364018850 con 0x7ff374102b20 2024-05-09T09:22:39.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.897+0000 7ff37b39e640 1 -- 172.21.15.12:0/409042749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37419ffa0 con 0x7ff374102b20 2024-05-09T09:22:39.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.898+0000 7ff37b39e640 1 -- 172.21.15.12:0/409042749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff374108bf0 con 0x7ff374102b20 2024-05-09T09:22:39.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.900+0000 7ff36a7fc640 1 -- 172.21.15.12:0/409042749 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff364018a10 con 0x7ff374102b20 2024-05-09T09:22:39.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.901+0000 7ff36a7fc640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff3600a2c10 0x7ff3600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:39.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.901+0000 7ff37913a640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff3600a2c10 0x7ff3600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:39.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.901+0000 7ff36a7fc640 1 -- 172.21.15.12:0/409042749 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff3640cec90 con 0x7ff374102b20 2024-05-09T09:22:39.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.901+0000 7ff37913a640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff3600a2c10 0x7ff3600a50d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff374069c70 tx=0x7ff370005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:39.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:39.905+0000 7ff36a7fc640 1 -- 172.21.15.12:0/409042749 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff364014030 con 0x7ff374102b20 2024-05-09T09:22:40.043 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:39 smithi012 ceph-mon[30140]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:40.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.145+0000 7ff37b39e640 1 -- 172.21.15.12:0/409042749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff37402d1c0 con 0x7ff374102b20 2024-05-09T09:22:40.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.146+0000 7ff36a7fc640 1 -- 172.21.15.12:0/409042749 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7ff36409c090 con 0x7ff374102b20 2024-05-09T09:22:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:40.146 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T09:22:40.146 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 4 2024-05-09T09:22:40.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.148+0000 7ff37b39e640 1 -- 172.21.15.12:0/409042749 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff3600a2c10 msgr2=0x7ff3600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:40.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.148+0000 7ff37b39e640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff3600a2c10 0x7ff3600a50d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff374069c70 tx=0x7ff370005f90 comp rx=0 tx=0).stop 2024-05-09T09:22:40.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.148+0000 7ff37b39e640 1 -- 172.21.15.12:0/409042749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff374102b20 msgr2=0x7ff374198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:40.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.148+0000 7ff37b39e640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff374102b20 0x7ff374198030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c920 tx=0x7ff36400cde0 comp rx=0 tx=0).stop 2024-05-09T09:22:40.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.148+0000 7ff37b39e640 1 -- 172.21.15.12:0/409042749 shutdown_connections 2024-05-09T09:22:40.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.148+0000 7ff37b39e640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff3600a2c10 0x7ff3600a50d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:40.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.148+0000 7ff37b39e640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3741034e0 0x7ff37419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:40.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.148+0000 7ff37b39e640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff374102b20 0x7ff374198030 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:40.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.148+0000 7ff37b39e640 1 --2- 172.21.15.12:0/409042749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3740ffb50 0x7ff374197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:40.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.148+0000 7ff37b39e640 1 -- 172.21.15.12:0/409042749 >> 172.21.15.12:0/409042749 conn(0x7ff3740fb460 msgr2=0x7ff374105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:40.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.148+0000 7ff37b39e640 1 -- 172.21.15.12:0/409042749 shutdown_connections 2024-05-09T09:22:40.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:40.148+0000 7ff37b39e640 1 -- 172.21.15.12:0/409042749 wait complete. 2024-05-09T09:22:40.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:39 smithi170 ceph-mon[33632]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:40.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:39 smithi016 ceph-mon[33335]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:40.454 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:22:40 smithi012 systemd[1]: Starting Ceph mds.d for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:22:40.734 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:22:40 smithi012 podman[71552]: 2024-05-09 09:22:40.357966146 +0000 UTC m=+0.018658869 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:22:40.734 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:22:40 smithi012 podman[71552]: 2024-05-09 09:22:40.474137801 +0000 UTC m=+0.134830565 container create b27c3a53d84589eac833b97c70c75f620c09353a6dcf13b9b3d53ee02ffeb554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-d, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-09T09:22:40.734 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:22:40 smithi012 podman[71552]: 2024-05-09 09:22:40.735298205 +0000 UTC m=+0.395990933 container init b27c3a53d84589eac833b97c70c75f620c09353a6dcf13b9b3d53ee02ffeb554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T09:22:40.989 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:22:40 smithi012 podman[71552]: 2024-05-09 09:22:40.738288656 +0000 UTC m=+0.398981380 container start b27c3a53d84589eac833b97c70c75f620c09353a6dcf13b9b3d53ee02ffeb554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc) 2024-05-09T09:22:40.990 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:22:40 smithi012 ceph-mds[71571]: -- 172.21.15.12:0/2717505010 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac7583da40 con 0x55ac75811000 2024-05-09T09:22:40.990 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:22:40 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-d[71567]: starting mds.d at 2024-05-09T09:22:40.990 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:22:40 smithi012 bash[71552]: b27c3a53d84589eac833b97c70c75f620c09353a6dcf13b9b3d53ee02ffeb554 2024-05-09T09:22:40.991 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:22:40 smithi012 systemd[1]: Started Ceph mds.d for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:22:41.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:40 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/409042749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:41.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:40 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/409042749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:41.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:40 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/409042749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:42.161 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:42.246 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:41 smithi170 ceph-mon[33632]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:42.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:41 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.12:6836/3490188582,v1:172.21.15.12:6837/3490188582] up:boot 2024-05-09T09:22:42.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:41 smithi170 ceph-mon[33632]: fsmap 4 up:standby 2024-05-09T09:22:42.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T09:22:42.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:42.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:42.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:42.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:42.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:42.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:41 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:42.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:41 smithi170 ceph-mon[33632]: Deploying daemon mds.e on smithi016 2024-05-09T09:22:42.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:41 smithi016 ceph-mon[33335]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:42.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:41 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.12:6836/3490188582,v1:172.21.15.12:6837/3490188582] up:boot 2024-05-09T09:22:42.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:41 smithi016 ceph-mon[33335]: fsmap 4 up:standby 2024-05-09T09:22:42.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T09:22:42.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:42.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:42.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:42.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:42.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:42.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:41 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:42.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:41 smithi016 ceph-mon[33335]: Deploying daemon mds.e on smithi016 2024-05-09T09:22:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.283+0000 7f9e58570640 1 -- 172.21.15.12:0/684870171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e50102a90 msgr2=0x7f9e50102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.283+0000 7f9e58570640 1 --2- 172.21.15.12:0/684870171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e50102a90 0x7f9e50102f10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009900 tx=0x7f9e4c02a410 comp rx=0 tx=0).stop 2024-05-09T09:22:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.283+0000 7f9e58570640 1 -- 172.21.15.12:0/684870171 shutdown_connections 2024-05-09T09:22:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.283+0000 7f9e58570640 1 --2- 172.21.15.12:0/684870171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e50103450 0x7f9e50109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.283+0000 7f9e58570640 1 --2- 172.21.15.12:0/684870171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e50102a90 0x7f9e50102f10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.283+0000 7f9e58570640 1 --2- 172.21.15.12:0/684870171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e501018a0 0x7f9e50101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.283+0000 7f9e58570640 1 -- 172.21.15.12:0/684870171 >> 172.21.15.12:0/684870171 conn(0x7f9e500fce30 msgr2=0x7f9e500ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.283+0000 7f9e58570640 1 -- 172.21.15.12:0/684870171 shutdown_connections 2024-05-09T09:22:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.283+0000 7f9e58570640 1 -- 172.21.15.12:0/684870171 wait complete. 2024-05-09T09:22:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e58570640 1 Processor -- start 2024-05-09T09:22:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e58570640 1 -- start start 2024-05-09T09:22:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e58570640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e501018a0 0x7f9e500755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e58570640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e50102a90 0x7f9e50073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e58570640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e50103450 0x7f9e50074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e58570640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5010d500 con 0x7f9e50102a90 2024-05-09T09:22:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e58570640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5010d380 con 0x7f9e50103450 2024-05-09T09:22:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e58570640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5010d680 con 0x7f9e501018a0 2024-05-09T09:22:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e55b0b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e50102a90 0x7f9e50073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e55b0b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e50102a90 0x7f9e50073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41982/0 (socket says 172.21.15.12:41982) 2024-05-09T09:22:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e55b0b640 1 -- 172.21.15.12:0/1474442024 learned_addr learned my addr 172.21.15.12:0/1474442024 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e56b0d640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e50103450 0x7f9e50074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e5630c640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e501018a0 0x7f9e500755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e55b0b640 1 -- 172.21.15.12:0/1474442024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e501018a0 msgr2=0x7f9e500755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e55b0b640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e501018a0 0x7f9e500755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e55b0b640 1 -- 172.21.15.12:0/1474442024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e50103450 msgr2=0x7f9e50074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e55b0b640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e50103450 0x7f9e50074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e55b0b640 1 -- 172.21.15.12:0/1474442024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e500749f0 con 0x7f9e50102a90 2024-05-09T09:22:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.284+0000 7f9e56b0d640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e50103450 0x7f9e50074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.285+0000 7f9e55b0b640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e50102a90 0x7f9e50073c20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009a20 tx=0x7f9e4c031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.285+0000 7f9e437fe640 1 -- 172.21.15.12:0/1474442024 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c03f030 con 0x7f9e50102a90 2024-05-09T09:22:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.285+0000 7f9e437fe640 1 -- 172.21.15.12:0/1474442024 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9e4c003ac0 con 0x7f9e50102a90 2024-05-09T09:22:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.285+0000 7f9e58570640 1 -- 172.21.15.12:0/1474442024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e501a1c60 con 0x7f9e50102a90 2024-05-09T09:22:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.285+0000 7f9e437fe640 1 -- 172.21.15.12:0/1474442024 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c035070 con 0x7f9e50102a90 2024-05-09T09:22:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.285+0000 7f9e58570640 1 -- 172.21.15.12:0/1474442024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e501a21b0 con 0x7f9e50102a90 2024-05-09T09:22:42.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.286+0000 7f9e58570640 1 -- 172.21.15.12:0/1474442024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e50108bc0 con 0x7f9e50102a90 2024-05-09T09:22:42.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.288+0000 7f9e437fe640 1 -- 172.21.15.12:0/1474442024 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9e4c042030 con 0x7f9e50102a90 2024-05-09T09:22:42.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.289+0000 7f9e437fe640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9e3c0a2b90 0x7f9e3c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:42.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.289+0000 7f9e5630c640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9e3c0a2b90 0x7f9e3c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:42.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.289+0000 7f9e437fe640 1 -- 172.21.15.12:0/1474442024 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9e4c0ef9b0 con 0x7f9e50102a90 2024-05-09T09:22:42.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.289+0000 7f9e5630c640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9e3c0a2b90 0x7f9e3c0a5050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e501028f0 tx=0x7f9e440098f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:42.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.292+0000 7f9e437fe640 1 -- 172.21.15.12:0/1474442024 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9e4c0bcd10 con 0x7f9e50102a90 2024-05-09T09:22:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:41 smithi012 ceph-mon[30140]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:41 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.12:6836/3490188582,v1:172.21.15.12:6837/3490188582] up:boot 2024-05-09T09:22:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:41 smithi012 ceph-mon[30140]: fsmap 4 up:standby 2024-05-09T09:22:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T09:22:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:42.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:41 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:42.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:41 smithi012 ceph-mon[30140]: Deploying daemon mds.e on smithi016 2024-05-09T09:22:42.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.517+0000 7f9e58570640 1 -- 172.21.15.12:0/1474442024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e5002d1c0 con 0x7f9e50102a90 2024-05-09T09:22:42.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.518+0000 7f9e437fe640 1 -- 172.21.15.12:0/1474442024 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f9e4c0bc610 con 0x7f9e50102a90 2024-05-09T09:22:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:42.518 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T09:22:42.518 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 5 2024-05-09T09:22:42.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.520+0000 7f9e58570640 1 -- 172.21.15.12:0/1474442024 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9e3c0a2b90 msgr2=0x7f9e3c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:42.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.520+0000 7f9e58570640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9e3c0a2b90 0x7f9e3c0a5050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e501028f0 tx=0x7f9e440098f0 comp rx=0 tx=0).stop 2024-05-09T09:22:42.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.520+0000 7f9e58570640 1 -- 172.21.15.12:0/1474442024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e50102a90 msgr2=0x7f9e50073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.520+0000 7f9e58570640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e50102a90 0x7f9e50073c20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009a20 tx=0x7f9e4c031ac0 comp rx=0 tx=0).stop 2024-05-09T09:22:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.520+0000 7f9e58570640 1 -- 172.21.15.12:0/1474442024 shutdown_connections 2024-05-09T09:22:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.520+0000 7f9e58570640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9e3c0a2b90 0x7f9e3c0a5050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.520+0000 7f9e58570640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e50103450 0x7f9e50074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.520+0000 7f9e58570640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e50102a90 0x7f9e50073c20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.520+0000 7f9e58570640 1 --2- 172.21.15.12:0/1474442024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e501018a0 0x7f9e500755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.520+0000 7f9e58570640 1 -- 172.21.15.12:0/1474442024 >> 172.21.15.12:0/1474442024 conn(0x7f9e500fce30 msgr2=0x7f9e50105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.520+0000 7f9e58570640 1 -- 172.21.15.12:0/1474442024 shutdown_connections 2024-05-09T09:22:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:42.520+0000 7f9e58570640 1 -- 172.21.15.12:0/1474442024 wait complete. 2024-05-09T09:22:43.213 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:42 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1474442024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:43.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:42 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1474442024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:43.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:42 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1474442024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:43.503 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:22:43 smithi016 systemd[1]: Starting Ceph mds.e for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:22:43.885 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:22:43 smithi016 podman[63383]: 2024-05-09 09:22:43.538441562 +0000 UTC m=+0.018980738 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:22:43.885 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:22:43 smithi016 podman[63383]: 2024-05-09 09:22:43.650307361 +0000 UTC m=+0.130846517 container create c2904ab19e3e8b0f272f1336553c4df10a5d156094ac92bcea0bfff143408229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-e, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-09T09:22:43.886 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:22:43 smithi016 podman[63383]: 2024-05-09 09:22:43.886807476 +0000 UTC m=+0.367346633 container init c2904ab19e3e8b0f272f1336553c4df10a5d156094ac92bcea0bfff143408229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:22:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:43 smithi016 ceph-mon[33335]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:44.253 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:22:43 smithi016 podman[63383]: 2024-05-09 09:22:43.889712564 +0000 UTC m=+0.370251730 container start c2904ab19e3e8b0f272f1336553c4df10a5d156094ac92bcea0bfff143408229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-05-09T09:22:44.253 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:22:43 smithi016 ceph-mds[63403]: -- 172.21.15.16:0/765080239 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c590579a40 con 0x55c59054c800 2024-05-09T09:22:44.253 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:22:43 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-e[63398]: starting mds.e at 2024-05-09T09:22:44.254 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:22:43 smithi016 bash[63383]: c2904ab19e3e8b0f272f1336553c4df10a5d156094ac92bcea0bfff143408229 2024-05-09T09:22:44.254 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:22:43 smithi016 systemd[1]: Started Ceph mds.e for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:22:44.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:43 smithi012 ceph-mon[30140]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:44.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:43 smithi170 ceph-mon[33632]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:44.531 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:44.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.650+0000 7f1818a38640 1 -- 172.21.15.12:0/1716762178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1814107570 msgr2=0x7f18141099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:44.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.650+0000 7f1818a38640 1 --2- 172.21.15.12:0/1716762178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1814107570 0x7f18141099c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f180c009ca0 tx=0x7f180c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:22:44.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.650+0000 7f1818a38640 1 -- 172.21.15.12:0/1716762178 shutdown_connections 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.650+0000 7f1818a38640 1 --2- 172.21.15.12:0/1716762178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1814107570 0x7f18141099c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.650+0000 7f1818a38640 1 --2- 172.21.15.12:0/1716762178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1814104c40 0x7f1814107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.650+0000 7f1818a38640 1 --2- 172.21.15.12:0/1716762178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1814102310 0x7f1814104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.650+0000 7f1818a38640 1 -- 172.21.15.12:0/1716762178 >> 172.21.15.12:0/1716762178 conn(0x7f18140fbd20 msgr2=0x7f18140fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.650+0000 7f1818a38640 1 -- 172.21.15.12:0/1716762178 shutdown_connections 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.650+0000 7f1818a38640 1 -- 172.21.15.12:0/1716762178 wait complete. 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.651+0000 7f1818a38640 1 Processor -- start 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.651+0000 7f1818a38640 1 -- start start 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.651+0000 7f1818a38640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1814102310 0x7f1814197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.651+0000 7f1818a38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1814104c40 0x7f18141980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.651+0000 7f1818a38640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1814107570 0x7f181419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.651+0000 7f1818a38640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f181410d560 con 0x7f1814102310 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.651+0000 7f1818a38640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f181410d3e0 con 0x7f1814104c40 2024-05-09T09:22:44.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.651+0000 7f1818a38640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f181410d6e0 con 0x7f1814107570 2024-05-09T09:22:44.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f181259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1814102310 0x7f1814197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:44.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f181259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1814102310 0x7f1814197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41996/0 (socket says 172.21.15.12:41996) 2024-05-09T09:22:44.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f181259c640 1 -- 172.21.15.12:0/2020319615 learned_addr learned my addr 172.21.15.12:0/2020319615 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:44.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f180bfff640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1814104c40 0x7f18141980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:44.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f1812d9d640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1814107570 0x7f181419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:44.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f181259c640 1 -- 172.21.15.12:0/2020319615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1814107570 msgr2=0x7f181419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:44.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f181259c640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1814107570 0x7f181419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:44.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f181259c640 1 -- 172.21.15.12:0/2020319615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1814104c40 msgr2=0x7f18141980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:44.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f181259c640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1814104c40 0x7f18141980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:44.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f181259c640 1 -- 172.21.15.12:0/2020319615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181419f7a0 con 0x7f1814102310 2024-05-09T09:22:44.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f181259c640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1814102310 0x7f1814197b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f180400e9f0 tx=0x7f180400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:44.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f180b7fe640 1 -- 172.21.15.12:0/2020319615 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180400cd80 con 0x7f1814102310 2024-05-09T09:22:44.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f180b7fe640 1 -- 172.21.15.12:0/2020319615 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f18040054f0 con 0x7f1814102310 2024-05-09T09:22:44.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f1818a38640 1 -- 172.21.15.12:0/2020319615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181419fa90 con 0x7f1814102310 2024-05-09T09:22:44.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f180b7fe640 1 -- 172.21.15.12:0/2020319615 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804010690 con 0x7f1814102310 2024-05-09T09:22:44.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.652+0000 7f1818a38640 1 -- 172.21.15.12:0/2020319615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181419ff90 con 0x7f1814102310 2024-05-09T09:22:44.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.653+0000 7f1818a38640 1 -- 172.21.15.12:0/2020319615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18140fd810 con 0x7f1814102310 2024-05-09T09:22:44.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.655+0000 7f180b7fe640 1 -- 172.21.15.12:0/2020319615 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1804004f20 con 0x7f1814102310 2024-05-09T09:22:44.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.655+0000 7f180b7fe640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f18000a2b90 0x7f18000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:44.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.655+0000 7f180bfff640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f18000a2b90 0x7f18000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:44.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.655+0000 7f180b7fe640 1 -- 172.21.15.12:0/2020319615 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f18040cef90 con 0x7f1814102310 2024-05-09T09:22:44.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.656+0000 7f180bfff640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f18000a2b90 0x7f18000a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1814199090 tx=0x7f17fc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:44.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.660+0000 7f180b7fe640 1 -- 172.21.15.12:0/2020319615 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f180409c390 con 0x7f1814102310 2024-05-09T09:22:44.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.882+0000 7f1818a38640 1 -- 172.21.15.12:0/2020319615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1814062430 con 0x7f1814102310 2024-05-09T09:22:44.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.883+0000 7f180b7fe640 1 -- 172.21.15.12:0/2020319615 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7f180409bc90 con 0x7f1814102310 2024-05-09T09:22:44.882 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:44.882 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-09T09:22:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 6 2024-05-09T09:22:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.884+0000 7f1818a38640 1 -- 172.21.15.12:0/2020319615 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f18000a2b90 msgr2=0x7f18000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.884+0000 7f1818a38640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f18000a2b90 0x7f18000a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1814199090 tx=0x7f17fc00b410 comp rx=0 tx=0).stop 2024-05-09T09:22:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.884+0000 7f1818a38640 1 -- 172.21.15.12:0/2020319615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1814102310 msgr2=0x7f1814197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.884+0000 7f1818a38640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1814102310 0x7f1814197b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f180400e9f0 tx=0x7f180400edf0 comp rx=0 tx=0).stop 2024-05-09T09:22:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.885+0000 7f1818a38640 1 -- 172.21.15.12:0/2020319615 shutdown_connections 2024-05-09T09:22:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.885+0000 7f1818a38640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f18000a2b90 0x7f18000a5050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.885+0000 7f1818a38640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1814107570 0x7f181419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.885+0000 7f1818a38640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1814104c40 0x7f18141980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.885+0000 7f1818a38640 1 --2- 172.21.15.12:0/2020319615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1814102310 0x7f1814197b80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.885+0000 7f1818a38640 1 -- 172.21.15.12:0/2020319615 >> 172.21.15.12:0/2020319615 conn(0x7f18140fbd20 msgr2=0x7f1814102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.885+0000 7f1818a38640 1 -- 172.21.15.12:0/2020319615 shutdown_connections 2024-05-09T09:22:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:44.885+0000 7f1818a38640 1 -- 172.21.15.12:0/2020319615 wait complete. 2024-05-09T09:22:45.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:45 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.16:6834/1098165328,v1:172.21.15.16:6835/1098165328] up:boot 2024-05-09T09:22:45.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:45 smithi012 ceph-mon[30140]: fsmap 5 up:standby 2024-05-09T09:22:45.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T09:22:45.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:45.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:45.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:45.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:45.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:45.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:45.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:45 smithi012 ceph-mon[30140]: Deploying daemon mds.f on smithi170 2024-05-09T09:22:45.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:45 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2020319615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:45.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:45 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.16:6834/1098165328,v1:172.21.15.16:6835/1098165328] up:boot 2024-05-09T09:22:45.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:45 smithi016 ceph-mon[33335]: fsmap 5 up:standby 2024-05-09T09:22:45.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T09:22:45.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:45.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:45.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:45.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:45.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:45.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:45.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:45 smithi016 ceph-mon[33335]: Deploying daemon mds.f on smithi170 2024-05-09T09:22:45.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:45 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2020319615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:45.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:45 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.16:6834/1098165328,v1:172.21.15.16:6835/1098165328] up:boot 2024-05-09T09:22:45.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:45 smithi170 ceph-mon[33632]: fsmap 5 up:standby 2024-05-09T09:22:45.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T09:22:45.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:45.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:45.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:45.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:45.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:45.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:45.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:45 smithi170 ceph-mon[33632]: Deploying daemon mds.f on smithi170 2024-05-09T09:22:45.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:45 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2020319615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:46.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:46 smithi170 ceph-mon[33632]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:46.243 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:22:46 smithi170 systemd[1]: Starting Ceph mds.f for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:22:46.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:46 smithi012 ceph-mon[30140]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:46.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:46 smithi016 ceph-mon[33335]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:46.597 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:22:46 smithi170 podman[64234]: 2024-05-09 09:22:46.221986977 +0000 UTC m=+0.018495202 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:22:46.597 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:22:46 smithi170 podman[64234]: 2024-05-09 09:22:46.353411377 +0000 UTC m=+0.149919592 container create d55293678f8990ff47ea365f0398ebcb339abdda236120f1cade439c01602a16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-05-09T09:22:46.597 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:22:46 smithi170 podman[64234]: 2024-05-09 09:22:46.597951451 +0000 UTC m=+0.394459663 container init d55293678f8990ff47ea365f0398ebcb339abdda236120f1cade439c01602a16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-f, io.buildah.version=1.33.5, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-09T09:22:46.896 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:46.993 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:22:46 smithi170 podman[64234]: 2024-05-09 09:22:46.601251459 +0000 UTC m=+0.397759671 container start d55293678f8990ff47ea365f0398ebcb339abdda236120f1cade439c01602a16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:22:46.993 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:22:46 smithi170 ceph-mds[64261]: -- 172.21.15.170:0/4254272552 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55930444fa40 con 0x559304422c00 2024-05-09T09:22:46.993 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:22:46 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-f[64252]: starting mds.f at 2024-05-09T09:22:46.993 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:22:46 smithi170 bash[64234]: d55293678f8990ff47ea365f0398ebcb339abdda236120f1cade439c01602a16 2024-05-09T09:22:46.993 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:22:46 smithi170 systemd[1]: Started Ceph mds.f for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:22:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.013+0000 7f391cf14640 1 -- 172.21.15.12:0/2961653952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3918103450 msgr2=0x7f3918109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.013+0000 7f391cf14640 1 --2- 172.21.15.12:0/2961653952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3918103450 0x7f3918109ce0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3910009ca0 tx=0x7f391002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:22:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.013+0000 7f391cf14640 1 -- 172.21.15.12:0/2961653952 shutdown_connections 2024-05-09T09:22:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.013+0000 7f391cf14640 1 --2- 172.21.15.12:0/2961653952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3918103450 0x7f3918109ce0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.013+0000 7f391cf14640 1 --2- 172.21.15.12:0/2961653952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3918102a90 0x7f3918102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.013+0000 7f391cf14640 1 --2- 172.21.15.12:0/2961653952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39181018a0 0x7f3918101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.013+0000 7f391cf14640 1 -- 172.21.15.12:0/2961653952 >> 172.21.15.12:0/2961653952 conn(0x7f39180fce30 msgr2=0x7f39180ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.013+0000 7f391cf14640 1 -- 172.21.15.12:0/2961653952 shutdown_connections 2024-05-09T09:22:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.013+0000 7f391cf14640 1 -- 172.21.15.12:0/2961653952 wait complete. 2024-05-09T09:22:47.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.014+0000 7f391cf14640 1 Processor -- start 2024-05-09T09:22:47.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.014+0000 7f391cf14640 1 -- start start 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.014+0000 7f391cf14640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39181018a0 0x7f3918197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.014+0000 7f391cf14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3918102a90 0x7f3918198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.014+0000 7f391cf14640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3918103450 0x7f391819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.014+0000 7f391cf14640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f391810d500 con 0x7f3918103450 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.014+0000 7f391cf14640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f391810d380 con 0x7f3918102a90 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.014+0000 7f391cf14640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f391810d680 con 0x7f39181018a0 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.014+0000 7f3916d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3918103450 0x7f391819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.014+0000 7f3916d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3918103450 0x7f391819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42014/0 (socket says 172.21.15.12:42014) 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.014+0000 7f3916d9d640 1 -- 172.21.15.12:0/2785444432 learned_addr learned my addr 172.21.15.12:0/2785444432 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.014+0000 7f391659c640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39181018a0 0x7f3918197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.014+0000 7f3915d9b640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3918102a90 0x7f3918198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.015+0000 7f3916d9d640 1 -- 172.21.15.12:0/2785444432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39181018a0 msgr2=0x7f3918197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.015+0000 7f3916d9d640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39181018a0 0x7f3918197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:47.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.015+0000 7f3916d9d640 1 -- 172.21.15.12:0/2785444432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3918102a90 msgr2=0x7f3918198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:47.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.015+0000 7f3916d9d640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3918102a90 0x7f3918198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:47.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.015+0000 7f3916d9d640 1 -- 172.21.15.12:0/2785444432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391819f770 con 0x7f3918103450 2024-05-09T09:22:47.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.015+0000 7f3916d9d640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3918103450 0x7f391819f100 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3910009430 tx=0x7f391002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:47.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.015+0000 7f390f7fe640 1 -- 172.21.15.12:0/2785444432 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910003bf0 con 0x7f3918103450 2024-05-09T09:22:47.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.015+0000 7f390f7fe640 1 -- 172.21.15.12:0/2785444432 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3910003d90 con 0x7f3918103450 2024-05-09T09:22:47.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.015+0000 7f390f7fe640 1 -- 172.21.15.12:0/2785444432 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391002fc90 con 0x7f3918103450 2024-05-09T09:22:47.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.015+0000 7f391cf14640 1 -- 172.21.15.12:0/2785444432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391819fa00 con 0x7f3918103450 2024-05-09T09:22:47.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.015+0000 7f391cf14640 1 -- 172.21.15.12:0/2785444432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391819ffa0 con 0x7f3918103450 2024-05-09T09:22:47.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.016+0000 7f391cf14640 1 -- 172.21.15.12:0/2785444432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3918108bc0 con 0x7f3918103450 2024-05-09T09:22:47.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.017+0000 7f390f7fe640 1 -- 172.21.15.12:0/2785444432 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3910004040 con 0x7f3918103450 2024-05-09T09:22:47.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.018+0000 7f390f7fe640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f39040a2c10 0x7f39040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:47.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.018+0000 7f391659c640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f39040a2c10 0x7f39040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:47.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.018+0000 7f390f7fe640 1 -- 172.21.15.12:0/2785444432 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f39100f0180 con 0x7f3918103450 2024-05-09T09:22:47.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.019+0000 7f391659c640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f39040a2c10 0x7f39040a50d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3908005fd0 tx=0x7f390800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:47.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.022+0000 7f390f7fe640 1 -- 172.21.15.12:0/2785444432 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f39100bd610 con 0x7f3918103450 2024-05-09T09:22:47.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.247+0000 7f391cf14640 1 -- 172.21.15.12:0/2785444432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f391802d1c0 con 0x7f3918103450 2024-05-09T09:22:47.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.247+0000 7f390f7fe640 1 -- 172.21.15.12:0/2785444432 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f39100bcf10 con 0x7f3918103450 2024-05-09T09:22:47.247 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:47.247 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-05-09T09:22:47.248 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 7 2024-05-09T09:22:47.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.250+0000 7f391cf14640 1 -- 172.21.15.12:0/2785444432 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f39040a2c10 msgr2=0x7f39040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.250+0000 7f391cf14640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f39040a2c10 0x7f39040a50d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3908005fd0 tx=0x7f390800b410 comp rx=0 tx=0).stop 2024-05-09T09:22:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.250+0000 7f391cf14640 1 -- 172.21.15.12:0/2785444432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3918103450 msgr2=0x7f391819f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.250+0000 7f391cf14640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3918103450 0x7f391819f100 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3910009430 tx=0x7f391002f380 comp rx=0 tx=0).stop 2024-05-09T09:22:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.250+0000 7f391cf14640 1 -- 172.21.15.12:0/2785444432 shutdown_connections 2024-05-09T09:22:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.250+0000 7f391cf14640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f39040a2c10 0x7f39040a50d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.250+0000 7f391cf14640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3918103450 0x7f391819f100 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.250+0000 7f391cf14640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3918102a90 0x7f3918198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.250+0000 7f391cf14640 1 --2- 172.21.15.12:0/2785444432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39181018a0 0x7f3918197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.250+0000 7f391cf14640 1 -- 172.21.15.12:0/2785444432 >> 172.21.15.12:0/2785444432 conn(0x7f39180fce30 msgr2=0x7f3918105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.250+0000 7f391cf14640 1 -- 172.21.15.12:0/2785444432 shutdown_connections 2024-05-09T09:22:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:47.250+0000 7f391cf14640 1 -- 172.21.15.12:0/2785444432 wait complete. 2024-05-09T09:22:47.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:47 smithi170 ceph-mon[33632]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:47.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:47.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:47.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:47.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:47.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:47.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:47.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:47 smithi170 ceph-mon[33632]: Deploying daemon mds.c on smithi170 2024-05-09T09:22:47.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:47 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] up:boot 2024-05-09T09:22:47.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:47 smithi170 ceph-mon[33632]: fsmap 6 up:standby 2024-05-09T09:22:47.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T09:22:47.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:47 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2785444432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:48.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:47 smithi016 ceph-mon[33335]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:48.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:48.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:48.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:48.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:48.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:48.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:48.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:47 smithi016 ceph-mon[33335]: Deploying daemon mds.c on smithi170 2024-05-09T09:22:48.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:47 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] up:boot 2024-05-09T09:22:48.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:47 smithi016 ceph-mon[33335]: fsmap 6 up:standby 2024-05-09T09:22:48.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T09:22:48.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:47 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2785444432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:48.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:47 smithi012 ceph-mon[30140]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:48.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:48.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:48.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:48.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:48.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:48.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:48.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:47 smithi012 ceph-mon[30140]: Deploying daemon mds.c on smithi170 2024-05-09T09:22:48.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:47 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] up:boot 2024-05-09T09:22:48.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:47 smithi012 ceph-mon[30140]: fsmap 6 up:standby 2024-05-09T09:22:48.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T09:22:48.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:47 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2785444432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:49.169 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:22:48 smithi170 systemd[1]: Starting Ceph mds.c for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:22:49.261 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:49.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.380+0000 7f1f7d914640 1 -- 172.21.15.12:0/49351591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f78102a90 msgr2=0x7f1f78102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:49.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.380+0000 7f1f7d914640 1 --2- 172.21.15.12:0/49351591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f78102a90 0x7f1f78102f10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009900 tx=0x7f1f6002a410 comp rx=0 tx=0).stop 2024-05-09T09:22:49.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.380+0000 7f1f7d914640 1 -- 172.21.15.12:0/49351591 shutdown_connections 2024-05-09T09:22:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.380+0000 7f1f7d914640 1 --2- 172.21.15.12:0/49351591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f78103450 0x7f1f78109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.380+0000 7f1f7d914640 1 --2- 172.21.15.12:0/49351591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f78102a90 0x7f1f78102f10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.380+0000 7f1f7d914640 1 --2- 172.21.15.12:0/49351591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f781018a0 0x7f1f78101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.380+0000 7f1f7d914640 1 -- 172.21.15.12:0/49351591 >> 172.21.15.12:0/49351591 conn(0x7f1f780fce30 msgr2=0x7f1f780ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.380+0000 7f1f7d914640 1 -- 172.21.15.12:0/49351591 shutdown_connections 2024-05-09T09:22:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.380+0000 7f1f7d914640 1 -- 172.21.15.12:0/49351591 wait complete. 2024-05-09T09:22:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.381+0000 7f1f7d914640 1 Processor -- start 2024-05-09T09:22:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.381+0000 7f1f7d914640 1 -- start start 2024-05-09T09:22:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.381+0000 7f1f7d914640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f781018a0 0x7f1f780755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.381+0000 7f1f7d914640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f78102a90 0x7f1f78073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.381+0000 7f1f7d914640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f78103450 0x7f1f78074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.381+0000 7f1f7d914640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7810d550 con 0x7f1f78102a90 2024-05-09T09:22:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.381+0000 7f1f7d914640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7810d3d0 con 0x7f1f781018a0 2024-05-09T09:22:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.381+0000 7f1f7d914640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7810d6d0 con 0x7f1f78103450 2024-05-09T09:22:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.381+0000 7f1f6e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f78102a90 0x7f1f78073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.381+0000 7f1f6e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f78102a90 0x7f1f78073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42026/0 (socket says 172.21.15.12:42026) 2024-05-09T09:22:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.381+0000 7f1f6e7fc640 1 -- 172.21.15.12:0/2807151665 learned_addr learned my addr 172.21.15.12:0/2807151665 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.381+0000 7f1f777fe640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f78103450 0x7f1f78074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.382+0000 7f1f76ffd640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f781018a0 0x7f1f780755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.382+0000 7f1f6e7fc640 1 -- 172.21.15.12:0/2807151665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f78103450 msgr2=0x7f1f78074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.382+0000 7f1f6e7fc640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f78103450 0x7f1f78074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.382+0000 7f1f6e7fc640 1 -- 172.21.15.12:0/2807151665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f781018a0 msgr2=0x7f1f780755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.382+0000 7f1f6e7fc640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f781018a0 0x7f1f780755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:49.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.382+0000 7f1f6e7fc640 1 -- 172.21.15.12:0/2807151665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f780749f0 con 0x7f1f78102a90 2024-05-09T09:22:49.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.382+0000 7f1f6e7fc640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f78102a90 0x7f1f78073c20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1f6002a930 tx=0x7f1f60004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:49.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.382+0000 7f1f7c912640 1 -- 172.21.15.12:0/2807151665 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6003f070 con 0x7f1f78102a90 2024-05-09T09:22:49.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.382+0000 7f1f7c912640 1 -- 172.21.15.12:0/2807151665 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1f60005530 con 0x7f1f78102a90 2024-05-09T09:22:49.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.382+0000 7f1f7d914640 1 -- 172.21.15.12:0/2807151665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f781a1c60 con 0x7f1f78102a90 2024-05-09T09:22:49.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.382+0000 7f1f7c912640 1 -- 172.21.15.12:0/2807151665 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60035030 con 0x7f1f78102a90 2024-05-09T09:22:49.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.382+0000 7f1f7d914640 1 -- 172.21.15.12:0/2807151665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f781a21b0 con 0x7f1f78102a90 2024-05-09T09:22:49.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.383+0000 7f1f7d914640 1 -- 172.21.15.12:0/2807151665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f78108b60 con 0x7f1f78102a90 2024-05-09T09:22:49.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.385+0000 7f1f7c912640 1 -- 172.21.15.12:0/2807151665 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1f60042070 con 0x7f1f78102a90 2024-05-09T09:22:49.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.385+0000 7f1f7c912640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1f640a2c10 0x7f1f640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:49.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.385+0000 7f1f76ffd640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1f640a2c10 0x7f1f640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:49.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.385+0000 7f1f7c912640 1 -- 172.21.15.12:0/2807151665 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1f600ef730 con 0x7f1f78102a90 2024-05-09T09:22:49.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.386+0000 7f1f76ffd640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1f640a2c10 0x7f1f640a50d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1f68005fd0 tx=0x7f1f6800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:49.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.389+0000 7f1f7c912640 1 -- 172.21.15.12:0/2807151665 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1f600bc4e0 con 0x7f1f78102a90 2024-05-09T09:22:49.436 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:22:49 smithi170 podman[64553]: 2024-05-09 09:22:49.072161292 +0000 UTC m=+0.018150027 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:22:49.436 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:22:49 smithi170 podman[64553]: 2024-05-09 09:22:49.192621161 +0000 UTC m=+0.138609877 container create 552f76904383249cd56aa58991f00dde4b771bb171bb877655c4997fd08046aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-96805dc) 2024-05-09T09:22:49.436 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:22:49 smithi170 podman[64553]: 2024-05-09 09:22:49.437095299 +0000 UTC m=+0.383084016 container init 552f76904383249cd56aa58991f00dde4b771bb171bb877655c4997fd08046aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-09T09:22:49.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.616+0000 7f1f7d914640 1 -- 172.21.15.12:0/2807151665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f7802d1c0 con 0x7f1f78102a90 2024-05-09T09:22:49.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.616+0000 7f1f7c912640 1 -- 172.21.15.12:0/2807151665 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f1f600c07e0 con 0x7f1f78102a90 2024-05-09T09:22:49.616 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:49.616 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-05-09T09:22:49.616 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 7 2024-05-09T09:22:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.619+0000 7f1f7d914640 1 -- 172.21.15.12:0/2807151665 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1f640a2c10 msgr2=0x7f1f640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.619+0000 7f1f7d914640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1f640a2c10 0x7f1f640a50d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1f68005fd0 tx=0x7f1f6800b390 comp rx=0 tx=0).stop 2024-05-09T09:22:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.619+0000 7f1f7d914640 1 -- 172.21.15.12:0/2807151665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f78102a90 msgr2=0x7f1f78073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.619+0000 7f1f7d914640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f78102a90 0x7f1f78073c20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1f6002a930 tx=0x7f1f60004f70 comp rx=0 tx=0).stop 2024-05-09T09:22:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.619+0000 7f1f7d914640 1 -- 172.21.15.12:0/2807151665 shutdown_connections 2024-05-09T09:22:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.619+0000 7f1f7d914640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1f640a2c10 0x7f1f640a50d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.619+0000 7f1f7d914640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f78103450 0x7f1f78074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.619+0000 7f1f7d914640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f78102a90 0x7f1f78073c20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.619+0000 7f1f7d914640 1 --2- 172.21.15.12:0/2807151665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f781018a0 0x7f1f780755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.619+0000 7f1f7d914640 1 -- 172.21.15.12:0/2807151665 >> 172.21.15.12:0/2807151665 conn(0x7f1f780fce30 msgr2=0x7f1f78105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.619+0000 7f1f7d914640 1 -- 172.21.15.12:0/2807151665 shutdown_connections 2024-05-09T09:22:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:49.619+0000 7f1f7d914640 1 -- 172.21.15.12:0/2807151665 wait complete. 2024-05-09T09:22:49.743 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:22:49 smithi170 podman[64553]: 2024-05-09 09:22:49.440084692 +0000 UTC m=+0.386073408 container start 552f76904383249cd56aa58991f00dde4b771bb171bb877655c4997fd08046aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc) 2024-05-09T09:22:49.743 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:22:49 smithi170 ceph-mds[64572]: -- 172.21.15.170:0/3717289078 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564cff895a40 con 0x564cff869000 2024-05-09T09:22:49.743 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:22:49 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-c[64568]: starting mds.c at 2024-05-09T09:22:49.743 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:22:49 smithi170 bash[64553]: 552f76904383249cd56aa58991f00dde4b771bb171bb877655c4997fd08046aa 2024-05-09T09:22:49.744 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:22:49 smithi170 systemd[1]: Started Ceph mds.c for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:22:49.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:49 smithi012 ceph-mon[30140]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:49.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:49 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:49.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:49 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:49.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:49 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2807151665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:49.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:49 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:49.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:49 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:49.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:49 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:49.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:49 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:50.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:49 smithi170 ceph-mon[33632]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:50.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:50.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:50.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:49 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2807151665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:50.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:50.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:50.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:50.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:50.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:49 smithi016 ceph-mon[33335]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:50.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:49 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:50.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:49 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:50.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:49 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2807151665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:50.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:49 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:50.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:49 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:50.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:49 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:50.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:49 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:51.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:50 smithi016 ceph-mon[33335]: Deploying daemon mds.h on smithi016 2024-05-09T09:22:51.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:50 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] up:boot 2024-05-09T09:22:51.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:50 smithi016 ceph-mon[33335]: fsmap 7 up:standby 2024-05-09T09:22:51.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:50 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T09:22:51.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:50 smithi012 ceph-mon[30140]: Deploying daemon mds.h on smithi016 2024-05-09T09:22:51.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:50 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] up:boot 2024-05-09T09:22:51.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:50 smithi012 ceph-mon[30140]: fsmap 7 up:standby 2024-05-09T09:22:51.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:50 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T09:22:51.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:50 smithi170 ceph-mon[33632]: Deploying daemon mds.h on smithi016 2024-05-09T09:22:51.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:50 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] up:boot 2024-05-09T09:22:51.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:50 smithi170 ceph-mon[33632]: fsmap 7 up:standby 2024-05-09T09:22:51.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:50 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T09:22:51.631 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.975+0000 7f686adee640 1 -- 172.21.15.12:0/3857141453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6864101960 msgr2=0x7f6864101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.975+0000 7f686adee640 1 --2- 172.21.15.12:0/3857141453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6864101960 0x7f6864101de0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6858009900 tx=0x7f685802a410 comp rx=0 tx=0).stop 2024-05-09T09:22:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.975+0000 7f686adee640 1 -- 172.21.15.12:0/3857141453 shutdown_connections 2024-05-09T09:22:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.975+0000 7f686adee640 1 --2- 172.21.15.12:0/3857141453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6864102320 0x7f6864108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.975+0000 7f686adee640 1 --2- 172.21.15.12:0/3857141453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6864101960 0x7f6864101de0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.975+0000 7f686adee640 1 --2- 172.21.15.12:0/3857141453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6864100770 0x7f6864100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.975+0000 7f686adee640 1 -- 172.21.15.12:0/3857141453 >> 172.21.15.12:0/3857141453 conn(0x7f68640fbce0 msgr2=0x7f68640fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.975+0000 7f686adee640 1 -- 172.21.15.12:0/3857141453 shutdown_connections 2024-05-09T09:22:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.975+0000 7f686adee640 1 -- 172.21.15.12:0/3857141453 wait complete. 2024-05-09T09:22:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.975+0000 7f686adee640 1 Processor -- start 2024-05-09T09:22:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.975+0000 7f686adee640 1 -- start start 2024-05-09T09:22:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f686adee640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6864100770 0x7f6864197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f686adee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6864101960 0x7f6864198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f686adee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6864102320 0x7f686419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f686adee640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f686410d520 con 0x7f6864100770 2024-05-09T09:22:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f686adee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f686410d3a0 con 0x7f6864102320 2024-05-09T09:22:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f686adee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f686410d6a0 con 0x7f6864101960 2024-05-09T09:22:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f6868b8a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6864100770 0x7f6864197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f6868b8a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6864100770 0x7f6864197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41254/0 (socket says 172.21.15.12:41254) 2024-05-09T09:22:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f6868b8a640 1 -- 172.21.15.12:0/3928957701 learned_addr learned my addr 172.21.15.12:0/3928957701 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f686938b640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6864102320 0x7f686419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f6863fff640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6864101960 0x7f6864198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f6868b8a640 1 -- 172.21.15.12:0/3928957701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6864101960 msgr2=0x7f6864198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f6868b8a640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6864101960 0x7f6864198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f6868b8a640 1 -- 172.21.15.12:0/3928957701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6864102320 msgr2=0x7f686419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f6868b8a640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6864102320 0x7f686419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f6868b8a640 1 -- 172.21.15.12:0/3928957701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686419f7e0 con 0x7f6864100770 2024-05-09T09:22:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f686938b640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6864102320 0x7f686419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:22:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f6868b8a640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6864100770 0x7f6864197bc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f685400d950 tx=0x7f685400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f6861ffb640 1 -- 172.21.15.12:0/3928957701 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854017070 con 0x7f6864100770 2024-05-09T09:22:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f6861ffb640 1 -- 172.21.15.12:0/3928957701 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6854009c30 con 0x7f6864100770 2024-05-09T09:22:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.976+0000 7f686adee640 1 -- 172.21.15.12:0/3928957701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686419fad0 con 0x7f6864100770 2024-05-09T09:22:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.977+0000 7f6861ffb640 1 -- 172.21.15.12:0/3928957701 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685400f620 con 0x7f6864100770 2024-05-09T09:22:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.977+0000 7f686adee640 1 -- 172.21.15.12:0/3928957701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686419ffa0 con 0x7f6864100770 2024-05-09T09:22:51.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.978+0000 7f686adee640 1 -- 172.21.15.12:0/3928957701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6864107a90 con 0x7f6864100770 2024-05-09T09:22:51.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.979+0000 7f6861ffb640 1 -- 172.21.15.12:0/3928957701 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f685401a070 con 0x7f6864100770 2024-05-09T09:22:51.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.980+0000 7f6861ffb640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f684c0a2b90 0x7f684c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:51.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.980+0000 7f6863fff640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f684c0a2b90 0x7f684c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:51.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.980+0000 7f6861ffb640 1 -- 172.21.15.12:0/3928957701 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f68540ce5f0 con 0x7f6864100770 2024-05-09T09:22:51.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.980+0000 7f6863fff640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f684c0a2b90 0x7f684c0a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6858006010 tx=0x7f6858031530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:51.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:51.984+0000 7f6861ffb640 1 -- 172.21.15.12:0/3928957701 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6854009de0 con 0x7f6864100770 2024-05-09T09:22:52.192 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:51 smithi016 ceph-mon[33335]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:52.193 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:22:51 smithi016 systemd[1]: Starting Ceph mds.h for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:22:52.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.212+0000 7f686adee640 1 -- 172.21.15.12:0/3928957701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6864062460 con 0x7f6864100770 2024-05-09T09:22:52.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.212+0000 7f6861ffb640 1 -- 172.21.15.12:0/3928957701 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f685409b300 con 0x7f6864100770 2024-05-09T09:22:52.212 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:52.212 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-09T09:22:52.213 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-05-09T09:22:52.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.215+0000 7f686adee640 1 -- 172.21.15.12:0/3928957701 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f684c0a2b90 msgr2=0x7f684c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:52.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.215+0000 7f686adee640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f684c0a2b90 0x7f684c0a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6858006010 tx=0x7f6858031530 comp rx=0 tx=0).stop 2024-05-09T09:22:52.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.215+0000 7f686adee640 1 -- 172.21.15.12:0/3928957701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6864100770 msgr2=0x7f6864197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:52.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.215+0000 7f686adee640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6864100770 0x7f6864197bc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f685400d950 tx=0x7f685400de10 comp rx=0 tx=0).stop 2024-05-09T09:22:52.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.215+0000 7f686adee640 1 -- 172.21.15.12:0/3928957701 shutdown_connections 2024-05-09T09:22:52.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.215+0000 7f686adee640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f684c0a2b90 0x7f684c0a5050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:52.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.215+0000 7f686adee640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6864102320 0x7f686419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:52.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.215+0000 7f686adee640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6864101960 0x7f6864198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:52.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.215+0000 7f686adee640 1 --2- 172.21.15.12:0/3928957701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6864100770 0x7f6864197bc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:52.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.215+0000 7f686adee640 1 -- 172.21.15.12:0/3928957701 >> 172.21.15.12:0/3928957701 conn(0x7f68640fbce0 msgr2=0x7f6864104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:52.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.215+0000 7f686adee640 1 -- 172.21.15.12:0/3928957701 shutdown_connections 2024-05-09T09:22:52.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:52.215+0000 7f686adee640 1 -- 172.21.15.12:0/3928957701 wait complete. 2024-05-09T09:22:52.224 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:51 smithi012 ceph-mon[30140]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:52.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:51 smithi170 ceph-mon[33632]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:52.467 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:22:52 smithi016 podman[63697]: 2024-05-09 09:22:52.096514243 +0000 UTC m=+0.018366484 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:22:52.467 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:22:52 smithi016 podman[63697]: 2024-05-09 09:22:52.214793498 +0000 UTC m=+0.136645783 container create 881bf909381e7c4c7e76b79f821364d0e6f47466f2b8be5dd3138d8b78c7815b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-05-09T09:22:52.467 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:22:52 smithi016 podman[63697]: 2024-05-09 09:22:52.467529054 +0000 UTC m=+0.389381297 container init 881bf909381e7c4c7e76b79f821364d0e6f47466f2b8be5dd3138d8b78c7815b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, ceph=True, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-09T09:22:52.753 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:22:52 smithi016 podman[63697]: 2024-05-09 09:22:52.471033089 +0000 UTC m=+0.392885338 container start 881bf909381e7c4c7e76b79f821364d0e6f47466f2b8be5dd3138d8b78c7815b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-09T09:22:52.753 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:22:52 smithi016 ceph-mds[63717]: -- 172.21.15.16:0/223149247 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cec96b3a40 con 0x55cec9686c00 2024-05-09T09:22:52.753 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:22:52 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h[63713]: starting mds.h at 2024-05-09T09:22:52.753 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:22:52 smithi016 bash[63697]: 881bf909381e7c4c7e76b79f821364d0e6f47466f2b8be5dd3138d8b78c7815b 2024-05-09T09:22:52.753 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:22:52 smithi016 systemd[1]: Started Ceph mds.h for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:22:53.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:52 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3928957701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:53.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:52 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:53.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:52 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:53.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:52 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:53.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:52 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:53.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:52 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:53.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:52 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:53.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:52 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3928957701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:53.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:52 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:53.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:52 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:53.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:52 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:53.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:52 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:53.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:52 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:53.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:52 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:53.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:52 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3928957701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:53.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:52 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:53.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:52 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:53.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:52 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:53.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:52 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:53.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:52 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:53.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:52 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:53.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:53 smithi170 ceph-mon[33632]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:53.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:53 smithi170 ceph-mon[33632]: Deploying daemon mds.i on smithi170 2024-05-09T09:22:53.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:53 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.16:6836/3413591353,v1:172.21.15.16:6837/3413591353] up:boot 2024-05-09T09:22:53.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:53 smithi170 ceph-mon[33632]: fsmap 8 up:standby 2024-05-09T09:22:53.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:53 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:22:54.227 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:54.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:53 smithi012 ceph-mon[30140]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:54.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:53 smithi012 ceph-mon[30140]: Deploying daemon mds.i on smithi170 2024-05-09T09:22:54.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:53 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.16:6836/3413591353,v1:172.21.15.16:6837/3413591353] up:boot 2024-05-09T09:22:54.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:53 smithi012 ceph-mon[30140]: fsmap 8 up:standby 2024-05-09T09:22:54.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:53 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:22:54.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:53 smithi016 ceph-mon[33335]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:54.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:53 smithi016 ceph-mon[33335]: Deploying daemon mds.i on smithi170 2024-05-09T09:22:54.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:53 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.16:6836/3413591353,v1:172.21.15.16:6837/3413591353] up:boot 2024-05-09T09:22:54.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:53 smithi016 ceph-mon[33335]: fsmap 8 up:standby 2024-05-09T09:22:54.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:53 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:22:54.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.346+0000 7fe60a161640 1 -- 172.21.15.12:0/4192715921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe604107570 msgr2=0x7fe6041099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:54.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.346+0000 7fe60a161640 1 --2- 172.21.15.12:0/4192715921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe604107570 0x7fe6041099c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009ca0 tx=0x7fe5fc02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:22:54.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.346+0000 7fe60a161640 1 -- 172.21.15.12:0/4192715921 shutdown_connections 2024-05-09T09:22:54.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.346+0000 7fe60a161640 1 --2- 172.21.15.12:0/4192715921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe604107570 0x7fe6041099c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:54.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.346+0000 7fe60a161640 1 --2- 172.21.15.12:0/4192715921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe604104c40 0x7fe604107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:54.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.346+0000 7fe60a161640 1 --2- 172.21.15.12:0/4192715921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe604102310 0x7fe604104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:54.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.346+0000 7fe60a161640 1 -- 172.21.15.12:0/4192715921 >> 172.21.15.12:0/4192715921 conn(0x7fe6040fbd20 msgr2=0x7fe6040fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:54.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.346+0000 7fe60a161640 1 -- 172.21.15.12:0/4192715921 shutdown_connections 2024-05-09T09:22:54.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.346+0000 7fe60a161640 1 -- 172.21.15.12:0/4192715921 wait complete. 2024-05-09T09:22:54.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.346+0000 7fe60a161640 1 Processor -- start 2024-05-09T09:22:54.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe60a161640 1 -- start start 2024-05-09T09:22:54.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe60a161640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe604102310 0x7fe604197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:54.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe60a161640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe604104c40 0x7fe6041980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:54.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe60a161640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe604107570 0x7fe60419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:54.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe60a161640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe60410d510 con 0x7fe604102310 2024-05-09T09:22:54.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe60a161640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe60410d390 con 0x7fe604104c40 2024-05-09T09:22:54.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe60a161640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe60410d690 con 0x7fe604107570 2024-05-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe6037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe604102310 0x7fe604197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe6037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe604102310 0x7fe604197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41258/0 (socket says 172.21.15.12:41258) 2024-05-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe6037fe640 1 -- 172.21.15.12:0/198931109 learned_addr learned my addr 172.21.15.12:0/198931109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe603fff640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe604107570 0x7fe60419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe6037fe640 1 -- 172.21.15.12:0/198931109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe604107570 msgr2=0x7fe60419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe6037fe640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe604107570 0x7fe60419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe602ffd640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe604104c40 0x7fe6041980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe6037fe640 1 -- 172.21.15.12:0/198931109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe604104c40 msgr2=0x7fe6041980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe6037fe640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe604104c40 0x7fe6041980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe6037fe640 1 -- 172.21.15.12:0/198931109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60419f7a0 con 0x7fe604102310 2024-05-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe602ffd640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe604104c40 0x7fe6041980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.347+0000 7fe6037fe640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe604102310 0x7fe604197b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400d950 tx=0x7fe5f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.348+0000 7fe600ff9640 1 -- 172.21.15.12:0/198931109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4017070 con 0x7fe604102310 2024-05-09T09:22:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.348+0000 7fe60a161640 1 -- 172.21.15.12:0/198931109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60419fa90 con 0x7fe604102310 2024-05-09T09:22:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.348+0000 7fe600ff9640 1 -- 172.21.15.12:0/198931109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe5f4009c30 con 0x7fe604102310 2024-05-09T09:22:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.348+0000 7fe600ff9640 1 -- 172.21.15.12:0/198931109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f400f6b0 con 0x7fe604102310 2024-05-09T09:22:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.348+0000 7fe60a161640 1 -- 172.21.15.12:0/198931109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60419ffe0 con 0x7fe604102310 2024-05-09T09:22:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.349+0000 7fe60a161640 1 -- 172.21.15.12:0/198931109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6040fd810 con 0x7fe604102310 2024-05-09T09:22:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.350+0000 7fe600ff9640 1 -- 172.21.15.12:0/198931109 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe5f4010a50 con 0x7fe604102310 2024-05-09T09:22:54.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.351+0000 7fe600ff9640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe5ec0a2b90 0x7fe5ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:54.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.351+0000 7fe602ffd640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe5ec0a2b90 0x7fe5ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:54.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.351+0000 7fe600ff9640 1 -- 172.21.15.12:0/198931109 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe5f40ce950 con 0x7fe604102310 2024-05-09T09:22:54.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.351+0000 7fe602ffd640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe5ec0a2b90 0x7fe5ec0a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe604199090 tx=0x7fe5f8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:54.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.356+0000 7fe600ff9640 1 -- 172.21.15.12:0/198931109 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe5f4009de0 con 0x7fe604102310 2024-05-09T09:22:54.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.580+0000 7fe60a161640 1 -- 172.21.15.12:0/198931109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe60402d1c0 con 0x7fe604102310 2024-05-09T09:22:54.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.581+0000 7fe600ff9640 1 -- 172.21.15.12:0/198931109 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7fe60402d1c0 con 0x7fe604102310 2024-05-09T09:22:54.581 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:54.581 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-09T09:22:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2024-05-09T09:22:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.583+0000 7fe60a161640 1 -- 172.21.15.12:0/198931109 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe5ec0a2b90 msgr2=0x7fe5ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.583+0000 7fe60a161640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe5ec0a2b90 0x7fe5ec0a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe604199090 tx=0x7fe5f8005e50 comp rx=0 tx=0).stop 2024-05-09T09:22:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.583+0000 7fe60a161640 1 -- 172.21.15.12:0/198931109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe604102310 msgr2=0x7fe604197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:54.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.583+0000 7fe60a161640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe604102310 0x7fe604197b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400d950 tx=0x7fe5f400de10 comp rx=0 tx=0).stop 2024-05-09T09:22:54.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.583+0000 7fe60a161640 1 -- 172.21.15.12:0/198931109 shutdown_connections 2024-05-09T09:22:54.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.583+0000 7fe60a161640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe5ec0a2b90 0x7fe5ec0a5050 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:54.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.583+0000 7fe60a161640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe604107570 0x7fe60419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:54.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.583+0000 7fe60a161640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe604104c40 0x7fe6041980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:54.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.583+0000 7fe60a161640 1 --2- 172.21.15.12:0/198931109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe604102310 0x7fe604197b80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:54.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.583+0000 7fe60a161640 1 -- 172.21.15.12:0/198931109 >> 172.21.15.12:0/198931109 conn(0x7fe6040fbd20 msgr2=0x7fe604102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:54.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.583+0000 7fe60a161640 1 -- 172.21.15.12:0/198931109 shutdown_connections 2024-05-09T09:22:54.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:54.583+0000 7fe60a161640 1 -- 172.21.15.12:0/198931109 wait complete. 2024-05-09T09:22:54.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:54 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/198931109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:54.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:54 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/198931109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:55.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:54 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/198931109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:55.743 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:22:55 smithi170 systemd[1]: Started Ceph mds.i for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:22:56.127 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:55 smithi012 ceph-mon[30140]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:56.127 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:56.127 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:56.128 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:56.128 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:56.128 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:56.128 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:55 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:55 smithi170 ceph-mon[33632]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:55 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:56.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:55 smithi016 ceph-mon[33335]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:56.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:56.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:56.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:56.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:56.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:56.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:55 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:56.595 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:56.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.706+0000 7efd18858640 1 -- 172.21.15.12:0/3940123141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd14100770 msgr2=0x7efd14100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:56.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.706+0000 7efd18858640 1 --2- 172.21.15.12:0/3940123141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd14100770 0x7efd14100b90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efd04009900 tx=0x7efd0402a410 comp rx=0 tx=0).stop 2024-05-09T09:22:56.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.706+0000 7efd18858640 1 -- 172.21.15.12:0/3940123141 shutdown_connections 2024-05-09T09:22:56.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.706+0000 7efd18858640 1 --2- 172.21.15.12:0/3940123141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd14102320 0x7efd14108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:56.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.706+0000 7efd18858640 1 --2- 172.21.15.12:0/3940123141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd14101960 0x7efd14101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.706+0000 7efd18858640 1 --2- 172.21.15.12:0/3940123141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd14100770 0x7efd14100b90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.706+0000 7efd18858640 1 -- 172.21.15.12:0/3940123141 >> 172.21.15.12:0/3940123141 conn(0x7efd140fbce0 msgr2=0x7efd140fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.706+0000 7efd18858640 1 -- 172.21.15.12:0/3940123141 shutdown_connections 2024-05-09T09:22:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.706+0000 7efd18858640 1 -- 172.21.15.12:0/3940123141 wait complete. 2024-05-09T09:22:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.706+0000 7efd18858640 1 Processor -- start 2024-05-09T09:22:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.706+0000 7efd18858640 1 -- start start 2024-05-09T09:22:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd18858640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd14100770 0x7efd14197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd18858640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd14101960 0x7efd141980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd18858640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd14102320 0x7efd1419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd18858640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efd1410d4e0 con 0x7efd14101960 2024-05-09T09:22:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd18858640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd1410d360 con 0x7efd14100770 2024-05-09T09:22:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd18858640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efd1410d660 con 0x7efd14102320 2024-05-09T09:22:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd0bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd14101960 0x7efd141980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd0bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd14101960 0x7efd141980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41264/0 (socket says 172.21.15.12:41264) 2024-05-09T09:22:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd0bfff640 1 -- 172.21.15.12:0/2526719324 learned_addr learned my addr 172.21.15.12:0/2526719324 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd12d9d640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd14102320 0x7efd1419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd1259c640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd14100770 0x7efd14197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd0bfff640 1 -- 172.21.15.12:0/2526719324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd14102320 msgr2=0x7efd1419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd0bfff640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd14102320 0x7efd1419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd0bfff640 1 -- 172.21.15.12:0/2526719324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd14100770 msgr2=0x7efd14197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd0bfff640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd14100770 0x7efd14197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd0bfff640 1 -- 172.21.15.12:0/2526719324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1419f7a0 con 0x7efd14101960 2024-05-09T09:22:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd1259c640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd14100770 0x7efd14197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:56.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd0bfff640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd14101960 0x7efd141980c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00c890 tx=0x7efcfc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:56.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.707+0000 7efd0b7fe640 1 -- 172.21.15.12:0/2526719324 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc004220 con 0x7efd14101960 2024-05-09T09:22:56.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.708+0000 7efd0b7fe640 1 -- 172.21.15.12:0/2526719324 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efcfc0043c0 con 0x7efd14101960 2024-05-09T09:22:56.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.708+0000 7efd18858640 1 -- 172.21.15.12:0/2526719324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1419fa90 con 0x7efd14101960 2024-05-09T09:22:56.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.708+0000 7efd0b7fe640 1 -- 172.21.15.12:0/2526719324 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc004a00 con 0x7efd14101960 2024-05-09T09:22:56.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.708+0000 7efd18858640 1 -- 172.21.15.12:0/2526719324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1419ffe0 con 0x7efd14101960 2024-05-09T09:22:56.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.709+0000 7efd18858640 1 -- 172.21.15.12:0/2526719324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd14107a90 con 0x7efd14101960 2024-05-09T09:22:56.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.710+0000 7efd0b7fe640 1 -- 172.21.15.12:0/2526719324 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efcfc004ba0 con 0x7efd14101960 2024-05-09T09:22:56.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.711+0000 7efd0b7fe640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efd000a2b90 0x7efd000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:56.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.711+0000 7efd1259c640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efd000a2b90 0x7efd000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:56.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.711+0000 7efd0b7fe640 1 -- 172.21.15.12:0/2526719324 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efcfc0ce9c0 con 0x7efd14101960 2024-05-09T09:22:56.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.712+0000 7efd1259c640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efd000a2b90 0x7efd000a5050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd04006010 tx=0x7efd04000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:56.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.716+0000 7efd0b7fe640 1 -- 172.21.15.12:0/2526719324 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efcfc014030 con 0x7efd14101960 2024-05-09T09:22:56.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.944+0000 7efd18858640 1 -- 172.21.15.12:0/2526719324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd14062430 con 0x7efd14101960 2024-05-09T09:22:56.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.945+0000 7efd0b7fe640 1 -- 172.21.15.12:0/2526719324 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7efcfc09bd20 con 0x7efd14101960 2024-05-09T09:22:56.945 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:56.945 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24523,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T09:22:56.945 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 10 2024-05-09T09:22:56.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.947+0000 7efd18858640 1 -- 172.21.15.12:0/2526719324 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efd000a2b90 msgr2=0x7efd000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:56.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.947+0000 7efd18858640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efd000a2b90 0x7efd000a5050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd04006010 tx=0x7efd04000bc0 comp rx=0 tx=0).stop 2024-05-09T09:22:56.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.947+0000 7efd18858640 1 -- 172.21.15.12:0/2526719324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd14101960 msgr2=0x7efd141980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:56.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.947+0000 7efd18858640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd14101960 0x7efd141980c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00c890 tx=0x7efcfc00cd50 comp rx=0 tx=0).stop 2024-05-09T09:22:56.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.947+0000 7efd18858640 1 -- 172.21.15.12:0/2526719324 shutdown_connections 2024-05-09T09:22:56.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.947+0000 7efd18858640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efd000a2b90 0x7efd000a5050 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:56.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.947+0000 7efd18858640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd14102320 0x7efd1419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:56.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.947+0000 7efd18858640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd14101960 0x7efd141980c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:56.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.947+0000 7efd18858640 1 --2- 172.21.15.12:0/2526719324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd14100770 0x7efd14197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:56.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.947+0000 7efd18858640 1 -- 172.21.15.12:0/2526719324 >> 172.21.15.12:0/2526719324 conn(0x7efd140fbce0 msgr2=0x7efd14104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:56.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.947+0000 7efd18858640 1 -- 172.21.15.12:0/2526719324 shutdown_connections 2024-05-09T09:22:56.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:56.947+0000 7efd18858640 1 -- 172.21.15.12:0/2526719324 wait complete. 2024-05-09T09:22:57.153 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:56 smithi012 ceph-mon[30140]: Deploying daemon mds.g on smithi012 2024-05-09T09:22:57.154 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:56 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] up:boot 2024-05-09T09:22:57.154 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:56 smithi012 ceph-mon[30140]: fsmap 9 up:standby 2024-05-09T09:22:57.154 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:56 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T09:22:57.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:56 smithi170 ceph-mon[33632]: Deploying daemon mds.g on smithi012 2024-05-09T09:22:57.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:56 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] up:boot 2024-05-09T09:22:57.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:56 smithi170 ceph-mon[33632]: fsmap 9 up:standby 2024-05-09T09:22:57.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:56 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T09:22:57.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:56 smithi016 ceph-mon[33335]: Deploying daemon mds.g on smithi012 2024-05-09T09:22:57.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:56 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] up:boot 2024-05-09T09:22:57.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:56 smithi016 ceph-mon[33335]: fsmap 9 up:standby 2024-05-09T09:22:57.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:56 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T09:22:57.796 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:22:57 smithi012 systemd[1]: Starting Ceph mds.g for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:22:58.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:57 smithi012 ceph-mon[30140]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:58.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:57 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2526719324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:58.063 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:22:57 smithi012 podman[72159]: 2024-05-09 09:22:57.845616499 +0000 UTC m=+0.018662756 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:22:58.063 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:22:57 smithi012 podman[72159]: 2024-05-09 09:22:57.946318264 +0000 UTC m=+0.119364508 container create 087bd705b6e3329a6cd246ab493d57e0291890f904ac14609e84063c37297f48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-g, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:22:58.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:57 smithi170 ceph-mon[33632]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:58.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:57 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2526719324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:58.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:57 smithi016 ceph-mon[33335]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:58.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:57 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2526719324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:58.484 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:22:58 smithi012 podman[72159]: 2024-05-09 09:22:58.192862276 +0000 UTC m=+0.365908531 container init 087bd705b6e3329a6cd246ab493d57e0291890f904ac14609e84063c37297f48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-g, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True) 2024-05-09T09:22:58.484 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:22:58 smithi012 podman[72159]: 2024-05-09 09:22:58.196005949 +0000 UTC m=+0.369052193 container start 087bd705b6e3329a6cd246ab493d57e0291890f904ac14609e84063c37297f48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-g, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-09T09:22:58.484 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:22:58 smithi012 ceph-mds[72178]: -- 172.21.15.12:0/659242555 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d966eb1a40 con 0x55d966e84c00 2024-05-09T09:22:58.484 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:22:58 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-g[72174]: starting mds.g at 2024-05-09T09:22:58.485 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:22:58 smithi012 bash[72159]: 087bd705b6e3329a6cd246ab493d57e0291890f904ac14609e84063c37297f48 2024-05-09T09:22:58.485 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:22:58 smithi012 systemd[1]: Started Ceph mds.g for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:22:58.960 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:59.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.069+0000 7f4a3c710640 1 -- 172.21.15.12:0/2845832272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34101970 msgr2=0x7f4a34101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:59.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.069+0000 7f4a3c710640 1 --2- 172.21.15.12:0/2845832272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34101970 0x7f4a34101df0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4a24009900 tx=0x7f4a2402a410 comp rx=0 tx=0).stop 2024-05-09T09:22:59.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 -- 172.21.15.12:0/2845832272 shutdown_connections 2024-05-09T09:22:59.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 --2- 172.21.15.12:0/2845832272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a34102330 0x7f4a34108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:59.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 --2- 172.21.15.12:0/2845832272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34101970 0x7f4a34101df0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 --2- 172.21.15.12:0/2845832272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34100780 0x7f4a34100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 -- 172.21.15.12:0/2845832272 >> 172.21.15.12:0/2845832272 conn(0x7f4a340fbd10 msgr2=0x7f4a340fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 -- 172.21.15.12:0/2845832272 shutdown_connections 2024-05-09T09:22:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 -- 172.21.15.12:0/2845832272 wait complete. 2024-05-09T09:22:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 Processor -- start 2024-05-09T09:22:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 -- start start 2024-05-09T09:22:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34100780 0x7f4a341046c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34101970 0x7f4a34102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a34102330 0x7f4a34103290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3410c480 con 0x7f4a34100780 2024-05-09T09:22:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3410c300 con 0x7f4a34101970 2024-05-09T09:22:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.070+0000 7f4a3c710640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3410c600 con 0x7f4a34102330 2024-05-09T09:22:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a3a4ac640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34100780 0x7f4a341046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a3a4ac640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34100780 0x7f4a341046c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41298/0 (socket says 172.21.15.12:41298) 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a3a4ac640 1 -- 172.21.15.12:0/1664648037 learned_addr learned my addr 172.21.15.12:0/1664648037 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a39cab640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34101970 0x7f4a34102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a3acad640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a34102330 0x7f4a34103290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a3a4ac640 1 -- 172.21.15.12:0/1664648037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a34102330 msgr2=0x7f4a34103290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a3a4ac640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a34102330 0x7f4a34103290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a3a4ac640 1 -- 172.21.15.12:0/1664648037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34101970 msgr2=0x7f4a34102d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a3a4ac640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34101970 0x7f4a34102d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a3a4ac640 1 -- 172.21.15.12:0/1664648037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a34103b50 con 0x7f4a34100780 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a3a4ac640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34100780 0x7f4a341046c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000b450 tx=0x7f4a3000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a2b7fe640 1 -- 172.21.15.12:0/1664648037 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30015020 con 0x7f4a34100780 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a2b7fe640 1 -- 172.21.15.12:0/1664648037 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4a30011d60 con 0x7f4a34100780 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a2b7fe640 1 -- 172.21.15.12:0/1664648037 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000f340 con 0x7f4a34100780 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a3c710640 1 -- 172.21.15.12:0/1664648037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a341a1da0 con 0x7f4a34100780 2024-05-09T09:22:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.071+0000 7f4a3c710640 1 -- 172.21.15.12:0/1664648037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a341a22e0 con 0x7f4a34100780 2024-05-09T09:22:59.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.072+0000 7f4a3c710640 1 -- 172.21.15.12:0/1664648037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a341a2570 con 0x7f4a34100780 2024-05-09T09:22:59.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.073+0000 7f4a2b7fe640 1 -- 172.21.15.12:0/1664648037 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4a30018020 con 0x7f4a34100780 2024-05-09T09:22:59.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.074+0000 7f4a2b7fe640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4a200a2c10 0x7f4a200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:59.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.074+0000 7f4a39cab640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4a200a2c10 0x7f4a200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:59.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.074+0000 7f4a2b7fe640 1 -- 172.21.15.12:0/1664648037 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4a300ce5e0 con 0x7f4a34100780 2024-05-09T09:22:59.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.075+0000 7f4a39cab640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4a200a2c10 0x7f4a200a50d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a24000c00 tx=0x7f4a24031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:59.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.078+0000 7f4a2b7fe640 1 -- 172.21.15.12:0/1664648037 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4a3001f020 con 0x7f4a34100780 2024-05-09T09:22:59.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.304+0000 7f4a3c710640 1 -- 172.21.15.12:0/1664648037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a341a2860 con 0x7f4a34100780 2024-05-09T09:22:59.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.305+0000 7f4a2b7fe640 1 -- 172.21.15.12:0/1664648037 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f4a3000d040 con 0x7f4a34100780 2024-05-09T09:22:59.305 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:22:59.305 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24523,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T09:22:59.305 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 11 2024-05-09T09:22:59.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.307+0000 7f4a3c710640 1 -- 172.21.15.12:0/1664648037 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4a200a2c10 msgr2=0x7f4a200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:59.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.307+0000 7f4a3c710640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4a200a2c10 0x7f4a200a50d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a24000c00 tx=0x7f4a24031510 comp rx=0 tx=0).stop 2024-05-09T09:22:59.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.307+0000 7f4a3c710640 1 -- 172.21.15.12:0/1664648037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34100780 msgr2=0x7f4a341046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:59.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.307+0000 7f4a3c710640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34100780 0x7f4a341046c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000b450 tx=0x7f4a3000b910 comp rx=0 tx=0).stop 2024-05-09T09:22:59.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.307+0000 7f4a3c710640 1 -- 172.21.15.12:0/1664648037 shutdown_connections 2024-05-09T09:22:59.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.307+0000 7f4a3c710640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4a200a2c10 0x7f4a200a50d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:59.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.307+0000 7f4a3c710640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a34102330 0x7f4a34103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:59.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.307+0000 7f4a3c710640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34101970 0x7f4a34102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:59.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.307+0000 7f4a3c710640 1 --2- 172.21.15.12:0/1664648037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34100780 0x7f4a341046c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:59.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.307+0000 7f4a3c710640 1 -- 172.21.15.12:0/1664648037 >> 172.21.15.12:0/1664648037 conn(0x7f4a340fbd10 msgr2=0x7f4a340fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:59.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.307+0000 7f4a3c710640 1 -- 172.21.15.12:0/1664648037 shutdown_connections 2024-05-09T09:22:59.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:22:59.307+0000 7f4a3c710640 1 -- 172.21.15.12:0/1664648037 wait complete. 2024-05-09T09:22:59.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:59 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:59.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:59 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:59.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:59 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:59.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:59 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:59.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:59 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:59.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:59 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:59.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:59 smithi012 ceph-mon[30140]: Deploying daemon mds.a on smithi012 2024-05-09T09:22:59.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:59 smithi012 ceph-mon[30140]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:59.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:59 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.12:6838/901613691,v1:172.21.15.12:6839/901613691] up:boot 2024-05-09T09:22:59.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:59 smithi012 ceph-mon[30140]: fsmap 10 up:standby 2024-05-09T09:22:59.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:59 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T09:22:59.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:22:59 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1664648037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:59.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:59 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:59.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:59 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:59.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:59 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:59.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:59 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:59.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:59 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:59.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:59 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:59.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:59 smithi170 ceph-mon[33632]: Deploying daemon mds.a on smithi012 2024-05-09T09:22:59.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:59 smithi170 ceph-mon[33632]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:59.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:59 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.12:6838/901613691,v1:172.21.15.12:6839/901613691] up:boot 2024-05-09T09:22:59.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:59 smithi170 ceph-mon[33632]: fsmap 10 up:standby 2024-05-09T09:22:59.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:59 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T09:22:59.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:22:59 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1664648037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:59.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:59 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:59.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:59 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:59.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:59 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:22:59.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:59 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:22:59.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:59 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:22:59.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:59 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:59.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:59 smithi016 ceph-mon[33335]: Deploying daemon mds.a on smithi012 2024-05-09T09:22:59.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:59 smithi016 ceph-mon[33335]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:22:59.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:59 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.12:6838/901613691,v1:172.21.15.12:6839/901613691] up:boot 2024-05-09T09:22:59.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:59 smithi016 ceph-mon[33335]: fsmap 10 up:standby 2024-05-09T09:22:59.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:59 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T09:22:59.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:22:59 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1664648037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:00.734 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:23:00 smithi012 systemd[1]: Starting Ceph mds.a for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:23:01.057 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:23:00 smithi012 podman[72508]: 2024-05-09 09:23:00.682055441 +0000 UTC m=+0.018752075 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:23:01.057 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:23:00 smithi012 podman[72508]: 2024-05-09 09:23:00.788274987 +0000 UTC m=+0.124971646 container create 512799093bbe69d806a619485538f6e80226fd5130309c0d9a4974e296226263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-a, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:23:01.057 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:23:01 smithi012 podman[72508]: 2024-05-09 09:23:01.057644789 +0000 UTC m=+0.394341413 container init 512799093bbe69d806a619485538f6e80226fd5130309c0d9a4974e296226263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-a, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:23:01.322 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:23:01.347 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:23:01 smithi012 podman[72508]: 2024-05-09 09:23:01.060650809 +0000 UTC m=+0.397347422 container start 512799093bbe69d806a619485538f6e80226fd5130309c0d9a4974e296226263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-09T09:23:01.347 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:23:01 smithi012 ceph-mds[72527]: -- 172.21.15.12:0/2932283404 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eb0ca13a40 con 0x55eb0c9e7000 2024-05-09T09:23:01.347 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:23:01 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-a[72523]: starting mds.a at 2024-05-09T09:23:01.347 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:23:01 smithi012 bash[72508]: 512799093bbe69d806a619485538f6e80226fd5130309c0d9a4974e296226263 2024-05-09T09:23:01.347 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:23:01 smithi012 systemd[1]: Started Ceph mds.a for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:23:01.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.432+0000 7ff6e724d640 1 -- 172.21.15.12:0/7344981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e0102a90 msgr2=0x7ff6e0102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:01.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.432+0000 7ff6e724d640 1 --2- 172.21.15.12:0/7344981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e0102a90 0x7ff6e0102f10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009900 tx=0x7ff6cc02a410 comp rx=0 tx=0).stop 2024-05-09T09:23:01.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.432+0000 7ff6e724d640 1 -- 172.21.15.12:0/7344981 shutdown_connections 2024-05-09T09:23:01.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.432+0000 7ff6e724d640 1 --2- 172.21.15.12:0/7344981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6e0103450 0x7ff6e0109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:01.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.432+0000 7ff6e724d640 1 --2- 172.21.15.12:0/7344981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e0102a90 0x7ff6e0102f10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:01.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.432+0000 7ff6e724d640 1 --2- 172.21.15.12:0/7344981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6e01018a0 0x7ff6e0101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:01.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.432+0000 7ff6e724d640 1 -- 172.21.15.12:0/7344981 >> 172.21.15.12:0/7344981 conn(0x7ff6e00fce30 msgr2=0x7ff6e00ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:01.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.432+0000 7ff6e724d640 1 -- 172.21.15.12:0/7344981 shutdown_connections 2024-05-09T09:23:01.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.432+0000 7ff6e724d640 1 -- 172.21.15.12:0/7344981 wait complete. 2024-05-09T09:23:01.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.433+0000 7ff6e724d640 1 Processor -- start 2024-05-09T09:23:01.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.433+0000 7ff6e724d640 1 -- start start 2024-05-09T09:23:01.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.433+0000 7ff6e724d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6e01018a0 0x7ff6e00755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:01.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.433+0000 7ff6e724d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e0102a90 0x7ff6e0073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:01.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.433+0000 7ff6e724d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6e0103450 0x7ff6e0074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:01.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.433+0000 7ff6e724d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e010d500 con 0x7ff6e0102a90 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.433+0000 7ff6e724d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e010d380 con 0x7ff6e0103450 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.433+0000 7ff6e724d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e010d680 con 0x7ff6e01018a0 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.433+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e0102a90 0x7ff6e0073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.433+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e0102a90 0x7ff6e0073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41346/0 (socket says 172.21.15.12:41346) 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.433+0000 7ff6dffff640 1 -- 172.21.15.12:0/480629946 learned_addr learned my addr 172.21.15.12:0/480629946 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.433+0000 7ff6e57ea640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6e0103450 0x7ff6e0074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.433+0000 7ff6e4fe9640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6e01018a0 0x7ff6e00755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.434+0000 7ff6dffff640 1 -- 172.21.15.12:0/480629946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6e01018a0 msgr2=0x7ff6e00755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.434+0000 7ff6dffff640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6e01018a0 0x7ff6e00755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.434+0000 7ff6dffff640 1 -- 172.21.15.12:0/480629946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6e0103450 msgr2=0x7ff6e0074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.434+0000 7ff6dffff640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6e0103450 0x7ff6e0074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.434+0000 7ff6dffff640 1 -- 172.21.15.12:0/480629946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e00749f0 con 0x7ff6e0102a90 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.434+0000 7ff6dffff640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e0102a90 0x7ff6e0073c20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009a20 tx=0x7ff6cc031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:01.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.434+0000 7ff6ddffb640 1 -- 172.21.15.12:0/480629946 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc03f030 con 0x7ff6e0102a90 2024-05-09T09:23:01.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.434+0000 7ff6e724d640 1 -- 172.21.15.12:0/480629946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e01a1c60 con 0x7ff6e0102a90 2024-05-09T09:23:01.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.434+0000 7ff6ddffb640 1 -- 172.21.15.12:0/480629946 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff6cc003ac0 con 0x7ff6e0102a90 2024-05-09T09:23:01.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.434+0000 7ff6ddffb640 1 -- 172.21.15.12:0/480629946 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc035070 con 0x7ff6e0102a90 2024-05-09T09:23:01.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.434+0000 7ff6e724d640 1 -- 172.21.15.12:0/480629946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e01a21b0 con 0x7ff6e0102a90 2024-05-09T09:23:01.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.435+0000 7ff6e724d640 1 -- 172.21.15.12:0/480629946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e0108bc0 con 0x7ff6e0102a90 2024-05-09T09:23:01.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.437+0000 7ff6ddffb640 1 -- 172.21.15.12:0/480629946 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff6cc042030 con 0x7ff6e0102a90 2024-05-09T09:23:01.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.437+0000 7ff6ddffb640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff6d00a2b90 0x7ff6d00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:01.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.437+0000 7ff6ddffb640 1 -- 172.21.15.12:0/480629946 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff6cc0ef7f0 con 0x7ff6e0102a90 2024-05-09T09:23:01.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.437+0000 7ff6e4fe9640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff6d00a2b90 0x7ff6d00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:01.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.438+0000 7ff6e4fe9640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff6d00a2b90 0x7ff6d00a5050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff6e01028f0 tx=0x7ff6d400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:01.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.441+0000 7ff6ddffb640 1 -- 172.21.15.12:0/480629946 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff6cc0bc5a0 con 0x7ff6e0102a90 2024-05-09T09:23:01.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.665+0000 7ff6e724d640 1 -- 172.21.15.12:0/480629946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6e002d430 con 0x7ff6e0102a90 2024-05-09T09:23:01.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.666+0000 7ff6ddffb640 1 -- 172.21.15.12:0/480629946 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7ff6cc0c08a0 con 0x7ff6e0102a90 2024-05-09T09:23:01.666 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24523,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T09:23:01.666 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 12 2024-05-09T09:23:01.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.668+0000 7ff6e724d640 1 -- 172.21.15.12:0/480629946 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff6d00a2b90 msgr2=0x7ff6d00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:01.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.668+0000 7ff6e724d640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff6d00a2b90 0x7ff6d00a5050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff6e01028f0 tx=0x7ff6d400b390 comp rx=0 tx=0).stop 2024-05-09T09:23:01.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.668+0000 7ff6e724d640 1 -- 172.21.15.12:0/480629946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e0102a90 msgr2=0x7ff6e0073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:01.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.668+0000 7ff6e724d640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e0102a90 0x7ff6e0073c20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009a20 tx=0x7ff6cc031ac0 comp rx=0 tx=0).stop 2024-05-09T09:23:01.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.668+0000 7ff6e724d640 1 -- 172.21.15.12:0/480629946 shutdown_connections 2024-05-09T09:23:01.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.668+0000 7ff6e724d640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff6d00a2b90 0x7ff6d00a5050 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:01.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.668+0000 7ff6e724d640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6e0103450 0x7ff6e0074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:01.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.668+0000 7ff6e724d640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e0102a90 0x7ff6e0073c20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:01.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.668+0000 7ff6e724d640 1 --2- 172.21.15.12:0/480629946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6e01018a0 0x7ff6e00755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:01.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.668+0000 7ff6e724d640 1 -- 172.21.15.12:0/480629946 >> 172.21.15.12:0/480629946 conn(0x7ff6e00fce30 msgr2=0x7ff6e0105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:01.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.669+0000 7ff6e724d640 1 -- 172.21.15.12:0/480629946 shutdown_connections 2024-05-09T09:23:01.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:01.669+0000 7ff6e724d640 1 -- 172.21.15.12:0/480629946 wait complete. 2024-05-09T09:23:02.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:02 smithi012 ceph-mon[30140]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:02.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:02.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:02.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:02.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:23:02.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:23:02.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:23:02.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:02 smithi012 ceph-mon[30140]: Deploying daemon mds.b on smithi016 2024-05-09T09:23:02.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:02 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] up:boot 2024-05-09T09:23:02.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:02 smithi012 ceph-mon[30140]: fsmap 11 up:standby 2024-05-09T09:23:02.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:02 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T09:23:02.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:02 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/480629946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:02 smithi170 ceph-mon[33632]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:23:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:23:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:23:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:02 smithi170 ceph-mon[33632]: Deploying daemon mds.b on smithi016 2024-05-09T09:23:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:02 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] up:boot 2024-05-09T09:23:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:02 smithi170 ceph-mon[33632]: fsmap 11 up:standby 2024-05-09T09:23:02.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:02 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T09:23:02.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:02 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/480629946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:02 smithi016 ceph-mon[33335]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:23:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:23:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:23:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:02 smithi016 ceph-mon[33335]: Deploying daemon mds.b on smithi016 2024-05-09T09:23:02.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:02 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] up:boot 2024-05-09T09:23:02.505 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:02 smithi016 ceph-mon[33335]: fsmap 11 up:standby 2024-05-09T09:23:02.505 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:02 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T09:23:02.505 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:02 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/480629946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:03.680 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:23:03.743 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:23:03 smithi016 systemd[1]: Starting Ceph mds.b for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:23:03.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.802+0000 7fb0a73dc640 1 -- 172.21.15.12:0/2710084288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a0102310 msgr2=0x7fb0a0104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:03.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.802+0000 7fb0a73dc640 1 --2- 172.21.15.12:0/2710084288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a0102310 0x7fb0a0104700 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009900 tx=0x7fb09c02a410 comp rx=0 tx=0).stop 2024-05-09T09:23:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.802+0000 7fb0a73dc640 1 -- 172.21.15.12:0/2710084288 shutdown_connections 2024-05-09T09:23:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.802+0000 7fb0a73dc640 1 --2- 172.21.15.12:0/2710084288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a0107570 0x7fb0a01099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.802+0000 7fb0a73dc640 1 --2- 172.21.15.12:0/2710084288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0a0104c40 0x7fb0a0107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.802+0000 7fb0a73dc640 1 --2- 172.21.15.12:0/2710084288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a0102310 0x7fb0a0104700 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.802+0000 7fb0a73dc640 1 -- 172.21.15.12:0/2710084288 >> 172.21.15.12:0/2710084288 conn(0x7fb0a00fbd20 msgr2=0x7fb0a00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.802+0000 7fb0a73dc640 1 -- 172.21.15.12:0/2710084288 shutdown_connections 2024-05-09T09:23:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.802+0000 7fb0a73dc640 1 -- 172.21.15.12:0/2710084288 wait complete. 2024-05-09T09:23:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.802+0000 7fb0a73dc640 1 Processor -- start 2024-05-09T09:23:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.802+0000 7fb0a73dc640 1 -- start start 2024-05-09T09:23:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a73dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a0102310 0x7fb0a0101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a73dc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a0104c40 0x7fb0a01002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a73dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0a0107570 0x7fb0a0100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a73dc640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a010d510 con 0x7fb0a0104c40 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a73dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a010d390 con 0x7fb0a0102310 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a73dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a010d690 con 0x7fb0a0107570 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a4977640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a0104c40 0x7fb0a01002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a4977640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a0104c40 0x7fb0a01002c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45492/0 (socket says 172.21.15.12:45492) 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a4977640 1 -- 172.21.15.12:0/212013713 learned_addr learned my addr 172.21.15.12:0/212013713 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a5178640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a0102310 0x7fb0a0101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a5979640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0a0107570 0x7fb0a0100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a4977640 1 -- 172.21.15.12:0/212013713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0a0107570 msgr2=0x7fb0a0100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a4977640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0a0107570 0x7fb0a0100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a4977640 1 -- 172.21.15.12:0/212013713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a0102310 msgr2=0x7fb0a0101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a4977640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a0102310 0x7fb0a0101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a4977640 1 -- 172.21.15.12:0/212013713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a0101090 con 0x7fb0a0104c40 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a4977640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a0104c40 0x7fb0a01002c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb09000d740 tx=0x7fb09000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0967fc640 1 -- 172.21.15.12:0/212013713 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090017070 con 0x7fb0a0104c40 2024-05-09T09:23:03.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0967fc640 1 -- 172.21.15.12:0/212013713 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb09000de50 con 0x7fb0a0104c40 2024-05-09T09:23:03.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0a73dc640 1 -- 172.21.15.12:0/212013713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a01a1d20 con 0x7fb0a0104c40 2024-05-09T09:23:03.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.803+0000 7fb0967fc640 1 -- 172.21.15.12:0/212013713 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090004e00 con 0x7fb0a0104c40 2024-05-09T09:23:03.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.804+0000 7fb0a73dc640 1 -- 172.21.15.12:0/212013713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a01a21c0 con 0x7fb0a0104c40 2024-05-09T09:23:03.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.804+0000 7fb0a73dc640 1 -- 172.21.15.12:0/212013713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a00fd810 con 0x7fb0a0104c40 2024-05-09T09:23:03.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.806+0000 7fb0967fc640 1 -- 172.21.15.12:0/212013713 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb090004fa0 con 0x7fb0a0104c40 2024-05-09T09:23:03.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.806+0000 7fb0967fc640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb08c0a2c10 0x7fb08c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:03.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.806+0000 7fb0a5178640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb08c0a2c10 0x7fb08c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:03.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.806+0000 7fb0967fc640 1 -- 172.21.15.12:0/212013713 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb0900ce4b0 con 0x7fb0a0104c40 2024-05-09T09:23:03.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.807+0000 7fb0a5178640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb08c0a2c10 0x7fb08c0a50d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb09c000c00 tx=0x7fb09c031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:03.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:03.811+0000 7fb0967fc640 1 -- 172.21.15.12:0/212013713 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb09009b8b0 con 0x7fb0a0104c40 2024-05-09T09:23:04.003 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:23:03 smithi016 podman[64003]: 2024-05-09 09:23:03.647137963 +0000 UTC m=+0.018937635 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:23:04.003 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:23:03 smithi016 podman[64003]: 2024-05-09 09:23:03.762037156 +0000 UTC m=+0.133836813 container create 19bdab9e4e0c64a83533b4d2eab29354ac91c7f2bd01fd90a5fd3af5047917e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-b, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T09:23:04.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.037+0000 7fb0a73dc640 1 -- 172.21.15.12:0/212013713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0a002d3c0 con 0x7fb0a0104c40 2024-05-09T09:23:04.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.038+0000 7fb0967fc640 1 -- 172.21.15.12:0/212013713 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7fb09009b1b0 con 0x7fb0a0104c40 2024-05-09T09:23:04.038 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:04.038 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24523,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T09:23:04.038 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 12 2024-05-09T09:23:04.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.040+0000 7fb0a73dc640 1 -- 172.21.15.12:0/212013713 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb08c0a2c10 msgr2=0x7fb08c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:04.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.040+0000 7fb0a73dc640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb08c0a2c10 0x7fb08c0a50d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb09c000c00 tx=0x7fb09c031510 comp rx=0 tx=0).stop 2024-05-09T09:23:04.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.040+0000 7fb0a73dc640 1 -- 172.21.15.12:0/212013713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a0104c40 msgr2=0x7fb0a01002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:04.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.040+0000 7fb0a73dc640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a0104c40 0x7fb0a01002c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb09000d740 tx=0x7fb09000dc00 comp rx=0 tx=0).stop 2024-05-09T09:23:04.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.040+0000 7fb0a73dc640 1 -- 172.21.15.12:0/212013713 shutdown_connections 2024-05-09T09:23:04.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.040+0000 7fb0a73dc640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb08c0a2c10 0x7fb08c0a50d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:04.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.040+0000 7fb0a73dc640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0a0107570 0x7fb0a0100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:04.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.040+0000 7fb0a73dc640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a0104c40 0x7fb0a01002c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:04.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.040+0000 7fb0a73dc640 1 --2- 172.21.15.12:0/212013713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a0102310 0x7fb0a0101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:04.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.040+0000 7fb0a73dc640 1 -- 172.21.15.12:0/212013713 >> 172.21.15.12:0/212013713 conn(0x7fb0a00fbd20 msgr2=0x7fb0a0102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:04.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.040+0000 7fb0a73dc640 1 -- 172.21.15.12:0/212013713 shutdown_connections 2024-05-09T09:23:04.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:04.040+0000 7fb0a73dc640 1 -- 172.21.15.12:0/212013713 wait complete. 2024-05-09T09:23:04.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:04 smithi012 ceph-mon[30140]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:04.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:04 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/212013713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:04.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:04 smithi170 ceph-mon[33632]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:04.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:04 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/212013713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:04.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:04 smithi016 ceph-mon[33335]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:04.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:04 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/212013713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:04.503 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:23:04 smithi016 podman[64003]: 2024-05-09 09:23:04.073375034 +0000 UTC m=+0.445174702 container init 19bdab9e4e0c64a83533b4d2eab29354ac91c7f2bd01fd90a5fd3af5047917e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-09T09:23:04.503 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:23:04 smithi016 podman[64003]: 2024-05-09 09:23:04.076344383 +0000 UTC m=+0.448144041 container start 19bdab9e4e0c64a83533b4d2eab29354ac91c7f2bd01fd90a5fd3af5047917e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T09:23:04.503 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:23:04 smithi016 ceph-mds[64022]: -- 172.21.15.16:0/3586453583 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557dce4eba40 con 0x557dce4be800 2024-05-09T09:23:04.503 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:23:04 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-b[64018]: starting mds.b at 2024-05-09T09:23:04.503 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:23:04 smithi016 bash[64003]: 19bdab9e4e0c64a83533b4d2eab29354ac91c7f2bd01fd90a5fd3af5047917e2 2024-05-09T09:23:04.503 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:23:04 smithi016 systemd[1]: Started Ceph mds.b for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:23:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:23:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:05 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] up:boot 2024-05-09T09:23:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:05 smithi012 ceph-mon[30140]: fsmap 12 up:standby 2024-05-09T09:23:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T09:23:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:23:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:05 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] up:boot 2024-05-09T09:23:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:05 smithi170 ceph-mon[33632]: fsmap 12 up:standby 2024-05-09T09:23:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T09:23:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:23:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:05 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] up:boot 2024-05-09T09:23:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:05 smithi016 ceph-mon[33335]: fsmap 12 up:standby 2024-05-09T09:23:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T09:23:06.052 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:23:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.169+0000 7f54568ee640 1 -- 172.21.15.12:0/1654259225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5450101960 msgr2=0x7f5450101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.169+0000 7f54568ee640 1 --2- 172.21.15.12:0/1654259225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5450101960 0x7f5450101de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5444009900 tx=0x7f544402a410 comp rx=0 tx=0).stop 2024-05-09T09:23:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.169+0000 7f54568ee640 1 -- 172.21.15.12:0/1654259225 shutdown_connections 2024-05-09T09:23:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.169+0000 7f54568ee640 1 --2- 172.21.15.12:0/1654259225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5450102320 0x7f5450108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.169+0000 7f54568ee640 1 --2- 172.21.15.12:0/1654259225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5450101960 0x7f5450101de0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.169+0000 7f54568ee640 1 --2- 172.21.15.12:0/1654259225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5450100770 0x7f5450100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.169+0000 7f54568ee640 1 -- 172.21.15.12:0/1654259225 >> 172.21.15.12:0/1654259225 conn(0x7f54500fbce0 msgr2=0x7f54500fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.169+0000 7f54568ee640 1 -- 172.21.15.12:0/1654259225 shutdown_connections 2024-05-09T09:23:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.169+0000 7f54568ee640 1 -- 172.21.15.12:0/1654259225 wait complete. 2024-05-09T09:23:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.169+0000 7f54568ee640 1 Processor -- start 2024-05-09T09:23:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.169+0000 7f54568ee640 1 -- start start 2024-05-09T09:23:06.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f54568ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5450100770 0x7f5450104690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f54568ee640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5450101960 0x7f5450102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f54568ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5450102320 0x7f5450103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f54568ee640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f545010c3d0 con 0x7f5450101960 2024-05-09T09:23:06.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f54568ee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f545010c250 con 0x7f5450100770 2024-05-09T09:23:06.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f54568ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f545010c550 con 0x7f5450102320 2024-05-09T09:23:06.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f544f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5450101960 0x7f5450102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f544f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5450101960 0x7f5450102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45500/0 (socket says 172.21.15.12:45500) 2024-05-09T09:23:06.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f544f7fe640 1 -- 172.21.15.12:0/3030085925 learned_addr learned my addr 172.21.15.12:0/3030085925 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f5454e8b640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5450102320 0x7f5450103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f544ffff640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5450100770 0x7f5450104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f5454e8b640 1 -- 172.21.15.12:0/3030085925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5450100770 msgr2=0x7f5450104690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f5454e8b640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5450100770 0x7f5450104690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f5454e8b640 1 -- 172.21.15.12:0/3030085925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5450101960 msgr2=0x7f5450102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f5454e8b640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5450101960 0x7f5450102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f5454e8b640 1 -- 172.21.15.12:0/3030085925 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5450103aa0 con 0x7f5450102320 2024-05-09T09:23:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f544f7fe640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5450101960 0x7f5450102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.170+0000 7f5454e8b640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5450102320 0x7f5450103280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5448009fd0 tx=0x7f544800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.171+0000 7f544d7fa640 1 -- 172.21.15.12:0/3030085925 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544800ce20 con 0x7f5450102320 2024-05-09T09:23:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.171+0000 7f54568ee640 1 -- 172.21.15.12:0/3030085925 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54501a1ce0 con 0x7f5450102320 2024-05-09T09:23:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.171+0000 7f544d7fa640 1 -- 172.21.15.12:0/3030085925 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5448005020 con 0x7f5450102320 2024-05-09T09:23:06.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.171+0000 7f544d7fa640 1 -- 172.21.15.12:0/3030085925 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448010690 con 0x7f5450102320 2024-05-09T09:23:06.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.171+0000 7f54568ee640 1 -- 172.21.15.12:0/3030085925 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54501a2220 con 0x7f5450102320 2024-05-09T09:23:06.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.172+0000 7f54568ee640 1 -- 172.21.15.12:0/3030085925 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54501a24b0 con 0x7f5450102320 2024-05-09T09:23:06.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.173+0000 7f544d7fa640 1 -- 172.21.15.12:0/3030085925 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f54480051d0 con 0x7f5450102320 2024-05-09T09:23:06.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.174+0000 7f544d7fa640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f54380a2b90 0x7f54380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.174+0000 7f544ffff640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f54380a2b90 0x7f54380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.174+0000 7f544d7fa640 1 -- 172.21.15.12:0/3030085925 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f54480cf2f0 con 0x7f5450102320 2024-05-09T09:23:06.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.174+0000 7f544ffff640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f54380a2b90 0x7f54380a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f54501017c0 tx=0x7f5440005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.177+0000 7f544d7fa640 1 -- 172.21.15.12:0/3030085925 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f544809c650 con 0x7f5450102320 2024-05-09T09:23:06.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.414+0000 7f54568ee640 1 -- 172.21.15.12:0/3030085925 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54501a2680 con 0x7f5450102320 2024-05-09T09:23:06.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.415+0000 7f544d7fa640 1 -- 172.21.15.12:0/3030085925 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10479 (secure 0 0 0) 0x7f544809bf50 con 0x7f5450102320 2024-05-09T09:23:06.414 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:06.414 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24523,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T09:23:06.414 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2024-05-09T09:23:06.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.417+0000 7f54568ee640 1 -- 172.21.15.12:0/3030085925 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f54380a2b90 msgr2=0x7f54380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.417+0000 7f54568ee640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f54380a2b90 0x7f54380a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f54501017c0 tx=0x7f5440005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:06.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.417+0000 7f54568ee640 1 -- 172.21.15.12:0/3030085925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5450102320 msgr2=0x7f5450103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.417+0000 7f54568ee640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5450102320 0x7f5450103280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5448009fd0 tx=0x7f544800ef10 comp rx=0 tx=0).stop 2024-05-09T09:23:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.417+0000 7f54568ee640 1 -- 172.21.15.12:0/3030085925 shutdown_connections 2024-05-09T09:23:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.417+0000 7f54568ee640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f54380a2b90 0x7f54380a5050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.417+0000 7f54568ee640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5450102320 0x7f5450103280 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.417+0000 7f54568ee640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5450101960 0x7f5450102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.417+0000 7f54568ee640 1 --2- 172.21.15.12:0/3030085925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5450100770 0x7f5450104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.417+0000 7f54568ee640 1 -- 172.21.15.12:0/3030085925 >> 172.21.15.12:0/3030085925 conn(0x7f54500fbce0 msgr2=0x7f54500fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.417+0000 7f54568ee640 1 -- 172.21.15.12:0/3030085925 shutdown_connections 2024-05-09T09:23:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.417+0000 7f54568ee640 1 -- 172.21.15.12:0/3030085925 wait complete. 2024-05-09T09:23:06.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:06 smithi012 ceph-mon[30140]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:06.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:06.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:06.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:06.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:06.428 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:23:06.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:06 smithi016 ceph-mon[33335]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.535+0000 7ff4fb563640 1 -- 172.21.15.12:0/174623509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4f4102320 msgr2=0x7ff4f4108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.535+0000 7ff4fb563640 1 --2- 172.21.15.12:0/174623509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4f4102320 0x7ff4f4108bb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009ca0 tx=0x7ff4ec02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.535+0000 7ff4fb563640 1 -- 172.21.15.12:0/174623509 shutdown_connections 2024-05-09T09:23:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.535+0000 7ff4fb563640 1 --2- 172.21.15.12:0/174623509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4f4102320 0x7ff4f4108bb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.535+0000 7ff4fb563640 1 --2- 172.21.15.12:0/174623509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4f4101960 0x7ff4f4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.535+0000 7ff4fb563640 1 --2- 172.21.15.12:0/174623509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4100770 0x7ff4f4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.535+0000 7ff4fb563640 1 -- 172.21.15.12:0/174623509 >> 172.21.15.12:0/174623509 conn(0x7ff4f40fbce0 msgr2=0x7ff4f40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.535+0000 7ff4fb563640 1 -- 172.21.15.12:0/174623509 shutdown_connections 2024-05-09T09:23:06.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.535+0000 7ff4fb563640 1 -- 172.21.15.12:0/174623509 wait complete. 2024-05-09T09:23:06.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.536+0000 7ff4fb563640 1 Processor -- start 2024-05-09T09:23:06.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.536+0000 7ff4fb563640 1 -- start start 2024-05-09T09:23:06.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.536+0000 7ff4fb563640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4f4100770 0x7ff4f4197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.536+0000 7ff4fb563640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4f4101960 0x7ff4f4198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.536+0000 7ff4fb563640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102320 0x7ff4f419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.536+0000 7ff4fb563640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f410c3d0 con 0x7ff4f4100770 2024-05-09T09:23:06.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.536+0000 7ff4fb563640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f410c250 con 0x7ff4f4102320 2024-05-09T09:23:06.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.536+0000 7ff4fb563640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f410c550 con 0x7ff4f4101960 2024-05-09T09:23:06.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.536+0000 7ff4f92ff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4f4100770 0x7ff4f4197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.536+0000 7ff4f92ff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4f4100770 0x7ff4f4197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45516/0 (socket says 172.21.15.12:45516) 2024-05-09T09:23:06.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.536+0000 7ff4f92ff640 1 -- 172.21.15.12:0/2201529289 learned_addr learned my addr 172.21.15.12:0/2201529289 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:06.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.536+0000 7ff4f9b00640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102320 0x7ff4f419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.536+0000 7ff4f8afe640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4f4101960 0x7ff4f4198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.537+0000 7ff4f92ff640 1 -- 172.21.15.12:0/2201529289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4f4101960 msgr2=0x7ff4f4198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.537+0000 7ff4f92ff640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4f4101960 0x7ff4f4198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.537+0000 7ff4f92ff640 1 -- 172.21.15.12:0/2201529289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102320 msgr2=0x7ff4f419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.537+0000 7ff4f92ff640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102320 0x7ff4f419f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.537+0000 7ff4f92ff640 1 -- 172.21.15.12:0/2201529289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f419f750 con 0x7ff4f4100770 2024-05-09T09:23:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.537+0000 7ff4f9b00640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102320 0x7ff4f419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.537+0000 7ff4f92ff640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4f4100770 0x7ff4f4197b30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000e9c0 tx=0x7ff4f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.537+0000 7ff4ea7fc640 1 -- 172.21.15.12:0/2201529289 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f000cd80 con 0x7ff4f4100770 2024-05-09T09:23:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.537+0000 7ff4ea7fc640 1 -- 172.21.15.12:0/2201529289 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff4f00035f0 con 0x7ff4f4100770 2024-05-09T09:23:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.537+0000 7ff4fb563640 1 -- 172.21.15.12:0/2201529289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f419fa40 con 0x7ff4f4100770 2024-05-09T09:23:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.537+0000 7ff4ea7fc640 1 -- 172.21.15.12:0/2201529289 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0010690 con 0x7ff4f4100770 2024-05-09T09:23:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.537+0000 7ff4fb563640 1 -- 172.21.15.12:0/2201529289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f419ff90 con 0x7ff4f4100770 2024-05-09T09:23:06.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.538+0000 7ff4fb563640 1 -- 172.21.15.12:0/2201529289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f41a01c0 con 0x7ff4f4100770 2024-05-09T09:23:06.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.539+0000 7ff4ea7fc640 1 -- 172.21.15.12:0/2201529289 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff4f00037a0 con 0x7ff4f4100770 2024-05-09T09:23:06.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.540+0000 7ff4ea7fc640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff4e00a2b90 0x7ff4e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.540+0000 7ff4ea7fc640 1 -- 172.21.15.12:0/2201529289 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff4f00cef70 con 0x7ff4f4100770 2024-05-09T09:23:06.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.540+0000 7ff4f8afe640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff4e00a2b90 0x7ff4e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.540+0000 7ff4f8afe640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff4e00a2b90 0x7ff4e00a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4199040 tx=0x7ff4e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.545+0000 7ff4ea7fc640 1 -- 172.21.15.12:0/2201529289 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff4f009c370 con 0x7ff4f4100770 2024-05-09T09:23:06.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:06 smithi170 ceph-mon[33632]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:06.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:06.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:06.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:06.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:06.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.765+0000 7ff4fb563640 1 -- 172.21.15.12:0/2201529289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4f41a0390 con 0x7ff4f4100770 2024-05-09T09:23:06.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.766+0000 7ff4ea7fc640 1 -- 172.21.15.12:0/2201529289 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18433 (secure 0 0 0) 0x7ff4f009bc70 con 0x7ff4f4100770 2024-05-09T09:23:06.765 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:06.766 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:22:21.034614+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:23:06.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.768+0000 7ff4fb563640 1 -- 172.21.15.12:0/2201529289 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff4e00a2b90 msgr2=0x7ff4e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.768+0000 7ff4fb563640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff4e00a2b90 0x7ff4e00a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4199040 tx=0x7ff4e4005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:06.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.768+0000 7ff4fb563640 1 -- 172.21.15.12:0/2201529289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4f4100770 msgr2=0x7ff4f4197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.768+0000 7ff4fb563640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4f4100770 0x7ff4f4197b30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000e9c0 tx=0x7ff4f000ee80 comp rx=0 tx=0).stop 2024-05-09T09:23:06.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.768+0000 7ff4fb563640 1 -- 172.21.15.12:0/2201529289 shutdown_connections 2024-05-09T09:23:06.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.768+0000 7ff4fb563640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff4e00a2b90 0x7ff4e00a5050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.768+0000 7ff4fb563640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102320 0x7ff4f419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.768+0000 7ff4fb563640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4f4101960 0x7ff4f4198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.768+0000 7ff4fb563640 1 --2- 172.21.15.12:0/2201529289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4f4100770 0x7ff4f4197b30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.768+0000 7ff4fb563640 1 -- 172.21.15.12:0/2201529289 >> 172.21.15.12:0/2201529289 conn(0x7ff4f40fbce0 msgr2=0x7ff4f40fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:06.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.768+0000 7ff4fb563640 1 -- 172.21.15.12:0/2201529289 shutdown_connections 2024-05-09T09:23:06.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.768+0000 7ff4fb563640 1 -- 172.21.15.12:0/2201529289 wait complete. 2024-05-09T09:23:06.779 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T09:23:06.780 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:23:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.890+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/848992343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e84102330 msgr2=0x7f1e84108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.890+0000 7f1e8b9f9640 1 --2- 172.21.15.12:0/848992343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e84102330 0x7f1e84108bc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009ca0 tx=0x7f1e7c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.890+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/848992343 shutdown_connections 2024-05-09T09:23:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.890+0000 7f1e8b9f9640 1 --2- 172.21.15.12:0/848992343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e84102330 0x7f1e84108bc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.890+0000 7f1e8b9f9640 1 --2- 172.21.15.12:0/848992343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e84101970 0x7f1e84101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.890+0000 7f1e8b9f9640 1 --2- 172.21.15.12:0/848992343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e84100780 0x7f1e84100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.890+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/848992343 >> 172.21.15.12:0/848992343 conn(0x7f1e840fbcf0 msgr2=0x7f1e840fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.890+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/848992343 shutdown_connections 2024-05-09T09:23:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.890+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/848992343 wait complete. 2024-05-09T09:23:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.890+0000 7f1e8b9f9640 1 Processor -- start 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.890+0000 7f1e8b9f9640 1 -- start start 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e8b9f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e84100780 0x7f1e84197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e8b9f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e84101970 0x7f1e84198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e8b9f9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e84102330 0x7f1e8419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e8b9f9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8410c3e0 con 0x7f1e84102330 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e8b9f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8410c260 con 0x7f1e84100780 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e8b9f9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8410c560 con 0x7f1e84101970 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e89f96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e84102330 0x7f1e8419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e89f96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e84102330 0x7f1e8419f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45520/0 (socket says 172.21.15.12:45520) 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e89f96640 1 -- 172.21.15.12:0/730142799 learned_addr learned my addr 172.21.15.12:0/730142799 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e89795640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e84100780 0x7f1e84197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e88f94640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e84101970 0x7f1e84198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e89f96640 1 -- 172.21.15.12:0/730142799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e84101970 msgr2=0x7f1e84198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e89f96640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e84101970 0x7f1e84198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e89f96640 1 -- 172.21.15.12:0/730142799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e84100780 msgr2=0x7f1e84197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e89f96640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e84100780 0x7f1e84197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e89f96640 1 -- 172.21.15.12:0/730142799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8419f7e0 con 0x7f1e84102330 2024-05-09T09:23:06.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e89f96640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e84102330 0x7f1e8419f170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0097b0 tx=0x7f1e7c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e7a7fc640 1 -- 172.21.15.12:0/730142799 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c031650 con 0x7f1e84102330 2024-05-09T09:23:06.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e7a7fc640 1 -- 172.21.15.12:0/730142799 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1e7c0317f0 con 0x7f1e84102330 2024-05-09T09:23:06.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e7a7fc640 1 -- 172.21.15.12:0/730142799 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c02fca0 con 0x7f1e84102330 2024-05-09T09:23:06.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.891+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/730142799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8419fa70 con 0x7f1e84102330 2024-05-09T09:23:06.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.892+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/730142799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8419ff70 con 0x7f1e84102330 2024-05-09T09:23:06.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.893+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/730142799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e841a01a0 con 0x7f1e84102330 2024-05-09T09:23:06.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.894+0000 7f1e7a7fc640 1 -- 172.21.15.12:0/730142799 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1e7c004040 con 0x7f1e84102330 2024-05-09T09:23:06.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.895+0000 7f1e7a7fc640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1e700a2b90 0x7f1e700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.895+0000 7f1e89795640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1e700a2b90 0x7f1e700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.895+0000 7f1e7a7fc640 1 -- 172.21.15.12:0/730142799 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1e7c0f01e0 con 0x7f1e84102330 2024-05-09T09:23:06.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.895+0000 7f1e89795640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1e700a2b90 0x7f1e700a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1e841017d0 tx=0x7f1e8000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:06.898+0000 7f1e7a7fc640 1 -- 172.21.15.12:0/730142799 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1e7c0bd5e0 con 0x7f1e84102330 2024-05-09T09:23:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.084+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/730142799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1e841a0490 con 0x7f1e84102330 2024-05-09T09:23:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.085+0000 7f1e7a7fc640 1 -- 172.21.15.12:0/730142799 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f1e7c0bcee0 con 0x7f1e84102330 2024-05-09T09:23:07.084 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:23:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.087+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/730142799 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1e700a2b90 msgr2=0x7f1e700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.087+0000 7f1e8b9f9640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1e700a2b90 0x7f1e700a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1e841017d0 tx=0x7f1e8000b390 comp rx=0 tx=0).stop 2024-05-09T09:23:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.087+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/730142799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e84102330 msgr2=0x7f1e8419f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.087+0000 7f1e8b9f9640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e84102330 0x7f1e8419f170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0097b0 tx=0x7f1e7c003f10 comp rx=0 tx=0).stop 2024-05-09T09:23:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.087+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/730142799 shutdown_connections 2024-05-09T09:23:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.087+0000 7f1e8b9f9640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1e700a2b90 0x7f1e700a5050 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.087+0000 7f1e8b9f9640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e84102330 0x7f1e8419f170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.087+0000 7f1e8b9f9640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e84101970 0x7f1e84198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.087+0000 7f1e8b9f9640 1 --2- 172.21.15.12:0/730142799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e84100780 0x7f1e84197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.087+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/730142799 >> 172.21.15.12:0/730142799 conn(0x7f1e840fbcf0 msgr2=0x7f1e84069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.087+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/730142799 shutdown_connections 2024-05-09T09:23:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.087+0000 7f1e8b9f9640 1 -- 172.21.15.12:0/730142799 wait complete. 2024-05-09T09:23:07.098 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T09:23:07.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.205+0000 7efc75d96640 1 -- 172.21.15.12:0/1835225272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc70102330 msgr2=0x7efc70104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.205+0000 7efc75d96640 1 --2- 172.21.15.12:0/1835225272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc70102330 0x7efc70104720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efc60009ca0 tx=0x7efc6002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.205+0000 7efc75d96640 1 -- 172.21.15.12:0/1835225272 shutdown_connections 2024-05-09T09:23:07.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.205+0000 7efc75d96640 1 --2- 172.21.15.12:0/1835225272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc70107590 0x7efc701099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.205+0000 7efc75d96640 1 --2- 172.21.15.12:0/1835225272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc70104c60 0x7efc70107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.205+0000 7efc75d96640 1 --2- 172.21.15.12:0/1835225272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc70102330 0x7efc70104720 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.205+0000 7efc75d96640 1 -- 172.21.15.12:0/1835225272 >> 172.21.15.12:0/1835225272 conn(0x7efc700fbce0 msgr2=0x7efc700fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.205+0000 7efc75d96640 1 -- 172.21.15.12:0/1835225272 shutdown_connections 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.205+0000 7efc75d96640 1 -- 172.21.15.12:0/1835225272 wait complete. 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc75d96640 1 Processor -- start 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc75d96640 1 -- start start 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc75d96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc70102330 0x7efc70101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc75d96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc70104c60 0x7efc701002e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc75d96640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc70107590 0x7efc70100820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc75d96640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efc7010d4d0 con 0x7efc70102330 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc75d96640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc7010d350 con 0x7efc70104c60 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc75d96640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efc7010d650 con 0x7efc70107590 2024-05-09T09:23:07.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc6f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc70102330 0x7efc70101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc6f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc70102330 0x7efc70101ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45546/0 (socket says 172.21.15.12:45546) 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc6f7fe640 1 -- 172.21.15.12:0/1467185583 learned_addr learned my addr 172.21.15.12:0/1467185583 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc6effd640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc70104c60 0x7efc701002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc6ffff640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc70107590 0x7efc70100820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc6f7fe640 1 -- 172.21.15.12:0/1467185583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc70107590 msgr2=0x7efc70100820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.206+0000 7efc6f7fe640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc70107590 0x7efc70100820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.207+0000 7efc6f7fe640 1 -- 172.21.15.12:0/1467185583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc70104c60 msgr2=0x7efc701002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.207+0000 7efc6f7fe640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc70104c60 0x7efc701002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.207+0000 7efc6f7fe640 1 -- 172.21.15.12:0/1467185583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc701010b0 con 0x7efc70102330 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.207+0000 7efc6effd640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc70104c60 0x7efc701002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.207+0000 7efc6f7fe640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc70102330 0x7efc70101ca0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc600097b0 tx=0x7efc6002f400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.207+0000 7efc6cff9640 1 -- 172.21.15.12:0/1467185583 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60003c70 con 0x7efc70102330 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.207+0000 7efc75d96640 1 -- 172.21.15.12:0/1467185583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc701a1cd0 con 0x7efc70102330 2024-05-09T09:23:07.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.207+0000 7efc6cff9640 1 -- 172.21.15.12:0/1467185583 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efc60003e10 con 0x7efc70102330 2024-05-09T09:23:07.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.207+0000 7efc6cff9640 1 -- 172.21.15.12:0/1467185583 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6002fd40 con 0x7efc70102330 2024-05-09T09:23:07.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.207+0000 7efc75d96640 1 -- 172.21.15.12:0/1467185583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc701a21c0 con 0x7efc70102330 2024-05-09T09:23:07.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.208+0000 7efc75d96640 1 -- 172.21.15.12:0/1467185583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc700fd7d0 con 0x7efc70102330 2024-05-09T09:23:07.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.210+0000 7efc6cff9640 1 -- 172.21.15.12:0/1467185583 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efc60004040 con 0x7efc70102330 2024-05-09T09:23:07.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.211+0000 7efc6cff9640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efc580a2b90 0x7efc580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.211+0000 7efc6effd640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efc580a2b90 0x7efc580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.211+0000 7efc6cff9640 1 -- 172.21.15.12:0/1467185583 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efc600f02e0 con 0x7efc70102330 2024-05-09T09:23:07.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.212+0000 7efc6effd640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efc580a2b90 0x7efc580a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc70101a10 tx=0x7efc6400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.214+0000 7efc6cff9640 1 -- 172.21.15.12:0/1467185583 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efc600bd6d0 con 0x7efc70102330 2024-05-09T09:23:07.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.403+0000 7efc75d96640 1 -- 172.21.15.12:0/1467185583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efc7002d390 con 0x7efc70102330 2024-05-09T09:23:07.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.403+0000 7efc6cff9640 1 -- 172.21.15.12:0/1467185583 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7efc600bcfd0 con 0x7efc70102330 2024-05-09T09:23:07.403 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:07.404 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2024-05-09T09:23:07.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:23:07.404 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 1, 2024-05-09T09:23:07.404 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": ".mgr" 2024-05-09T09:23:07.404 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:23:07.404 INFO:teuthology.orchestra.run.smithi012.stdout:] 2024-05-09T09:23:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.405+0000 7efc75d96640 1 -- 172.21.15.12:0/1467185583 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efc580a2b90 msgr2=0x7efc580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.405+0000 7efc75d96640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efc580a2b90 0x7efc580a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc70101a10 tx=0x7efc6400b410 comp rx=0 tx=0).stop 2024-05-09T09:23:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.405+0000 7efc75d96640 1 -- 172.21.15.12:0/1467185583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc70102330 msgr2=0x7efc70101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.405+0000 7efc75d96640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc70102330 0x7efc70101ca0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc600097b0 tx=0x7efc6002f400 comp rx=0 tx=0).stop 2024-05-09T09:23:07.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.406+0000 7efc75d96640 1 -- 172.21.15.12:0/1467185583 shutdown_connections 2024-05-09T09:23:07.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.406+0000 7efc75d96640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efc580a2b90 0x7efc580a5050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.406+0000 7efc75d96640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc70107590 0x7efc70100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.406+0000 7efc75d96640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc70104c60 0x7efc701002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.406+0000 7efc75d96640 1 --2- 172.21.15.12:0/1467185583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc70102330 0x7efc70101ca0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.406+0000 7efc75d96640 1 -- 172.21.15.12:0/1467185583 >> 172.21.15.12:0/1467185583 conn(0x7efc700fbce0 msgr2=0x7efc70102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.406+0000 7efc75d96640 1 -- 172.21.15.12:0/1467185583 shutdown_connections 2024-05-09T09:23:07.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.406+0000 7efc75d96640 1 -- 172.21.15.12:0/1467185583 wait complete. 2024-05-09T09:23:07.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:07.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:07.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:07 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3030085925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:07.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:07 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2201529289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:07.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:23:07.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:23:07.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:07.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:07 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/730142799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:23:07.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:07 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:23:07.417 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T09:23:07.417 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T09:23:07.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.528+0000 7f9dd645d640 1 -- 172.21.15.12:0/2085424141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0102a70 msgr2=0x7f9dd0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.528+0000 7f9dd645d640 1 --2- 172.21.15.12:0/2085424141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0102a70 0x7f9dd0102ef0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009900 tx=0x7f9dc402a410 comp rx=0 tx=0).stop 2024-05-09T09:23:07.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.528+0000 7f9dd645d640 1 -- 172.21.15.12:0/2085424141 shutdown_connections 2024-05-09T09:23:07.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.528+0000 7f9dd645d640 1 --2- 172.21.15.12:0/2085424141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dd0103430 0x7f9dd0109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.528+0000 7f9dd645d640 1 --2- 172.21.15.12:0/2085424141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0102a70 0x7f9dd0102ef0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.528+0000 7f9dd645d640 1 --2- 172.21.15.12:0/2085424141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd0101880 0x7f9dd0101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.528+0000 7f9dd645d640 1 -- 172.21.15.12:0/2085424141 >> 172.21.15.12:0/2085424141 conn(0x7f9dd00fce10 msgr2=0x7f9dd00ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.528+0000 7f9dd645d640 1 -- 172.21.15.12:0/2085424141 shutdown_connections 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.528+0000 7f9dd645d640 1 -- 172.21.15.12:0/2085424141 wait complete. 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.529+0000 7f9dd645d640 1 Processor -- start 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.529+0000 7f9dd645d640 1 -- start start 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.529+0000 7f9dd645d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dd0101880 0x7f9dd0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.529+0000 7f9dd645d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd0102a70 0x7f9dd01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.529+0000 7f9dd645d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0103430 0x7f9dd019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.529+0000 7f9dd645d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd010d4e0 con 0x7f9dd0103430 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.529+0000 7f9dd645d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd010d360 con 0x7f9dd0102a70 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.529+0000 7f9dd645d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd010d660 con 0x7f9dd0101880 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.529+0000 7f9dcf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd0102a70 0x7f9dd01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.529+0000 7f9dcf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd0102a70 0x7f9dd01980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:33732/0 (socket says 172.21.15.12:33732) 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.529+0000 7f9dcf7fe640 1 -- 172.21.15.12:0/1792964914 learned_addr learned my addr 172.21.15.12:0/1792964914 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.529+0000 7f9dcffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dd0101880 0x7f9dd0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.529+0000 7f9dd49fa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0103430 0x7f9dd019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.530+0000 7f9dcf7fe640 1 -- 172.21.15.12:0/1792964914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dd0101880 msgr2=0x7f9dd0197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.530+0000 7f9dcf7fe640 1 --2- 172.21.15.12:0/1792964914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dd0101880 0x7f9dd0197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.530+0000 7f9dcf7fe640 1 -- 172.21.15.12:0/1792964914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0103430 msgr2=0x7f9dd019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.530+0000 7f9dcf7fe640 1 --2- 172.21.15.12:0/1792964914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0103430 0x7f9dd019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.530+0000 7f9dcf7fe640 1 -- 172.21.15.12:0/1792964914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd019f7a0 con 0x7f9dd0102a70 2024-05-09T09:23:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.530+0000 7f9dd49fa640 1 --2- 172.21.15.12:0/1792964914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0103430 0x7f9dd019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.530+0000 7f9dcf7fe640 1 --2- 172.21.15.12:0/1792964914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd0102a70 0x7f9dd01980c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009230 tx=0x7f9dc40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.530+0000 7f9dcd7fa640 1 -- 172.21.15.12:0/1792964914 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc403f030 con 0x7f9dd0102a70 2024-05-09T09:23:07.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.530+0000 7f9dd645d640 1 -- 172.21.15.12:0/1792964914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd019fa30 con 0x7f9dd0102a70 2024-05-09T09:23:07.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.530+0000 7f9dcd7fa640 1 -- 172.21.15.12:0/1792964914 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9dc4031980 con 0x7f9dd0102a70 2024-05-09T09:23:07.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.530+0000 7f9dcd7fa640 1 -- 172.21.15.12:0/1792964914 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4035070 con 0x7f9dd0102a70 2024-05-09T09:23:07.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.530+0000 7f9dd645d640 1 -- 172.21.15.12:0/1792964914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd019ff70 con 0x7f9dd0102a70 2024-05-09T09:23:07.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.531+0000 7f9dd645d640 1 -- 172.21.15.12:0/1792964914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd0108ba0 con 0x7f9dd0102a70 2024-05-09T09:23:07.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.532+0000 7f9dcd7fa640 1 -- 172.21.15.12:0/1792964914 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9dc4042030 con 0x7f9dd0102a70 2024-05-09T09:23:07.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.533+0000 7f9dcd7fa640 1 --2- 172.21.15.12:0/1792964914 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9db80a2b90 0x7f9db80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.533+0000 7f9dcffff640 1 --2- 172.21.15.12:0/1792964914 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9db80a2b90 0x7f9db80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.533+0000 7f9dcd7fa640 1 -- 172.21.15.12:0/1792964914 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9dc40ef0e0 con 0x7f9dd0102a70 2024-05-09T09:23:07.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.534+0000 7f9dcffff640 1 --2- 172.21.15.12:0/1792964914 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9db80a2b90 0x7f9db80a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9dd01028d0 tx=0x7f9dc0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.536+0000 7f9dcd7fa640 1 -- 172.21.15.12:0/1792964914 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9dc40bbdf0 con 0x7f9dd0102a70 2024-05-09T09:23:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:07.733+0000 7f9dd645d640 1 -- 172.21.15.12:0/1792964914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9dd002d1c0 con 0x7f9dd0102a70 2024-05-09T09:23:07.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:07.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:07.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:07 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3030085925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:07.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:07 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2201529289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:07.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:23:07.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:23:07.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:07.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:07 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/730142799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:23:07.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:07 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:23:07.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:07.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:07.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:07 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3030085925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:07.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:07 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2201529289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:07.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:23:07.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:23:07.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:07.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:07 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/730142799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:23:07.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:07 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:23:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.281+0000 7f9dcd7fa640 1 -- 172.21.15.12:0/1792964914 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f9dc40c00f0 con 0x7f9dd0102a70 2024-05-09T09:23:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' created 2024-05-09T09:23:08.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.285+0000 7f9dd645d640 1 -- 172.21.15.12:0/1792964914 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9db80a2b90 msgr2=0x7f9db80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.285+0000 7f9dd645d640 1 --2- 172.21.15.12:0/1792964914 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9db80a2b90 0x7f9db80a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9dd01028d0 tx=0x7f9dc0005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:08.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.285+0000 7f9dd645d640 1 -- 172.21.15.12:0/1792964914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd0102a70 msgr2=0x7f9dd01980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.285+0000 7f9dd645d640 1 --2- 172.21.15.12:0/1792964914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd0102a70 0x7f9dd01980c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009230 tx=0x7f9dc40042f0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.285+0000 7f9dd645d640 1 -- 172.21.15.12:0/1792964914 shutdown_connections 2024-05-09T09:23:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.285+0000 7f9dd645d640 1 --2- 172.21.15.12:0/1792964914 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9db80a2b90 0x7f9db80a5050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.285+0000 7f9dd645d640 1 --2- 172.21.15.12:0/1792964914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0103430 0x7f9dd019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.285+0000 7f9dd645d640 1 --2- 172.21.15.12:0/1792964914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd0102a70 0x7f9dd01980c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.285+0000 7f9dd645d640 1 --2- 172.21.15.12:0/1792964914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dd0101880 0x7f9dd0197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.285+0000 7f9dd645d640 1 -- 172.21.15.12:0/1792964914 >> 172.21.15.12:0/1792964914 conn(0x7f9dd00fce10 msgr2=0x7f9dd0105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.285+0000 7f9dd645d640 1 -- 172.21.15.12:0/1792964914 shutdown_connections 2024-05-09T09:23:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.285+0000 7f9dd645d640 1 -- 172.21.15.12:0/1792964914 wait complete. 2024-05-09T09:23:08.296 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-09T09:23:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.412+0000 7f66f0812640 1 -- 172.21.15.12:0/1682277323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66ec102a50 msgr2=0x7f66ec102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.412+0000 7f66f0812640 1 --2- 172.21.15.12:0/1682277323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66ec102a50 0x7f66ec102ed0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009900 tx=0x7f66d402a410 comp rx=0 tx=0).stop 2024-05-09T09:23:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.412+0000 7f66f0812640 1 -- 172.21.15.12:0/1682277323 shutdown_connections 2024-05-09T09:23:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.412+0000 7f66f0812640 1 --2- 172.21.15.12:0/1682277323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66ec103410 0x7f66ec109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.412+0000 7f66f0812640 1 --2- 172.21.15.12:0/1682277323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66ec102a50 0x7f66ec102ed0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.412+0000 7f66f0812640 1 --2- 172.21.15.12:0/1682277323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66ec101860 0x7f66ec101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.412+0000 7f66f0812640 1 -- 172.21.15.12:0/1682277323 >> 172.21.15.12:0/1682277323 conn(0x7f66ec0fcdf0 msgr2=0x7f66ec0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.412+0000 7f66f0812640 1 -- 172.21.15.12:0/1682277323 shutdown_connections 2024-05-09T09:23:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.412+0000 7f66f0812640 1 -- 172.21.15.12:0/1682277323 wait complete. 2024-05-09T09:23:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.413+0000 7f66f0812640 1 Processor -- start 2024-05-09T09:23:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.413+0000 7f66f0812640 1 -- start start 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66f0812640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66ec101860 0x7f66ec197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66f0812640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66ec102a50 0x7f66ec1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66f0812640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66ec103410 0x7f66ec19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66f0812640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec10d4c0 con 0x7f66ec102a50 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66f0812640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec10d340 con 0x7f66ec101860 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66f0812640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec10d640 con 0x7f66ec103410 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66e3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66ec102a50 0x7f66ec1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66e3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66ec102a50 0x7f66ec1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45572/0 (socket says 172.21.15.12:45572) 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66e3fff640 1 -- 172.21.15.12:0/2337189071 learned_addr learned my addr 172.21.15.12:0/2337189071 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66ea59c640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66ec101860 0x7f66ec197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66ead9d640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66ec103410 0x7f66ec19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66e3fff640 1 -- 172.21.15.12:0/2337189071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66ec103410 msgr2=0x7f66ec19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66e3fff640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66ec103410 0x7f66ec19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66e3fff640 1 -- 172.21.15.12:0/2337189071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66ec101860 msgr2=0x7f66ec197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66e3fff640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66ec101860 0x7f66ec197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66e3fff640 1 -- 172.21.15.12:0/2337189071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ec19f780 con 0x7f66ec102a50 2024-05-09T09:23:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66ea59c640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66ec101860 0x7f66ec197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66e3fff640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66ec102a50 0x7f66ec1980a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009230 tx=0x7f66d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.414+0000 7f66e37fe640 1 -- 172.21.15.12:0/2337189071 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d403f030 con 0x7f66ec102a50 2024-05-09T09:23:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.415+0000 7f66f0812640 1 -- 172.21.15.12:0/2337189071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ec19fa10 con 0x7f66ec102a50 2024-05-09T09:23:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.415+0000 7f66e37fe640 1 -- 172.21.15.12:0/2337189071 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f66d4031980 con 0x7f66ec102a50 2024-05-09T09:23:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.415+0000 7f66e37fe640 1 -- 172.21.15.12:0/2337189071 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4035070 con 0x7f66ec102a50 2024-05-09T09:23:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.415+0000 7f66f0812640 1 -- 172.21.15.12:0/2337189071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ec19ff50 con 0x7f66ec102a50 2024-05-09T09:23:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.416+0000 7f66f0812640 1 -- 172.21.15.12:0/2337189071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66ec108b80 con 0x7f66ec102a50 2024-05-09T09:23:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.418+0000 7f66e37fe640 1 -- 172.21.15.12:0/2337189071 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f66d4042030 con 0x7f66ec102a50 2024-05-09T09:23:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.418+0000 7f66e37fe640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f66d80a2b90 0x7f66d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.418+0000 7f66ea59c640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f66d80a2b90 0x7f66d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.418+0000 7f66e37fe640 1 -- 172.21.15.12:0/2337189071 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f66d40ef270 con 0x7f66ec102a50 2024-05-09T09:23:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.419+0000 7f66ea59c640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f66d80a2b90 0x7f66d80a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f66ec1028b0 tx=0x7f66dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.423+0000 7f66e37fe640 1 -- 172.21.15.12:0/2337189071 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f66d40bbe10 con 0x7f66ec102a50 2024-05-09T09:23:08.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:08.629+0000 7f66f0812640 1 -- 172.21.15.12:0/2337189071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f66ec062430 con 0x7f66ec102a50 2024-05-09T09:23:08.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:08 smithi012 ceph-mon[30140]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:08.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:08 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1467185583' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T09:23:08.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:08 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1792964914' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T09:23:08.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:08 smithi012 ceph-mon[30140]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T09:23:08.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:08 smithi170 ceph-mon[33632]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:08.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:08 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1467185583' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T09:23:08.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:08 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1792964914' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T09:23:08.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:08 smithi170 ceph-mon[33632]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T09:23:08.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:08 smithi016 ceph-mon[33335]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:08.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:08 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1467185583' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T09:23:08.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:08 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1792964914' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T09:23:08.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:08 smithi016 ceph-mon[33335]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T09:23:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.282+0000 7f66e37fe640 1 -- 172.21.15.12:0/2337189071 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7f66d40c0110 con 0x7f66ec102a50 2024-05-09T09:23:09.282 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' created 2024-05-09T09:23:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.284+0000 7f66f0812640 1 -- 172.21.15.12:0/2337189071 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f66d80a2b90 msgr2=0x7f66d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.284+0000 7f66f0812640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f66d80a2b90 0x7f66d80a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f66ec1028b0 tx=0x7f66dc005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.284+0000 7f66f0812640 1 -- 172.21.15.12:0/2337189071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66ec102a50 msgr2=0x7f66ec1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.284+0000 7f66f0812640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66ec102a50 0x7f66ec1980a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009230 tx=0x7f66d40042f0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.284+0000 7f66f0812640 1 -- 172.21.15.12:0/2337189071 shutdown_connections 2024-05-09T09:23:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.284+0000 7f66f0812640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f66d80a2b90 0x7f66d80a5050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.284+0000 7f66f0812640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66ec103410 0x7f66ec19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.284+0000 7f66f0812640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66ec102a50 0x7f66ec1980a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.284+0000 7f66f0812640 1 --2- 172.21.15.12:0/2337189071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66ec101860 0x7f66ec197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.284+0000 7f66f0812640 1 -- 172.21.15.12:0/2337189071 >> 172.21.15.12:0/2337189071 conn(0x7f66ec0fcdf0 msgr2=0x7f66ec105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.284+0000 7f66f0812640 1 -- 172.21.15.12:0/2337189071 shutdown_connections 2024-05-09T09:23:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.284+0000 7f66f0812640 1 -- 172.21.15.12:0/2337189071 wait complete. 2024-05-09T09:23:09.306 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T09:23:09.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.416+0000 7f5adcae1640 1 -- 172.21.15.12:0/3732752893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad8100790 msgr2=0x7f5ad8100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.416+0000 7f5adcae1640 1 --2- 172.21.15.12:0/3732752893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad8100790 0x7f5ad8100bb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009900 tx=0x7f5ac802a410 comp rx=0 tx=0).stop 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.416+0000 7f5adcae1640 1 -- 172.21.15.12:0/3732752893 shutdown_connections 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.416+0000 7f5adcae1640 1 --2- 172.21.15.12:0/3732752893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8102340 0x7f5ad8108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.417+0000 7f5adcae1640 1 --2- 172.21.15.12:0/3732752893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ad8101980 0x7f5ad8101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.417+0000 7f5adcae1640 1 --2- 172.21.15.12:0/3732752893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad8100790 0x7f5ad8100bb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.417+0000 7f5adcae1640 1 -- 172.21.15.12:0/3732752893 >> 172.21.15.12:0/3732752893 conn(0x7f5ad80fbd20 msgr2=0x7f5ad80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.417+0000 7f5adcae1640 1 -- 172.21.15.12:0/3732752893 shutdown_connections 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.417+0000 7f5adcae1640 1 -- 172.21.15.12:0/3732752893 wait complete. 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.417+0000 7f5adcae1640 1 Processor -- start 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.417+0000 7f5adcae1640 1 -- start start 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.417+0000 7f5adcae1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ad8100790 0x7f5ad8197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5adcae1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad8101980 0x7f5ad81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5adcae1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8102340 0x7f5ad819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5adcae1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad810c3f0 con 0x7f5ad8101980 2024-05-09T09:23:09.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5adcae1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad810c270 con 0x7f5ad8102340 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5adcae1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad810c570 con 0x7f5ad8100790 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5ad5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad8101980 0x7f5ad81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5ad5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad8101980 0x7f5ad81981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45582/0 (socket says 172.21.15.12:45582) 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5ad5d9b640 1 -- 172.21.15.12:0/3985891705 learned_addr learned my addr 172.21.15.12:0/3985891705 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5ad659c640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ad8100790 0x7f5ad8197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5ad6d9d640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8102340 0x7f5ad819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5ad5d9b640 1 -- 172.21.15.12:0/3985891705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ad8100790 msgr2=0x7f5ad8197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5ad5d9b640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ad8100790 0x7f5ad8197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5ad5d9b640 1 -- 172.21.15.12:0/3985891705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8102340 msgr2=0x7f5ad819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5ad5d9b640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8102340 0x7f5ad819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5ad5d9b640 1 -- 172.21.15.12:0/3985891705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad819f830 con 0x7f5ad8101980 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5ad6d9d640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8102340 0x7f5ad819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5ad5d9b640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad8101980 0x7f5ad81981b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000d950 tx=0x7f5ac000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5acf7fe640 1 -- 172.21.15.12:0/3985891705 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0017030 con 0x7f5ad8101980 2024-05-09T09:23:09.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5acf7fe640 1 -- 172.21.15.12:0/3985891705 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5ac0004020 con 0x7f5ad8101980 2024-05-09T09:23:09.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5adcae1640 1 -- 172.21.15.12:0/3985891705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad819fb20 con 0x7f5ad8101980 2024-05-09T09:23:09.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5acf7fe640 1 -- 172.21.15.12:0/3985891705 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00047a0 con 0x7f5ad8101980 2024-05-09T09:23:09.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.418+0000 7f5adcae1640 1 -- 172.21.15.12:0/3985891705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad81a00c0 con 0x7f5ad8101980 2024-05-09T09:23:09.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.419+0000 7f5adcae1640 1 -- 172.21.15.12:0/3985891705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad81a02f0 con 0x7f5ad8101980 2024-05-09T09:23:09.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.421+0000 7f5acf7fe640 1 -- 172.21.15.12:0/3985891705 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5ac00041d0 con 0x7f5ad8101980 2024-05-09T09:23:09.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.421+0000 7f5acf7fe640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5ac40a2c10 0x7f5ac40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.421+0000 7f5ad659c640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5ac40a2c10 0x7f5ac40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.421+0000 7f5acf7fe640 1 -- 172.21.15.12:0/3985891705 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f5ac00cea10 con 0x7f5ad8101980 2024-05-09T09:23:09.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.422+0000 7f5ad659c640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5ac40a2c10 0x7f5ac40a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5ad81990f0 tx=0x7f5ac8031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.426+0000 7f5acf7fe640 1 -- 172.21.15.12:0/3985891705 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5ac009ba80 con 0x7f5ad8101980 2024-05-09T09:23:09.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:09 smithi012 ceph-mon[30140]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T09:23:09.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:09 smithi012 ceph-mon[30140]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T09:23:09.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:09 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2337189071' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T09:23:09.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:23:09.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:23:09.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:09.654+0000 7f5adcae1640 1 -- 172.21.15.12:0/3985891705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5ad81a04c0 con 0x7f5ad8101980 2024-05-09T09:23:09.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:09 smithi170 ceph-mon[33632]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T09:23:09.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:09 smithi170 ceph-mon[33632]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T09:23:09.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:09 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2337189071' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T09:23:09.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:23:09.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:23:09.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:09 smithi016 ceph-mon[33335]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:09 smithi016 ceph-mon[33335]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:09 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2337189071' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:23:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.307+0000 7f5acf7fe640 1 -- 172.21.15.12:0/3985891705 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f5ac009b380 con 0x7f5ad8101980 2024-05-09T09:23:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T09:23:10.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.311+0000 7f5adcae1640 1 -- 172.21.15.12:0/3985891705 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5ac40a2c10 msgr2=0x7f5ac40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.311+0000 7f5adcae1640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5ac40a2c10 0x7f5ac40a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5ad81990f0 tx=0x7f5ac8031510 comp rx=0 tx=0).stop 2024-05-09T09:23:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.311+0000 7f5adcae1640 1 -- 172.21.15.12:0/3985891705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad8101980 msgr2=0x7f5ad81981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.311+0000 7f5adcae1640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad8101980 0x7f5ad81981b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000d950 tx=0x7f5ac000de10 comp rx=0 tx=0).stop 2024-05-09T09:23:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.311+0000 7f5adcae1640 1 -- 172.21.15.12:0/3985891705 shutdown_connections 2024-05-09T09:23:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.311+0000 7f5adcae1640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5ac40a2c10 0x7f5ac40a50d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.311+0000 7f5adcae1640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8102340 0x7f5ad819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.311+0000 7f5adcae1640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ad8101980 0x7f5ad81981b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.311+0000 7f5adcae1640 1 --2- 172.21.15.12:0/3985891705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ad8100790 0x7f5ad8197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.311+0000 7f5adcae1640 1 -- 172.21.15.12:0/3985891705 >> 172.21.15.12:0/3985891705 conn(0x7f5ad80fbd20 msgr2=0x7f5ad80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:10.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.311+0000 7f5adcae1640 1 -- 172.21.15.12:0/3985891705 shutdown_connections 2024-05-09T09:23:10.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.312+0000 7f5adcae1640 1 -- 172.21.15.12:0/3985891705 wait complete. 2024-05-09T09:23:10.335 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-09T09:23:10.336 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-09T09:23:10.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.455+0000 7f2282b5a640 1 -- 172.21.15.12:0/2613300655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f227c1022f0 msgr2=0x7f227c1046e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.455+0000 7f2282b5a640 1 --2- 172.21.15.12:0/2613300655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f227c1022f0 0x7f227c1046e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2270009900 tx=0x7f227002a410 comp rx=0 tx=0).stop 2024-05-09T09:23:10.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.455+0000 7f2282b5a640 1 -- 172.21.15.12:0/2613300655 shutdown_connections 2024-05-09T09:23:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.455+0000 7f2282b5a640 1 --2- 172.21.15.12:0/2613300655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f227c107550 0x7f227c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.455+0000 7f2282b5a640 1 --2- 172.21.15.12:0/2613300655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227c104c20 0x7f227c107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.455+0000 7f2282b5a640 1 --2- 172.21.15.12:0/2613300655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f227c1022f0 0x7f227c1046e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.455+0000 7f2282b5a640 1 -- 172.21.15.12:0/2613300655 >> 172.21.15.12:0/2613300655 conn(0x7f227c0fbce0 msgr2=0x7f227c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.455+0000 7f2282b5a640 1 -- 172.21.15.12:0/2613300655 shutdown_connections 2024-05-09T09:23:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.455+0000 7f2282b5a640 1 -- 172.21.15.12:0/2613300655 wait complete. 2024-05-09T09:23:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.455+0000 7f2282b5a640 1 Processor -- start 2024-05-09T09:23:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.455+0000 7f2282b5a640 1 -- start start 2024-05-09T09:23:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f2282b5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227c1022f0 0x7f227c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f2282b5a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f227c104c20 0x7f227c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f2282b5a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f227c107550 0x7f227c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f2282b5a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f227c10d4d0 con 0x7f227c104c20 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f2282b5a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f227c10d350 con 0x7f227c1022f0 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f2282b5a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f227c10d650 con 0x7f227c107550 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f227bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f227c104c20 0x7f227c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f227bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f227c104c20 0x7f227c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45600/0 (socket says 172.21.15.12:45600) 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f227bfff640 1 -- 172.21.15.12:0/4037557973 learned_addr learned my addr 172.21.15.12:0/4037557973 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f22808f6640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227c1022f0 0x7f227c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f22810f7640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f227c107550 0x7f227c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f227bfff640 1 -- 172.21.15.12:0/4037557973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f227c107550 msgr2=0x7f227c19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f227bfff640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f227c107550 0x7f227c19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f227bfff640 1 -- 172.21.15.12:0/4037557973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227c1022f0 msgr2=0x7f227c197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f227bfff640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227c1022f0 0x7f227c197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f227bfff640 1 -- 172.21.15.12:0/4037557973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227c19f800 con 0x7f227c104c20 2024-05-09T09:23:10.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f227bfff640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f227c104c20 0x7f227c198120 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f226800d770 tx=0x7f226800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:10.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f2279ffb640 1 -- 172.21.15.12:0/4037557973 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268017070 con 0x7f227c104c20 2024-05-09T09:23:10.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f2282b5a640 1 -- 172.21.15.12:0/4037557973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227c19faf0 con 0x7f227c104c20 2024-05-09T09:23:10.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f2279ffb640 1 -- 172.21.15.12:0/4037557973 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2268009c30 con 0x7f227c104c20 2024-05-09T09:23:10.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.456+0000 7f2279ffb640 1 -- 172.21.15.12:0/4037557973 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226800f620 con 0x7f227c104c20 2024-05-09T09:23:10.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.457+0000 7f2282b5a640 1 -- 172.21.15.12:0/4037557973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f227c1a0040 con 0x7f227c104c20 2024-05-09T09:23:10.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.457+0000 7f2282b5a640 1 -- 172.21.15.12:0/4037557973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f227c0fd7d0 con 0x7f227c104c20 2024-05-09T09:23:10.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.459+0000 7f2279ffb640 1 -- 172.21.15.12:0/4037557973 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f226801a070 con 0x7f227c104c20 2024-05-09T09:23:10.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.459+0000 7f2279ffb640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f226c0a2b90 0x7f226c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.460+0000 7f22808f6640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f226c0a2b90 0x7f226c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.460+0000 7f2279ffb640 1 -- 172.21.15.12:0/4037557973 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f22680ce8f0 con 0x7f227c104c20 2024-05-09T09:23:10.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.460+0000 7f22808f6640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f226c0a2b90 0x7f226c0a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2270000c00 tx=0x7f2270031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:10.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.463+0000 7f2279ffb640 1 -- 172.21.15.12:0/4037557973 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2268009de0 con 0x7f227c104c20 2024-05-09T09:23:10.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:10.659+0000 7f2282b5a640 1 -- 172.21.15.12:0/4037557973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f227c02d3c0 con 0x7f227c104c20 2024-05-09T09:23:10.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:10 smithi012 ceph-mon[30140]: pgmap v284: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:10.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:10 smithi012 ceph-mon[30140]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:23:10.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:10 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2337189071' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-09T09:23:10.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:10 smithi012 ceph-mon[30140]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T09:23:10.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:10 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3985891705' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T09:23:10.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:10 smithi012 ceph-mon[30140]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T09:23:10.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:10 smithi170 ceph-mon[33632]: pgmap v284: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:10.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:10 smithi170 ceph-mon[33632]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:23:10.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:10 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2337189071' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-09T09:23:10.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:10 smithi170 ceph-mon[33632]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T09:23:10.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:10 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3985891705' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T09:23:10.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:10 smithi170 ceph-mon[33632]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T09:23:10.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:10 smithi016 ceph-mon[33335]: pgmap v284: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:10.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:10 smithi016 ceph-mon[33335]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:23:10.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:10 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2337189071' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-09T09:23:10.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:10 smithi016 ceph-mon[33335]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T09:23:10.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:10 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3985891705' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T09:23:10.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:10 smithi016 ceph-mon[33335]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T09:23:11.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.309+0000 7f2279ffb640 1 -- 172.21.15.12:0/4037557973 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f2268095e20 con 0x7f227c104c20 2024-05-09T09:23:11.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.313+0000 7f2282b5a640 1 -- 172.21.15.12:0/4037557973 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f226c0a2b90 msgr2=0x7f226c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.313+0000 7f2282b5a640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f226c0a2b90 0x7f226c0a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2270000c00 tx=0x7f2270031510 comp rx=0 tx=0).stop 2024-05-09T09:23:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.313+0000 7f2282b5a640 1 -- 172.21.15.12:0/4037557973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f227c104c20 msgr2=0x7f227c198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.313+0000 7f2282b5a640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f227c104c20 0x7f227c198120 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f226800d770 tx=0x7f226800dc30 comp rx=0 tx=0).stop 2024-05-09T09:23:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.313+0000 7f2282b5a640 1 -- 172.21.15.12:0/4037557973 shutdown_connections 2024-05-09T09:23:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.313+0000 7f2282b5a640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f226c0a2b90 0x7f226c0a5050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.313+0000 7f2282b5a640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f227c107550 0x7f227c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.313+0000 7f2282b5a640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f227c104c20 0x7f227c198120 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.313+0000 7f2282b5a640 1 --2- 172.21.15.12:0/4037557973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227c1022f0 0x7f227c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.313+0000 7f2282b5a640 1 -- 172.21.15.12:0/4037557973 >> 172.21.15.12:0/4037557973 conn(0x7f227c0fbce0 msgr2=0x7f227c102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.313+0000 7f2282b5a640 1 -- 172.21.15.12:0/4037557973 shutdown_connections 2024-05-09T09:23:11.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.313+0000 7f2282b5a640 1 -- 172.21.15.12:0/4037557973 wait complete. 2024-05-09T09:23:11.329 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T09:23:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.450+0000 7f6761789640 1 -- 172.21.15.12:0/879494215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675c0ffb50 msgr2=0x7f675c0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.450+0000 7f6761789640 1 --2- 172.21.15.12:0/879494215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675c0ffb50 0x7f675c0fff70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f674c009930 tx=0x7f674c02a420 comp rx=0 tx=0).stop 2024-05-09T09:23:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.450+0000 7f6761789640 1 -- 172.21.15.12:0/879494215 shutdown_connections 2024-05-09T09:23:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.450+0000 7f6761789640 1 --2- 172.21.15.12:0/879494215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f675c1034e0 0x7f675c109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.450+0000 7f6761789640 1 --2- 172.21.15.12:0/879494215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f675c102b20 0x7f675c102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.450+0000 7f6761789640 1 --2- 172.21.15.12:0/879494215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675c0ffb50 0x7f675c0fff70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.450+0000 7f6761789640 1 -- 172.21.15.12:0/879494215 >> 172.21.15.12:0/879494215 conn(0x7f675c0fb4a0 msgr2=0x7f675c0fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.450+0000 7f6761789640 1 -- 172.21.15.12:0/879494215 shutdown_connections 2024-05-09T09:23:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.450+0000 7f6761789640 1 -- 172.21.15.12:0/879494215 wait complete. 2024-05-09T09:23:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f6761789640 1 Processor -- start 2024-05-09T09:23:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f6761789640 1 -- start start 2024-05-09T09:23:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f6761789640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675c0ffb50 0x7f675c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f6761789640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f675c102b20 0x7f675c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f6761789640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f675c1034e0 0x7f675c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f6761789640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f675c10d5c0 con 0x7f675c0ffb50 2024-05-09T09:23:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f6761789640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f675c10d440 con 0x7f675c1034e0 2024-05-09T09:23:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f6761789640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f675c10d740 con 0x7f675c102b20 2024-05-09T09:23:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f675affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675c0ffb50 0x7f675c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f675affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675c0ffb50 0x7f675c197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45618/0 (socket says 172.21.15.12:45618) 2024-05-09T09:23:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f675affd640 1 -- 172.21.15.12:0/3780625930 learned_addr learned my addr 172.21.15.12:0/3780625930 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f675b7fe640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f675c1034e0 0x7f675c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f6753fff640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f675c102b20 0x7f675c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f675affd640 1 -- 172.21.15.12:0/3780625930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f675c102b20 msgr2=0x7f675c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f675affd640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f675c102b20 0x7f675c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f675affd640 1 -- 172.21.15.12:0/3780625930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f675c1034e0 msgr2=0x7f675c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f675affd640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f675c1034e0 0x7f675c19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.451+0000 7f675affd640 1 -- 172.21.15.12:0/3780625930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c19f860 con 0x7f675c0ffb50 2024-05-09T09:23:11.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.452+0000 7f675affd640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675c0ffb50 0x7f675c197c50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f674c009230 tx=0x7f674c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:11.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.452+0000 7f6758ff9640 1 -- 172.21.15.12:0/3780625930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c03f070 con 0x7f675c0ffb50 2024-05-09T09:23:11.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.452+0000 7f6758ff9640 1 -- 172.21.15.12:0/3780625930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f674c005620 con 0x7f675c0ffb50 2024-05-09T09:23:11.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.452+0000 7f6761789640 1 -- 172.21.15.12:0/3780625930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675c19faf0 con 0x7f675c0ffb50 2024-05-09T09:23:11.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.452+0000 7f6758ff9640 1 -- 172.21.15.12:0/3780625930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c035030 con 0x7f675c0ffb50 2024-05-09T09:23:11.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.452+0000 7f6761789640 1 -- 172.21.15.12:0/3780625930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675c1a0030 con 0x7f675c0ffb50 2024-05-09T09:23:11.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.453+0000 7f6761789640 1 -- 172.21.15.12:0/3780625930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f675c108ca0 con 0x7f675c0ffb50 2024-05-09T09:23:11.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.454+0000 7f6758ff9640 1 -- 172.21.15.12:0/3780625930 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f674c042070 con 0x7f675c0ffb50 2024-05-09T09:23:11.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.455+0000 7f6758ff9640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67480a2c10 0x7f67480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:11.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.455+0000 7f6753fff640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67480a2c10 0x7f67480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:11.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.455+0000 7f6758ff9640 1 -- 172.21.15.12:0/3780625930 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f674c0efb00 con 0x7f675c0ffb50 2024-05-09T09:23:11.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.455+0000 7f6753fff640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67480a2c10 0x7f67480a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6744009890 tx=0x7f6744008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:11.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.459+0000 7f6758ff9640 1 -- 172.21.15.12:0/3780625930 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f674c0bca30 con 0x7f675c0ffb50 2024-05-09T09:23:11.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:11.649+0000 7f6761789640 1 -- 172.21.15.12:0/3780625930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f675c0624d0 con 0x7f675c0ffb50 2024-05-09T09:23:11.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:11 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3985891705' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T09:23:11.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:11 smithi012 ceph-mon[30140]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T09:23:11.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:11 smithi012 ceph-mon[30140]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T09:23:11.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:11 smithi012 ceph-mon[30140]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T09:23:11.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:11 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4037557973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-09T09:23:11.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:11 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4037557973' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-09T09:23:11.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:11 smithi012 ceph-mon[30140]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T09:23:11.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:11 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3985891705' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T09:23:11.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:11 smithi170 ceph-mon[33632]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T09:23:11.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:11 smithi170 ceph-mon[33632]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T09:23:11.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:11 smithi170 ceph-mon[33632]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T09:23:11.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:11 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/4037557973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-09T09:23:11.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:11 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/4037557973' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-09T09:23:11.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:11 smithi170 ceph-mon[33632]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T09:23:11.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:11 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3985891705' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T09:23:11.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:11 smithi016 ceph-mon[33335]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T09:23:11.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:11 smithi016 ceph-mon[33335]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T09:23:11.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:11 smithi016 ceph-mon[33335]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T09:23:11.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:11 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/4037557973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-09T09:23:11.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:11 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/4037557973' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-09T09:23:11.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:11 smithi016 ceph-mon[33335]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T09:23:12.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:12 smithi012 ceph-mon[30140]: pgmap v287: 97 pgs: 32 creating+peering, 14 active+clean, 51 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:12.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:12 smithi012 ceph-mon[30140]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T09:23:12.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:12 smithi012 ceph-mon[30140]: Cluster is now healthy 2024-05-09T09:23:12.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:12 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] up:active 2024-05-09T09:23:12.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:12 smithi012 ceph-mon[30140]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:12.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:12 smithi012 ceph-mon[30140]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:12.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:12 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3780625930' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T09:23:12.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:12 smithi170 ceph-mon[33632]: pgmap v287: 97 pgs: 32 creating+peering, 14 active+clean, 51 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:12.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:12 smithi170 ceph-mon[33632]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T09:23:12.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:12 smithi170 ceph-mon[33632]: Cluster is now healthy 2024-05-09T09:23:12.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:12 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] up:active 2024-05-09T09:23:12.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:12 smithi170 ceph-mon[33632]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:12.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:12 smithi170 ceph-mon[33632]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:12.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:12 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3780625930' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T09:23:12.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:12 smithi016 ceph-mon[33335]: pgmap v287: 97 pgs: 32 creating+peering, 14 active+clean, 51 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:23:12.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:12 smithi016 ceph-mon[33335]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T09:23:12.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:12 smithi016 ceph-mon[33335]: Cluster is now healthy 2024-05-09T09:23:12.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:12 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] up:active 2024-05-09T09:23:12.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:12 smithi016 ceph-mon[33335]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:12.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:12 smithi016 ceph-mon[33335]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:12.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:12 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3780625930' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T09:23:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.337+0000 7f6758ff9640 1 -- 172.21.15.12:0/3780625930 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f674c0bc330 con 0x7f675c0ffb50 2024-05-09T09:23:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data_ec' created 2024-05-09T09:23:13.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.340+0000 7f6761789640 1 -- 172.21.15.12:0/3780625930 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67480a2c10 msgr2=0x7f67480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:13.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.340+0000 7f6761789640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67480a2c10 0x7f67480a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6744009890 tx=0x7f6744008040 comp rx=0 tx=0).stop 2024-05-09T09:23:13.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.340+0000 7f6761789640 1 -- 172.21.15.12:0/3780625930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675c0ffb50 msgr2=0x7f675c197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:13.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.340+0000 7f6761789640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675c0ffb50 0x7f675c197c50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f674c009230 tx=0x7f674c005020 comp rx=0 tx=0).stop 2024-05-09T09:23:13.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.340+0000 7f6761789640 1 -- 172.21.15.12:0/3780625930 shutdown_connections 2024-05-09T09:23:13.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.340+0000 7f6761789640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67480a2c10 0x7f67480a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:13.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.341+0000 7f6761789640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f675c1034e0 0x7f675c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:13.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.341+0000 7f6761789640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f675c102b20 0x7f675c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:13.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.341+0000 7f6761789640 1 --2- 172.21.15.12:0/3780625930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675c0ffb50 0x7f675c197c50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:13.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.341+0000 7f6761789640 1 -- 172.21.15.12:0/3780625930 >> 172.21.15.12:0/3780625930 conn(0x7f675c0fb4a0 msgr2=0x7f675c105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:13.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.341+0000 7f6761789640 1 -- 172.21.15.12:0/3780625930 shutdown_connections 2024-05-09T09:23:13.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.341+0000 7f6761789640 1 -- 172.21.15.12:0/3780625930 wait complete. 2024-05-09T09:23:13.359 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-09T09:23:13.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.475+0000 7f021f5c8640 1 -- 172.21.15.12:0/1702451690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0218100a60 msgr2=0x7f0218100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:13.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.475+0000 7f021f5c8640 1 --2- 172.21.15.12:0/1702451690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0218100a60 0x7f0218100ec0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f020c009900 tx=0x7f020c02a410 comp rx=0 tx=0).stop 2024-05-09T09:23:13.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.475+0000 7f021f5c8640 1 -- 172.21.15.12:0/1702451690 shutdown_connections 2024-05-09T09:23:13.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.475+0000 7f021f5c8640 1 --2- 172.21.15.12:0/1702451690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0218101400 0x7f0218101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:13.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.475+0000 7f021f5c8640 1 --2- 172.21.15.12:0/1702451690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0218100a60 0x7f0218100ec0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:13.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.475+0000 7f021f5c8640 1 --2- 172.21.15.12:0/1702451690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0218100100 0x7f0218100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:13.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.475+0000 7f021f5c8640 1 -- 172.21.15.12:0/1702451690 >> 172.21.15.12:0/1702451690 conn(0x7f02180fbd20 msgr2=0x7f02180fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:13.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.475+0000 7f021f5c8640 1 -- 172.21.15.12:0/1702451690 shutdown_connections 2024-05-09T09:23:13.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.475+0000 7f021f5c8640 1 -- 172.21.15.12:0/1702451690 wait complete. 2024-05-09T09:23:13.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.476+0000 7f021f5c8640 1 Processor -- start 2024-05-09T09:23:13.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.476+0000 7f021f5c8640 1 -- start start 2024-05-09T09:23:13.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.476+0000 7f021f5c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0218100100 0x7f0218199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:13.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.476+0000 7f021f5c8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0218100a60 0x7f021819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:13.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.476+0000 7f021f5c8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0218101400 0x7f02181a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:13.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.476+0000 7f021f5c8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f021810f6c0 con 0x7f0218101400 2024-05-09T09:23:13.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.476+0000 7f021f5c8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f021810f540 con 0x7f0218100100 2024-05-09T09:23:13.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.476+0000 7f021f5c8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f021810f840 con 0x7f0218100a60 2024-05-09T09:23:13.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.476+0000 7f021db65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0218101400 0x7f02181a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.476+0000 7f021db65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0218101400 0x7f02181a1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37444/0 (socket says 172.21.15.12:37444) 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.476+0000 7f021db65640 1 -- 172.21.15.12:0/3075777014 learned_addr learned my addr 172.21.15.12:0/3075777014 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.476+0000 7f021d364640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0218100100 0x7f0218199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.476+0000 7f021cb63640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0218100a60 0x7f021819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.477+0000 7f021db65640 1 -- 172.21.15.12:0/3075777014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0218100a60 msgr2=0x7f021819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.477+0000 7f021db65640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0218100a60 0x7f021819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.477+0000 7f021db65640 1 -- 172.21.15.12:0/3075777014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0218100100 msgr2=0x7f0218199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.477+0000 7f021db65640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0218100100 0x7f0218199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.477+0000 7f021db65640 1 -- 172.21.15.12:0/3075777014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02181a19d0 con 0x7f0218101400 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.477+0000 7f021db65640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0218101400 0x7f02181a1360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f021000cad0 tx=0x7f021000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.477+0000 7f020a7fc640 1 -- 172.21.15.12:0/3075777014 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021000ae20 con 0x7f0218101400 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.477+0000 7f020a7fc640 1 -- 172.21.15.12:0/3075777014 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0210005020 con 0x7f0218101400 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.477+0000 7f021f5c8640 1 -- 172.21.15.12:0/3075777014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02181a1cc0 con 0x7f0218101400 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.477+0000 7f020a7fc640 1 -- 172.21.15.12:0/3075777014 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021000f690 con 0x7f0218101400 2024-05-09T09:23:13.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.477+0000 7f021f5c8640 1 -- 172.21.15.12:0/3075777014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02181a21c0 con 0x7f0218101400 2024-05-09T09:23:13.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.478+0000 7f021f5c8640 1 -- 172.21.15.12:0/3075777014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f021810ae50 con 0x7f0218101400 2024-05-09T09:23:13.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.479+0000 7f020a7fc640 1 -- 172.21.15.12:0/3075777014 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f02100051d0 con 0x7f0218101400 2024-05-09T09:23:13.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.480+0000 7f020a7fc640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f02040a2b90 0x7f02040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:13.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.480+0000 7f021d364640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f02040a2b90 0x7f02040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:13.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.480+0000 7f020a7fc640 1 -- 172.21.15.12:0/3075777014 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7f02100cf650 con 0x7f0218101400 2024-05-09T09:23:13.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.480+0000 7f021d364640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f02040a2b90 0x7f02040a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f021400bbe0 tx=0x7f021400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:13.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.484+0000 7f020a7fc640 1 -- 172.21.15.12:0/3075777014 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0210014030 con 0x7f0218101400 2024-05-09T09:23:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:13.672+0000 7f021f5c8640 1 -- 172.21.15.12:0/3075777014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f021802d1c0 con 0x7f0218101400 2024-05-09T09:23:13.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:13 smithi012 ceph-mon[30140]: pgmap v290: 97 pgs: 32 creating+peering, 59 active+clean, 6 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-05-09T09:23:13.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:13 smithi012 ceph-mon[30140]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T09:23:13.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:13 smithi170 ceph-mon[33632]: pgmap v290: 97 pgs: 32 creating+peering, 59 active+clean, 6 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-05-09T09:23:13.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:13 smithi170 ceph-mon[33632]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T09:23:13.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:13 smithi016 ceph-mon[33335]: pgmap v290: 97 pgs: 32 creating+peering, 59 active+clean, 6 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-05-09T09:23:13.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:13 smithi016 ceph-mon[33335]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T09:23:14.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.346+0000 7f020a7fc640 1 -- 172.21.15.12:0/3075777014 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7f021009c390 con 0x7f0218101400 2024-05-09T09:23:14.346 INFO:teuthology.orchestra.run.smithi012.stderr:set pool 4 allow_ec_overwrites to true 2024-05-09T09:23:14.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.349+0000 7f021f5c8640 1 -- 172.21.15.12:0/3075777014 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f02040a2b90 msgr2=0x7f02040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:14.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.349+0000 7f021f5c8640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f02040a2b90 0x7f02040a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f021400bbe0 tx=0x7f021400b390 comp rx=0 tx=0).stop 2024-05-09T09:23:14.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.349+0000 7f021f5c8640 1 -- 172.21.15.12:0/3075777014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0218101400 msgr2=0x7f02181a1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:14.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.349+0000 7f021f5c8640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0218101400 0x7f02181a1360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f021000cad0 tx=0x7f021000cf90 comp rx=0 tx=0).stop 2024-05-09T09:23:14.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.350+0000 7f021f5c8640 1 -- 172.21.15.12:0/3075777014 shutdown_connections 2024-05-09T09:23:14.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.350+0000 7f021f5c8640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f02040a2b90 0x7f02040a5050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:14.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.350+0000 7f021f5c8640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0218101400 0x7f02181a1360 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:14.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.350+0000 7f021f5c8640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0218100a60 0x7f021819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:14.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.350+0000 7f021f5c8640 1 --2- 172.21.15.12:0/3075777014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0218100100 0x7f0218199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:14.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.350+0000 7f021f5c8640 1 -- 172.21.15.12:0/3075777014 >> 172.21.15.12:0/3075777014 conn(0x7f02180fbd20 msgr2=0x7f02180ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:14.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.350+0000 7f021f5c8640 1 -- 172.21.15.12:0/3075777014 shutdown_connections 2024-05-09T09:23:14.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.350+0000 7f021f5c8640 1 -- 172.21.15.12:0/3075777014 wait complete. 2024-05-09T09:23:14.373 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-09T09:23:14.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.487+0000 7f859fd77640 1 -- 172.21.15.12:0/1111870778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8598075870 msgr2=0x7f8598073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:14.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.487+0000 7f859fd77640 1 --2- 172.21.15.12:0/1111870778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8598075870 0x7f8598073cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8594009900 tx=0x7f859402a3f0 comp rx=0 tx=0).stop 2024-05-09T09:23:14.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.487+0000 7f859fd77640 1 -- 172.21.15.12:0/1111870778 shutdown_connections 2024-05-09T09:23:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.487+0000 7f859fd77640 1 --2- 172.21.15.12:0/1111870778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8598109730 0x7f859810bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.487+0000 7f859fd77640 1 --2- 172.21.15.12:0/1111870778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8598074290 0x7f8598074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.487+0000 7f859fd77640 1 --2- 172.21.15.12:0/1111870778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8598075870 0x7f8598073cc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.487+0000 7f859fd77640 1 -- 172.21.15.12:0/1111870778 >> 172.21.15.12:0/1111870778 conn(0x7f85980fcde0 msgr2=0x7f85980ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.487+0000 7f859fd77640 1 -- 172.21.15.12:0/1111870778 shutdown_connections 2024-05-09T09:23:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.487+0000 7f859fd77640 1 -- 172.21.15.12:0/1111870778 wait complete. 2024-05-09T09:23:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.487+0000 7f859fd77640 1 Processor -- start 2024-05-09T09:23:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.487+0000 7f859fd77640 1 -- start start 2024-05-09T09:23:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859fd77640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8598074290 0x7f8598199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859fd77640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8598075870 0x7f859819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859fd77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8598109730 0x7f85981a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859fd77640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f859810f6a0 con 0x7f8598075870 2024-05-09T09:23:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859fd77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f859810f520 con 0x7f8598109730 2024-05-09T09:23:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859fd77640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f859810f820 con 0x7f8598074290 2024-05-09T09:23:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859d312640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8598075870 0x7f859819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859d312640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8598075870 0x7f859819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37454/0 (socket says 172.21.15.12:37454) 2024-05-09T09:23:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859d312640 1 -- 172.21.15.12:0/2015889338 learned_addr learned my addr 172.21.15.12:0/2015889338 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859db13640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8598074290 0x7f8598199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859e314640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8598109730 0x7f85981a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859d312640 1 -- 172.21.15.12:0/2015889338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8598074290 msgr2=0x7f8598199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859d312640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8598074290 0x7f8598199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859d312640 1 -- 172.21.15.12:0/2015889338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8598109730 msgr2=0x7f85981a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859d312640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8598109730 0x7f85981a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859d312640 1 -- 172.21.15.12:0/2015889338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85981a1960 con 0x7f8598075870 2024-05-09T09:23:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859d312640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8598075870 0x7f859819a280 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f858800d950 tx=0x7f858800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f858effd640 1 -- 172.21.15.12:0/2015889338 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588017070 con 0x7f8598075870 2024-05-09T09:23:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f858effd640 1 -- 172.21.15.12:0/2015889338 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8588009c30 con 0x7f8598075870 2024-05-09T09:23:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859fd77640 1 -- 172.21.15.12:0/2015889338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85981a1c50 con 0x7f8598075870 2024-05-09T09:23:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f858effd640 1 -- 172.21.15.12:0/2015889338 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858800f6b0 con 0x7f8598075870 2024-05-09T09:23:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.488+0000 7f859fd77640 1 -- 172.21.15.12:0/2015889338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85981a21a0 con 0x7f8598075870 2024-05-09T09:23:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.489+0000 7f859fd77640 1 -- 172.21.15.12:0/2015889338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8598108460 con 0x7f8598075870 2024-05-09T09:23:14.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.491+0000 7f858effd640 1 -- 172.21.15.12:0/2015889338 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8588010a50 con 0x7f8598075870 2024-05-09T09:23:14.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.491+0000 7f858effd640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85840a2b90 0x7f85840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:14.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.491+0000 7f859db13640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85840a2b90 0x7f85840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:14.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.491+0000 7f858effd640 1 -- 172.21.15.12:0/2015889338 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f85880cef20 con 0x7f8598075870 2024-05-09T09:23:14.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.492+0000 7f859db13640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85840a2b90 0x7f85840a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8594000c00 tx=0x7f8594031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:14.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.496+0000 7f858effd640 1 -- 172.21.15.12:0/2015889338 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8588009de0 con 0x7f8598075870 2024-05-09T09:23:14.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:14.725+0000 7f859fd77640 1 -- 172.21.15.12:0/2015889338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f8598062430 con 0x7f8598075870 2024-05-09T09:23:14.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:14 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3780625930' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T09:23:14.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:14 smithi012 ceph-mon[30140]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T09:23:14.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:14 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3075777014' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-09T09:23:14.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:14 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3780625930' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T09:23:14.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:14 smithi170 ceph-mon[33632]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T09:23:14.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:14 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3075777014' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-09T09:23:14.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:14 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3780625930' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T09:23:14.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:14 smithi016 ceph-mon[33335]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T09:23:14.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:14 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3075777014' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-09T09:23:15.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.352+0000 7f858effd640 1 -- 172.21.15.12:0/2015889338 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f858809bc60 con 0x7f8598075870 2024-05-09T09:23:15.353 INFO:teuthology.orchestra.run.smithi012.stderr:added data pool 4 to fsmap 2024-05-09T09:23:15.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.356+0000 7f859fd77640 1 -- 172.21.15.12:0/2015889338 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85840a2b90 msgr2=0x7f85840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:15.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.356+0000 7f859fd77640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85840a2b90 0x7f85840a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8594000c00 tx=0x7f8594031510 comp rx=0 tx=0).stop 2024-05-09T09:23:15.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.356+0000 7f859fd77640 1 -- 172.21.15.12:0/2015889338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8598075870 msgr2=0x7f859819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:15.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.356+0000 7f859fd77640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8598075870 0x7f859819a280 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f858800d950 tx=0x7f858800de10 comp rx=0 tx=0).stop 2024-05-09T09:23:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.356+0000 7f859fd77640 1 -- 172.21.15.12:0/2015889338 shutdown_connections 2024-05-09T09:23:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.356+0000 7f859fd77640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85840a2b90 0x7f85840a5050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.356+0000 7f859fd77640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8598109730 0x7f85981a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.356+0000 7f859fd77640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8598075870 0x7f859819a280 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.356+0000 7f859fd77640 1 --2- 172.21.15.12:0/2015889338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8598074290 0x7f8598199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.356+0000 7f859fd77640 1 -- 172.21.15.12:0/2015889338 >> 172.21.15.12:0/2015889338 conn(0x7f85980fcde0 msgr2=0x7f8598105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.356+0000 7f859fd77640 1 -- 172.21.15.12:0/2015889338 shutdown_connections 2024-05-09T09:23:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.356+0000 7f859fd77640 1 -- 172.21.15.12:0/2015889338 wait complete. 2024-05-09T09:23:15.370 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:23:15.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.482+0000 7f806ef29640 1 -- 172.21.15.12:0/3351937132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8068102a90 msgr2=0x7f8068102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:15.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.482+0000 7f806ef29640 1 --2- 172.21.15.12:0/3351937132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8068102a90 0x7f8068102f10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8054009900 tx=0x7f805402a410 comp rx=0 tx=0).stop 2024-05-09T09:23:15.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.482+0000 7f806ef29640 1 -- 172.21.15.12:0/3351937132 shutdown_connections 2024-05-09T09:23:15.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.482+0000 7f806ef29640 1 --2- 172.21.15.12:0/3351937132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8068103450 0x7f8068109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.482+0000 7f806ef29640 1 --2- 172.21.15.12:0/3351937132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8068102a90 0x7f8068102f10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.482+0000 7f806ef29640 1 --2- 172.21.15.12:0/3351937132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80681018a0 0x7f8068101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.482+0000 7f806ef29640 1 -- 172.21.15.12:0/3351937132 >> 172.21.15.12:0/3351937132 conn(0x7f80680fce30 msgr2=0x7f80680ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.482+0000 7f806ef29640 1 -- 172.21.15.12:0/3351937132 shutdown_connections 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.482+0000 7f806ef29640 1 -- 172.21.15.12:0/3351937132 wait complete. 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ef29640 1 Processor -- start 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ef29640 1 -- start start 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ef29640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80681018a0 0x7f8068197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ef29640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8068102a90 0x7f8068198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ef29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8068103450 0x7f806819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ef29640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f806810d560 con 0x7f80681018a0 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ef29640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f806810d3e0 con 0x7f8068103450 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ef29640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f806810d6e0 con 0x7f8068102a90 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ccc5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80681018a0 0x7f8068197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ccc5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80681018a0 0x7f8068197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37464/0 (socket says 172.21.15.12:37464) 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ccc5640 1 -- 172.21.15.12:0/2819900223 learned_addr learned my addr 172.21.15.12:0/2819900223 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806d4c6640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8068103450 0x7f806819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ccc5640 1 -- 172.21.15.12:0/2819900223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8068102a90 msgr2=0x7f8068198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f8067fff640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8068102a90 0x7f8068198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ccc5640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8068102a90 0x7f8068198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ccc5640 1 -- 172.21.15.12:0/2819900223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8068103450 msgr2=0x7f806819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ccc5640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8068103450 0x7f806819f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806ccc5640 1 -- 172.21.15.12:0/2819900223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806819f860 con 0x7f80681018a0 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.483+0000 7f806d4c6640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8068103450 0x7f806819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.484+0000 7f806ccc5640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80681018a0 0x7f8068197c50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f805c00c970 tx=0x7f805c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.484+0000 7f8065ffb640 1 -- 172.21.15.12:0/2819900223 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c004220 con 0x7f80681018a0 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.484+0000 7f806ef29640 1 -- 172.21.15.12:0/2819900223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806819fb50 con 0x7f80681018a0 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.484+0000 7f8065ffb640 1 -- 172.21.15.12:0/2819900223 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f805c0043c0 con 0x7f80681018a0 2024-05-09T09:23:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.484+0000 7f8065ffb640 1 -- 172.21.15.12:0/2819900223 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c00f450 con 0x7f80681018a0 2024-05-09T09:23:15.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.484+0000 7f806ef29640 1 -- 172.21.15.12:0/2819900223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80681a00a0 con 0x7f80681018a0 2024-05-09T09:23:15.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.485+0000 7f806ef29640 1 -- 172.21.15.12:0/2819900223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8068108bc0 con 0x7f80681018a0 2024-05-09T09:23:15.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.486+0000 7f8065ffb640 1 -- 172.21.15.12:0/2819900223 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f805c0079b0 con 0x7f80681018a0 2024-05-09T09:23:15.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.487+0000 7f8065ffb640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f80580a2b90 0x7f80580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:15.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.487+0000 7f8067fff640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f80580a2b90 0x7f80580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:15.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.487+0000 7f8065ffb640 1 -- 172.21.15.12:0/2819900223 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f805c0cf160 con 0x7f80681018a0 2024-05-09T09:23:15.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.487+0000 7f8067fff640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f80580a2b90 0x7f80580a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8054005c40 tx=0x7f8054000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:15.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.491+0000 7f8065ffb640 1 -- 172.21.15.12:0/2819900223 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f805c014030 con 0x7f80681018a0 2024-05-09T09:23:15.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:15 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3075777014' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-09T09:23:15.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:15 smithi170 ceph-mon[33632]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T09:23:15.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:15 smithi170 ceph-mon[33632]: pgmap v293: 129 pgs: 19 creating+peering, 92 active+clean, 18 unknown; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-05-09T09:23:15.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:15 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2015889338' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T09:23:15.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.717+0000 7f806ef29640 1 -- 172.21.15.12:0/2819900223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8068062430 con 0x7f80681018a0 2024-05-09T09:23:15.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.717+0000 7f8065ffb640 1 -- 172.21.15.12:0/2819900223 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11797 (secure 0 0 0) 0x7f805c09be80 con 0x7f80681018a0 2024-05-09T09:23:15.717 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:15.718 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:23:14.726503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24523,"qdb_cluster":[24523]},"id":1}]} 2024-05-09T09:23:15.718 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2024-05-09T09:23:15.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.719+0000 7f806ef29640 1 -- 172.21.15.12:0/2819900223 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f80580a2b90 msgr2=0x7f80580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:15.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.719+0000 7f806ef29640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f80580a2b90 0x7f80580a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8054005c40 tx=0x7f8054000bc0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.719+0000 7f806ef29640 1 -- 172.21.15.12:0/2819900223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80681018a0 msgr2=0x7f8068197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.719+0000 7f806ef29640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80681018a0 0x7f8068197c50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f805c00c970 tx=0x7f805c00ce30 comp rx=0 tx=0).stop 2024-05-09T09:23:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.719+0000 7f806ef29640 1 -- 172.21.15.12:0/2819900223 shutdown_connections 2024-05-09T09:23:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.719+0000 7f806ef29640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f80580a2b90 0x7f80580a5050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.719+0000 7f806ef29640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8068103450 0x7f806819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.719+0000 7f806ef29640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8068102a90 0x7f8068198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.719+0000 7f806ef29640 1 --2- 172.21.15.12:0/2819900223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80681018a0 0x7f8068197c50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.719+0000 7f806ef29640 1 -- 172.21.15.12:0/2819900223 >> 172.21.15.12:0/2819900223 conn(0x7f80680fce30 msgr2=0x7f8068105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.720+0000 7f806ef29640 1 -- 172.21.15.12:0/2819900223 shutdown_connections 2024-05-09T09:23:15.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.720+0000 7f806ef29640 1 -- 172.21.15.12:0/2819900223 wait complete. 2024-05-09T09:23:15.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:15 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3075777014' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-09T09:23:15.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:15 smithi012 ceph-mon[30140]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T09:23:15.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:15 smithi012 ceph-mon[30140]: pgmap v293: 129 pgs: 19 creating+peering, 92 active+clean, 18 unknown; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-05-09T09:23:15.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:15 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2015889338' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T09:23:15.731 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:23:15.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:15 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3075777014' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-09T09:23:15.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:15 smithi016 ceph-mon[33335]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T09:23:15.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:15 smithi016 ceph-mon[33335]: pgmap v293: 129 pgs: 19 creating+peering, 92 active+clean, 18 unknown; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-05-09T09:23:15.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:15 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2015889338' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T09:23:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.836+0000 7fc46dd89640 1 -- 172.21.15.12:0/3771147068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc468107570 msgr2=0x7fc4681099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.836+0000 7fc46dd89640 1 --2- 172.21.15.12:0/3771147068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc468107570 0x7fc4681099a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc460009ca0 tx=0x7fc46002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.836+0000 7fc46dd89640 1 -- 172.21.15.12:0/3771147068 shutdown_connections 2024-05-09T09:23:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.836+0000 7fc46dd89640 1 --2- 172.21.15.12:0/3771147068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc468107570 0x7fc4681099a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.836+0000 7fc46dd89640 1 --2- 172.21.15.12:0/3771147068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc468104c40 0x7fc468107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.836+0000 7fc46dd89640 1 --2- 172.21.15.12:0/3771147068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc468102310 0x7fc468104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.836+0000 7fc46dd89640 1 -- 172.21.15.12:0/3771147068 >> 172.21.15.12:0/3771147068 conn(0x7fc4680fbce0 msgr2=0x7fc4680fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.836+0000 7fc46dd89640 1 -- 172.21.15.12:0/3771147068 shutdown_connections 2024-05-09T09:23:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.836+0000 7fc46dd89640 1 -- 172.21.15.12:0/3771147068 wait complete. 2024-05-09T09:23:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.836+0000 7fc46dd89640 1 Processor -- start 2024-05-09T09:23:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.836+0000 7fc46dd89640 1 -- start start 2024-05-09T09:23:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc46dd89640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc468102310 0x7fc468197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc46dd89640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc468104c40 0x7fc468198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc46dd89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc468107570 0x7fc46819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc46dd89640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc46810d4d0 con 0x7fc468102310 2024-05-09T09:23:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc46dd89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc46810d350 con 0x7fc468107570 2024-05-09T09:23:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc46dd89640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc46810d650 con 0x7fc468104c40 2024-05-09T09:23:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc4677fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc468102310 0x7fc468197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc4677fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc468102310 0x7fc468197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37476/0 (socket says 172.21.15.12:37476) 2024-05-09T09:23:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc4677fe640 1 -- 172.21.15.12:0/105948985 learned_addr learned my addr 172.21.15.12:0/105948985 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc45effd640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc468104c40 0x7fc468198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc467fff640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc468107570 0x7fc46819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc4677fe640 1 -- 172.21.15.12:0/105948985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc468104c40 msgr2=0x7fc468198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc4677fe640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc468104c40 0x7fc468198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc4677fe640 1 -- 172.21.15.12:0/105948985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc468107570 msgr2=0x7fc46819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc4677fe640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc468107570 0x7fc46819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc4677fe640 1 -- 172.21.15.12:0/105948985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46819f770 con 0x7fc468102310 2024-05-09T09:23:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc467fff640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc468107570 0x7fc46819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc4677fe640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc468102310 0x7fc468197b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc45800d950 tx=0x7fc45800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.837+0000 7fc4657fa640 1 -- 172.21.15.12:0/105948985 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458017070 con 0x7fc468102310 2024-05-09T09:23:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.838+0000 7fc46dd89640 1 -- 172.21.15.12:0/105948985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46819fa60 con 0x7fc468102310 2024-05-09T09:23:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.838+0000 7fc4657fa640 1 -- 172.21.15.12:0/105948985 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc458009c30 con 0x7fc468102310 2024-05-09T09:23:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.838+0000 7fc4657fa640 1 -- 172.21.15.12:0/105948985 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45800f6b0 con 0x7fc468102310 2024-05-09T09:23:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.838+0000 7fc46dd89640 1 -- 172.21.15.12:0/105948985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4681a0000 con 0x7fc468102310 2024-05-09T09:23:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.838+0000 7fc46dd89640 1 -- 172.21.15.12:0/105948985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4680fd7d0 con 0x7fc468102310 2024-05-09T09:23:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.840+0000 7fc4657fa640 1 -- 172.21.15.12:0/105948985 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc458010a50 con 0x7fc468102310 2024-05-09T09:23:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.840+0000 7fc4657fa640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc4540a2b90 0x7fc4540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.841+0000 7fc45effd640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc4540a2b90 0x7fc4540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.841+0000 7fc4657fa640 1 -- 172.21.15.12:0/105948985 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc4580ceee0 con 0x7fc468102310 2024-05-09T09:23:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.841+0000 7fc45effd640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc4540a2b90 0x7fc4540a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc468199060 tx=0x7fc450005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:15.845+0000 7fc4657fa640 1 -- 172.21.15.12:0/105948985 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc458009de0 con 0x7fc468102310 2024-05-09T09:23:15.921 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:23:15 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i[64878]: 2024-05-09T09:23:15.795+0000 7f86049b6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T09:23:16.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.033+0000 7fc46dd89640 1 -- 172.21.15.12:0/105948985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc468062460 con 0x7fc468102310 2024-05-09T09:23:16.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.034+0000 7fc4657fa640 1 -- 172.21.15.12:0/105948985 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24105 (secure 0 0 0) 0x7fc45809bc00 con 0x7fc468102310 2024-05-09T09:23:16.034 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:16.034 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:23:14.726332+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:23:16.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.036+0000 7fc46dd89640 1 -- 172.21.15.12:0/105948985 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc4540a2b90 msgr2=0x7fc4540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.036+0000 7fc46dd89640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc4540a2b90 0x7fc4540a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc468199060 tx=0x7fc450005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:16.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.036+0000 7fc46dd89640 1 -- 172.21.15.12:0/105948985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc468102310 msgr2=0x7fc468197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.036+0000 7fc46dd89640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc468102310 0x7fc468197b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc45800d950 tx=0x7fc45800de10 comp rx=0 tx=0).stop 2024-05-09T09:23:16.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.036+0000 7fc46dd89640 1 -- 172.21.15.12:0/105948985 shutdown_connections 2024-05-09T09:23:16.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.036+0000 7fc46dd89640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc4540a2b90 0x7fc4540a5050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.036+0000 7fc46dd89640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc468107570 0x7fc46819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.036+0000 7fc46dd89640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc468104c40 0x7fc468198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.036+0000 7fc46dd89640 1 --2- 172.21.15.12:0/105948985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc468102310 0x7fc468197b50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.036+0000 7fc46dd89640 1 -- 172.21.15.12:0/105948985 >> 172.21.15.12:0/105948985 conn(0x7fc4680fbce0 msgr2=0x7fc468102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:16.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.036+0000 7fc46dd89640 1 -- 172.21.15.12:0/105948985 shutdown_connections 2024-05-09T09:23:16.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.036+0000 7fc46dd89640 1 -- 172.21.15.12:0/105948985 wait complete. 2024-05-09T09:23:16.048 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:23:16.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.153+0000 7f99626ba640 1 -- 172.21.15.12:0/3635578639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c102340 msgr2=0x7f995c108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.153+0000 7f99626ba640 1 --2- 172.21.15.12:0/3635578639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c102340 0x7f995c108bd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9954009ca0 tx=0x7f995402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.153+0000 7f99626ba640 1 -- 172.21.15.12:0/3635578639 shutdown_connections 2024-05-09T09:23:16.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.153+0000 7f99626ba640 1 --2- 172.21.15.12:0/3635578639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c102340 0x7f995c108bd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.153+0000 7f99626ba640 1 --2- 172.21.15.12:0/3635578639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c101980 0x7f995c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.153+0000 7f99626ba640 1 --2- 172.21.15.12:0/3635578639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f995c100790 0x7f995c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.153+0000 7f99626ba640 1 -- 172.21.15.12:0/3635578639 >> 172.21.15.12:0/3635578639 conn(0x7f995c0fbd20 msgr2=0x7f995c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.153+0000 7f99626ba640 1 -- 172.21.15.12:0/3635578639 shutdown_connections 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.153+0000 7f99626ba640 1 -- 172.21.15.12:0/3635578639 wait complete. 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.154+0000 7f99626ba640 1 Processor -- start 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.154+0000 7f99626ba640 1 -- start start 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.154+0000 7f99626ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c100790 0x7f995c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.154+0000 7f99626ba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f995c101980 0x7f995c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.154+0000 7f99626ba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c102340 0x7f995c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.154+0000 7f99626ba640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f995c10d500 con 0x7f995c102340 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.154+0000 7f99626ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f995c10d380 con 0x7f995c100790 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.154+0000 7f99626ba640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f995c10d680 con 0x7f995c101980 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.154+0000 7f9960c57640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c102340 0x7f995c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.154+0000 7f9960c57640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c102340 0x7f995c19f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37494/0 (socket says 172.21.15.12:37494) 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.154+0000 7f9960c57640 1 -- 172.21.15.12:0/1283632881 learned_addr learned my addr 172.21.15.12:0/1283632881 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.155+0000 7f995b7fe640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f995c101980 0x7f995c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.155+0000 7f995bfff640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c100790 0x7f995c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.155+0000 7f9960c57640 1 -- 172.21.15.12:0/1283632881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f995c101980 msgr2=0x7f995c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.155+0000 7f9960c57640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f995c101980 0x7f995c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.155+0000 7f9960c57640 1 -- 172.21.15.12:0/1283632881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c100790 msgr2=0x7f995c197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.155+0000 7f9960c57640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c100790 0x7f995c197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.155+0000 7f9960c57640 1 -- 172.21.15.12:0/1283632881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c19f790 con 0x7f995c102340 2024-05-09T09:23:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.155+0000 7f9960c57640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c102340 0x7f995c19f120 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9954009430 tx=0x7f995402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.155+0000 7f99597fa640 1 -- 172.21.15.12:0/1283632881 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954003bf0 con 0x7f995c102340 2024-05-09T09:23:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.155+0000 7f99626ba640 1 -- 172.21.15.12:0/1283632881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995c19fa20 con 0x7f995c102340 2024-05-09T09:23:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.155+0000 7f99597fa640 1 -- 172.21.15.12:0/1283632881 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9954003d90 con 0x7f995c102340 2024-05-09T09:23:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.155+0000 7f99597fa640 1 -- 172.21.15.12:0/1283632881 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995402fc90 con 0x7f995c102340 2024-05-09T09:23:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.155+0000 7f99626ba640 1 -- 172.21.15.12:0/1283632881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f995c19ffc0 con 0x7f995c102340 2024-05-09T09:23:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.156+0000 7f99626ba640 1 -- 172.21.15.12:0/1283632881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f995c107ab0 con 0x7f995c102340 2024-05-09T09:23:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.157+0000 7f99597fa640 1 -- 172.21.15.12:0/1283632881 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9954004040 con 0x7f995c102340 2024-05-09T09:23:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.158+0000 7f99597fa640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99440a2b90 0x7f99440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.158+0000 7f995bfff640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99440a2b90 0x7f99440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.158+0000 7f99597fa640 1 -- 172.21.15.12:0/1283632881 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f99540f0900 con 0x7f995c102340 2024-05-09T09:23:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.158+0000 7f995bfff640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99440a2b90 0x7f99440a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f995c1017e0 tx=0x7f994c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:16.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.162+0000 7f99597fa640 1 -- 172.21.15.12:0/1283632881 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f99540bd6b0 con 0x7f995c102340 2024-05-09T09:23:16.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.350+0000 7f99626ba640 1 -- 172.21.15.12:0/1283632881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f995c062430 con 0x7f995c102340 2024-05-09T09:23:16.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.352+0000 7f99597fa640 1 -- 172.21.15.12:0/1283632881 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24105 (secure 0 0 0) 0x7f99540bcfb0 con 0x7f995c102340 2024-05-09T09:23:16.352 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:16.352 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:23:14.726332+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:23:16.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.354+0000 7f99626ba640 1 -- 172.21.15.12:0/1283632881 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99440a2b90 msgr2=0x7f99440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.354+0000 7f99626ba640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99440a2b90 0x7f99440a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f995c1017e0 tx=0x7f994c005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:16.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.354+0000 7f99626ba640 1 -- 172.21.15.12:0/1283632881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c102340 msgr2=0x7f995c19f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.354+0000 7f99626ba640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c102340 0x7f995c19f120 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9954009430 tx=0x7f995402f380 comp rx=0 tx=0).stop 2024-05-09T09:23:16.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.354+0000 7f99626ba640 1 -- 172.21.15.12:0/1283632881 shutdown_connections 2024-05-09T09:23:16.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.354+0000 7f99626ba640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99440a2b90 0x7f99440a5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.354+0000 7f99626ba640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c102340 0x7f995c19f120 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.354+0000 7f99626ba640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f995c101980 0x7f995c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.354+0000 7f99626ba640 1 --2- 172.21.15.12:0/1283632881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c100790 0x7f995c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.354+0000 7f99626ba640 1 -- 172.21.15.12:0/1283632881 >> 172.21.15.12:0/1283632881 conn(0x7f995c0fbd20 msgr2=0x7f995c104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:16.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.354+0000 7f99626ba640 1 -- 172.21.15.12:0/1283632881 shutdown_connections 2024-05-09T09:23:16.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.355+0000 7f99626ba640 1 -- 172.21.15.12:0/1283632881 wait complete. 2024-05-09T09:23:16.382 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:23:16.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.491+0000 7fa99888c640 1 -- 172.21.15.12:0/3831072757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa994100770 msgr2=0x7fa994100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.491+0000 7fa99888c640 1 --2- 172.21.15.12:0/3831072757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa994100770 0x7fa994100b90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa984009900 tx=0x7fa98402a410 comp rx=0 tx=0).stop 2024-05-09T09:23:16.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.491+0000 7fa99888c640 1 -- 172.21.15.12:0/3831072757 shutdown_connections 2024-05-09T09:23:16.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.491+0000 7fa99888c640 1 --2- 172.21.15.12:0/3831072757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994102320 0x7fa994108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.491+0000 7fa99888c640 1 --2- 172.21.15.12:0/3831072757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994101960 0x7fa994101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.491+0000 7fa99888c640 1 --2- 172.21.15.12:0/3831072757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa994100770 0x7fa994100b90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.491+0000 7fa99888c640 1 -- 172.21.15.12:0/3831072757 >> 172.21.15.12:0/3831072757 conn(0x7fa9940fbce0 msgr2=0x7fa9940fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:16.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.491+0000 7fa99888c640 1 -- 172.21.15.12:0/3831072757 shutdown_connections 2024-05-09T09:23:16.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.491+0000 7fa99888c640 1 -- 172.21.15.12:0/3831072757 wait complete. 2024-05-09T09:23:16.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.491+0000 7fa99888c640 1 Processor -- start 2024-05-09T09:23:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.491+0000 7fa99888c640 1 -- start start 2024-05-09T09:23:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa99888c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994100770 0x7fa994197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa99888c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994101960 0x7fa9941980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa99888c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa994102320 0x7fa99419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa99888c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa99410d520 con 0x7fa994102320 2024-05-09T09:23:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa99888c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa99410d3a0 con 0x7fa994100770 2024-05-09T09:23:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa99888c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa99410d6a0 con 0x7fa994101960 2024-05-09T09:23:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa992d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa994102320 0x7fa99419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa992d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa994102320 0x7fa99419f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37520/0 (socket says 172.21.15.12:37520) 2024-05-09T09:23:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa992d9d640 1 -- 172.21.15.12:0/1275477933 learned_addr learned my addr 172.21.15.12:0/1275477933 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa98bfff640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994101960 0x7fa9941980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa99259c640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994100770 0x7fa994197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa992d9d640 1 -- 172.21.15.12:0/1275477933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994101960 msgr2=0x7fa9941980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa992d9d640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994101960 0x7fa9941980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa992d9d640 1 -- 172.21.15.12:0/1275477933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994100770 msgr2=0x7fa994197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa992d9d640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994100770 0x7fa994197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa992d9d640 1 -- 172.21.15.12:0/1275477933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99419f7b0 con 0x7fa994102320 2024-05-09T09:23:16.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa992d9d640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa994102320 0x7fa99419f140 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00eaa0 tx=0x7fa98c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:16.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa98b7fe640 1 -- 172.21.15.12:0/1275477933 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00cdb0 con 0x7fa994102320 2024-05-09T09:23:16.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa98b7fe640 1 -- 172.21.15.12:0/1275477933 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa98c005020 con 0x7fa994102320 2024-05-09T09:23:16.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.492+0000 7fa99888c640 1 -- 172.21.15.12:0/1275477933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99419faa0 con 0x7fa994102320 2024-05-09T09:23:16.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.493+0000 7fa98b7fe640 1 -- 172.21.15.12:0/1275477933 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0116b0 con 0x7fa994102320 2024-05-09T09:23:16.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.493+0000 7fa99888c640 1 -- 172.21.15.12:0/1275477933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99419ffa0 con 0x7fa994102320 2024-05-09T09:23:16.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.493+0000 7fa99888c640 1 -- 172.21.15.12:0/1275477933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa994107a90 con 0x7fa994102320 2024-05-09T09:23:16.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.495+0000 7fa98b7fe640 1 -- 172.21.15.12:0/1275477933 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa98c0051d0 con 0x7fa994102320 2024-05-09T09:23:16.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.495+0000 7fa98b7fe640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa9800a2b90 0x7fa9800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:16.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.495+0000 7fa99259c640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa9800a2b90 0x7fa9800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:16.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.495+0000 7fa98b7fe640 1 -- 172.21.15.12:0/1275477933 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa98c0cf440 con 0x7fa994102320 2024-05-09T09:23:16.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.496+0000 7fa99259c640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa9800a2b90 0x7fa9800a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa984004770 tx=0x7fa984031530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:16.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.500+0000 7fa98b7fe640 1 -- 172.21.15.12:0/1275477933 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa98c016030 con 0x7fa994102320 2024-05-09T09:23:16.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.724+0000 7fa99888c640 1 -- 172.21.15.12:0/1275477933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa99402d1c0 con 0x7fa994102320 2024-05-09T09:23:16.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.725+0000 7fa98b7fe640 1 -- 172.21.15.12:0/1275477933 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11797 (secure 0 0 0) 0x7fa98c09bb10 con 0x7fa994102320 2024-05-09T09:23:16.724 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:16.724 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:23:14.726503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24523,"qdb_cluster":[24523]},"id":1}]} 2024-05-09T09:23:16.725 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2024-05-09T09:23:16.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.727+0000 7fa99888c640 1 -- 172.21.15.12:0/1275477933 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa9800a2b90 msgr2=0x7fa9800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.727+0000 7fa99888c640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa9800a2b90 0x7fa9800a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa984004770 tx=0x7fa984031530 comp rx=0 tx=0).stop 2024-05-09T09:23:16.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.727+0000 7fa99888c640 1 -- 172.21.15.12:0/1275477933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa994102320 msgr2=0x7fa99419f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.727+0000 7fa99888c640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa994102320 0x7fa99419f140 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00eaa0 tx=0x7fa98c00ef60 comp rx=0 tx=0).stop 2024-05-09T09:23:16.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.727+0000 7fa99888c640 1 -- 172.21.15.12:0/1275477933 shutdown_connections 2024-05-09T09:23:16.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.727+0000 7fa99888c640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa9800a2b90 0x7fa9800a5050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.727+0000 7fa99888c640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa994102320 0x7fa99419f140 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.727+0000 7fa99888c640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994101960 0x7fa9941980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.727+0000 7fa99888c640 1 --2- 172.21.15.12:0/1275477933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994100770 0x7fa994197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.727+0000 7fa99888c640 1 -- 172.21.15.12:0/1275477933 >> 172.21.15.12:0/1275477933 conn(0x7fa9940fbce0 msgr2=0x7fa994104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:16.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.727+0000 7fa99888c640 1 -- 172.21.15.12:0/1275477933 shutdown_connections 2024-05-09T09:23:16.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.727+0000 7fa99888c640 1 -- 172.21.15.12:0/1275477933 wait complete. 2024-05-09T09:23:16.733 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:16 smithi012 ceph-mon[30140]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:23:16.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:16 smithi012 ceph-mon[30140]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T09:23:16.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:16 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2015889338' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-09T09:23:16.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:16 smithi012 ceph-mon[30140]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:16.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:16 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2819900223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:16.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:16 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/105948985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:16.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:16 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1283632881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:16.738 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:23:16.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:16 smithi170 ceph-mon[33632]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:23:16.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:16 smithi170 ceph-mon[33632]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T09:23:16.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:16 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2015889338' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-09T09:23:16.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:16 smithi170 ceph-mon[33632]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:16.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:16 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2819900223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:16.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:16 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/105948985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:16.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:16 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1283632881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:16.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:16 smithi016 ceph-mon[33335]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:23:16.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:16 smithi016 ceph-mon[33335]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T09:23:16.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:16 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2015889338' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-09T09:23:16.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:16 smithi016 ceph-mon[33335]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:16.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:16 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2819900223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:16.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:16 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/105948985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:16.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:16 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1283632881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.844+0000 7f2b94d5a640 1 -- 172.21.15.12:0/2054000150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b901023d0 msgr2=0x7f2b901047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.844+0000 7f2b94d5a640 1 --2- 172.21.15.12:0/2054000150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b901023d0 0x7f2b901047c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2b80009900 tx=0x7f2b8002a410 comp rx=0 tx=0).stop 2024-05-09T09:23:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.844+0000 7f2b94d5a640 1 -- 172.21.15.12:0/2054000150 shutdown_connections 2024-05-09T09:23:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.844+0000 7f2b94d5a640 1 --2- 172.21.15.12:0/2054000150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b90107630 0x7f2b90109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.844+0000 7f2b94d5a640 1 --2- 172.21.15.12:0/2054000150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b90104d00 0x7f2b901070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.844+0000 7f2b94d5a640 1 --2- 172.21.15.12:0/2054000150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b901023d0 0x7f2b901047c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.844+0000 7f2b94d5a640 1 -- 172.21.15.12:0/2054000150 >> 172.21.15.12:0/2054000150 conn(0x7f2b900fbd20 msgr2=0x7f2b900fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.844+0000 7f2b94d5a640 1 -- 172.21.15.12:0/2054000150 shutdown_connections 2024-05-09T09:23:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.844+0000 7f2b94d5a640 1 -- 172.21.15.12:0/2054000150 wait complete. 2024-05-09T09:23:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b94d5a640 1 Processor -- start 2024-05-09T09:23:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b94d5a640 1 -- start start 2024-05-09T09:23:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b94d5a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b901023d0 0x7f2b90197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:16.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b94d5a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b90104d00 0x7f2b90198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:16.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b94d5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b90107630 0x7f2b9019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:16.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b94d5a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9010d500 con 0x7f2b901023d0 2024-05-09T09:23:16.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b94d5a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9010d380 con 0x7f2b90107630 2024-05-09T09:23:16.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b94d5a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9010d680 con 0x7f2b90104d00 2024-05-09T09:23:16.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b8e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b901023d0 0x7f2b90197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b8e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b901023d0 0x7f2b90197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37528/0 (socket says 172.21.15.12:37528) 2024-05-09T09:23:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b8e59c640 1 -- 172.21.15.12:0/3951712060 learned_addr learned my addr 172.21.15.12:0/3951712060 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b87fff640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b90104d00 0x7f2b90198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b8ed9d640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b90107630 0x7f2b9019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b8e59c640 1 -- 172.21.15.12:0/3951712060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b90104d00 msgr2=0x7f2b90198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b8e59c640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b90104d00 0x7f2b90198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b8e59c640 1 -- 172.21.15.12:0/3951712060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b90107630 msgr2=0x7f2b9019f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b8e59c640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b90107630 0x7f2b9019f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b8e59c640 1 -- 172.21.15.12:0/3951712060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9019f840 con 0x7f2b901023d0 2024-05-09T09:23:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.845+0000 7f2b8ed9d640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b90107630 0x7f2b9019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.846+0000 7f2b8e59c640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b901023d0 0x7f2b90197c30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2b80009a20 tx=0x7f2b80031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:16.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.846+0000 7f2b877fe640 1 -- 172.21.15.12:0/3951712060 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8003f030 con 0x7f2b901023d0 2024-05-09T09:23:16.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.846+0000 7f2b94d5a640 1 -- 172.21.15.12:0/3951712060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9019fad0 con 0x7f2b901023d0 2024-05-09T09:23:16.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.846+0000 7f2b877fe640 1 -- 172.21.15.12:0/3951712060 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2b80003ac0 con 0x7f2b901023d0 2024-05-09T09:23:16.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.846+0000 7f2b877fe640 1 -- 172.21.15.12:0/3951712060 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80035070 con 0x7f2b901023d0 2024-05-09T09:23:16.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.846+0000 7f2b94d5a640 1 -- 172.21.15.12:0/3951712060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b9019ffc0 con 0x7f2b901023d0 2024-05-09T09:23:16.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.847+0000 7f2b94d5a640 1 -- 172.21.15.12:0/3951712060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b900fd810 con 0x7f2b901023d0 2024-05-09T09:23:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.848+0000 7f2b877fe640 1 -- 172.21.15.12:0/3951712060 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2b80042030 con 0x7f2b901023d0 2024-05-09T09:23:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.849+0000 7f2b877fe640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2b7c0a2b90 0x7f2b7c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.849+0000 7f2b87fff640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2b7c0a2b90 0x7f2b7c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.849+0000 7f2b877fe640 1 -- 172.21.15.12:0/3951712060 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f2b800ef9c0 con 0x7f2b901023d0 2024-05-09T09:23:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.849+0000 7f2b87fff640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2b7c0a2b90 0x7f2b7c0a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b78005d00 tx=0x7f2b7800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:16.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:16.853+0000 7f2b877fe640 1 -- 172.21.15.12:0/3951712060 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2b800f1e20 con 0x7f2b901023d0 2024-05-09T09:23:17.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.039+0000 7f2b94d5a640 1 -- 172.21.15.12:0/3951712060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b90062460 con 0x7f2b901023d0 2024-05-09T09:23:17.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.041+0000 7f2b877fe640 1 -- 172.21.15.12:0/3951712060 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24105 (secure 0 0 0) 0x7f2b800bc6e0 con 0x7f2b901023d0 2024-05-09T09:23:17.041 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:17.042 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:23:14.726332+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:23:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.043+0000 7f2b94d5a640 1 -- 172.21.15.12:0/3951712060 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2b7c0a2b90 msgr2=0x7f2b7c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.043+0000 7f2b94d5a640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2b7c0a2b90 0x7f2b7c0a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b78005d00 tx=0x7f2b7800b410 comp rx=0 tx=0).stop 2024-05-09T09:23:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.043+0000 7f2b94d5a640 1 -- 172.21.15.12:0/3951712060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b901023d0 msgr2=0x7f2b90197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.043+0000 7f2b94d5a640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b901023d0 0x7f2b90197c30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2b80009a20 tx=0x7f2b80031ac0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.043+0000 7f2b94d5a640 1 -- 172.21.15.12:0/3951712060 shutdown_connections 2024-05-09T09:23:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.043+0000 7f2b94d5a640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2b7c0a2b90 0x7f2b7c0a5050 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.043+0000 7f2b94d5a640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b90107630 0x7f2b9019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.043+0000 7f2b94d5a640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b90104d00 0x7f2b90198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.043+0000 7f2b94d5a640 1 --2- 172.21.15.12:0/3951712060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b901023d0 0x7f2b90197c30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.043+0000 7f2b94d5a640 1 -- 172.21.15.12:0/3951712060 >> 172.21.15.12:0/3951712060 conn(0x7f2b900fbd20 msgr2=0x7f2b90102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.043+0000 7f2b94d5a640 1 -- 172.21.15.12:0/3951712060 shutdown_connections 2024-05-09T09:23:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.043+0000 7f2b94d5a640 1 -- 172.21.15.12:0/3951712060 wait complete. 2024-05-09T09:23:17.055 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T09:23:09.656296+0000', 'modified': '2024-05-09T09:23:14.726503+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24523}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24523': {'gid': 24523, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.170:6839/3731005466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 3731005466}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 3731005466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24523, 'qdb_cluster': [24523]} 2024-05-09T09:23:17.055 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T09:23:17.055 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-09T09:23:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.161+0000 7f4546163640 1 -- 172.21.15.12:0/2366951189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4540102310 msgr2=0x7f4540104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.161+0000 7f4546163640 1 --2- 172.21.15.12:0/2366951189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4540102310 0x7f4540104700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4530009900 tx=0x7f453002a410 comp rx=0 tx=0).stop 2024-05-09T09:23:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.162+0000 7f4546163640 1 -- 172.21.15.12:0/2366951189 shutdown_connections 2024-05-09T09:23:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.162+0000 7f4546163640 1 --2- 172.21.15.12:0/2366951189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4540107570 0x7f45401099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.162+0000 7f4546163640 1 --2- 172.21.15.12:0/2366951189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4540104c40 0x7f4540107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.162+0000 7f4546163640 1 --2- 172.21.15.12:0/2366951189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4540102310 0x7f4540104700 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.162+0000 7f4546163640 1 -- 172.21.15.12:0/2366951189 >> 172.21.15.12:0/2366951189 conn(0x7f45400fbce0 msgr2=0x7f45400fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.162+0000 7f4546163640 1 -- 172.21.15.12:0/2366951189 shutdown_connections 2024-05-09T09:23:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.162+0000 7f4546163640 1 -- 172.21.15.12:0/2366951189 wait complete. 2024-05-09T09:23:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.162+0000 7f4546163640 1 Processor -- start 2024-05-09T09:23:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.162+0000 7f4546163640 1 -- start start 2024-05-09T09:23:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.162+0000 7f4546163640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4540102310 0x7f4540197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.162+0000 7f4546163640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4540104c40 0x7f4540198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f4546163640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4540107570 0x7f454019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f4546163640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f454010d4f0 con 0x7f4540104c40 2024-05-09T09:23:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f4546163640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f454010d370 con 0x7f4540107570 2024-05-09T09:23:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f4546163640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f454010d670 con 0x7f4540102310 2024-05-09T09:23:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4540104c40 0x7f4540198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4540104c40 0x7f4540198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37544/0 (socket says 172.21.15.12:37544) 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453effd640 1 -- 172.21.15.12:0/1014908167 learned_addr learned my addr 172.21.15.12:0/1014908167 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453f7fe640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4540102310 0x7f4540197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453ffff640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4540107570 0x7f454019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453effd640 1 -- 172.21.15.12:0/1014908167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4540102310 msgr2=0x7f4540197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453effd640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4540102310 0x7f4540197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453effd640 1 -- 172.21.15.12:0/1014908167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4540107570 msgr2=0x7f454019f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453effd640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4540107570 0x7f454019f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453effd640 1 -- 172.21.15.12:0/1014908167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454019f7e0 con 0x7f4540104c40 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453ffff640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4540107570 0x7f454019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453effd640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4540104c40 0x7f4540198100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f453400c970 tx=0x7f453400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453cff9640 1 -- 172.21.15.12:0/1014908167 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534007bb0 con 0x7f4540104c40 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453cff9640 1 -- 172.21.15.12:0/1014908167 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4534007d50 con 0x7f4540104c40 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f453cff9640 1 -- 172.21.15.12:0/1014908167 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453400f450 con 0x7f4540104c40 2024-05-09T09:23:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f4546163640 1 -- 172.21.15.12:0/1014908167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454019fad0 con 0x7f4540104c40 2024-05-09T09:23:17.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.163+0000 7f4546163640 1 -- 172.21.15.12:0/1014908167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45401a0070 con 0x7f4540104c40 2024-05-09T09:23:17.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f453cff9640 1 -- 172.21.15.12:0/1014908167 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4534004f50 con 0x7f4540104c40 2024-05-09T09:23:17.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f4548e79740 1 Processor -- start 2024-05-09T09:23:17.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f4548e79740 1 -- start start 2024-05-09T09:23:17.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f4548e79740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 0x5582bb92cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f4548e79740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92d2b0 0x5582bb931db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f4548e79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb9348b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f4548e79740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5582bb8904d0 con 0x5582bb92c8c0 2024-05-09T09:23:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f4548e79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5582bb8383e0 con 0x5582bb932420 2024-05-09T09:23:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f4548e79740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5582bb88c490 con 0x5582bb92d2b0 2024-05-09T09:23:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f453ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 0x5582bb92cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f453ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 0x5582bb92cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37552/0 (socket says 172.21.15.12:37552) 2024-05-09T09:23:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f453ffff640 1 -- 172.21.15.12:0/2403278949 learned_addr learned my addr 172.21.15.12:0/2403278949 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f453f7fe640 1 --2- 172.21.15.12:0/2403278949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92d2b0 0x5582bb931db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.166+0000 7f453cff9640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f45280a2c10 0x7f45280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.167+0000 7f453effd640 1 --2- 172.21.15.12:0/2403278949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb9348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.167+0000 7f453ffff640 1 -- 172.21.15.12:0/2403278949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92d2b0 msgr2=0x5582bb931db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.167+0000 7f453ffff640 1 --2- 172.21.15.12:0/2403278949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92d2b0 0x5582bb931db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.167+0000 7f453ffff640 1 -- 172.21.15.12:0/2403278949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 msgr2=0x5582bb9348b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.167+0000 7f453ffff640 1 --2- 172.21.15.12:0/2403278949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb9348b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.167+0000 7f453ffff640 1 -- 172.21.15.12:0/2403278949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582bb5a42d0 con 0x5582bb92c8c0 2024-05-09T09:23:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.167+0000 7f453cff9640 1 -- 172.21.15.12:0/1014908167 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f45340cf220 con 0x7f4540104c40 2024-05-09T09:23:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.167+0000 7f453ffff640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f45280a2c10 0x7f45280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.167+0000 7f453ffff640 1 --2- 172.21.15.12:0/2403278949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 0x5582bb92cce0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f453800f390 tx=0x7f453800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3a0499cf22efc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.167+0000 7f4525ffb640 1 -- 172.21.15.12:0/2403278949 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453801a610 con 0x5582bb92c8c0 2024-05-09T09:23:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.167+0000 7f4525ffb640 1 -- 172.21.15.12:0/2403278949 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f453801ac80 con 0x5582bb92c8c0 2024-05-09T09:23:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.167+0000 7f4525ffb640 1 -- 172.21.15.12:0/2403278949 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4538010a80 con 0x5582bb92c8c0 2024-05-09T09:23:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.167+0000 7f453ffff640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f45280a2c10 0x7f45280a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4538020000 tx=0x7f453801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.168+0000 7f4548e79740 1 -- 172.21.15.12:0/2403278949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 msgr2=0x5582bb92cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.168+0000 7f4548e79740 1 --2- 172.21.15.12:0/2403278949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 0x5582bb92cce0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f453800f390 tx=0x7f453800f850 comp rx=0 tx=0).stop 2024-05-09T09:23:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.168+0000 7f4548e79740 1 -- 172.21.15.12:0/2403278949 shutdown_connections 2024-05-09T09:23:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.168+0000 7f4548e79740 1 --2- 172.21.15.12:0/2403278949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb9348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.168+0000 7f4548e79740 1 --2- 172.21.15.12:0/2403278949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92d2b0 0x5582bb931db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.168+0000 7f4548e79740 1 --2- 172.21.15.12:0/2403278949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 0x5582bb92cce0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.168+0000 7f4548e79740 1 -- 172.21.15.12:0/2403278949 >> 172.21.15.12:0/2403278949 conn(0x5582bb927f30 msgr2=0x5582bb928330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.168+0000 7f4548e79740 1 -- 172.21.15.12:0/2403278949 shutdown_connections 2024-05-09T09:23:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.168+0000 7f4548e79740 1 -- 172.21.15.12:0/2403278949 wait complete. 2024-05-09T09:23:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.168+0000 7f4548e79740 1 -- start start 2024-05-09T09:23:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.168+0000 7f4548e79740 1 Processor -- start 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f4548e79740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 0x5582bb936ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f4548e79740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92d2b0 0x5582bb937220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f4548e79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb93e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f4548e79740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5582bb9353d0 con 0x5582bb92c8c0 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f4548e79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5582bb84b8e0 con 0x5582bb932420 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f4548e79740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5582bb935550 con 0x5582bb92d2b0 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f4548e79740 10 client.0 _ll_register_callbacks cb 0x5582bb8189c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f453f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 0x5582bb936ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f453f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 0x5582bb936ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37566/0 (socket says 172.21.15.12:37566) 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f453f7fe640 1 -- 172.21.15.12:0/107489933 learned_addr learned my addr 172.21.15.12:0/107489933 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f453effd640 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92d2b0 0x5582bb937220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f453ffff640 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb93e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f453f7fe640 1 -- 172.21.15.12:0/107489933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92d2b0 msgr2=0x5582bb937220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f453f7fe640 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92d2b0 0x5582bb937220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f453f7fe640 1 -- 172.21.15.12:0/107489933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 msgr2=0x5582bb93e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f453f7fe640 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb93e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f453f7fe640 1 -- 172.21.15.12:0/107489933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582bb66e6b0 con 0x5582bb92c8c0 2024-05-09T09:23:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f453effd640 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92d2b0 0x5582bb937220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:23:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f453f7fe640 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 0x5582bb936ce0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4530009900 tx=0x7f4530031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d42c15ebe4bbe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f451effd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:23:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f451effd640 1 -- 172.21.15.12:0/107489933 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530035040 con 0x5582bb92c8c0 2024-05-09T09:23:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f451effd640 1 -- 172.21.15.12:0/107489933 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4530004040 con 0x5582bb92c8c0 2024-05-09T09:23:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.169+0000 7f4548e79740 1 -- 172.21.15.12:0/107489933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f453801ac80 con 0x5582bb92c8c0 2024-05-09T09:23:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.170+0000 7f451effd640 1 -- 172.21.15.12:0/107489933 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f4530004240 con 0x5582bb92c8c0 2024-05-09T09:23:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.170+0000 7f4548e79740 10 client.14754 fetch_fsmap learned FSMap version 17 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.170+0000 7f4548e79740 1 -- 172.21.15.12:0/107489933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5582bb9412a0 con 0x5582bb92c8c0 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.170+0000 7f451effd640 1 -- 172.21.15.12:0/107489933 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f453801ac80 con 0x5582bb92c8c0 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.170+0000 7f451effd640 1 -- 172.21.15.12:0/107489933 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f453003d570 con 0x5582bb92c8c0 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.170+0000 7f4548e79740 10 client.14754 fetch_fsmap finished waiting for FSMap version 17 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.170+0000 7f4548e79740 10 client.14754 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.170+0000 7f4548e79740 20 client.14754 populate_metadata read hostname 'smithi012' 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.170+0000 7f4548e79740 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5582bb943900 0x5582bb944d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.170+0000 7f4548e79740 4 client.14754 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.170+0000 7f4548e79740 1 -- 172.21.15.12:0/107489933 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582bb89dc00 con 0x5582bb943900 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.171+0000 7f453effd640 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5582bb943900 0x5582bb944d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.171+0000 7f453effd640 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5582bb943900 0x5582bb944d70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a32f6c3af019ecfc server_cookie=a1bdd554e50c9b05 in_seq=0 out_seq=0 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.171+0000 7f451effd640 10 client.14754 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f451effd640 1 -- 172.21.15.12:0/107489933 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582bb89dc00 con 0x5582bb943900 2024-05-09T09:23:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f451effd640 10 client.14754 handle_command_reply: tid=0 2024-05-09T09:23:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f4548e79740 1 client.14754 shutdown 2024-05-09T09:23:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f4548e79740 1 -- 172.21.15.12:0/107489933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 msgr2=0x5582bb936ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f4548e79740 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 0x5582bb936ce0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4530009900 tx=0x7f4530031840 comp rx=0 tx=0).stop 2024-05-09T09:23:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f4548e79740 1 -- 172.21.15.12:0/107489933 shutdown_connections 2024-05-09T09:23:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f4548e79740 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5582bb943900 0x5582bb944d70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f4548e79740 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb93e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f4548e79740 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92d2b0 0x5582bb937220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f4548e79740 1 --2- 172.21.15.12:0/107489933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92c8c0 0x5582bb936ce0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f451effd640 0 client.14754 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:23:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f4548e79740 1 -- 172.21.15.12:0/107489933 >> 172.21.15.12:0/107489933 conn(0x5582bb927f30 msgr2=0x5582bb927000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:17.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f4548e79740 1 -- 172.21.15.12:0/107489933 shutdown_connections 2024-05-09T09:23:17.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f4548e79740 1 -- 172.21.15.12:0/107489933 wait complete. 2024-05-09T09:23:17.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.172+0000 7f4548e79740 20 client.14754 trim_cache size 0 max 16384 2024-05-09T09:23:17.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.190+0000 7f4548e79740 1 Processor -- start 2024-05-09T09:23:17.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.190+0000 7f4548e79740 1 -- start start 2024-05-09T09:23:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.190+0000 7f4548e79740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92c8c0 0x5582bb92e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f4548e79740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 0x5582bb92ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f4548e79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb93e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f4548e79740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5582bb9430b0 con 0x5582bb92d2b0 2024-05-09T09:23:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f4548e79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5582bb943780 con 0x5582bb932420 2024-05-09T09:23:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f4548e79740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5582bb943e50 con 0x5582bb92c8c0 2024-05-09T09:23:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f453effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 0x5582bb92ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f453effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 0x5582bb92ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37572/0 (socket says 172.21.15.12:37572) 2024-05-09T09:23:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f453effd640 1 -- 172.21.15.12:0/1887021859 learned_addr learned my addr 172.21.15.12:0/1887021859 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f453ffff640 1 --2- 172.21.15.12:0/1887021859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb93e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f453effd640 1 -- 172.21.15.12:0/1887021859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92c8c0 msgr2=0x5582bb92e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f453effd640 1 --2- 172.21.15.12:0/1887021859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92c8c0 0x5582bb92e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f453effd640 1 -- 172.21.15.12:0/1887021859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 msgr2=0x5582bb93e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f453effd640 1 --2- 172.21.15.12:0/1887021859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb93e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f453effd640 1 -- 172.21.15.12:0/1887021859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582bb92c500 con 0x5582bb92d2b0 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f453ffff640 1 --2- 172.21.15.12:0/1887021859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb93e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f453effd640 1 --2- 172.21.15.12:0/1887021859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 0x5582bb92ec20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f45340ccae0 tx=0x7f4534086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26be69faa3802fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f45257fa640 1 -- 172.21.15.12:0/1887021859 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453401d040 con 0x5582bb92d2b0 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f45257fa640 1 -- 172.21.15.12:0/1887021859 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f45340a44b0 con 0x5582bb92d2b0 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.191+0000 7f45257fa640 1 -- 172.21.15.12:0/1887021859 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45340860b0 con 0x5582bb92d2b0 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.192+0000 7f4548e79740 1 -- 172.21.15.12:0/1887021859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 msgr2=0x5582bb92ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.192+0000 7f4548e79740 1 --2- 172.21.15.12:0/1887021859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 0x5582bb92ec20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f45340ccae0 tx=0x7f4534086040 comp rx=0 tx=0).stop 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.192+0000 7f4548e79740 1 -- 172.21.15.12:0/1887021859 shutdown_connections 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.192+0000 7f4548e79740 1 --2- 172.21.15.12:0/1887021859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb93e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.192+0000 7f4548e79740 1 --2- 172.21.15.12:0/1887021859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 0x5582bb92ec20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.192+0000 7f4548e79740 1 --2- 172.21.15.12:0/1887021859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92c8c0 0x5582bb92e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.192+0000 7f4548e79740 1 -- 172.21.15.12:0/1887021859 >> 172.21.15.12:0/1887021859 conn(0x5582bb927f30 msgr2=0x5582bb928310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.192+0000 7f4548e79740 1 -- 172.21.15.12:0/1887021859 shutdown_connections 2024-05-09T09:23:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.192+0000 7f4548e79740 1 -- 172.21.15.12:0/1887021859 wait complete. 2024-05-09T09:23:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.192+0000 7f4548e79740 1 -- start start 2024-05-09T09:23:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.192+0000 7f4548e79740 1 Processor -- start 2024-05-09T09:23:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.192+0000 7f4548e79740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92c8c0 0x5582bb93a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f4548e79740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 0x5582bb93a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f4548e79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb93d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f4548e79740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5582bb944bf0 con 0x5582bb92d2b0 2024-05-09T09:23:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f4548e79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5582bb944520 con 0x5582bb932420 2024-05-09T09:23:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f4548e79740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5582bb9452c0 con 0x5582bb92c8c0 2024-05-09T09:23:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f4548e79740 10 client.0 _ll_register_callbacks cb 0x7f4530005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f453effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 0x5582bb93a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f453effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 0x5582bb93a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37578/0 (socket says 172.21.15.12:37578) 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f453effd640 1 -- 172.21.15.12:0/2582805087 learned_addr learned my addr 172.21.15.12:0/2582805087 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f453f7fe640 1 --2- 172.21.15.12:0/2582805087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92c8c0 0x5582bb93a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f453ffff640 1 --2- 172.21.15.12:0/2582805087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb93d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f453effd640 1 -- 172.21.15.12:0/2582805087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92c8c0 msgr2=0x5582bb93a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f453effd640 1 --2- 172.21.15.12:0/2582805087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92c8c0 0x5582bb93a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f453effd640 1 -- 172.21.15.12:0/2582805087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 msgr2=0x5582bb93d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f453effd640 1 --2- 172.21.15.12:0/2582805087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb93d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f453effd640 1 -- 172.21.15.12:0/2582805087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582bb932170 con 0x5582bb92d2b0 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f453effd640 1 --2- 172.21.15.12:0/2582805087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 0x5582bb93a8e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f453409b7d0 tx=0x7f4534004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c266d696d1a542d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f451effd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f451effd640 1 -- 172.21.15.12:0/2582805087 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453401d040 con 0x5582bb92d2b0 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f451effd640 1 -- 172.21.15.12:0/2582805087 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f45340860b0 con 0x5582bb92d2b0 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f4548e79740 1 -- 172.21.15.12:0/2582805087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45340a44b0 con 0x5582bb92d2b0 2024-05-09T09:23:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f451effd640 1 -- 172.21.15.12:0/2582805087 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f4534099960 con 0x5582bb92d2b0 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f4548e79740 10 client.14760 fetch_fsmap learned FSMap version 17 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.193+0000 7f4548e79740 1 -- 172.21.15.12:0/2582805087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5582bb9a9c80 con 0x5582bb92d2b0 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.194+0000 7f451effd640 1 -- 172.21.15.12:0/2582805087 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f45340a44b0 con 0x5582bb92d2b0 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.194+0000 7f451effd640 1 -- 172.21.15.12:0/2582805087 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f4534005240 con 0x5582bb92d2b0 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.194+0000 7f4548e79740 10 client.14760 fetch_fsmap finished waiting for FSMap version 17 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.194+0000 7f4548e79740 10 client.14760 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.194+0000 7f4548e79740 20 client.14760 populate_metadata read hostname 'smithi012' 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.194+0000 7f4548e79740 1 --2- 172.21.15.12:0/2582805087 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5582bb9ae350 0x5582bb9ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.194+0000 7f4548e79740 4 client.14760 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.194+0000 7f4548e79740 1 -- 172.21.15.12:0/2582805087 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "status"}) -- 0x5582bb897f50 con 0x5582bb9ae350 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.194+0000 7f453f7fe640 1 --2- 172.21.15.12:0/2582805087 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5582bb9ae350 0x5582bb9ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.194+0000 7f453f7fe640 1 --2- 172.21.15.12:0/2582805087 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5582bb9ae350 0x5582bb9ad060 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbd9613f499d0cdb server_cookie=8404c6519adf5b05 in_seq=0 out_seq=0 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.194+0000 7f451effd640 10 client.14760 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f451effd640 1 -- 172.21.15.12:0/2582805087 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5582bb897f50 con 0x5582bb9ae350 2024-05-09T09:23:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f451effd640 10 client.14760 handle_command_reply: tid=0 2024-05-09T09:23:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4548e79740 1 client.14760 shutdown 2024-05-09T09:23:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4548e79740 1 -- 172.21.15.12:0/2582805087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 msgr2=0x5582bb93a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4548e79740 1 --2- 172.21.15.12:0/2582805087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 0x5582bb93a8e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f453409b7d0 tx=0x7f4534004740 comp rx=0 tx=0).stop 2024-05-09T09:23:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4548e79740 1 -- 172.21.15.12:0/2582805087 shutdown_connections 2024-05-09T09:23:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4548e79740 1 --2- 172.21.15.12:0/2582805087 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5582bb9ae350 0x5582bb9ad060 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4548e79740 1 --2- 172.21.15.12:0/2582805087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb932420 0x5582bb93d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4548e79740 1 --2- 172.21.15.12:0/2582805087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582bb92d2b0 0x5582bb93a8e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4548e79740 1 --2- 172.21.15.12:0/2582805087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582bb92c8c0 0x5582bb93a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f451effd640 0 client.14760 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:23:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4548e79740 1 -- 172.21.15.12:0/2582805087 >> 172.21.15.12:0/2582805087 conn(0x5582bb927f30 msgr2=0x5582bb92e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4548e79740 1 -- 172.21.15.12:0/2582805087 shutdown_connections 2024-05-09T09:23:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4548e79740 1 -- 172.21.15.12:0/2582805087 wait complete. 2024-05-09T09:23:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4548e79740 20 client.14760 trim_cache size 0 max 16384 2024-05-09T09:23:17.203 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:23:17.203 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:23:17.203 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-05-09T09:23:17.203 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24523, 2024-05-09T09:23:17.203 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-09T09:23:17.203 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-09T09:23:17.203 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-09T09:23:17.203 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 6.8859004480000001, 2024-05-09T09:23:17.203 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 17, 2024-05-09T09:23:17.203 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-05-09T09:23:17.203 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T09:23:17.203 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 21.882999338000001 2024-05-09T09:23:17.204 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:23:17.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4546163640 1 -- 172.21.15.12:0/1014908167 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f45280a2c10 msgr2=0x7f45280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:17.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4546163640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f45280a2c10 0x7f45280a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4538020000 tx=0x7f453801d000 comp rx=0 tx=0).stop 2024-05-09T09:23:17.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4546163640 1 -- 172.21.15.12:0/1014908167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4540104c40 msgr2=0x7f4540198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:17.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4546163640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4540104c40 0x7f4540198100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f453400c970 tx=0x7f453400ce30 comp rx=0 tx=0).stop 2024-05-09T09:23:17.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4546163640 1 -- 172.21.15.12:0/1014908167 shutdown_connections 2024-05-09T09:23:17.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4546163640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f45280a2c10 0x7f45280a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4546163640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4540107570 0x7f454019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4546163640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4540104c40 0x7f4540198100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4546163640 1 --2- 172.21.15.12:0/1014908167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4540102310 0x7f4540197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:17.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.195+0000 7f4546163640 1 -- 172.21.15.12:0/1014908167 >> 172.21.15.12:0/1014908167 conn(0x7f45400fbce0 msgr2=0x7f4540102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:17.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.196+0000 7f4546163640 1 -- 172.21.15.12:0/1014908167 shutdown_connections 2024-05-09T09:23:17.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:17.196+0000 7f4546163640 1 -- 172.21.15.12:0/1014908167 wait complete. 2024-05-09T09:23:17.206 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T09:23:17.206 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:23:17.207 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-09T09:23:17.454 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:17.703 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:17 smithi012 ceph-mon[30140]: pgmap v295: 129 pgs: 10 creating+peering, 116 active+clean, 3 unknown; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 5 op/s 2024-05-09T09:23:17.704 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:17 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1275477933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:17.704 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:17 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3951712060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:17.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:17 smithi170 ceph-mon[33632]: pgmap v295: 129 pgs: 10 creating+peering, 116 active+clean, 3 unknown; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 5 op/s 2024-05-09T09:23:17.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:17 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1275477933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:17.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:17 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3951712060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:17.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:17 smithi016 ceph-mon[33335]: pgmap v295: 129 pgs: 10 creating+peering, 116 active+clean, 3 unknown; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 5 op/s 2024-05-09T09:23:17.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:17 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1275477933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:17.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:17 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3951712060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:18.658 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:18 smithi016 ceph-mon[33335]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T09:23:18.658 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:18 smithi016 ceph-mon[33335]: Cluster is now healthy 2024-05-09T09:23:18.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:18 smithi012 ceph-mon[30140]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T09:23:18.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:18 smithi012 ceph-mon[30140]: Cluster is now healthy 2024-05-09T09:23:18.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:18 smithi170 ceph-mon[33632]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T09:23:18.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:18 smithi170 ceph-mon[33632]: Cluster is now healthy 2024-05-09T09:23:19.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.391+0000 7f897caf1700 1 -- 172.21.15.12:0/2525679008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8978102fb0 msgr2=0x7f8978106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:19.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.391+0000 7f897caf1700 1 --2- 172.21.15.12:0/2525679008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8978102fb0 0x7f8978106ff0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f89600099c0 tx=0x7f8960009cd0 comp rx=0 tx=0).stop 2024-05-09T09:23:19.392 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:19 smithi012 ceph-mon[30140]: pgmap v296: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-05-09T09:23:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.391+0000 7f897caf1700 1 -- 172.21.15.12:0/2525679008 shutdown_connections 2024-05-09T09:23:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.391+0000 7f897caf1700 1 --2- 172.21.15.12:0/2525679008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89781076d0 0x7f897810f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.391+0000 7f897caf1700 1 --2- 172.21.15.12:0/2525679008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8978102600 0x7f89781029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.391+0000 7f897caf1700 1 --2- 172.21.15.12:0/2525679008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8978102fb0 0x7f8978106ff0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.391+0000 7f897caf1700 1 -- 172.21.15.12:0/2525679008 >> 172.21.15.12:0/2525679008 conn(0x7f89780794d0 msgr2=0x7f89780798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.391+0000 7f897caf1700 1 -- 172.21.15.12:0/2525679008 shutdown_connections 2024-05-09T09:23:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.391+0000 7f897caf1700 1 -- 172.21.15.12:0/2525679008 wait complete. 2024-05-09T09:23:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.392+0000 7f897caf1700 1 Processor -- start 2024-05-09T09:23:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.392+0000 7f897caf1700 1 -- start start 2024-05-09T09:23:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.392+0000 7f897caf1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8978102600 0x7f897819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.392+0000 7f897caf1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8978102fb0 0x7f897819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.392+0000 7f897caf1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89781076d0 0x7f8978198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.392+0000 7f897caf1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8978112720 con 0x7f8978102fb0 2024-05-09T09:23:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.392+0000 7f897caf1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89781125a0 con 0x7f89781076d0 2024-05-09T09:23:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.392+0000 7f897caf1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89781128a0 con 0x7f8978102600 2024-05-09T09:23:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.392+0000 7f896dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8978102fb0 0x7f897819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.392+0000 7f896dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8978102fb0 0x7f897819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37602/0 (socket says 172.21.15.12:37602) 2024-05-09T09:23:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.392+0000 7f896dd9b700 1 -- 172.21.15.12:0/621510470 learned_addr learned my addr 172.21.15.12:0/621510470 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.392+0000 7f897659c700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8978102600 0x7f897819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.393+0000 7f8976d9d700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89781076d0 0x7f8978198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.393+0000 7f896dd9b700 1 -- 172.21.15.12:0/621510470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8978102600 msgr2=0x7f897819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.393+0000 7f896dd9b700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8978102600 0x7f897819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.393+0000 7f896dd9b700 1 -- 172.21.15.12:0/621510470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89781076d0 msgr2=0x7f8978198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.393+0000 7f896dd9b700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89781076d0 0x7f8978198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.393+0000 7f896dd9b700 1 -- 172.21.15.12:0/621510470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8978199180 con 0x7f8978102fb0 2024-05-09T09:23:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.393+0000 7f8976d9d700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89781076d0 0x7f8978198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.393+0000 7f896dd9b700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8978102fb0 0x7f897819eda0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f896000c010 tx=0x7f89600057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.393+0000 7f896ffff700 1 -- 172.21.15.12:0/621510470 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960027070 con 0x7f8978102fb0 2024-05-09T09:23:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.393+0000 7f897caf1700 1 -- 172.21.15.12:0/621510470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8978199410 con 0x7f8978102fb0 2024-05-09T09:23:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.393+0000 7f896ffff700 1 -- 172.21.15.12:0/621510470 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8960003ea0 con 0x7f8978102fb0 2024-05-09T09:23:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.393+0000 7f896ffff700 1 -- 172.21.15.12:0/621510470 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960022bf0 con 0x7f8978102fb0 2024-05-09T09:23:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.393+0000 7f897caf1700 1 -- 172.21.15.12:0/621510470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89781a5730 con 0x7f8978102fb0 2024-05-09T09:23:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.394+0000 7f897caf1700 1 -- 172.21.15.12:0/621510470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8978069010 con 0x7f8978102fb0 2024-05-09T09:23:19.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.396+0000 7f896ffff700 1 -- 172.21.15.12:0/621510470 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f896002a070 con 0x7f8978102fb0 2024-05-09T09:23:19.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.396+0000 7f896ffff700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f89640a2ca0 0x7f89640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:19.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.397+0000 7f897659c700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f89640a2ca0 0x7f89640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:19.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.397+0000 7f896ffff700 1 -- 172.21.15.12:0/621510470 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f89600cfe70 con 0x7f8978102fb0 2024-05-09T09:23:19.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.397+0000 7f897659c700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f89640a2ca0 0x7f89640a5160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8978199b60 tx=0x7f896800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:19.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.401+0000 7f896ffff700 1 -- 172.21.15.12:0/621510470 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8960013070 con 0x7f8978102fb0 2024-05-09T09:23:19.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.652+0000 7f897caf1700 1 -- 172.21.15.12:0/621510470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8978199f50 con 0x7f8978102fb0 2024-05-09T09:23:19.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.653+0000 7f896ffff700 1 -- 172.21.15.12:0/621510470 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f896009cc20 con 0x7f8978102fb0 2024-05-09T09:23:19.653 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T09:23:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.655+0000 7f897caf1700 1 -- 172.21.15.12:0/621510470 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f89640a2ca0 msgr2=0x7f89640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.655+0000 7f897caf1700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f89640a2ca0 0x7f89640a5160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8978199b60 tx=0x7f896800b3c0 comp rx=0 tx=0).stop 2024-05-09T09:23:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.655+0000 7f897caf1700 1 -- 172.21.15.12:0/621510470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8978102fb0 msgr2=0x7f897819eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.655+0000 7f897caf1700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8978102fb0 0x7f897819eda0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f896000c010 tx=0x7f89600057c0 comp rx=0 tx=0).stop 2024-05-09T09:23:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.655+0000 7f897caf1700 1 -- 172.21.15.12:0/621510470 shutdown_connections 2024-05-09T09:23:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.655+0000 7f897caf1700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f89640a2ca0 0x7f89640a5160 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.655+0000 7f897caf1700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89781076d0 0x7f8978198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.655+0000 7f897caf1700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8978102600 0x7f897819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.655+0000 7f897caf1700 1 --2- 172.21.15.12:0/621510470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8978102fb0 0x7f897819eda0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.655+0000 7f897caf1700 1 -- 172.21.15.12:0/621510470 >> 172.21.15.12:0/621510470 conn(0x7f89780794d0 msgr2=0x7f8978100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.655+0000 7f897caf1700 1 -- 172.21.15.12:0/621510470 shutdown_connections 2024-05-09T09:23:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:19.655+0000 7f897caf1700 1 -- 172.21.15.12:0/621510470 wait complete. 2024-05-09T09:23:19.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:19 smithi170 ceph-mon[33632]: pgmap v296: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-05-09T09:23:19.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:19 smithi016 ceph-mon[33335]: pgmap v296: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-05-09T09:23:20.541 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:20 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/621510470' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T09:23:20.575 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-09T09:23:20.575 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T09:23:20.575 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-05-09T09:23:20.575 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-09T09:23:20.575 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:23:20.576 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-09T09:23:20.576 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T09:23:20.576 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T09:23:20.576 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T09:23:20.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:20.576 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-05-09T09:23:20.595 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T09:23:20.595 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T09:23:20.595 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T09:23:20.596 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42226sec preferred_lft 42226sec 2024-05-09T09:23:20.597 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-05-09T09:23:20.597 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-09T09:23:20.597 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T09:23:20.597 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-05-09T09:23:20.597 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-05-09T09:23:20.597 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T09:23:20.598 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:23:20.598 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-09T09:23:20.598 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2024-05-09T09:23:20.598 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2024-05-09T09:23:20.598 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2024-05-09T09:23:20.598 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T09:23:20.598 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-09T09:23:20.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:23:20.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:20 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/621510470' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T09:23:20.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:20 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/621510470' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T09:23:20.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:23:20.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:20.924 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T09:23:20.951 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-05-09T09:23:20.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:20.952 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-05-09T09:23:21.024 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-05-09T09:23:21.025 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T09:23:21.025 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T09:23:21.025 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T09:23:21.025 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T09:23:21.025 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:23:21.026 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-09T09:23:21.026 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T09:23:21.026 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T09:23:21.026 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T09:23:21.026 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-09T09:23:21.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:23:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:23:21.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:21.370 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-05-09T09:23:21.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:21.387 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2024-05-09T09:23:21.442 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:23:21.443 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-09T09:23:21.443 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:23:21.443 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-09T09:23:21.443 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-09T09:23:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:23:21.541 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:21 smithi012 ceph-mon[30140]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 3 op/s 2024-05-09T09:23:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:23:21.563 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-09T09:23:21.564 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:23:21.564 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-09T09:23:21.564 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-09T09:23:21.564 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T09:23:21.564 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-09T09:23:21.564 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-09T09:23:21.564 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-09T09:23:21.564 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-09T09:23:21.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:23:21.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:23:21.694 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:23:21.694 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-09T09:23:21.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-05-09T09:23:21.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T09:23:21.694 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-09T09:23:21.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:21 smithi170 ceph-mon[33632]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 3 op/s 2024-05-09T09:23:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:23:21.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:21 smithi016 ceph-mon[33335]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 3 op/s 2024-05-09T09:23:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:23:21.773 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-09T09:23:21.774 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T09:23:21.774 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:23:21.789 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T09:23:21.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T09:23:21.790 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:23:21.846 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-05-09T09:23:21.914 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T09:23:21.971 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-05-09T09:23:21.971 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-05-09T09:23:21.972 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-05-09T09:23:21.972 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-05-09T09:23:21.972 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-05-09T09:23:21.972 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-05-09T09:23:21.972 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-05-09T09:23:21.972 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-05-09T09:23:21.972 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-05-09T09:23:21.972 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-05-09T09:23:21.972 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-05-09T09:23:21.972 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-05-09T09:23:21.972 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:23:21.972 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e88de82f85e98df57e228d7fe5d2b464e921cd3fbb86b352dc1f8680310d2fc5/merged 2024-05-09T09:23:21.973 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/47eb820a76be2ed00b0b2bc110d5ddcb3ba3cf21aeb046cba3fa97500223e232/merged 2024-05-09T09:23:21.974 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-05-09T09:23:21.974 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:23:21.974 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/f53bf39be5eb886f0c543698bd5796a232fbe79c324e29998c6fce40c8424a3a/merged 2024-05-09T09:23:21.974 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/4c2954d02a4a8aae844dd087075ae7189c1efe2363e62e195ff316bd2b233f72/merged 2024-05-09T09:23:21.974 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e86ea0c2dbb511b70d72ff40ff8767c6e5c13ca820da6152cdf9dcc8bbeda890/merged 2024-05-09T09:23:21.974 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/d329092564ce4ad60562f3d9a0ca5a315c01c60bbd3fb776d3760e89f0c8e393/merged 2024-05-09T09:23:21.974 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/a76753f2da79e4dd54a726d9da6582f14c0f0ea3885cdd7abb165d85cf305e5c/merged 2024-05-09T09:23:21.974 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e0710f4ce6abaa0e66e2ffe05da939b27fb50b3ca2ed31ad8a5d5674efedec36/merged 2024-05-09T09:23:21.974 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e6f0b9765e9120f74f8a213a2b59acafc93d309a03fb4f9d4ae2cc3ca5db1398/merged 2024-05-09T09:23:21.974 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/ef4517c95f8187307254aa2de801d47b459dc6bd5826e09086e0e3ba1fda2f9e/merged 2024-05-09T09:23:21.974 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-05-09T09:23:21.974 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:23:21.974 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:23:21.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:21.975 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-05-09T09:23:22.027 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T09:23:22.028 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-09T09:23:22.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-05-09T09:23:22.096 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T09:23:22.153 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-05-09T09:23:22.153 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-05-09T09:23:22.153 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-05-09T09:23:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e88de82f85e98df57e228d7fe5d2b464e921cd3fbb86b352dc1f8680310d2fc5/merged 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/47eb820a76be2ed00b0b2bc110d5ddcb3ba3cf21aeb046cba3fa97500223e232/merged 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/f53bf39be5eb886f0c543698bd5796a232fbe79c324e29998c6fce40c8424a3a/merged 2024-05-09T09:23:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/4c2954d02a4a8aae844dd087075ae7189c1efe2363e62e195ff316bd2b233f72/merged 2024-05-09T09:23:22.156 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e86ea0c2dbb511b70d72ff40ff8767c6e5c13ca820da6152cdf9dcc8bbeda890/merged 2024-05-09T09:23:22.156 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/d329092564ce4ad60562f3d9a0ca5a315c01c60bbd3fb776d3760e89f0c8e393/merged 2024-05-09T09:23:22.156 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/a76753f2da79e4dd54a726d9da6582f14c0f0ea3885cdd7abb165d85cf305e5c/merged 2024-05-09T09:23:22.156 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e0710f4ce6abaa0e66e2ffe05da939b27fb50b3ca2ed31ad8a5d5674efedec36/merged 2024-05-09T09:23:22.156 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e6f0b9765e9120f74f8a213a2b59acafc93d309a03fb4f9d4ae2cc3ca5db1398/merged 2024-05-09T09:23:22.156 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/ef4517c95f8187307254aa2de801d47b459dc6bd5826e09086e0e3ba1fda2f9e/merged 2024-05-09T09:23:22.156 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-05-09T09:23:22.156 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:23:22.156 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:23:22.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:22.157 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-05-09T09:23:22.220 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:2024-05-09T09:23:22.220+0000 7f5a03d94480 -1 init, newargv = 0x55ca65bdc3c0 newargc=15 2024-05-09T09:23:22.220 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[74789]: starting ceph client 2024-05-09T09:23:22.226 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[74789]: starting fuse 2024-05-09T09:23:23.311 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-05-09T09:23:23.345 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T09:23:23.405 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-05-09T09:23:23.405 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-05-09T09:23:23.405 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-05-09T09:23:23.405 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-05-09T09:23:23.405 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-05-09T09:23:23.406 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e88de82f85e98df57e228d7fe5d2b464e921cd3fbb86b352dc1f8680310d2fc5/merged 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/47eb820a76be2ed00b0b2bc110d5ddcb3ba3cf21aeb046cba3fa97500223e232/merged 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/f53bf39be5eb886f0c543698bd5796a232fbe79c324e29998c6fce40c8424a3a/merged 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/4c2954d02a4a8aae844dd087075ae7189c1efe2363e62e195ff316bd2b233f72/merged 2024-05-09T09:23:23.407 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e86ea0c2dbb511b70d72ff40ff8767c6e5c13ca820da6152cdf9dcc8bbeda890/merged 2024-05-09T09:23:23.408 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/d329092564ce4ad60562f3d9a0ca5a315c01c60bbd3fb776d3760e89f0c8e393/merged 2024-05-09T09:23:23.408 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/a76753f2da79e4dd54a726d9da6582f14c0f0ea3885cdd7abb165d85cf305e5c/merged 2024-05-09T09:23:23.408 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e0710f4ce6abaa0e66e2ffe05da939b27fb50b3ca2ed31ad8a5d5674efedec36/merged 2024-05-09T09:23:23.408 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e6f0b9765e9120f74f8a213a2b59acafc93d309a03fb4f9d4ae2cc3ca5db1398/merged 2024-05-09T09:23:23.408 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/ef4517c95f8187307254aa2de801d47b459dc6bd5826e09086e0e3ba1fda2f9e/merged 2024-05-09T09:23:23.408 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-05-09T09:23:23.408 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:23:23.408 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:23:23.408 INFO:teuthology.orchestra.run.smithi012.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-09T09:23:23.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:23.408 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-05-09T09:23:23.464 INFO:teuthology.orchestra.run.smithi012.stdout:114 2024-05-09T09:23:23.464 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-09T09:23:23.464 DEBUG:teuthology.orchestra.run.smithi012:> sudo stdin-killer -- python3 -c ' 2024-05-09T09:23:23.464 DEBUG:teuthology.orchestra.run.smithi012:> import glob 2024-05-09T09:23:23.464 DEBUG:teuthology.orchestra.run.smithi012:> import re 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> import os 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> import subprocess 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> def _find_admin_socket(client_name): 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> files = glob.glob(asok_path) 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> # Given a non-glob path, it better be there 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> if "*" not in asok_path: 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> assert(len(files) == 1) 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> return files[0] 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> for f in files: 2024-05-09T09:23:23.465 DEBUG:teuthology.orchestra.run.smithi012:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T09:23:23.466 DEBUG:teuthology.orchestra.run.smithi012:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T09:23:23.466 DEBUG:teuthology.orchestra.run.smithi012:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T09:23:23.466 DEBUG:teuthology.orchestra.run.smithi012:> contents = proc_f.read() 2024-05-09T09:23:23.466 DEBUG:teuthology.orchestra.run.smithi012:> if mountpoint in contents: 2024-05-09T09:23:23.466 DEBUG:teuthology.orchestra.run.smithi012:> return f 2024-05-09T09:23:23.466 DEBUG:teuthology.orchestra.run.smithi012:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T09:23:23.466 DEBUG:teuthology.orchestra.run.smithi012:> 2024-05-09T09:23:23.466 DEBUG:teuthology.orchestra.run.smithi012:> print(_find_admin_socket("client.admin")) 2024-05-09T09:23:23.466 DEBUG:teuthology.orchestra.run.smithi012:> ' 2024-05-09T09:23:23.595 INFO:teuthology.orchestra.run.smithi012.stdout:/var/run/ceph/ceph-client.admin.74789.asok 2024-05-09T09:23:23.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:23:23.607 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74789.asok 2024-05-09T09:23:23.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:23.607 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74789.asok status 2024-05-09T09:23:23.748 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:23:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "metadata": { 2024-05-09T09:23:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T09:23:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T09:23:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "entity_id": "admin", 2024-05-09T09:23:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": "smithi012", 2024-05-09T09:23:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-09T09:23:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "pid": "74789", 2024-05-09T09:23:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "root": "/" 2024-05-09T09:23:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:23:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_count": 0, 2024-05-09T09:23:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_pinned_count": 0, 2024-05-09T09:23:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 14772, 2024-05-09T09:23:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "inst": { 2024-05-09T09:23:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "name": { 2024-05-09T09:23:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "client", 2024-05-09T09:23:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "num": 14772 2024-05-09T09:23:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:23:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-05-09T09:23:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-05-09T09:23:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2024-05-09T09:23:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2857510741 2024-05-09T09:23:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:23:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:23:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-05-09T09:23:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-05-09T09:23:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2024-05-09T09:23:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2857510741 2024-05-09T09:23:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:23:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "inst_str": "client.14772 192.168.0.1:0/2857510741", 2024-05-09T09:23:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "addr_str": "192.168.0.1:0/2857510741", 2024-05-09T09:23:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "inode_count": 1, 2024-05-09T09:23:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_epoch": 17, 2024-05-09T09:23:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch": 89, 2024-05-09T09:23:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch_barrier": 0, 2024-05-09T09:23:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklisted": false, 2024-05-09T09:23:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs" 2024-05-09T09:23:23.753 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:23:23.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:23.757 DEBUG:teuthology.orchestra.run.smithi012:> stat --file-system '--printf=%T 2024-05-09T09:23:23.757 DEBUG:teuthology.orchestra.run.smithi012:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:23:23.774 INFO:teuthology.orchestra.run.smithi012.stdout:fuseblk 2024-05-09T09:23:23.774 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:23:23.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:23.774 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:23:23.842 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-09T09:23:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T09:23:23.961 INFO:teuthology.orchestra.run.smithi012.stdout:# file: . 2024-05-09T09:23:23.961 INFO:teuthology.orchestra.run.smithi012.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-09T09:23:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:23.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:23:23.971 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:23:23.971 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T09:23:23.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:23 smithi012 ceph-mon[30140]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 249 B/s wr, 0 op/s 2024-05-09T09:23:23.989 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:23:23.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:23 smithi170 ceph-mon[33632]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 249 B/s wr, 0 op/s 2024-05-09T09:23:24.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:23 smithi016 ceph-mon[33335]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 249 B/s wr, 0 op/s 2024-05-09T09:23:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.135+0000 7f462271d640 1 -- 172.21.15.12:0/588841970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f461c102310 msgr2=0x7f461c104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.135+0000 7f462271d640 1 --2- 172.21.15.12:0/588841970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f461c102310 0x7f461c104700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f460c009900 tx=0x7f460c02a410 comp rx=0 tx=0).stop 2024-05-09T09:23:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.135+0000 7f462271d640 1 -- 172.21.15.12:0/588841970 shutdown_connections 2024-05-09T09:23:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.135+0000 7f462271d640 1 --2- 172.21.15.12:0/588841970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f461c107570 0x7f461c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.135+0000 7f462271d640 1 --2- 172.21.15.12:0/588841970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f461c104c40 0x7f461c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.135+0000 7f462271d640 1 --2- 172.21.15.12:0/588841970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f461c102310 0x7f461c104700 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.135+0000 7f462271d640 1 -- 172.21.15.12:0/588841970 >> 172.21.15.12:0/588841970 conn(0x7f461c0fbd20 msgr2=0x7f461c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.135+0000 7f462271d640 1 -- 172.21.15.12:0/588841970 shutdown_connections 2024-05-09T09:23:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.135+0000 7f462271d640 1 -- 172.21.15.12:0/588841970 wait complete. 2024-05-09T09:23:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f462271d640 1 Processor -- start 2024-05-09T09:23:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f462271d640 1 -- start start 2024-05-09T09:23:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f462271d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f461c102310 0x7f461c101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f462271d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f461c104c40 0x7f461c1002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f462271d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f461c107570 0x7f461c100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f462271d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f461c10d540 con 0x7f461c104c40 2024-05-09T09:23:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f462271d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f461c10d3c0 con 0x7f461c102310 2024-05-09T09:23:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f462271d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f461c10d6c0 con 0x7f461c107570 2024-05-09T09:23:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f461b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f461c104c40 0x7f461c1002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f461b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f461c104c40 0x7f461c1002c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45318/0 (socket says 172.21.15.12:45318) 2024-05-09T09:23:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f461b7fe640 1 -- 172.21.15.12:0/1997538092 learned_addr learned my addr 172.21.15.12:0/1997538092 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f4620cba640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f461c107570 0x7f461c100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f461bfff640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f461c102310 0x7f461c101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:24.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f461b7fe640 1 -- 172.21.15.12:0/1997538092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f461c107570 msgr2=0x7f461c100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f461b7fe640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f461c107570 0x7f461c100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f461b7fe640 1 -- 172.21.15.12:0/1997538092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f461c102310 msgr2=0x7f461c101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f461b7fe640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f461c102310 0x7f461c101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.136+0000 7f461b7fe640 1 -- 172.21.15.12:0/1997538092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461c101090 con 0x7f461c104c40 2024-05-09T09:23:24.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.137+0000 7f461bfff640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f461c102310 0x7f461c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:23:24.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.137+0000 7f461b7fe640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f461c104c40 0x7f461c1002c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f461000c970 tx=0x7f461000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:24.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.137+0000 7f46197fa640 1 -- 172.21.15.12:0/1997538092 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610007bb0 con 0x7f461c104c40 2024-05-09T09:23:24.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.137+0000 7f46197fa640 1 -- 172.21.15.12:0/1997538092 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4610007d50 con 0x7f461c104c40 2024-05-09T09:23:24.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.137+0000 7f462271d640 1 -- 172.21.15.12:0/1997538092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461c1a1d30 con 0x7f461c104c40 2024-05-09T09:23:24.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.137+0000 7f46197fa640 1 -- 172.21.15.12:0/1997538092 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461000f450 con 0x7f461c104c40 2024-05-09T09:23:24.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.137+0000 7f462271d640 1 -- 172.21.15.12:0/1997538092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f461c1a21d0 con 0x7f461c104c40 2024-05-09T09:23:24.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.138+0000 7f462271d640 1 -- 172.21.15.12:0/1997538092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f461c0fd810 con 0x7f461c104c40 2024-05-09T09:23:24.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.139+0000 7f46197fa640 1 -- 172.21.15.12:0/1997538092 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4610004f50 con 0x7f461c104c40 2024-05-09T09:23:24.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.140+0000 7f46197fa640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46040a2b90 0x7f46040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:24.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.140+0000 7f46197fa640 1 -- 172.21.15.12:0/1997538092 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f46100cf360 con 0x7f461c104c40 2024-05-09T09:23:24.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.140+0000 7f461bfff640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46040a2b90 0x7f46040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:24.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.141+0000 7f461bfff640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46040a2b90 0x7f46040a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f460c02a930 tx=0x7f460c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:24.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.143+0000 7f46197fa640 1 -- 172.21.15.12:0/1997538092 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4610014030 con 0x7f461c104c40 2024-05-09T09:23:24.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.332+0000 7f462271d640 1 -- 172.21.15.12:0/1997538092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f461c02d340 con 0x7f461c104c40 2024-05-09T09:23:24.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.333+0000 7f46197fa640 1 -- 172.21.15.12:0/1997538092 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24105 (secure 0 0 0) 0x7f461009c080 con 0x7f461c104c40 2024-05-09T09:23:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:24.333 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:23:14.726332+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:23:24.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.335+0000 7f462271d640 1 -- 172.21.15.12:0/1997538092 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46040a2b90 msgr2=0x7f46040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.335+0000 7f462271d640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46040a2b90 0x7f46040a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f460c02a930 tx=0x7f460c000bc0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.336+0000 7f462271d640 1 -- 172.21.15.12:0/1997538092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f461c104c40 msgr2=0x7f461c1002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.336+0000 7f462271d640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f461c104c40 0x7f461c1002c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f461000c970 tx=0x7f461000ce30 comp rx=0 tx=0).stop 2024-05-09T09:23:24.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.336+0000 7f462271d640 1 -- 172.21.15.12:0/1997538092 shutdown_connections 2024-05-09T09:23:24.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.336+0000 7f462271d640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46040a2b90 0x7f46040a5050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.336+0000 7f462271d640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f461c107570 0x7f461c100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.336+0000 7f462271d640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f461c104c40 0x7f461c1002c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.336+0000 7f462271d640 1 --2- 172.21.15.12:0/1997538092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f461c102310 0x7f461c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.336+0000 7f462271d640 1 -- 172.21.15.12:0/1997538092 >> 172.21.15.12:0/1997538092 conn(0x7f461c0fbd20 msgr2=0x7f461c102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:24.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.336+0000 7f462271d640 1 -- 172.21.15.12:0/1997538092 shutdown_connections 2024-05-09T09:23:24.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.336+0000 7f462271d640 1 -- 172.21.15.12:0/1997538092 wait complete. 2024-05-09T09:23:24.346 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:23:08.167107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:23:09.065670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:23:12.773640+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:23:24.347 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:23:24.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.456+0000 7fc712f7e640 1 -- 172.21.15.12:0/965736819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c102320 msgr2=0x7fc70c108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.456+0000 7fc712f7e640 1 --2- 172.21.15.12:0/965736819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c102320 0x7fc70c108bb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc704009900 tx=0x7fc70402a410 comp rx=0 tx=0).stop 2024-05-09T09:23:24.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.456+0000 7fc712f7e640 1 -- 172.21.15.12:0/965736819 shutdown_connections 2024-05-09T09:23:24.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.456+0000 7fc712f7e640 1 --2- 172.21.15.12:0/965736819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c102320 0x7fc70c108bb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.456+0000 7fc712f7e640 1 --2- 172.21.15.12:0/965736819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc70c101960 0x7fc70c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.456+0000 7fc712f7e640 1 --2- 172.21.15.12:0/965736819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70c100770 0x7fc70c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.456+0000 7fc712f7e640 1 -- 172.21.15.12:0/965736819 >> 172.21.15.12:0/965736819 conn(0x7fc70c0fbce0 msgr2=0x7fc70c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.456+0000 7fc712f7e640 1 -- 172.21.15.12:0/965736819 shutdown_connections 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.456+0000 7fc712f7e640 1 -- 172.21.15.12:0/965736819 wait complete. 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.456+0000 7fc712f7e640 1 Processor -- start 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.456+0000 7fc712f7e640 1 -- start start 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.456+0000 7fc712f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70c100770 0x7fc70c199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.456+0000 7fc712f7e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc70c101960 0x7fc70c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc712f7e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c102320 0x7fc70c1a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc712f7e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c10c3d0 con 0x7fc70c102320 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc712f7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c10c250 con 0x7fc70c100770 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc712f7e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c10c550 con 0x7fc70c101960 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc71151b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c102320 0x7fc70c1a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:24.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc71151b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c102320 0x7fc70c1a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45334/0 (socket says 172.21.15.12:45334) 2024-05-09T09:23:24.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc71151b640 1 -- 172.21.15.12:0/4266758456 learned_addr learned my addr 172.21.15.12:0/4266758456 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:24.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc710d1a640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70c100770 0x7fc70c199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:24.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc70bfff640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc70c101960 0x7fc70c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:24.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc71151b640 1 -- 172.21.15.12:0/4266758456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc70c101960 msgr2=0x7fc70c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc71151b640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc70c101960 0x7fc70c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc71151b640 1 -- 172.21.15.12:0/4266758456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70c100770 msgr2=0x7fc70c199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc71151b640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70c100770 0x7fc70c199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc71151b640 1 -- 172.21.15.12:0/4266758456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c1a1910 con 0x7fc70c102320 2024-05-09T09:23:24.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc71151b640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c102320 0x7fc70c1a1270 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc704006010 tx=0x7fc7040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:24.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc709ffb640 1 -- 172.21.15.12:0/4266758456 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70403f030 con 0x7fc70c102320 2024-05-09T09:23:24.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc709ffb640 1 -- 172.21.15.12:0/4266758456 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc704003ac0 con 0x7fc70c102320 2024-05-09T09:23:24.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc709ffb640 1 -- 172.21.15.12:0/4266758456 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704035070 con 0x7fc70c102320 2024-05-09T09:23:24.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.457+0000 7fc712f7e640 1 -- 172.21.15.12:0/4266758456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c1a1ba0 con 0x7fc70c102320 2024-05-09T09:23:24.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.458+0000 7fc712f7e640 1 -- 172.21.15.12:0/4266758456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c1a2040 con 0x7fc70c102320 2024-05-09T09:23:24.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.459+0000 7fc712f7e640 1 -- 172.21.15.12:0/4266758456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c1a2210 con 0x7fc70c102320 2024-05-09T09:23:24.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.460+0000 7fc709ffb640 1 -- 172.21.15.12:0/4266758456 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc704042030 con 0x7fc70c102320 2024-05-09T09:23:24.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.461+0000 7fc709ffb640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc6f80a2b90 0x7fc6f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:24.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.461+0000 7fc709ffb640 1 -- 172.21.15.12:0/4266758456 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc7040ef850 con 0x7fc70c102320 2024-05-09T09:23:24.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.461+0000 7fc710d1a640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc6f80a2b90 0x7fc6f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:24.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.461+0000 7fc710d1a640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc6f80a2b90 0x7fc6f80a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc70c19b1d0 tx=0x7fc6f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:24.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.465+0000 7fc709ffb640 1 -- 172.21.15.12:0/4266758456 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc7040bc570 con 0x7fc70c102320 2024-05-09T09:23:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.655+0000 7fc712f7e640 1 -- 172.21.15.12:0/4266758456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc70c1a25a0 con 0x7fc70c102320 2024-05-09T09:23:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.655+0000 7fc709ffb640 1 -- 172.21.15.12:0/4266758456 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fc7040bbe70 con 0x7fc70c102320 2024-05-09T09:23:24.655 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:23:24.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.658+0000 7fc712f7e640 1 -- 172.21.15.12:0/4266758456 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc6f80a2b90 msgr2=0x7fc6f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.658+0000 7fc712f7e640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc6f80a2b90 0x7fc6f80a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc70c19b1d0 tx=0x7fc6f4005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:24.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.658+0000 7fc712f7e640 1 -- 172.21.15.12:0/4266758456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c102320 msgr2=0x7fc70c1a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.658+0000 7fc712f7e640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c102320 0x7fc70c1a1270 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc704006010 tx=0x7fc7040042f0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.658+0000 7fc712f7e640 1 -- 172.21.15.12:0/4266758456 shutdown_connections 2024-05-09T09:23:24.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.658+0000 7fc712f7e640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc6f80a2b90 0x7fc6f80a5050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.658+0000 7fc712f7e640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c102320 0x7fc70c1a1270 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.658+0000 7fc712f7e640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc70c101960 0x7fc70c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.658+0000 7fc712f7e640 1 --2- 172.21.15.12:0/4266758456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70c100770 0x7fc70c199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.658+0000 7fc712f7e640 1 -- 172.21.15.12:0/4266758456 >> 172.21.15.12:0/4266758456 conn(0x7fc70c0fbce0 msgr2=0x7fc70c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:24.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.658+0000 7fc712f7e640 1 -- 172.21.15.12:0/4266758456 shutdown_connections 2024-05-09T09:23:24.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.658+0000 7fc712f7e640 1 -- 172.21.15.12:0/4266758456 wait complete. 2024-05-09T09:23:24.668 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:23:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.777+0000 7f5136753640 1 -- 172.21.15.12:0/1829240504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5130102310 msgr2=0x7f5130104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.777+0000 7f5136753640 1 --2- 172.21.15.12:0/1829240504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5130102310 0x7f5130104700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5120009900 tx=0x7f512002a410 comp rx=0 tx=0).stop 2024-05-09T09:23:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.777+0000 7f5136753640 1 -- 172.21.15.12:0/1829240504 shutdown_connections 2024-05-09T09:23:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.777+0000 7f5136753640 1 --2- 172.21.15.12:0/1829240504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5130107570 0x7f51301099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.777+0000 7f5136753640 1 --2- 172.21.15.12:0/1829240504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130104c40 0x7f5130107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.777+0000 7f5136753640 1 --2- 172.21.15.12:0/1829240504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5130102310 0x7f5130104700 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.777+0000 7f5136753640 1 -- 172.21.15.12:0/1829240504 >> 172.21.15.12:0/1829240504 conn(0x7f51300fbd20 msgr2=0x7f51300fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.777+0000 7f5136753640 1 -- 172.21.15.12:0/1829240504 shutdown_connections 2024-05-09T09:23:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.777+0000 7f5136753640 1 -- 172.21.15.12:0/1829240504 wait complete. 2024-05-09T09:23:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.777+0000 7f5136753640 1 Processor -- start 2024-05-09T09:23:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f5136753640 1 -- start start 2024-05-09T09:23:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f5136753640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5130102310 0x7f5130101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f5136753640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5130104c40 0x7f51301002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f5136753640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130107570 0x7f5130100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f5136753640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f513010d570 con 0x7f5130102310 2024-05-09T09:23:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f5136753640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f513010d3f0 con 0x7f5130104c40 2024-05-09T09:23:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f5136753640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f513010d6f0 con 0x7f5130107570 2024-05-09T09:23:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f512ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5130102310 0x7f5130101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f512ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5130102310 0x7f5130101c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45352/0 (socket says 172.21.15.12:45352) 2024-05-09T09:23:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f512ffff640 1 -- 172.21.15.12:0/2055533542 learned_addr learned my addr 172.21.15.12:0/2055533542 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f5134cf0640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130107570 0x7f5130100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f512f7fe640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5130104c40 0x7f51301002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f512ffff640 1 -- 172.21.15.12:0/2055533542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130107570 msgr2=0x7f5130100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f512ffff640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130107570 0x7f5130100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f512ffff640 1 -- 172.21.15.12:0/2055533542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5130104c40 msgr2=0x7f51301002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f512ffff640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5130104c40 0x7f51301002c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f512ffff640 1 -- 172.21.15.12:0/2055533542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51301010c0 con 0x7f5130102310 2024-05-09T09:23:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.778+0000 7f5134cf0640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130107570 0x7f5130100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:23:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.779+0000 7f512ffff640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5130102310 0x7f5130101c30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5120009230 tx=0x7f5120004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.779+0000 7f512d7fa640 1 -- 172.21.15.12:0/2055533542 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512003f030 con 0x7f5130102310 2024-05-09T09:23:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.779+0000 7f512d7fa640 1 -- 172.21.15.12:0/2055533542 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5120003ac0 con 0x7f5130102310 2024-05-09T09:23:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.779+0000 7f512d7fa640 1 -- 172.21.15.12:0/2055533542 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120035070 con 0x7f5130102310 2024-05-09T09:23:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.779+0000 7f5136753640 1 -- 172.21.15.12:0/2055533542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51301a1d40 con 0x7f5130102310 2024-05-09T09:23:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.779+0000 7f5136753640 1 -- 172.21.15.12:0/2055533542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51301a21b0 con 0x7f5130102310 2024-05-09T09:23:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.780+0000 7f5136753640 1 -- 172.21.15.12:0/2055533542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51300fd810 con 0x7f5130102310 2024-05-09T09:23:24.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.781+0000 7f512d7fa640 1 -- 172.21.15.12:0/2055533542 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5120042030 con 0x7f5130102310 2024-05-09T09:23:24.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.782+0000 7f512d7fa640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f51180a2b90 0x7f51180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:24.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.782+0000 7f512f7fe640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f51180a2b90 0x7f51180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:24.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.782+0000 7f512d7fa640 1 -- 172.21.15.12:0/2055533542 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f51200efae0 con 0x7f5130102310 2024-05-09T09:23:24.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.782+0000 7f512f7fe640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f51180a2b90 0x7f51180a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f51301019a0 tx=0x7f5124008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:24.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.786+0000 7f512d7fa640 1 -- 172.21.15.12:0/2055533542 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f51200bc800 con 0x7f5130102310 2024-05-09T09:23:24.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.971+0000 7f5136753640 1 -- 172.21.15.12:0/2055533542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f51300624d0 con 0x7f5130102310 2024-05-09T09:23:24.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.971+0000 7f512d7fa640 1 -- 172.21.15.12:0/2055533542 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f5120003c70 con 0x7f5130102310 2024-05-09T09:23:24.971 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:23:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.973+0000 7f5136753640 1 -- 172.21.15.12:0/2055533542 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f51180a2b90 msgr2=0x7f51180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.973+0000 7f5136753640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f51180a2b90 0x7f51180a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f51301019a0 tx=0x7f5124008040 comp rx=0 tx=0).stop 2024-05-09T09:23:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.973+0000 7f5136753640 1 -- 172.21.15.12:0/2055533542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5130102310 msgr2=0x7f5130101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.973+0000 7f5136753640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5130102310 0x7f5130101c30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5120009230 tx=0x7f5120004360 comp rx=0 tx=0).stop 2024-05-09T09:23:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.973+0000 7f5136753640 1 -- 172.21.15.12:0/2055533542 shutdown_connections 2024-05-09T09:23:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.973+0000 7f5136753640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f51180a2b90 0x7f51180a5050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.973+0000 7f5136753640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130107570 0x7f5130100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.973+0000 7f5136753640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5130104c40 0x7f51301002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.973+0000 7f5136753640 1 --2- 172.21.15.12:0/2055533542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5130102310 0x7f5130101c30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.973+0000 7f5136753640 1 -- 172.21.15.12:0/2055533542 >> 172.21.15.12:0/2055533542 conn(0x7f51300fbd20 msgr2=0x7f5130102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.973+0000 7f5136753640 1 -- 172.21.15.12:0/2055533542 shutdown_connections 2024-05-09T09:23:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:24.973+0000 7f5136753640 1 -- 172.21.15.12:0/2055533542 wait complete. 2024-05-09T09:23:24.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:24 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1997538092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:24.984 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:23:24.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:24 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1997538092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:25.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:24 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1997538092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.093+0000 7f37d3838640 1 -- 172.21.15.12:0/2908378041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37cc102a70 msgr2=0x7f37cc102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.093+0000 7f37d3838640 1 --2- 172.21.15.12:0/2908378041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37cc102a70 0x7f37cc102ef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009900 tx=0x7f37bc02a410 comp rx=0 tx=0).stop 2024-05-09T09:23:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.093+0000 7f37d3838640 1 -- 172.21.15.12:0/2908378041 shutdown_connections 2024-05-09T09:23:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.093+0000 7f37d3838640 1 --2- 172.21.15.12:0/2908378041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37cc103430 0x7f37cc109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.093+0000 7f37d3838640 1 --2- 172.21.15.12:0/2908378041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37cc102a70 0x7f37cc102ef0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.093+0000 7f37d3838640 1 --2- 172.21.15.12:0/2908378041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37cc101880 0x7f37cc101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.093+0000 7f37d3838640 1 -- 172.21.15.12:0/2908378041 >> 172.21.15.12:0/2908378041 conn(0x7f37cc0fce10 msgr2=0x7f37cc0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.093+0000 7f37d3838640 1 -- 172.21.15.12:0/2908378041 shutdown_connections 2024-05-09T09:23:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.093+0000 7f37d3838640 1 -- 172.21.15.12:0/2908378041 wait complete. 2024-05-09T09:23:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.093+0000 7f37d3838640 1 Processor -- start 2024-05-09T09:23:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.093+0000 7f37d3838640 1 -- start start 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d3838640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37cc101880 0x7f37cc197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d3838640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37cc102a70 0x7f37cc198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d3838640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37cc103430 0x7f37cc19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d3838640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc10d4e0 con 0x7f37cc103430 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d3838640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc10d360 con 0x7f37cc102a70 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d3838640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc10d660 con 0x7f37cc101880 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d1dd5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37cc103430 0x7f37cc19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d1dd5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37cc103430 0x7f37cc19f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45368/0 (socket says 172.21.15.12:45368) 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d1dd5640 1 -- 172.21.15.12:0/85501537 learned_addr learned my addr 172.21.15.12:0/85501537 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d15d4640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37cc101880 0x7f37cc197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d0dd3640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37cc102a70 0x7f37cc198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d1dd5640 1 -- 172.21.15.12:0/85501537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37cc101880 msgr2=0x7f37cc197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d1dd5640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37cc101880 0x7f37cc197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d1dd5640 1 -- 172.21.15.12:0/85501537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37cc102a70 msgr2=0x7f37cc198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d1dd5640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37cc102a70 0x7f37cc198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d1dd5640 1 -- 172.21.15.12:0/85501537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc19f720 con 0x7f37cc103430 2024-05-09T09:23:25.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d1dd5640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37cc103430 0x7f37cc19f0b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f37c400cce0 tx=0x7f37c400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:25.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37c27fc640 1 -- 172.21.15.12:0/85501537 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c400f450 con 0x7f37cc103430 2024-05-09T09:23:25.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37d3838640 1 -- 172.21.15.12:0/85501537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc19fa10 con 0x7f37cc103430 2024-05-09T09:23:25.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37c27fc640 1 -- 172.21.15.12:0/85501537 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f37c400fac0 con 0x7f37cc103430 2024-05-09T09:23:25.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.094+0000 7f37c27fc640 1 -- 172.21.15.12:0/85501537 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c4018be0 con 0x7f37cc103430 2024-05-09T09:23:25.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.095+0000 7f37d3838640 1 -- 172.21.15.12:0/85501537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc19ff10 con 0x7f37cc103430 2024-05-09T09:23:25.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.096+0000 7f37d3838640 1 -- 172.21.15.12:0/85501537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37cc108b40 con 0x7f37cc103430 2024-05-09T09:23:25.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.097+0000 7f37c27fc640 1 -- 172.21.15.12:0/85501537 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f37c400fc70 con 0x7f37cc103430 2024-05-09T09:23:25.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.097+0000 7f37c27fc640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37b80a2b90 0x7f37b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:25.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.098+0000 7f37d15d4640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37b80a2b90 0x7f37b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:25.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.098+0000 7f37c27fc640 1 -- 172.21.15.12:0/85501537 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f37c40cf900 con 0x7f37cc103430 2024-05-09T09:23:25.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.098+0000 7f37d15d4640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37b80a2b90 0x7f37b80a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f37c8005fd0 tx=0x7f37c800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:25.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.102+0000 7f37c27fc640 1 -- 172.21.15.12:0/85501537 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f37c4014030 con 0x7f37cc103430 2024-05-09T09:23:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.291+0000 7f37d3838640 1 -- 172.21.15.12:0/85501537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f37cc062430 con 0x7f37cc103430 2024-05-09T09:23:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.291+0000 7f37c27fc640 1 -- 172.21.15.12:0/85501537 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f37c409c620 con 0x7f37cc103430 2024-05-09T09:23:25.291 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T09:23:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.293+0000 7f37d3838640 1 -- 172.21.15.12:0/85501537 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37b80a2b90 msgr2=0x7f37b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.293+0000 7f37d3838640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37b80a2b90 0x7f37b80a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f37c8005fd0 tx=0x7f37c800b410 comp rx=0 tx=0).stop 2024-05-09T09:23:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.293+0000 7f37d3838640 1 -- 172.21.15.12:0/85501537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37cc103430 msgr2=0x7f37cc19f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.293+0000 7f37d3838640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37cc103430 0x7f37cc19f0b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f37c400cce0 tx=0x7f37c400a620 comp rx=0 tx=0).stop 2024-05-09T09:23:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.293+0000 7f37d3838640 1 -- 172.21.15.12:0/85501537 shutdown_connections 2024-05-09T09:23:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.293+0000 7f37d3838640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37b80a2b90 0x7f37b80a5050 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.293+0000 7f37d3838640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37cc103430 0x7f37cc19f0b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.293+0000 7f37d3838640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37cc102a70 0x7f37cc198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.293+0000 7f37d3838640 1 --2- 172.21.15.12:0/85501537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37cc101880 0x7f37cc197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.293+0000 7f37d3838640 1 -- 172.21.15.12:0/85501537 >> 172.21.15.12:0/85501537 conn(0x7f37cc0fce10 msgr2=0x7f37cc105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.294+0000 7f37d3838640 1 -- 172.21.15.12:0/85501537 shutdown_connections 2024-05-09T09:23:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.294+0000 7f37d3838640 1 -- 172.21.15.12:0/85501537 wait complete. 2024-05-09T09:23:25.305 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:23:25.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.416+0000 7fc030c58640 1 -- 172.21.15.12:0/3534960089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c102320 msgr2=0x7fc02c108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.416+0000 7fc030c58640 1 --2- 172.21.15.12:0/3534960089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c102320 0x7fc02c108bb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc024009ca0 tx=0x7fc02402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.416+0000 7fc030c58640 1 -- 172.21.15.12:0/3534960089 shutdown_connections 2024-05-09T09:23:25.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.416+0000 7fc030c58640 1 --2- 172.21.15.12:0/3534960089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c102320 0x7fc02c108bb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.416+0000 7fc030c58640 1 --2- 172.21.15.12:0/3534960089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c101960 0x7fc02c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.416+0000 7fc030c58640 1 --2- 172.21.15.12:0/3534960089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c100770 0x7fc02c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.416+0000 7fc030c58640 1 -- 172.21.15.12:0/3534960089 >> 172.21.15.12:0/3534960089 conn(0x7fc02c0fbce0 msgr2=0x7fc02c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:25.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.416+0000 7fc030c58640 1 -- 172.21.15.12:0/3534960089 shutdown_connections 2024-05-09T09:23:25.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.416+0000 7fc030c58640 1 -- 172.21.15.12:0/3534960089 wait complete. 2024-05-09T09:23:25.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.417+0000 7fc030c58640 1 Processor -- start 2024-05-09T09:23:25.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.417+0000 7fc030c58640 1 -- start start 2024-05-09T09:23:25.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.417+0000 7fc030c58640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c100770 0x7fc02c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:25.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.417+0000 7fc030c58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c101960 0x7fc02c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:25.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.417+0000 7fc030c58640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c102320 0x7fc02c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:25.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.417+0000 7fc030c58640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c10d4c0 con 0x7fc02c102320 2024-05-09T09:23:25.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.417+0000 7fc030c58640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c10d340 con 0x7fc02c101960 2024-05-09T09:23:25.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.417+0000 7fc030c58640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c10d640 con 0x7fc02c100770 2024-05-09T09:23:25.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.417+0000 7fc02ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c102320 0x7fc02c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.417+0000 7fc02ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c102320 0x7fc02c19f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45388/0 (socket says 172.21.15.12:45388) 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.417+0000 7fc02ad9d640 1 -- 172.21.15.12:0/1516533791 learned_addr learned my addr 172.21.15.12:0/1516533791 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.418+0000 7fc029d9b640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c101960 0x7fc02c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.418+0000 7fc02a59c640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c100770 0x7fc02c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.418+0000 7fc02ad9d640 1 -- 172.21.15.12:0/1516533791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c100770 msgr2=0x7fc02c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.418+0000 7fc02ad9d640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c100770 0x7fc02c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.418+0000 7fc02ad9d640 1 -- 172.21.15.12:0/1516533791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c101960 msgr2=0x7fc02c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.418+0000 7fc02ad9d640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c101960 0x7fc02c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.418+0000 7fc02ad9d640 1 -- 172.21.15.12:0/1516533791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02c19f780 con 0x7fc02c102320 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.418+0000 7fc02ad9d640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c102320 0x7fc02c19f110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc024009430 tx=0x7fc02402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.418+0000 7fc01b7fe640 1 -- 172.21.15.12:0/1516533791 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024003bf0 con 0x7fc02c102320 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.418+0000 7fc030c58640 1 -- 172.21.15.12:0/1516533791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02c19fa10 con 0x7fc02c102320 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.418+0000 7fc01b7fe640 1 -- 172.21.15.12:0/1516533791 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc024003d90 con 0x7fc02c102320 2024-05-09T09:23:25.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.418+0000 7fc030c58640 1 -- 172.21.15.12:0/1516533791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02c19ffb0 con 0x7fc02c102320 2024-05-09T09:23:25.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.418+0000 7fc01b7fe640 1 -- 172.21.15.12:0/1516533791 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02402fc90 con 0x7fc02c102320 2024-05-09T09:23:25.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.419+0000 7fc030c58640 1 -- 172.21.15.12:0/1516533791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc02c107b10 con 0x7fc02c102320 2024-05-09T09:23:25.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.420+0000 7fc01b7fe640 1 -- 172.21.15.12:0/1516533791 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc024004040 con 0x7fc02c102320 2024-05-09T09:23:25.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.421+0000 7fc01b7fe640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc0140a2b90 0x7fc0140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:25.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.421+0000 7fc02a59c640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc0140a2b90 0x7fc0140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:25.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.421+0000 7fc01b7fe640 1 -- 172.21.15.12:0/1516533791 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc0240f0a30 con 0x7fc02c102320 2024-05-09T09:23:25.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.422+0000 7fc02a59c640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc0140a2b90 0x7fc0140a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc01c005d00 tx=0x7fc01c005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:25.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.426+0000 7fc01b7fe640 1 -- 172.21.15.12:0/1516533791 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc0240bd780 con 0x7fc02c102320 2024-05-09T09:23:25.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.614+0000 7fc030c58640 1 -- 172.21.15.12:0/1516533791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc02c02d1c0 con 0x7fc02c102320 2024-05-09T09:23:25.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.615+0000 7fc01b7fe640 1 -- 172.21.15.12:0/1516533791 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7fc0240bd080 con 0x7fc02c102320 2024-05-09T09:23:25.615 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:23:25.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.617+0000 7fc030c58640 1 -- 172.21.15.12:0/1516533791 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc0140a2b90 msgr2=0x7fc0140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.617+0000 7fc030c58640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc0140a2b90 0x7fc0140a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc01c005d00 tx=0x7fc01c005c50 comp rx=0 tx=0).stop 2024-05-09T09:23:25.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.617+0000 7fc030c58640 1 -- 172.21.15.12:0/1516533791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c102320 msgr2=0x7fc02c19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.617+0000 7fc030c58640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c102320 0x7fc02c19f110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc024009430 tx=0x7fc02402f380 comp rx=0 tx=0).stop 2024-05-09T09:23:25.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.617+0000 7fc030c58640 1 -- 172.21.15.12:0/1516533791 shutdown_connections 2024-05-09T09:23:25.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.617+0000 7fc030c58640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc0140a2b90 0x7fc0140a5050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.617+0000 7fc030c58640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c102320 0x7fc02c19f110 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.617+0000 7fc030c58640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c101960 0x7fc02c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.617+0000 7fc030c58640 1 --2- 172.21.15.12:0/1516533791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c100770 0x7fc02c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.617+0000 7fc030c58640 1 -- 172.21.15.12:0/1516533791 >> 172.21.15.12:0/1516533791 conn(0x7fc02c0fbce0 msgr2=0x7fc02c104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:25.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.618+0000 7fc030c58640 1 -- 172.21.15.12:0/1516533791 shutdown_connections 2024-05-09T09:23:25.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.618+0000 7fc030c58640 1 -- 172.21.15.12:0/1516533791 wait complete. 2024-05-09T09:23:25.630 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T09:23:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.739+0000 7efc573a9640 1 -- 172.21.15.12:0/2176910530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc50100770 msgr2=0x7efc50100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.739+0000 7efc573a9640 1 --2- 172.21.15.12:0/2176910530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc50100770 0x7efc50100b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc44009900 tx=0x7efc4402a410 comp rx=0 tx=0).stop 2024-05-09T09:23:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.739+0000 7efc573a9640 1 -- 172.21.15.12:0/2176910530 shutdown_connections 2024-05-09T09:23:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.739+0000 7efc573a9640 1 --2- 172.21.15.12:0/2176910530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc50102320 0x7efc50108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.739+0000 7efc573a9640 1 --2- 172.21.15.12:0/2176910530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc50101960 0x7efc50101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.739+0000 7efc573a9640 1 --2- 172.21.15.12:0/2176910530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc50100770 0x7efc50100b90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.739+0000 7efc573a9640 1 -- 172.21.15.12:0/2176910530 >> 172.21.15.12:0/2176910530 conn(0x7efc500fbce0 msgr2=0x7efc500fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.739+0000 7efc573a9640 1 -- 172.21.15.12:0/2176910530 shutdown_connections 2024-05-09T09:23:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.739+0000 7efc573a9640 1 -- 172.21.15.12:0/2176910530 wait complete. 2024-05-09T09:23:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc573a9640 1 Processor -- start 2024-05-09T09:23:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc573a9640 1 -- start start 2024-05-09T09:23:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc573a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc50100770 0x7efc50197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc573a9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc50101960 0x7efc50198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc573a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc50102320 0x7efc5019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc573a9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efc5010c3d0 con 0x7efc50101960 2024-05-09T09:23:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc573a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc5010c250 con 0x7efc50100770 2024-05-09T09:23:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc573a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efc5010c550 con 0x7efc50102320 2024-05-09T09:23:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc54944640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc50101960 0x7efc50198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc54944640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc50101960 0x7efc50198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45404/0 (socket says 172.21.15.12:45404) 2024-05-09T09:23:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc54944640 1 -- 172.21.15.12:0/212068319 learned_addr learned my addr 172.21.15.12:0/212068319 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc55946640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc50102320 0x7efc5019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc55145640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc50100770 0x7efc50197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc54944640 1 -- 172.21.15.12:0/212068319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc50102320 msgr2=0x7efc5019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc54944640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc50102320 0x7efc5019f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc54944640 1 -- 172.21.15.12:0/212068319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc50100770 msgr2=0x7efc50197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc54944640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc50100770 0x7efc50197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc54944640 1 -- 172.21.15.12:0/212068319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5019f770 con 0x7efc50101960 2024-05-09T09:23:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.740+0000 7efc54944640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc50101960 0x7efc50198090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00c920 tx=0x7efc4c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.741+0000 7efc427fc640 1 -- 172.21.15.12:0/212068319 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c007c70 con 0x7efc50101960 2024-05-09T09:23:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.741+0000 7efc427fc640 1 -- 172.21.15.12:0/212068319 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efc4c007e10 con 0x7efc50101960 2024-05-09T09:23:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.741+0000 7efc573a9640 1 -- 172.21.15.12:0/212068319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5019fa60 con 0x7efc50101960 2024-05-09T09:23:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.741+0000 7efc427fc640 1 -- 172.21.15.12:0/212068319 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c018850 con 0x7efc50101960 2024-05-09T09:23:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.741+0000 7efc573a9640 1 -- 172.21.15.12:0/212068319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5019ffb0 con 0x7efc50101960 2024-05-09T09:23:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.742+0000 7efc573a9640 1 -- 172.21.15.12:0/212068319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc501a0240 con 0x7efc50101960 2024-05-09T09:23:25.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.743+0000 7efc427fc640 1 -- 172.21.15.12:0/212068319 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efc4c018a30 con 0x7efc50101960 2024-05-09T09:23:25.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.743+0000 7efc427fc640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efc3c0a2b90 0x7efc3c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:25.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.744+0000 7efc427fc640 1 -- 172.21.15.12:0/212068319 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7efc4c0cf500 con 0x7efc50101960 2024-05-09T09:23:25.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.744+0000 7efc55145640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efc3c0a2b90 0x7efc3c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:25.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.744+0000 7efc55145640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efc3c0a2b90 0x7efc3c0a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efc44005c40 tx=0x7efc44000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.748+0000 7efc427fc640 1 -- 172.21.15.12:0/212068319 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efc4c014030 con 0x7efc50101960 2024-05-09T09:23:25.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.950+0000 7efc573a9640 1 -- 172.21.15.12:0/212068319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efc501a0410 con 0x7efc50101960 2024-05-09T09:23:25.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.951+0000 7efc427fc640 1 -- 172.21.15.12:0/212068319 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+591 (secure 0 0 0) 0x7efc4c01f300 con 0x7efc50101960 2024-05-09T09:23:25.951 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1454579683 2024-05-10T09:16:04.466876+0000 2024-05-09T09:23:25.951 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/2168557623 2024-05-10T09:16:04.466876+0000 2024-05-09T09:23:25.951 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/2168557623 2024-05-10T09:16:04.466876+0000 2024-05-09T09:23:25.951 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/545109421 2024-05-10T09:15:36.182637+0000 2024-05-09T09:23:25.951 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1153421462 2024-05-10T09:16:04.466876+0000 2024-05-09T09:23:25.951 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1731765200 2024-05-10T09:15:36.182637+0000 2024-05-09T09:23:25.951 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2508497448 2024-05-10T09:15:36.182637+0000 2024-05-09T09:23:25.951 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1876427478 2024-05-10T09:15:36.182637+0000 2024-05-09T09:23:25.951 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3479121957 2024-05-10T09:16:04.466876+0000 2024-05-09T09:23:25.951 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1876427478 2024-05-10T09:15:36.182637+0000 2024-05-09T09:23:25.952 INFO:teuthology.orchestra.run.smithi012.stderr:listed 10 entries 2024-05-09T09:23:25.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.953+0000 7efc573a9640 1 -- 172.21.15.12:0/212068319 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efc3c0a2b90 msgr2=0x7efc3c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.953+0000 7efc573a9640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efc3c0a2b90 0x7efc3c0a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efc44005c40 tx=0x7efc44000bc0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.953+0000 7efc573a9640 1 -- 172.21.15.12:0/212068319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc50101960 msgr2=0x7efc50198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:25.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.953+0000 7efc573a9640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc50101960 0x7efc50198090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00c920 tx=0x7efc4c00cde0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.953+0000 7efc573a9640 1 -- 172.21.15.12:0/212068319 shutdown_connections 2024-05-09T09:23:25.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.953+0000 7efc573a9640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efc3c0a2b90 0x7efc3c0a5050 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.953+0000 7efc573a9640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc50102320 0x7efc5019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.953+0000 7efc573a9640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc50101960 0x7efc50198090 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.953+0000 7efc573a9640 1 --2- 172.21.15.12:0/212068319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc50100770 0x7efc50197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:25.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.953+0000 7efc573a9640 1 -- 172.21.15.12:0/212068319 >> 172.21.15.12:0/212068319 conn(0x7efc500fbce0 msgr2=0x7efc500689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:25.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.953+0000 7efc573a9640 1 -- 172.21.15.12:0/212068319 shutdown_connections 2024-05-09T09:23:25.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:25.953+0000 7efc573a9640 1 -- 172.21.15.12:0/212068319 wait complete. 2024-05-09T09:23:25.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:25 smithi012 ceph-mon[30140]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-05-09T09:23:25.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:25 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4266758456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:23:25.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:25 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2055533542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:23:25.963 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:25 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/85501537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:23:25.964 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:23:25.964 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T09:23:25.982 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:23:25.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:25 smithi170 ceph-mon[33632]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-05-09T09:23:25.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:25 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/4266758456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:23:25.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:25 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2055533542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:23:25.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:25 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/85501537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:23:26.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:25 smithi016 ceph-mon[33335]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-05-09T09:23:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:25 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/4266758456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:23:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:25 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2055533542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:23:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:25 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/85501537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:23:26.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.128+0000 7ff72baee640 1 -- 172.21.15.12:0/2314732436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff724100770 msgr2=0x7ff724100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.128+0000 7ff72baee640 1 --2- 172.21.15.12:0/2314732436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff724100770 0x7ff724100b90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff720009900 tx=0x7ff72002a3f0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.128+0000 7ff72baee640 1 -- 172.21.15.12:0/2314732436 shutdown_connections 2024-05-09T09:23:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.128+0000 7ff72baee640 1 --2- 172.21.15.12:0/2314732436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff724102320 0x7ff724108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.128+0000 7ff72baee640 1 --2- 172.21.15.12:0/2314732436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff724101960 0x7ff724101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.128+0000 7ff72baee640 1 --2- 172.21.15.12:0/2314732436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff724100770 0x7ff724100b90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.128+0000 7ff72baee640 1 -- 172.21.15.12:0/2314732436 >> 172.21.15.12:0/2314732436 conn(0x7ff7240fbce0 msgr2=0x7ff7240fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.128+0000 7ff72baee640 1 -- 172.21.15.12:0/2314732436 shutdown_connections 2024-05-09T09:23:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.128+0000 7ff72baee640 1 -- 172.21.15.12:0/2314732436 wait complete. 2024-05-09T09:23:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.128+0000 7ff72baee640 1 Processor -- start 2024-05-09T09:23:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.128+0000 7ff72baee640 1 -- start start 2024-05-09T09:23:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72baee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff724100770 0x7ff724197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72baee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff724101960 0x7ff724198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72baee640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff724102320 0x7ff72419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72baee640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff72410c3d0 con 0x7ff724102320 2024-05-09T09:23:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72baee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff72410c250 con 0x7ff724100770 2024-05-09T09:23:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72baee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff72410c550 con 0x7ff724101960 2024-05-09T09:23:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72a08b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff724102320 0x7ff72419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72a08b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff724102320 0x7ff72419f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45424/0 (socket says 172.21.15.12:45424) 2024-05-09T09:23:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72a08b640 1 -- 172.21.15.12:0/1214277878 learned_addr learned my addr 172.21.15.12:0/1214277878 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff729089640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff724101960 0x7ff724198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:26.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72988a640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff724100770 0x7ff724197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:26.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72a08b640 1 -- 172.21.15.12:0/1214277878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff724101960 msgr2=0x7ff724198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72a08b640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff724101960 0x7ff724198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72a08b640 1 -- 172.21.15.12:0/1214277878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff724100770 msgr2=0x7ff724197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72a08b640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff724100770 0x7ff724197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72a08b640 1 -- 172.21.15.12:0/1214277878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72419f800 con 0x7ff724102320 2024-05-09T09:23:26.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff729089640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff724101960 0x7ff724198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:26.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff72a08b640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff724102320 0x7ff72419f190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00eb10 tx=0x7ff71c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:26.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.129+0000 7ff716ffd640 1 -- 172.21.15.12:0/1214277878 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c004b20 con 0x7ff724102320 2024-05-09T09:23:26.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.130+0000 7ff72baee640 1 -- 172.21.15.12:0/1214277878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72419faf0 con 0x7ff724102320 2024-05-09T09:23:26.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.130+0000 7ff716ffd640 1 -- 172.21.15.12:0/1214277878 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff71c004cc0 con 0x7ff724102320 2024-05-09T09:23:26.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.130+0000 7ff716ffd640 1 -- 172.21.15.12:0/1214277878 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c010700 con 0x7ff724102320 2024-05-09T09:23:26.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.130+0000 7ff72baee640 1 -- 172.21.15.12:0/1214277878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7241a0040 con 0x7ff724102320 2024-05-09T09:23:26.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.131+0000 7ff72baee640 1 -- 172.21.15.12:0/1214277878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7241a0270 con 0x7ff724102320 2024-05-09T09:23:26.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.132+0000 7ff716ffd640 1 -- 172.21.15.12:0/1214277878 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff71c004e70 con 0x7ff724102320 2024-05-09T09:23:26.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.133+0000 7ff716ffd640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7100a2b90 0x7ff7100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:26.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.134+0000 7ff716ffd640 1 -- 172.21.15.12:0/1214277878 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff71c0cf990 con 0x7ff724102320 2024-05-09T09:23:26.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.134+0000 7ff72988a640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7100a2b90 0x7ff7100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:26.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.134+0000 7ff72988a640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7100a2b90 0x7ff7100a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff720000c00 tx=0x7ff720031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:26.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.138+0000 7ff716ffd640 1 -- 172.21.15.12:0/1214277878 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff71c09c650 con 0x7ff724102320 2024-05-09T09:23:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.328+0000 7ff72baee640 1 -- 172.21.15.12:0/1214277878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7241a04e0 con 0x7ff724102320 2024-05-09T09:23:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.330+0000 7ff716ffd640 1 -- 172.21.15.12:0/1214277878 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24105 (secure 0 0 0) 0x7ff71c09bf50 con 0x7ff724102320 2024-05-09T09:23:26.330 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:26.330 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:23:14.726332+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:23:26.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.333+0000 7ff72baee640 1 -- 172.21.15.12:0/1214277878 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7100a2b90 msgr2=0x7ff7100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.333+0000 7ff72baee640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7100a2b90 0x7ff7100a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff720000c00 tx=0x7ff720031510 comp rx=0 tx=0).stop 2024-05-09T09:23:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.333+0000 7ff72baee640 1 -- 172.21.15.12:0/1214277878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff724102320 msgr2=0x7ff72419f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.333+0000 7ff72baee640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff724102320 0x7ff72419f190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00eb10 tx=0x7ff71c00c2d0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.333+0000 7ff72baee640 1 -- 172.21.15.12:0/1214277878 shutdown_connections 2024-05-09T09:23:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.333+0000 7ff72baee640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7100a2b90 0x7ff7100a5050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.333+0000 7ff72baee640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff724102320 0x7ff72419f190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.333+0000 7ff72baee640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff724101960 0x7ff724198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.333+0000 7ff72baee640 1 --2- 172.21.15.12:0/1214277878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff724100770 0x7ff724197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.333+0000 7ff72baee640 1 -- 172.21.15.12:0/1214277878 >> 172.21.15.12:0/1214277878 conn(0x7ff7240fbce0 msgr2=0x7ff7240689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.333+0000 7ff72baee640 1 -- 172.21.15.12:0/1214277878 shutdown_connections 2024-05-09T09:23:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.333+0000 7ff72baee640 1 -- 172.21.15.12:0/1214277878 wait complete. 2024-05-09T09:23:26.345 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:23:08.167107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:23:09.065670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:23:12.773640+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:23:26.346 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:23:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.455+0000 7f62ef909640 1 -- 172.21.15.12:0/1042496904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62e8101960 msgr2=0x7f62e8101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.455+0000 7f62ef909640 1 --2- 172.21.15.12:0/1042496904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62e8101960 0x7f62e8101de0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009900 tx=0x7f62d802a410 comp rx=0 tx=0).stop 2024-05-09T09:23:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.455+0000 7f62ef909640 1 -- 172.21.15.12:0/1042496904 shutdown_connections 2024-05-09T09:23:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.455+0000 7f62ef909640 1 --2- 172.21.15.12:0/1042496904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e8102320 0x7f62e8108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.455+0000 7f62ef909640 1 --2- 172.21.15.12:0/1042496904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62e8101960 0x7f62e8101de0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.455+0000 7f62ef909640 1 --2- 172.21.15.12:0/1042496904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62e8100770 0x7f62e8100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.455+0000 7f62ef909640 1 -- 172.21.15.12:0/1042496904 >> 172.21.15.12:0/1042496904 conn(0x7f62e80fbce0 msgr2=0x7f62e80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.455+0000 7f62ef909640 1 -- 172.21.15.12:0/1042496904 shutdown_connections 2024-05-09T09:23:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.455+0000 7f62ef909640 1 -- 172.21.15.12:0/1042496904 wait complete. 2024-05-09T09:23:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.455+0000 7f62ef909640 1 Processor -- start 2024-05-09T09:23:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ef909640 1 -- start start 2024-05-09T09:23:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ef909640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62e8100770 0x7f62e8199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ef909640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e8101960 0x7f62e819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ef909640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62e8102320 0x7f62e81a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ef909640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f62e810c3d0 con 0x7f62e8100770 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ef909640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62e810c250 con 0x7f62e8101960 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ef909640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f62e810c550 con 0x7f62e8102320 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ed6a5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62e8100770 0x7f62e8199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ed6a5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62e8100770 0x7f62e8199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45438/0 (socket says 172.21.15.12:45438) 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ed6a5640 1 -- 172.21.15.12:0/1026908225 learned_addr learned my addr 172.21.15.12:0/1026908225 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ecea4640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e8101960 0x7f62e819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62edea6640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62e8102320 0x7f62e81a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ed6a5640 1 -- 172.21.15.12:0/1026908225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62e8102320 msgr2=0x7f62e81a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ed6a5640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62e8102320 0x7f62e81a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ed6a5640 1 -- 172.21.15.12:0/1026908225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e8101960 msgr2=0x7f62e819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ed6a5640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e8101960 0x7f62e819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ed6a5640 1 -- 172.21.15.12:0/1026908225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e81a1a20 con 0x7f62e8100770 2024-05-09T09:23:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.456+0000 7f62ed6a5640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62e8100770 0x7f62e8199e00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62e400e9c0 tx=0x7f62e400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.457+0000 7f62de7fc640 1 -- 172.21.15.12:0/1026908225 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e400cd80 con 0x7f62e8100770 2024-05-09T09:23:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.457+0000 7f62de7fc640 1 -- 172.21.15.12:0/1026908225 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f62e40035f0 con 0x7f62e8100770 2024-05-09T09:23:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.457+0000 7f62ef909640 1 -- 172.21.15.12:0/1026908225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e81a1d10 con 0x7f62e8100770 2024-05-09T09:23:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.457+0000 7f62de7fc640 1 -- 172.21.15.12:0/1026908225 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4010690 con 0x7f62e8100770 2024-05-09T09:23:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.457+0000 7f62ef909640 1 -- 172.21.15.12:0/1026908225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e81a2180 con 0x7f62e8100770 2024-05-09T09:23:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.458+0000 7f62ef909640 1 -- 172.21.15.12:0/1026908225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e81a2410 con 0x7f62e8100770 2024-05-09T09:23:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.459+0000 7f62de7fc640 1 -- 172.21.15.12:0/1026908225 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f62e40037a0 con 0x7f62e8100770 2024-05-09T09:23:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.460+0000 7f62de7fc640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f62d40a2b90 0x7f62d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.460+0000 7f62de7fc640 1 -- 172.21.15.12:0/1026908225 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f62e40cf690 con 0x7f62e8100770 2024-05-09T09:23:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.460+0000 7f62ecea4640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f62d40a2b90 0x7f62d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.460+0000 7f62ecea4640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f62d40a2b90 0x7f62d40a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f62d8000c00 tx=0x7f62d8031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:26.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.464+0000 7f62de7fc640 1 -- 172.21.15.12:0/1026908225 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f62e409c350 con 0x7f62e8100770 2024-05-09T09:23:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.666+0000 7f62ef909640 1 -- 172.21.15.12:0/1026908225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62e81a25e0 con 0x7f62e8100770 2024-05-09T09:23:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.666+0000 7f62de7fc640 1 -- 172.21.15.12:0/1026908225 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f62e409bc50 con 0x7f62e8100770 2024-05-09T09:23:26.666 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:23:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.668+0000 7f62ef909640 1 -- 172.21.15.12:0/1026908225 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f62d40a2b90 msgr2=0x7f62d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.668+0000 7f62ef909640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f62d40a2b90 0x7f62d40a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f62d8000c00 tx=0x7f62d8031510 comp rx=0 tx=0).stop 2024-05-09T09:23:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.668+0000 7f62ef909640 1 -- 172.21.15.12:0/1026908225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62e8100770 msgr2=0x7f62e8199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.668+0000 7f62ef909640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62e8100770 0x7f62e8199e00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62e400e9c0 tx=0x7f62e400ee80 comp rx=0 tx=0).stop 2024-05-09T09:23:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.668+0000 7f62ef909640 1 -- 172.21.15.12:0/1026908225 shutdown_connections 2024-05-09T09:23:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.668+0000 7f62ef909640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f62d40a2b90 0x7f62d40a5050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.668+0000 7f62ef909640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62e8102320 0x7f62e81a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.668+0000 7f62ef909640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e8101960 0x7f62e819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.668+0000 7f62ef909640 1 --2- 172.21.15.12:0/1026908225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62e8100770 0x7f62e8199e00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.668+0000 7f62ef909640 1 -- 172.21.15.12:0/1026908225 >> 172.21.15.12:0/1026908225 conn(0x7f62e80fbce0 msgr2=0x7f62e80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.668+0000 7f62ef909640 1 -- 172.21.15.12:0/1026908225 shutdown_connections 2024-05-09T09:23:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.668+0000 7f62ef909640 1 -- 172.21.15.12:0/1026908225 wait complete. 2024-05-09T09:23:26.679 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:23:26.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.788+0000 7f5ddb712640 1 -- 172.21.15.12:0/3800621912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dd4101400 msgr2=0x7f5dd4101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.788+0000 7f5ddb712640 1 --2- 172.21.15.12:0/3800621912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dd4101400 0x7f5dd4101880 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009900 tx=0x7f5dcc02a410 comp rx=0 tx=0).stop 2024-05-09T09:23:26.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.788+0000 7f5ddb712640 1 -- 172.21.15.12:0/3800621912 shutdown_connections 2024-05-09T09:23:26.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.788+0000 7f5ddb712640 1 --2- 172.21.15.12:0/3800621912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dd4101400 0x7f5dd4101880 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.788+0000 7f5ddb712640 1 --2- 172.21.15.12:0/3800621912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dd4100a60 0x7f5dd4100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.788+0000 7f5ddb712640 1 --2- 172.21.15.12:0/3800621912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dd4100100 0x7f5dd4100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.788+0000 7f5ddb712640 1 -- 172.21.15.12:0/3800621912 >> 172.21.15.12:0/3800621912 conn(0x7f5dd40fbd20 msgr2=0x7f5dd40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.788+0000 7f5ddb712640 1 -- 172.21.15.12:0/3800621912 shutdown_connections 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.788+0000 7f5ddb712640 1 -- 172.21.15.12:0/3800621912 wait complete. 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.789+0000 7f5ddb712640 1 Processor -- start 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.789+0000 7f5ddb712640 1 -- start start 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.789+0000 7f5ddb712640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dd4100100 0x7f5dd4199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.789+0000 7f5ddb712640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dd4100a60 0x7f5dd419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.789+0000 7f5ddb712640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dd4101400 0x7f5dd41a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.789+0000 7f5ddb712640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd410f700 con 0x7f5dd4101400 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.789+0000 7f5ddb712640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd410f580 con 0x7f5dd4100a60 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.789+0000 7f5ddb712640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd410f880 con 0x7f5dd4100100 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.789+0000 7f5dd9caf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dd4101400 0x7f5dd41a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.789+0000 7f5dd9caf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dd4101400 0x7f5dd41a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45462/0 (socket says 172.21.15.12:45462) 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.789+0000 7f5dd9caf640 1 -- 172.21.15.12:0/2772400013 learned_addr learned my addr 172.21.15.12:0/2772400013 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.789+0000 7f5dd8cad640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dd4100a60 0x7f5dd419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:26.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.790+0000 7f5dd94ae640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dd4100100 0x7f5dd4199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:26.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.790+0000 7f5dd9caf640 1 -- 172.21.15.12:0/2772400013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dd4100100 msgr2=0x7f5dd4199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.790+0000 7f5dd9caf640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dd4100100 0x7f5dd4199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.790+0000 7f5dd9caf640 1 -- 172.21.15.12:0/2772400013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dd4100a60 msgr2=0x7f5dd419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.790+0000 7f5dd9caf640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dd4100a60 0x7f5dd419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.790+0000 7f5dd9caf640 1 -- 172.21.15.12:0/2772400013 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd41a18b0 con 0x7f5dd4101400 2024-05-09T09:23:26.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.790+0000 7f5dd94ae640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dd4100100 0x7f5dd4199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:26.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.790+0000 7f5dd9caf640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dd4101400 0x7f5dd41a1210 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc006010 tx=0x7f5dcc031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:26.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.790+0000 7f5dca7fc640 1 -- 172.21.15.12:0/2772400013 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc03f030 con 0x7f5dd4101400 2024-05-09T09:23:26.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.790+0000 7f5dca7fc640 1 -- 172.21.15.12:0/2772400013 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5dcc003ac0 con 0x7f5dd4101400 2024-05-09T09:23:26.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.790+0000 7f5dca7fc640 1 -- 172.21.15.12:0/2772400013 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc035070 con 0x7f5dd4101400 2024-05-09T09:23:26.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.790+0000 7f5ddb712640 1 -- 172.21.15.12:0/2772400013 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd41a1b40 con 0x7f5dd4101400 2024-05-09T09:23:26.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.790+0000 7f5ddb712640 1 -- 172.21.15.12:0/2772400013 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd41a2030 con 0x7f5dd4101400 2024-05-09T09:23:26.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.791+0000 7f5ddb712640 1 -- 172.21.15.12:0/2772400013 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd410ae50 con 0x7f5dd4101400 2024-05-09T09:23:26.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.792+0000 7f5dca7fc640 1 -- 172.21.15.12:0/2772400013 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5dcc042030 con 0x7f5dd4101400 2024-05-09T09:23:26.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.793+0000 7f5dca7fc640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5dc00a2b90 0x7f5dc00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:26.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.793+0000 7f5dd94ae640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5dc00a2b90 0x7f5dc00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:26.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.793+0000 7f5dca7fc640 1 -- 172.21.15.12:0/2772400013 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f5dcc0efd70 con 0x7f5dd4101400 2024-05-09T09:23:26.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.794+0000 7f5dd94ae640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5dc00a2b90 0x7f5dc00a5050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5dd419ab40 tx=0x7f5dd0009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:26.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.796+0000 7f5dca7fc640 1 -- 172.21.15.12:0/2772400013 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5dcc0bca30 con 0x7f5dd4101400 2024-05-09T09:23:26.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:26 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1516533791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:23:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:26 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/212068319' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:23:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:26 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1214277878' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:26.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.981+0000 7f5ddb712640 1 -- 172.21.15.12:0/2772400013 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5dd402d410 con 0x7f5dd4101400 2024-05-09T09:23:26.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.982+0000 7f5dca7fc640 1 -- 172.21.15.12:0/2772400013 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f5dcc0bc330 con 0x7f5dd4101400 2024-05-09T09:23:26.982 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:23:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.984+0000 7f5ddb712640 1 -- 172.21.15.12:0/2772400013 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5dc00a2b90 msgr2=0x7f5dc00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.984+0000 7f5ddb712640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5dc00a2b90 0x7f5dc00a5050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5dd419ab40 tx=0x7f5dd0009520 comp rx=0 tx=0).stop 2024-05-09T09:23:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.984+0000 7f5ddb712640 1 -- 172.21.15.12:0/2772400013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dd4101400 msgr2=0x7f5dd41a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.984+0000 7f5ddb712640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dd4101400 0x7f5dd41a1210 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc006010 tx=0x7f5dcc031ac0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.984+0000 7f5ddb712640 1 -- 172.21.15.12:0/2772400013 shutdown_connections 2024-05-09T09:23:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.984+0000 7f5ddb712640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5dc00a2b90 0x7f5dc00a5050 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.984+0000 7f5ddb712640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dd4101400 0x7f5dd41a1210 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.984+0000 7f5ddb712640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dd4100a60 0x7f5dd419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.984+0000 7f5ddb712640 1 --2- 172.21.15.12:0/2772400013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dd4100100 0x7f5dd4199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.984+0000 7f5ddb712640 1 -- 172.21.15.12:0/2772400013 >> 172.21.15.12:0/2772400013 conn(0x7f5dd40fbd20 msgr2=0x7f5dd40fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:26.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:26 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1516533791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:23:26.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:26 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/212068319' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:23:26.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:26 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1214277878' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:26.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.984+0000 7f5ddb712640 1 -- 172.21.15.12:0/2772400013 shutdown_connections 2024-05-09T09:23:26.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:26.984+0000 7f5ddb712640 1 -- 172.21.15.12:0/2772400013 wait complete. 2024-05-09T09:23:26.995 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:23:27.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:26 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1516533791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:23:27.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:26 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/212068319' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:23:27.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:26 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1214277878' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:27.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.103+0000 7f48fbf9e640 1 -- 172.21.15.12:0/3805730977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48f4107570 msgr2=0x7f48f41099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.103+0000 7f48fbf9e640 1 --2- 172.21.15.12:0/3805730977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48f4107570 0x7f48f41099c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009ca0 tx=0x7f48ec02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.103+0000 7f48fbf9e640 1 -- 172.21.15.12:0/3805730977 shutdown_connections 2024-05-09T09:23:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.103+0000 7f48fbf9e640 1 --2- 172.21.15.12:0/3805730977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48f4107570 0x7f48f41099c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.103+0000 7f48fbf9e640 1 --2- 172.21.15.12:0/3805730977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f4104c40 0x7f48f4107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.103+0000 7f48fbf9e640 1 --2- 172.21.15.12:0/3805730977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f4102310 0x7f48f4104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.103+0000 7f48fbf9e640 1 -- 172.21.15.12:0/3805730977 >> 172.21.15.12:0/3805730977 conn(0x7f48f40fbd20 msgr2=0x7f48f40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.104+0000 7f48fbf9e640 1 -- 172.21.15.12:0/3805730977 shutdown_connections 2024-05-09T09:23:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.104+0000 7f48fbf9e640 1 -- 172.21.15.12:0/3805730977 wait complete. 2024-05-09T09:23:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.104+0000 7f48fbf9e640 1 Processor -- start 2024-05-09T09:23:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.104+0000 7f48fbf9e640 1 -- start start 2024-05-09T09:23:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.104+0000 7f48fbf9e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48f4102310 0x7f48f4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.104+0000 7f48fbf9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f4104c40 0x7f48f41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.104+0000 7f48fbf9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f4107570 0x7f48f419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.104+0000 7f48fbf9e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f48f410d560 con 0x7f48f4102310 2024-05-09T09:23:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.104+0000 7f48fbf9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48f410d3e0 con 0x7f48f4104c40 2024-05-09T09:23:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.104+0000 7f48fbf9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f48f410d6e0 con 0x7f48f4107570 2024-05-09T09:23:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.104+0000 7f48f9d3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48f4102310 0x7f48f4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.104+0000 7f48f9d3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48f4102310 0x7f48f4197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45490/0 (socket says 172.21.15.12:45490) 2024-05-09T09:23:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.104+0000 7f48f9d3a640 1 -- 172.21.15.12:0/2903191758 learned_addr learned my addr 172.21.15.12:0/2903191758 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.105+0000 7f48f9539640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f4104c40 0x7f48f41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.105+0000 7f48fa53b640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f4107570 0x7f48f419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.105+0000 7f48f9d3a640 1 -- 172.21.15.12:0/2903191758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f4107570 msgr2=0x7f48f419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.105+0000 7f48f9d3a640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f4107570 0x7f48f419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.105+0000 7f48f9d3a640 1 -- 172.21.15.12:0/2903191758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f4104c40 msgr2=0x7f48f41980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.105+0000 7f48f9d3a640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f4104c40 0x7f48f41980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.105+0000 7f48f9d3a640 1 -- 172.21.15.12:0/2903191758 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f419f7a0 con 0x7f48f4102310 2024-05-09T09:23:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.105+0000 7f48f9d3a640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48f4102310 0x7f48f4197b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f48f000d770 tx=0x7f48f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.105+0000 7f48eaffd640 1 -- 172.21.15.12:0/2903191758 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0017070 con 0x7f48f4102310 2024-05-09T09:23:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.105+0000 7f48eaffd640 1 -- 172.21.15.12:0/2903191758 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f48f00054f0 con 0x7f48f4102310 2024-05-09T09:23:27.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.105+0000 7f48fbf9e640 1 -- 172.21.15.12:0/2903191758 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f419fa90 con 0x7f48f4102310 2024-05-09T09:23:27.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.105+0000 7f48eaffd640 1 -- 172.21.15.12:0/2903191758 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0004d60 con 0x7f48f4102310 2024-05-09T09:23:27.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.105+0000 7f48fbf9e640 1 -- 172.21.15.12:0/2903191758 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f419ff90 con 0x7f48f4102310 2024-05-09T09:23:27.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.106+0000 7f48fbf9e640 1 -- 172.21.15.12:0/2903191758 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f40fd810 con 0x7f48f4102310 2024-05-09T09:23:27.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.107+0000 7f48eaffd640 1 -- 172.21.15.12:0/2903191758 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f48f0004f00 con 0x7f48f4102310 2024-05-09T09:23:27.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.108+0000 7f48eaffd640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f48e00a2b90 0x7f48e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:27.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.108+0000 7f48eaffd640 1 -- 172.21.15.12:0/2903191758 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f48f00cf1e0 con 0x7f48f4102310 2024-05-09T09:23:27.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.108+0000 7f48f9539640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f48e00a2b90 0x7f48e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:27.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.108+0000 7f48f9539640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f48e00a2b90 0x7f48e00a5050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f48f4199090 tx=0x7f48e400b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:27.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.111+0000 7f48eaffd640 1 -- 172.21.15.12:0/2903191758 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f48f009bea0 con 0x7f48f4102310 2024-05-09T09:23:27.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.296+0000 7f48fbf9e640 1 -- 172.21.15.12:0/2903191758 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f48f4062430 con 0x7f48f4102310 2024-05-09T09:23:27.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.296+0000 7f48eaffd640 1 -- 172.21.15.12:0/2903191758 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f48f009b7a0 con 0x7f48f4102310 2024-05-09T09:23:27.296 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T09:23:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.298+0000 7f48fbf9e640 1 -- 172.21.15.12:0/2903191758 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f48e00a2b90 msgr2=0x7f48e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.298+0000 7f48fbf9e640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f48e00a2b90 0x7f48e00a5050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f48f4199090 tx=0x7f48e400b590 comp rx=0 tx=0).stop 2024-05-09T09:23:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.298+0000 7f48fbf9e640 1 -- 172.21.15.12:0/2903191758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48f4102310 msgr2=0x7f48f4197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.298+0000 7f48fbf9e640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48f4102310 0x7f48f4197b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f48f000d770 tx=0x7f48f000dc30 comp rx=0 tx=0).stop 2024-05-09T09:23:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.298+0000 7f48fbf9e640 1 -- 172.21.15.12:0/2903191758 shutdown_connections 2024-05-09T09:23:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.298+0000 7f48fbf9e640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f48e00a2b90 0x7f48e00a5050 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.298+0000 7f48fbf9e640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f4107570 0x7f48f419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.298+0000 7f48fbf9e640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f4104c40 0x7f48f41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.298+0000 7f48fbf9e640 1 --2- 172.21.15.12:0/2903191758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48f4102310 0x7f48f4197b80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.298+0000 7f48fbf9e640 1 -- 172.21.15.12:0/2903191758 >> 172.21.15.12:0/2903191758 conn(0x7f48f40fbd20 msgr2=0x7f48f4102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.298+0000 7f48fbf9e640 1 -- 172.21.15.12:0/2903191758 shutdown_connections 2024-05-09T09:23:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.298+0000 7f48fbf9e640 1 -- 172.21.15.12:0/2903191758 wait complete. 2024-05-09T09:23:27.309 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:23:27.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 -- 172.21.15.12:0/4195833103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a14100790 msgr2=0x7f8a14100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 --2- 172.21.15.12:0/4195833103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a14100790 0x7f8a14100bb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009900 tx=0x7f8a1002a410 comp rx=0 tx=0).stop 2024-05-09T09:23:27.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 -- 172.21.15.12:0/4195833103 shutdown_connections 2024-05-09T09:23:27.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 --2- 172.21.15.12:0/4195833103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a14102340 0x7f8a14108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 --2- 172.21.15.12:0/4195833103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a14101980 0x7f8a14101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 --2- 172.21.15.12:0/4195833103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a14100790 0x7f8a14100bb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 -- 172.21.15.12:0/4195833103 >> 172.21.15.12:0/4195833103 conn(0x7f8a140fbd20 msgr2=0x7f8a140fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 -- 172.21.15.12:0/4195833103 shutdown_connections 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 -- 172.21.15.12:0/4195833103 wait complete. 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 Processor -- start 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 -- start start 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a14100790 0x7f8a14197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a14101980 0x7f8a141981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a14102340 0x7f8a1419f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1410c3f0 con 0x7f8a14102340 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.418+0000 7f8a1bb42640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1410c270 con 0x7f8a14101980 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a1bb42640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1410c570 con 0x7f8a14100790 2024-05-09T09:23:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a1a0df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a14102340 0x7f8a1419f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a1a0df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a14102340 0x7f8a1419f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45496/0 (socket says 172.21.15.12:45496) 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a1a0df640 1 -- 172.21.15.12:0/2866506719 learned_addr learned my addr 172.21.15.12:0/2866506719 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a190dd640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a14101980 0x7f8a141981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a198de640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a14100790 0x7f8a14197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a1a0df640 1 -- 172.21.15.12:0/2866506719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a14100790 msgr2=0x7f8a14197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a1a0df640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a14100790 0x7f8a14197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a1a0df640 1 -- 172.21.15.12:0/2866506719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a14101980 msgr2=0x7f8a141981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a1a0df640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a14101980 0x7f8a141981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a1a0df640 1 -- 172.21.15.12:0/2866506719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1419f850 con 0x7f8a14102340 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a1a0df640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a14102340 0x7f8a1419f1b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00eaa0 tx=0x7f8a0c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a0affd640 1 -- 172.21.15.12:0/2866506719 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c00cdb0 con 0x7f8a14102340 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a1bb42640 1 -- 172.21.15.12:0/2866506719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1419fb40 con 0x7f8a14102340 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a0affd640 1 -- 172.21.15.12:0/2866506719 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8a0c005020 con 0x7f8a14102340 2024-05-09T09:23:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a0affd640 1 -- 172.21.15.12:0/2866506719 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0116b0 con 0x7f8a14102340 2024-05-09T09:23:27.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.419+0000 7f8a1bb42640 1 -- 172.21.15.12:0/2866506719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a141a0090 con 0x7f8a14102340 2024-05-09T09:23:27.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.420+0000 7f8a1bb42640 1 -- 172.21.15.12:0/2866506719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a141a0260 con 0x7f8a14102340 2024-05-09T09:23:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.422+0000 7f8a0affd640 1 -- 172.21.15.12:0/2866506719 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8a0c0051d0 con 0x7f8a14102340 2024-05-09T09:23:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.422+0000 7f8a0affd640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8a000a2b90 0x7f8a000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.422+0000 7f8a198de640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8a000a2b90 0x7f8a000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.422+0000 7f8a0affd640 1 -- 172.21.15.12:0/2866506719 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f8a0c0cf580 con 0x7f8a14102340 2024-05-09T09:23:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.423+0000 7f8a198de640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8a000a2b90 0x7f8a000a5050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8a10006010 tx=0x7f8a100095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.427+0000 7f8a0affd640 1 -- 172.21.15.12:0/2866506719 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8a0c016030 con 0x7f8a14102340 2024-05-09T09:23:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.611+0000 7f8a1bb42640 1 -- 172.21.15.12:0/2866506719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8a141a04d0 con 0x7f8a14102340 2024-05-09T09:23:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.612+0000 7f8a0affd640 1 -- 172.21.15.12:0/2866506719 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f8a0c09c2a0 con 0x7f8a14102340 2024-05-09T09:23:27.612 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:23:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.614+0000 7f8a1bb42640 1 -- 172.21.15.12:0/2866506719 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8a000a2b90 msgr2=0x7f8a000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.614+0000 7f8a1bb42640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8a000a2b90 0x7f8a000a5050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8a10006010 tx=0x7f8a100095f0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.614+0000 7f8a1bb42640 1 -- 172.21.15.12:0/2866506719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a14102340 msgr2=0x7f8a1419f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.614+0000 7f8a1bb42640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a14102340 0x7f8a1419f1b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00eaa0 tx=0x7f8a0c00ef60 comp rx=0 tx=0).stop 2024-05-09T09:23:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.614+0000 7f8a1bb42640 1 -- 172.21.15.12:0/2866506719 shutdown_connections 2024-05-09T09:23:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.614+0000 7f8a1bb42640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8a000a2b90 0x7f8a000a5050 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.614+0000 7f8a1bb42640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a14102340 0x7f8a1419f1b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.614+0000 7f8a1bb42640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a14101980 0x7f8a141981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.614+0000 7f8a1bb42640 1 --2- 172.21.15.12:0/2866506719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a14100790 0x7f8a14197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.614+0000 7f8a1bb42640 1 -- 172.21.15.12:0/2866506719 >> 172.21.15.12:0/2866506719 conn(0x7f8a140fbd20 msgr2=0x7f8a140fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.614+0000 7f8a1bb42640 1 -- 172.21.15.12:0/2866506719 shutdown_connections 2024-05-09T09:23:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.614+0000 7f8a1bb42640 1 -- 172.21.15.12:0/2866506719 wait complete. 2024-05-09T09:23:27.626 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T09:23:27.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.732+0000 7fc1e0a93640 1 -- 172.21.15.12:0/902191522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1dc103420 msgr2=0x7fc1dc109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.732+0000 7fc1e0a93640 1 --2- 172.21.15.12:0/902191522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1dc103420 0x7fc1dc109cb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009ca0 tx=0x7fc1d402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.732+0000 7fc1e0a93640 1 -- 172.21.15.12:0/902191522 shutdown_connections 2024-05-09T09:23:27.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.732+0000 7fc1e0a93640 1 --2- 172.21.15.12:0/902191522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1dc103420 0x7fc1dc109cb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.732+0000 7fc1e0a93640 1 --2- 172.21.15.12:0/902191522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1dc102a60 0x7fc1dc102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.732+0000 7fc1e0a93640 1 --2- 172.21.15.12:0/902191522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc101870 0x7fc1dc101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.732+0000 7fc1e0a93640 1 -- 172.21.15.12:0/902191522 >> 172.21.15.12:0/902191522 conn(0x7fc1dc0fcde0 msgr2=0x7fc1dc0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:27.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.733+0000 7fc1e0a93640 1 -- 172.21.15.12:0/902191522 shutdown_connections 2024-05-09T09:23:27.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.733+0000 7fc1e0a93640 1 -- 172.21.15.12:0/902191522 wait complete. 2024-05-09T09:23:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.733+0000 7fc1e0a93640 1 Processor -- start 2024-05-09T09:23:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.733+0000 7fc1e0a93640 1 -- start start 2024-05-09T09:23:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.733+0000 7fc1e0a93640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1dc101870 0x7fc1dc197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.733+0000 7fc1e0a93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc102a60 0x7fc1dc198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1e0a93640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1dc103420 0x7fc1dc19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1e0a93640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc10d4d0 con 0x7fc1dc101870 2024-05-09T09:23:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1e0a93640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc10d350 con 0x7fc1dc102a60 2024-05-09T09:23:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1e0a93640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc10d650 con 0x7fc1dc103420 2024-05-09T09:23:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1da59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1dc101870 0x7fc1dc197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1da59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1dc101870 0x7fc1dc197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45522/0 (socket says 172.21.15.12:45522) 2024-05-09T09:23:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1da59c640 1 -- 172.21.15.12:0/2424091312 learned_addr learned my addr 172.21.15.12:0/2424091312 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1dad9d640 1 --2- 172.21.15.12:0/2424091312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1dc103420 0x7fc1dc19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1da59c640 1 -- 172.21.15.12:0/2424091312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1dc103420 msgr2=0x7fc1dc19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1da59c640 1 --2- 172.21.15.12:0/2424091312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1dc103420 0x7fc1dc19f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1da59c640 1 -- 172.21.15.12:0/2424091312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc102a60 msgr2=0x7fc1dc198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1da59c640 1 --2- 172.21.15.12:0/2424091312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc102a60 0x7fc1dc198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1da59c640 1 -- 172.21.15.12:0/2424091312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc19f740 con 0x7fc1dc101870 2024-05-09T09:23:27.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1da59c640 1 --2- 172.21.15.12:0/2424091312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1dc101870 0x7fc1dc197b20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00d770 tx=0x7fc1cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:27.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1cb7fe640 1 -- 172.21.15.12:0/2424091312 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc017070 con 0x7fc1dc101870 2024-05-09T09:23:27.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1e0a93640 1 -- 172.21.15.12:0/2424091312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1dc19fa30 con 0x7fc1dc101870 2024-05-09T09:23:27.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1cb7fe640 1 -- 172.21.15.12:0/2424091312 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc1cc009c30 con 0x7fc1dc101870 2024-05-09T09:23:27.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1cb7fe640 1 -- 172.21.15.12:0/2424091312 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc003e10 con 0x7fc1dc101870 2024-05-09T09:23:27.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.734+0000 7fc1e0a93640 1 -- 172.21.15.12:0/2424091312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1dc19ff80 con 0x7fc1dc101870 2024-05-09T09:23:27.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.735+0000 7fc1e0a93640 1 -- 172.21.15.12:0/2424091312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1dc108c10 con 0x7fc1dc101870 2024-05-09T09:23:27.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.736+0000 7fc1cb7fe640 1 -- 172.21.15.12:0/2424091312 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc1cc003fb0 con 0x7fc1dc101870 2024-05-09T09:23:27.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.737+0000 7fc1cb7fe640 1 --2- 172.21.15.12:0/2424091312 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc1c40a2b90 0x7fc1c40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:27.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.737+0000 7fc1d9d9b640 1 --2- 172.21.15.12:0/2424091312 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc1c40a2b90 0x7fc1c40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:27.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.737+0000 7fc1cb7fe640 1 -- 172.21.15.12:0/2424091312 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc1cc0cf0d0 con 0x7fc1dc101870 2024-05-09T09:23:27.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.738+0000 7fc1d9d9b640 1 --2- 172.21.15.12:0/2424091312 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc1c40a2b90 0x7fc1c40a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0005fd0 tx=0x7fc1d0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:27.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.741+0000 7fc1cb7fe640 1 -- 172.21.15.12:0/2424091312 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc1cc09bdf0 con 0x7fc1dc101870 2024-05-09T09:23:27.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.940+0000 7fc1e0a93640 1 -- 172.21.15.12:0/2424091312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc1dc062530 con 0x7fc1dc101870 2024-05-09T09:23:27.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.941+0000 7fc1cb7fe640 1 -- 172.21.15.12:0/2424091312 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+591 (secure 0 0 0) 0x7fc1cc09b6f0 con 0x7fc1dc101870 2024-05-09T09:23:27.941 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1454579683 2024-05-10T09:16:04.466876+0000 2024-05-09T09:23:27.942 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/2168557623 2024-05-10T09:16:04.466876+0000 2024-05-09T09:23:27.942 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/2168557623 2024-05-10T09:16:04.466876+0000 2024-05-09T09:23:27.942 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/545109421 2024-05-10T09:15:36.182637+0000 2024-05-09T09:23:27.942 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1153421462 2024-05-10T09:16:04.466876+0000 2024-05-09T09:23:27.942 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1731765200 2024-05-10T09:15:36.182637+0000 2024-05-09T09:23:27.942 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2508497448 2024-05-10T09:15:36.182637+0000 2024-05-09T09:23:27.942 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1876427478 2024-05-10T09:15:36.182637+0000 2024-05-09T09:23:27.942 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3479121957 2024-05-10T09:16:04.466876+0000 2024-05-09T09:23:27.942 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1876427478 2024-05-10T09:15:36.182637+0000 2024-05-09T09:23:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:listed 10 entries 2024-05-09T09:23:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.943+0000 7fc1e0a93640 1 -- 172.21.15.12:0/2424091312 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc1c40a2b90 msgr2=0x7fc1c40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.943+0000 7fc1e0a93640 1 --2- 172.21.15.12:0/2424091312 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc1c40a2b90 0x7fc1c40a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0005fd0 tx=0x7fc1d0005850 comp rx=0 tx=0).stop 2024-05-09T09:23:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.943+0000 7fc1e0a93640 1 -- 172.21.15.12:0/2424091312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1dc101870 msgr2=0x7fc1dc197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.943+0000 7fc1e0a93640 1 --2- 172.21.15.12:0/2424091312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1dc101870 0x7fc1dc197b20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00d770 tx=0x7fc1cc00dc30 comp rx=0 tx=0).stop 2024-05-09T09:23:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.943+0000 7fc1e0a93640 1 -- 172.21.15.12:0/2424091312 shutdown_connections 2024-05-09T09:23:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.943+0000 7fc1e0a93640 1 --2- 172.21.15.12:0/2424091312 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc1c40a2b90 0x7fc1c40a5050 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.943+0000 7fc1e0a93640 1 --2- 172.21.15.12:0/2424091312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1dc103420 0x7fc1dc19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.943+0000 7fc1e0a93640 1 --2- 172.21.15.12:0/2424091312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc102a60 0x7fc1dc198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.943+0000 7fc1e0a93640 1 --2- 172.21.15.12:0/2424091312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1dc101870 0x7fc1dc197b20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.943+0000 7fc1e0a93640 1 -- 172.21.15.12:0/2424091312 >> 172.21.15.12:0/2424091312 conn(0x7fc1dc0fcde0 msgr2=0x7fc1dc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.943+0000 7fc1e0a93640 1 -- 172.21.15.12:0/2424091312 shutdown_connections 2024-05-09T09:23:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:27.943+0000 7fc1e0a93640 1 -- 172.21.15.12:0/2424091312 wait complete. 2024-05-09T09:23:27.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:27 smithi012 ceph-mon[30140]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s wr, 0 op/s 2024-05-09T09:23:27.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:27 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1026908225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:23:27.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:27 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2772400013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:23:27.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:27 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2903191758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:23:27.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:27 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2866506719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:23:27.954 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi012.front.sepia.ceph.com... 2024-05-09T09:23:27.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:27.954 DEBUG:teuthology.orchestra.run.smithi012:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:23:27.978 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[74789]: fuse finished with error 0 and tester_r 0 2024-05-09T09:23:27.980 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T09:23:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:27 smithi170 ceph-mon[33632]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s wr, 0 op/s 2024-05-09T09:23:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:27 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1026908225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:23:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:27 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2772400013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:23:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:27 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2903191758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:23:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:27 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2866506719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:23:28.003 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-05-09T09:23:28.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:28.003 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-09T09:23:28.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:27 smithi016 ceph-mon[33335]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s wr, 0 op/s 2024-05-09T09:23:28.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:27 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1026908225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:23:28.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:27 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2772400013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:23:28.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:27 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2903191758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:23:28.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:27 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2866506719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:23:28.072 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:23:28.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.217+0000 7f63d5810640 1 -- 172.21.15.12:0/2319218268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63d0102a90 msgr2=0x7f63d0102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:28.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.217+0000 7f63d5810640 1 --2- 172.21.15.12:0/2319218268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63d0102a90 0x7f63d0102f10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009900 tx=0x7f63b802a410 comp rx=0 tx=0).stop 2024-05-09T09:23:28.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.217+0000 7f63d5810640 1 -- 172.21.15.12:0/2319218268 shutdown_connections 2024-05-09T09:23:28.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.217+0000 7f63d5810640 1 --2- 172.21.15.12:0/2319218268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d0103450 0x7f63d0109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.217+0000 7f63d5810640 1 --2- 172.21.15.12:0/2319218268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63d0102a90 0x7f63d0102f10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.217+0000 7f63d5810640 1 --2- 172.21.15.12:0/2319218268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63d01018a0 0x7f63d0101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.217+0000 7f63d5810640 1 -- 172.21.15.12:0/2319218268 >> 172.21.15.12:0/2319218268 conn(0x7f63d00fce30 msgr2=0x7f63d00ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.217+0000 7f63d5810640 1 -- 172.21.15.12:0/2319218268 shutdown_connections 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.217+0000 7f63d5810640 1 -- 172.21.15.12:0/2319218268 wait complete. 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.218+0000 7f63d5810640 1 Processor -- start 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.218+0000 7f63d5810640 1 -- start start 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.218+0000 7f63d5810640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d01018a0 0x7f63d0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.218+0000 7f63d5810640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63d0102a90 0x7f63d01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.218+0000 7f63d5810640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63d0103450 0x7f63d019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.218+0000 7f63d5810640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f63d010d500 con 0x7f63d0103450 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.218+0000 7f63d5810640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63d010d380 con 0x7f63d01018a0 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.218+0000 7f63d5810640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f63d010d680 con 0x7f63d0102a90 2024-05-09T09:23:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.218+0000 7f63cf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63d0103450 0x7f63d019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63cf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63d0103450 0x7f63d019f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45538/0 (socket says 172.21.15.12:45538) 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63cf7fe640 1 -- 172.21.15.12:0/164698474 learned_addr learned my addr 172.21.15.12:0/164698474 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63ceffd640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d01018a0 0x7f63d0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63ce7fc640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63d0102a90 0x7f63d01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63cf7fe640 1 -- 172.21.15.12:0/164698474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63d0102a90 msgr2=0x7f63d01980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63cf7fe640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63d0102a90 0x7f63d01980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63cf7fe640 1 -- 172.21.15.12:0/164698474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d01018a0 msgr2=0x7f63d0197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63cf7fe640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d01018a0 0x7f63d0197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63cf7fe640 1 -- 172.21.15.12:0/164698474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d019f790 con 0x7f63d0103450 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63cf7fe640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63d0103450 0x7f63d019f120 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f63c000c970 tx=0x7f63c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63d480e640 1 -- 172.21.15.12:0/164698474 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0007bb0 con 0x7f63d0103450 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63d480e640 1 -- 172.21.15.12:0/164698474 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f63c0007d50 con 0x7f63d0103450 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63d5810640 1 -- 172.21.15.12:0/164698474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d019fa80 con 0x7f63d0103450 2024-05-09T09:23:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63d480e640 1 -- 172.21.15.12:0/164698474 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0018790 con 0x7f63d0103450 2024-05-09T09:23:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.219+0000 7f63d5810640 1 -- 172.21.15.12:0/164698474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d019ff50 con 0x7f63d0103450 2024-05-09T09:23:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.220+0000 7f63d5810640 1 -- 172.21.15.12:0/164698474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63d0108bc0 con 0x7f63d0103450 2024-05-09T09:23:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.221+0000 7f63d480e640 1 -- 172.21.15.12:0/164698474 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f63c000fab0 con 0x7f63d0103450 2024-05-09T09:23:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.222+0000 7f63d480e640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f63bc0a2b90 0x7f63bc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.222+0000 7f63d480e640 1 -- 172.21.15.12:0/164698474 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f63c00cf330 con 0x7f63d0103450 2024-05-09T09:23:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.222+0000 7f63ceffd640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f63bc0a2b90 0x7f63bc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.223+0000 7f63ceffd640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f63bc0a2b90 0x7f63bc0a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f63d01028f0 tx=0x7f63c8009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:28.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.227+0000 7f63d480e640 1 -- 172.21.15.12:0/164698474 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f63c0014030 con 0x7f63d0103450 2024-05-09T09:23:28.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.415+0000 7f63d5810640 1 -- 172.21.15.12:0/164698474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63d00624d0 con 0x7f63d0103450 2024-05-09T09:23:28.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.416+0000 7f63d480e640 1 -- 172.21.15.12:0/164698474 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24105 (secure 0 0 0) 0x7f63c009bff0 con 0x7f63d0103450 2024-05-09T09:23:28.416 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:28.416 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:23:14.726332+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:23:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.419+0000 7f63d5810640 1 -- 172.21.15.12:0/164698474 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f63bc0a2b90 msgr2=0x7f63bc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.419+0000 7f63d5810640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f63bc0a2b90 0x7f63bc0a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f63d01028f0 tx=0x7f63c8009680 comp rx=0 tx=0).stop 2024-05-09T09:23:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.419+0000 7f63d5810640 1 -- 172.21.15.12:0/164698474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63d0103450 msgr2=0x7f63d019f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.419+0000 7f63d5810640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63d0103450 0x7f63d019f120 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f63c000c970 tx=0x7f63c000ce30 comp rx=0 tx=0).stop 2024-05-09T09:23:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.419+0000 7f63d5810640 1 -- 172.21.15.12:0/164698474 shutdown_connections 2024-05-09T09:23:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.419+0000 7f63d5810640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f63bc0a2b90 0x7f63bc0a5050 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.419+0000 7f63d5810640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63d0103450 0x7f63d019f120 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.419+0000 7f63d5810640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63d0102a90 0x7f63d01980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.419+0000 7f63d5810640 1 --2- 172.21.15.12:0/164698474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d01018a0 0x7f63d0197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.419+0000 7f63d5810640 1 -- 172.21.15.12:0/164698474 >> 172.21.15.12:0/164698474 conn(0x7f63d00fce30 msgr2=0x7f63d0105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.419+0000 7f63d5810640 1 -- 172.21.15.12:0/164698474 shutdown_connections 2024-05-09T09:23:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.419+0000 7f63d5810640 1 -- 172.21.15.12:0/164698474 wait complete. 2024-05-09T09:23:28.430 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:23:28.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.535+0000 7f5529742640 1 -- 172.21.15.12:0/1056213018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5524100770 msgr2=0x7f5524100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:28.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.535+0000 7f5529742640 1 --2- 172.21.15.12:0/1056213018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5524100770 0x7f5524100b90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5514009ca0 tx=0x7f551402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.536+0000 7f5529742640 1 -- 172.21.15.12:0/1056213018 shutdown_connections 2024-05-09T09:23:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.536+0000 7f5529742640 1 --2- 172.21.15.12:0/1056213018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5524102320 0x7f5524108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.536+0000 7f5529742640 1 --2- 172.21.15.12:0/1056213018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5524101960 0x7f5524101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.536+0000 7f5529742640 1 --2- 172.21.15.12:0/1056213018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5524100770 0x7f5524100b90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.536+0000 7f5529742640 1 -- 172.21.15.12:0/1056213018 >> 172.21.15.12:0/1056213018 conn(0x7f55240fbce0 msgr2=0x7f55240fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.536+0000 7f5529742640 1 -- 172.21.15.12:0/1056213018 shutdown_connections 2024-05-09T09:23:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.536+0000 7f5529742640 1 -- 172.21.15.12:0/1056213018 wait complete. 2024-05-09T09:23:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.536+0000 7f5529742640 1 Processor -- start 2024-05-09T09:23:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.536+0000 7f5529742640 1 -- start start 2024-05-09T09:23:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.536+0000 7f5529742640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5524100770 0x7f5524104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.536+0000 7f5529742640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5524101960 0x7f5524102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.536+0000 7f5529742640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5524102320 0x7f5524103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.536+0000 7f5529742640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f552410c480 con 0x7f5524101960 2024-05-09T09:23:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f5529742640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f552410c300 con 0x7f5524102320 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f5529742640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f552410c600 con 0x7f5524100770 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f55227fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5524101960 0x7f5524102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f55227fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5524101960 0x7f5524102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45556/0 (socket says 172.21.15.12:45556) 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f55227fc640 1 -- 172.21.15.12:0/464769605 learned_addr learned my addr 172.21.15.12:0/464769605 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f55237fe640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5524102320 0x7f5524103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f5522ffd640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5524100770 0x7f5524104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f55227fc640 1 -- 172.21.15.12:0/464769605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5524100770 msgr2=0x7f5524104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f55227fc640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5524100770 0x7f5524104700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f55227fc640 1 -- 172.21.15.12:0/464769605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5524102320 msgr2=0x7f5524103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f55227fc640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5524102320 0x7f5524103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f55227fc640 1 -- 172.21.15.12:0/464769605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5524103b10 con 0x7f5524101960 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f55227fc640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5524101960 0x7f5524102d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f551800d950 tx=0x7f551800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f550bfff640 1 -- 172.21.15.12:0/464769605 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518017070 con 0x7f5524101960 2024-05-09T09:23:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f5529742640 1 -- 172.21.15.12:0/464769605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55241a1d40 con 0x7f5524101960 2024-05-09T09:23:28.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f550bfff640 1 -- 172.21.15.12:0/464769605 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5518009c30 con 0x7f5524101960 2024-05-09T09:23:28.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f550bfff640 1 -- 172.21.15.12:0/464769605 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551800f6b0 con 0x7f5524101960 2024-05-09T09:23:28.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.537+0000 7f5529742640 1 -- 172.21.15.12:0/464769605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55241a22a0 con 0x7f5524101960 2024-05-09T09:23:28.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.538+0000 7f5529742640 1 -- 172.21.15.12:0/464769605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55241a2530 con 0x7f5524101960 2024-05-09T09:23:28.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.540+0000 7f550bfff640 1 -- 172.21.15.12:0/464769605 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5518010a50 con 0x7f5524101960 2024-05-09T09:23:28.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.540+0000 7f550bfff640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f550c0a2b90 0x7f550c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:28.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.540+0000 7f550bfff640 1 -- 172.21.15.12:0/464769605 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f55180ceee0 con 0x7f5524101960 2024-05-09T09:23:28.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.540+0000 7f5522ffd640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f550c0a2b90 0x7f550c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:28.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.541+0000 7f5522ffd640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f550c0a2b90 0x7f550c0a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5514006010 tx=0x7f5514031ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:28.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.544+0000 7f550bfff640 1 -- 172.21.15.12:0/464769605 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5518009de0 con 0x7f5524101960 2024-05-09T09:23:28.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.732+0000 7f5529742640 1 -- 172.21.15.12:0/464769605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55241a27a0 con 0x7f5524101960 2024-05-09T09:23:28.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.733+0000 7f550bfff640 1 -- 172.21.15.12:0/464769605 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24105 (secure 0 0 0) 0x7f551809bc00 con 0x7f5524101960 2024-05-09T09:23:28.733 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:28.733 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:23:14.726332+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:23:28.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:28 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2424091312' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:23:28.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:28 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/164698474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:28.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.735+0000 7f5529742640 1 -- 172.21.15.12:0/464769605 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f550c0a2b90 msgr2=0x7f550c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:28.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.735+0000 7f5529742640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f550c0a2b90 0x7f550c0a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5514006010 tx=0x7f5514031ba0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.735+0000 7f5529742640 1 -- 172.21.15.12:0/464769605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5524101960 msgr2=0x7f5524102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:28.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.735+0000 7f5529742640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5524101960 0x7f5524102d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f551800d950 tx=0x7f551800de10 comp rx=0 tx=0).stop 2024-05-09T09:23:28.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.736+0000 7f5529742640 1 -- 172.21.15.12:0/464769605 shutdown_connections 2024-05-09T09:23:28.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.736+0000 7f5529742640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f550c0a2b90 0x7f550c0a5050 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.736+0000 7f5529742640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5524102320 0x7f5524103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.736+0000 7f5529742640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5524101960 0x7f5524102d40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.736+0000 7f5529742640 1 --2- 172.21.15.12:0/464769605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5524100770 0x7f5524104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.736+0000 7f5529742640 1 -- 172.21.15.12:0/464769605 >> 172.21.15.12:0/464769605 conn(0x7f55240fbce0 msgr2=0x7f55240fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:28.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.736+0000 7f5529742640 1 -- 172.21.15.12:0/464769605 shutdown_connections 2024-05-09T09:23:28.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.736+0000 7f5529742640 1 -- 172.21.15.12:0/464769605 wait complete. 2024-05-09T09:23:28.747 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T09:23:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.856+0000 7ff6ca64d640 1 -- 172.21.15.12:0/182440307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4102340 msgr2=0x7ff6c4108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.856+0000 7ff6ca64d640 1 --2- 172.21.15.12:0/182440307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4102340 0x7ff6c4108bd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009ca0 tx=0x7ff6bc02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.856+0000 7ff6ca64d640 1 -- 172.21.15.12:0/182440307 shutdown_connections 2024-05-09T09:23:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.856+0000 7ff6ca64d640 1 --2- 172.21.15.12:0/182440307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4102340 0x7ff6c4108bd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.856+0000 7ff6ca64d640 1 --2- 172.21.15.12:0/182440307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c4101980 0x7ff6c4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.856+0000 7ff6ca64d640 1 --2- 172.21.15.12:0/182440307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6c4100790 0x7ff6c4100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.856+0000 7ff6ca64d640 1 -- 172.21.15.12:0/182440307 >> 172.21.15.12:0/182440307 conn(0x7ff6c40fbd20 msgr2=0x7ff6c40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.856+0000 7ff6ca64d640 1 -- 172.21.15.12:0/182440307 shutdown_connections 2024-05-09T09:23:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.856+0000 7ff6ca64d640 1 -- 172.21.15.12:0/182440307 wait complete. 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6ca64d640 1 Processor -- start 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6ca64d640 1 -- start start 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6ca64d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4100790 0x7ff6c4199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6ca64d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6c4101980 0x7ff6c419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6ca64d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c4102340 0x7ff6c41a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6ca64d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410d500 con 0x7ff6c4100790 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6ca64d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410d380 con 0x7ff6c4102340 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6ca64d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410d680 con 0x7ff6c4101980 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6c3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4100790 0x7ff6c4199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6c3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4100790 0x7ff6c4199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45574/0 (socket says 172.21.15.12:45574) 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6c3fff640 1 -- 172.21.15.12:0/898644260 learned_addr learned my addr 172.21.15.12:0/898644260 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6bbfff640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6c4101980 0x7ff6c419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6c8bea640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c4102340 0x7ff6c41a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6c3fff640 1 -- 172.21.15.12:0/898644260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6c4101980 msgr2=0x7ff6c419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6c3fff640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6c4101980 0x7ff6c419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6c3fff640 1 -- 172.21.15.12:0/898644260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c4102340 msgr2=0x7ff6c41a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6c3fff640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c4102340 0x7ff6c41a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6c3fff640 1 -- 172.21.15.12:0/898644260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c41a1960 con 0x7ff6c4100790 2024-05-09T09:23:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6c8bea640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c4102340 0x7ff6c41a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:23:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.857+0000 7ff6bbfff640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6c4101980 0x7ff6c419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.858+0000 7ff6c3fff640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4100790 0x7ff6c4199d50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400c970 tx=0x7ff6b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.858+0000 7ff6c1ffb640 1 -- 172.21.15.12:0/898644260 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4004220 con 0x7ff6c4100790 2024-05-09T09:23:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.858+0000 7ff6c1ffb640 1 -- 172.21.15.12:0/898644260 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff6b40043c0 con 0x7ff6c4100790 2024-05-09T09:23:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.858+0000 7ff6ca64d640 1 -- 172.21.15.12:0/898644260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c41a1c50 con 0x7ff6c4100790 2024-05-09T09:23:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.858+0000 7ff6c1ffb640 1 -- 172.21.15.12:0/898644260 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4004a00 con 0x7ff6c4100790 2024-05-09T09:23:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.858+0000 7ff6ca64d640 1 -- 172.21.15.12:0/898644260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c41a2150 con 0x7ff6c4100790 2024-05-09T09:23:28.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.859+0000 7ff6ca64d640 1 -- 172.21.15.12:0/898644260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c41a2320 con 0x7ff6c4100790 2024-05-09T09:23:28.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.860+0000 7ff6c1ffb640 1 -- 172.21.15.12:0/898644260 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff6b4004ba0 con 0x7ff6c4100790 2024-05-09T09:23:28.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.861+0000 7ff6c1ffb640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff6b00a2b90 0x7ff6b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:28.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.861+0000 7ff6bbfff640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff6b00a2b90 0x7ff6b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:28.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.861+0000 7ff6c1ffb640 1 -- 172.21.15.12:0/898644260 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff6b40cf510 con 0x7ff6c4100790 2024-05-09T09:23:28.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.861+0000 7ff6bbfff640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff6b00a2b90 0x7ff6b00a5050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00abe0 tx=0x7ff6ac00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:28.865+0000 7ff6c1ffb640 1 -- 172.21.15.12:0/898644260 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff6b4014030 con 0x7ff6c4100790 2024-05-09T09:23:28.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:28 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2424091312' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:23:28.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:28 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/164698474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:29.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:28 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2424091312' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:23:29.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:28 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/164698474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:29.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.087+0000 7ff6ca64d640 1 -- 172.21.15.12:0/898644260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff6c41a24f0 con 0x7ff6c4100790 2024-05-09T09:23:29.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.629+0000 7ff6c1ffb640 1 -- 172.21.15.12:0/898644260 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7ff6b409c230 con 0x7ff6c4100790 2024-05-09T09:23:29.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.632+0000 7ff6ca64d640 1 -- 172.21.15.12:0/898644260 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff6b00a2b90 msgr2=0x7ff6b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:29.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.632+0000 7ff6ca64d640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff6b00a2b90 0x7ff6b00a5050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00abe0 tx=0x7ff6ac00a5c0 comp rx=0 tx=0).stop 2024-05-09T09:23:29.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.632+0000 7ff6ca64d640 1 -- 172.21.15.12:0/898644260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4100790 msgr2=0x7ff6c4199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:29.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.632+0000 7ff6ca64d640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4100790 0x7ff6c4199d50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400c970 tx=0x7ff6b400ce30 comp rx=0 tx=0).stop 2024-05-09T09:23:29.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.632+0000 7ff6ca64d640 1 -- 172.21.15.12:0/898644260 shutdown_connections 2024-05-09T09:23:29.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.632+0000 7ff6ca64d640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff6b00a2b90 0x7ff6b00a5050 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:29.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.632+0000 7ff6ca64d640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c4102340 0x7ff6c41a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:29.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.632+0000 7ff6ca64d640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6c4101980 0x7ff6c419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:29.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.632+0000 7ff6ca64d640 1 --2- 172.21.15.12:0/898644260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4100790 0x7ff6c4199d50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:29.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.632+0000 7ff6ca64d640 1 -- 172.21.15.12:0/898644260 >> 172.21.15.12:0/898644260 conn(0x7ff6c40fbd20 msgr2=0x7ff6c4109410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:29.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.632+0000 7ff6ca64d640 1 -- 172.21.15.12:0/898644260 shutdown_connections 2024-05-09T09:23:29.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.632+0000 7ff6ca64d640 1 -- 172.21.15.12:0/898644260 wait complete. 2024-05-09T09:23:29.645 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-09T09:23:29.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-05-09T09:23:29.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.761+0000 7fd3346bd640 1 -- 172.21.15.12:0/2177957150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd32c101890 msgr2=0x7fd32c101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.761+0000 7fd3346bd640 1 --2- 172.21.15.12:0/2177957150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd32c101890 0x7fd32c101cb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd328009900 tx=0x7fd32802a410 comp rx=0 tx=0).stop 2024-05-09T09:23:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.761+0000 7fd3346bd640 1 -- 172.21.15.12:0/2177957150 shutdown_connections 2024-05-09T09:23:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.761+0000 7fd3346bd640 1 --2- 172.21.15.12:0/2177957150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd32c103440 0x7fd32c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.761+0000 7fd3346bd640 1 --2- 172.21.15.12:0/2177957150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c102a80 0x7fd32c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.761+0000 7fd3346bd640 1 --2- 172.21.15.12:0/2177957150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd32c101890 0x7fd32c101cb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.761+0000 7fd3346bd640 1 -- 172.21.15.12:0/2177957150 >> 172.21.15.12:0/2177957150 conn(0x7fd32c0fce00 msgr2=0x7fd32c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.761+0000 7fd3346bd640 1 -- 172.21.15.12:0/2177957150 shutdown_connections 2024-05-09T09:23:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.761+0000 7fd3346bd640 1 -- 172.21.15.12:0/2177957150 wait complete. 2024-05-09T09:23:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd3346bd640 1 Processor -- start 2024-05-09T09:23:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd3346bd640 1 -- start start 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd3346bd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd32c101890 0x7fd32c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd3346bd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd32c102a80 0x7fd32c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd3346bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c103440 0x7fd32c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd3346bd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c10d4e0 con 0x7fd32c101890 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd3346bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c10d360 con 0x7fd32c103440 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd3346bd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c10d660 con 0x7fd32c102a80 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd332459640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd32c101890 0x7fd32c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd332459640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd32c101890 0x7fd32c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45596/0 (socket says 172.21.15.12:45596) 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd332459640 1 -- 172.21.15.12:0/1591430918 learned_addr learned my addr 172.21.15.12:0/1591430918 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd332c5a640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c103440 0x7fd32c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd331c58640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd32c102a80 0x7fd32c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd332459640 1 -- 172.21.15.12:0/1591430918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd32c102a80 msgr2=0x7fd32c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd332459640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd32c102a80 0x7fd32c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:29.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd332459640 1 -- 172.21.15.12:0/1591430918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c103440 msgr2=0x7fd32c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:29.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd332459640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c103440 0x7fd32c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:29.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.762+0000 7fd332459640 1 -- 172.21.15.12:0/1591430918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c19f770 con 0x7fd32c101890 2024-05-09T09:23:29.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.763+0000 7fd332459640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd32c101890 0x7fd32c197b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd328009230 tx=0x7fd328031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:29.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.763+0000 7fd3237fe640 1 -- 172.21.15.12:0/1591430918 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32803f030 con 0x7fd32c101890 2024-05-09T09:23:29.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.763+0000 7fd3237fe640 1 -- 172.21.15.12:0/1591430918 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd328003ac0 con 0x7fd32c101890 2024-05-09T09:23:29.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.763+0000 7fd3346bd640 1 -- 172.21.15.12:0/1591430918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c19fa00 con 0x7fd32c101890 2024-05-09T09:23:29.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.763+0000 7fd3237fe640 1 -- 172.21.15.12:0/1591430918 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd328035070 con 0x7fd32c101890 2024-05-09T09:23:29.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.763+0000 7fd3346bd640 1 -- 172.21.15.12:0/1591430918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c19fef0 con 0x7fd32c101890 2024-05-09T09:23:29.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.764+0000 7fd3346bd640 1 -- 172.21.15.12:0/1591430918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32c108bb0 con 0x7fd32c101890 2024-05-09T09:23:29.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.765+0000 7fd3237fe640 1 -- 172.21.15.12:0/1591430918 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd328042030 con 0x7fd32c101890 2024-05-09T09:23:29.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.766+0000 7fd3237fe640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd3180a2b90 0x7fd3180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:29.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.766+0000 7fd331c58640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd3180a2b90 0x7fd3180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:29.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.766+0000 7fd3237fe640 1 -- 172.21.15.12:0/1591430918 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd3280efcb0 con 0x7fd32c101890 2024-05-09T09:23:29.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.766+0000 7fd331c58640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd3180a2b90 0x7fd3180a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd32c199060 tx=0x7fd31c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:29.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.769+0000 7fd3237fe640 1 -- 172.21.15.12:0/1591430918 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd3280bc380 con 0x7fd32c101890 2024-05-09T09:23:29.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:29 smithi012 ceph-mon[30140]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-09T09:23:29.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:29 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/464769605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:29.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:29 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/898644260' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T09:23:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:29 smithi170 ceph-mon[33632]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-09T09:23:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:29 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/464769605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:29 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/898644260' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T09:23:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:29.997+0000 7fd3346bd640 1 -- 172.21.15.12:0/1591430918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fd32c02d350 con 0x7fd32c101890 2024-05-09T09:23:30.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:29 smithi016 ceph-mon[33335]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-09T09:23:30.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:29 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/464769605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:30.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:29 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/898644260' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T09:23:30.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.635+0000 7fd3237fe640 1 -- 172.21.15.12:0/1591430918 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fd3280c0680 con 0x7fd32c101890 2024-05-09T09:23:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.638+0000 7fd3346bd640 1 -- 172.21.15.12:0/1591430918 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd3180a2b90 msgr2=0x7fd3180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.638+0000 7fd3346bd640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd3180a2b90 0x7fd3180a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd32c199060 tx=0x7fd31c00b390 comp rx=0 tx=0).stop 2024-05-09T09:23:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.638+0000 7fd3346bd640 1 -- 172.21.15.12:0/1591430918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd32c101890 msgr2=0x7fd32c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.638+0000 7fd3346bd640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd32c101890 0x7fd32c197b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd328009230 tx=0x7fd328031ac0 comp rx=0 tx=0).stop 2024-05-09T09:23:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.638+0000 7fd3346bd640 1 -- 172.21.15.12:0/1591430918 shutdown_connections 2024-05-09T09:23:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.638+0000 7fd3346bd640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd3180a2b90 0x7fd3180a5050 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.638+0000 7fd3346bd640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c103440 0x7fd32c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.638+0000 7fd3346bd640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd32c102a80 0x7fd32c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.638+0000 7fd3346bd640 1 --2- 172.21.15.12:0/1591430918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd32c101890 0x7fd32c197b50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.638+0000 7fd3346bd640 1 -- 172.21.15.12:0/1591430918 >> 172.21.15.12:0/1591430918 conn(0x7fd32c0fce00 msgr2=0x7fd32c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.638+0000 7fd3346bd640 1 -- 172.21.15.12:0/1591430918 shutdown_connections 2024-05-09T09:23:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.638+0000 7fd3346bd640 1 -- 172.21.15.12:0/1591430918 wait complete. 2024-05-09T09:23:30.655 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-09T09:23:30.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.772+0000 7fb39423f640 1 -- 172.21.15.12:0/83268603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb38c100790 msgr2=0x7fb38c100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:30.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.772+0000 7fb39423f640 1 --2- 172.21.15.12:0/83268603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb38c100790 0x7fb38c100bb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb388009900 tx=0x7fb38802a410 comp rx=0 tx=0).stop 2024-05-09T09:23:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.772+0000 7fb39423f640 1 -- 172.21.15.12:0/83268603 shutdown_connections 2024-05-09T09:23:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.772+0000 7fb39423f640 1 --2- 172.21.15.12:0/83268603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c102340 0x7fb38c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.772+0000 7fb39423f640 1 --2- 172.21.15.12:0/83268603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb38c101980 0x7fb38c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.772+0000 7fb39423f640 1 --2- 172.21.15.12:0/83268603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb38c100790 0x7fb38c100bb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.772+0000 7fb39423f640 1 -- 172.21.15.12:0/83268603 >> 172.21.15.12:0/83268603 conn(0x7fb38c0fbd00 msgr2=0x7fb38c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.772+0000 7fb39423f640 1 -- 172.21.15.12:0/83268603 shutdown_connections 2024-05-09T09:23:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.772+0000 7fb39423f640 1 -- 172.21.15.12:0/83268603 wait complete. 2024-05-09T09:23:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.772+0000 7fb39423f640 1 Processor -- start 2024-05-09T09:23:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.772+0000 7fb39423f640 1 -- start start 2024-05-09T09:23:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb39423f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c100790 0x7fb38c197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb39423f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb38c101980 0x7fb38c198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb39423f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb38c102340 0x7fb38c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb39423f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c06a550 con 0x7fb38c101980 2024-05-09T09:23:30.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb39423f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c06a3d0 con 0x7fb38c100790 2024-05-09T09:23:30.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb39423f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c06a6d0 con 0x7fb38c102340 2024-05-09T09:23:30.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb3917da640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb38c101980 0x7fb38c198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:30.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb3917da640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb38c101980 0x7fb38c198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45610/0 (socket says 172.21.15.12:45610) 2024-05-09T09:23:30.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb3917da640 1 -- 172.21.15.12:0/2190585372 learned_addr learned my addr 172.21.15.12:0/2190585372 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:30.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb3927dc640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb38c102340 0x7fb38c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:30.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb391fdb640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c100790 0x7fb38c197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:30.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb3917da640 1 -- 172.21.15.12:0/2190585372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb38c102340 msgr2=0x7fb38c19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:30.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb3917da640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb38c102340 0x7fb38c19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:30.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb3917da640 1 -- 172.21.15.12:0/2190585372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c100790 msgr2=0x7fb38c197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:30.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb3917da640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c100790 0x7fb38c197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:30.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb3917da640 1 -- 172.21.15.12:0/2190585372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38c19f830 con 0x7fb38c101980 2024-05-09T09:23:30.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb3917da640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb38c101980 0x7fb38c198160 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00d770 tx=0x7fb37c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:30.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.773+0000 7fb382ffd640 1 -- 172.21.15.12:0/2190585372 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c017070 con 0x7fb38c101980 2024-05-09T09:23:30.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.774+0000 7fb39423f640 1 -- 172.21.15.12:0/2190585372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38c19fb20 con 0x7fb38c101980 2024-05-09T09:23:30.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.774+0000 7fb382ffd640 1 -- 172.21.15.12:0/2190585372 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb37c0054f0 con 0x7fb38c101980 2024-05-09T09:23:30.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.774+0000 7fb382ffd640 1 -- 172.21.15.12:0/2190585372 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c004d60 con 0x7fb38c101980 2024-05-09T09:23:30.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.774+0000 7fb39423f640 1 -- 172.21.15.12:0/2190585372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38c1a0040 con 0x7fb38c101980 2024-05-09T09:23:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.775+0000 7fb39423f640 1 -- 172.21.15.12:0/2190585372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38c107b30 con 0x7fb38c101980 2024-05-09T09:23:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.776+0000 7fb382ffd640 1 -- 172.21.15.12:0/2190585372 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb37c004f00 con 0x7fb38c101980 2024-05-09T09:23:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.777+0000 7fb382ffd640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb3780a2b90 0x7fb3780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.777+0000 7fb382ffd640 1 -- 172.21.15.12:0/2190585372 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb37c0cf0b0 con 0x7fb38c101980 2024-05-09T09:23:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.777+0000 7fb391fdb640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb3780a2b90 0x7fb3780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.777+0000 7fb391fdb640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb3780a2b90 0x7fb3780a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb388004770 tx=0x7fb388031530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:30.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:30.780+0000 7fb382ffd640 1 -- 172.21.15.12:0/2190585372 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb37c09b780 con 0x7fb38c101980 2024-05-09T09:23:30.954 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:30 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/898644260' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T09:23:30.954 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:30 smithi012 ceph-mon[30140]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:30.954 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:30 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1591430918' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T09:23:30.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:30 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/898644260' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T09:23:30.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:30 smithi170 ceph-mon[33632]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:30.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:30 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1591430918' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T09:23:31.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:30 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/898644260' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T09:23:31.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:30 smithi016 ceph-mon[33335]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:31.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:30 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1591430918' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T09:23:31.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.009+0000 7fb39423f640 1 -- 172.21.15.12:0/2190585372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fb38c02d1c0 con 0x7fb38c101980 2024-05-09T09:23:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.645+0000 7fb382ffd640 1 -- 172.21.15.12:0/2190585372 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fb37c09fa80 con 0x7fb38c101980 2024-05-09T09:23:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.649+0000 7fb39423f640 1 -- 172.21.15.12:0/2190585372 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb3780a2b90 msgr2=0x7fb3780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.649+0000 7fb39423f640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb3780a2b90 0x7fb3780a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb388004770 tx=0x7fb388031530 comp rx=0 tx=0).stop 2024-05-09T09:23:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.649+0000 7fb39423f640 1 -- 172.21.15.12:0/2190585372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb38c101980 msgr2=0x7fb38c198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.649+0000 7fb39423f640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb38c101980 0x7fb38c198160 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00d770 tx=0x7fb37c00dc30 comp rx=0 tx=0).stop 2024-05-09T09:23:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.650+0000 7fb39423f640 1 -- 172.21.15.12:0/2190585372 shutdown_connections 2024-05-09T09:23:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.650+0000 7fb39423f640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb3780a2b90 0x7fb3780a5050 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.650+0000 7fb39423f640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb38c102340 0x7fb38c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.650+0000 7fb39423f640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb38c101980 0x7fb38c198160 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.650+0000 7fb39423f640 1 --2- 172.21.15.12:0/2190585372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c100790 0x7fb38c197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.650+0000 7fb39423f640 1 -- 172.21.15.12:0/2190585372 >> 172.21.15.12:0/2190585372 conn(0x7fb38c0fbd00 msgr2=0x7fb38c104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.650+0000 7fb39423f640 1 -- 172.21.15.12:0/2190585372 shutdown_connections 2024-05-09T09:23:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.650+0000 7fb39423f640 1 -- 172.21.15.12:0/2190585372 wait complete. 2024-05-09T09:23:31.668 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-09T09:23:31.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.783+0000 7f2c8a488640 1 -- 172.21.15.12:0/3691815707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c84103590 msgr2=0x7f2c84109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:31.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.783+0000 7f2c8a488640 1 --2- 172.21.15.12:0/3691815707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c84103590 0x7f2c84109e20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009cd0 tx=0x7f2c7c02a8e0 comp rx=0 tx=0).stop 2024-05-09T09:23:31.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.783+0000 7f2c8a488640 1 -- 172.21.15.12:0/3691815707 shutdown_connections 2024-05-09T09:23:31.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.783+0000 7f2c8a488640 1 --2- 172.21.15.12:0/3691815707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c84103590 0x7f2c84109e20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:31.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.783+0000 7f2c8a488640 1 --2- 172.21.15.12:0/3691815707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c84102bd0 0x7f2c84103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:31.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.783+0000 7f2c8a488640 1 --2- 172.21.15.12:0/3691815707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840ffb60 0x7f2c840fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:31.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.783+0000 7f2c8a488640 1 -- 172.21.15.12:0/3691815707 >> 172.21.15.12:0/3691815707 conn(0x7f2c840fb470 msgr2=0x7f2c840fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:31.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.783+0000 7f2c8a488640 1 -- 172.21.15.12:0/3691815707 shutdown_connections 2024-05-09T09:23:31.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.783+0000 7f2c8a488640 1 -- 172.21.15.12:0/3691815707 wait complete. 2024-05-09T09:23:31.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c8a488640 1 Processor -- start 2024-05-09T09:23:31.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c8a488640 1 -- start start 2024-05-09T09:23:31.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c8a488640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840ffb60 0x7f2c84102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:31.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c8a488640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c84102bd0 0x7f2c84100b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:31.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c8a488640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c84103590 0x7f2c841010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:31.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c8a488640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8410d4f0 con 0x7f2c84103590 2024-05-09T09:23:31.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c8a488640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8410d370 con 0x7f2c840ffb60 2024-05-09T09:23:31.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c8a488640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8410d670 con 0x7f2c84102bd0 2024-05-09T09:23:31.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c88a25640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c84103590 0x7f2c841010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:31.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c88a25640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c84103590 0x7f2c841010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55932/0 (socket says 172.21.15.12:55932) 2024-05-09T09:23:31.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c88a25640 1 -- 172.21.15.12:0/941327036 learned_addr learned my addr 172.21.15.12:0/941327036 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:31.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c83fff640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840ffb60 0x7f2c84102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:31.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c7bfff640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c84102bd0 0x7f2c84100b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:31.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c88a25640 1 -- 172.21.15.12:0/941327036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c84102bd0 msgr2=0x7f2c84100b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c88a25640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c84102bd0 0x7f2c84100b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c88a25640 1 -- 172.21.15.12:0/941327036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840ffb60 msgr2=0x7f2c84102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c88a25640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840ffb60 0x7f2c84102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c88a25640 1 -- 172.21.15.12:0/941327036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c84101950 con 0x7f2c84103590 2024-05-09T09:23:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.784+0000 7f2c7bfff640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c84102bd0 0x7f2c84100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.785+0000 7f2c88a25640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c84103590 0x7f2c841010c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009ca0 tx=0x7f2c7c003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.785+0000 7f2c81ffb640 1 -- 172.21.15.12:0/941327036 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c004040 con 0x7f2c84103590 2024-05-09T09:23:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.785+0000 7f2c81ffb640 1 -- 172.21.15.12:0/941327036 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2c7c0046b0 con 0x7f2c84103590 2024-05-09T09:23:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.785+0000 7f2c8a488640 1 -- 172.21.15.12:0/941327036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c841a1bc0 con 0x7f2c84103590 2024-05-09T09:23:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.785+0000 7f2c81ffb640 1 -- 172.21.15.12:0/941327036 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c03a420 con 0x7f2c84103590 2024-05-09T09:23:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.785+0000 7f2c8a488640 1 -- 172.21.15.12:0/941327036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c841a2110 con 0x7f2c84103590 2024-05-09T09:23:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.786+0000 7f2c8a488640 1 -- 172.21.15.12:0/941327036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c84108c30 con 0x7f2c84103590 2024-05-09T09:23:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.787+0000 7f2c81ffb640 1 -- 172.21.15.12:0/941327036 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2c7c030730 con 0x7f2c84103590 2024-05-09T09:23:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.788+0000 7f2c81ffb640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2c700a2b90 0x7f2c700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.788+0000 7f2c81ffb640 1 -- 172.21.15.12:0/941327036 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f2c7c02e030 con 0x7f2c84103590 2024-05-09T09:23:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.788+0000 7f2c83fff640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2c700a2b90 0x7f2c700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:31.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.788+0000 7f2c83fff640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2c700a2b90 0x7f2c700a5050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2c84069bf0 tx=0x7f2c74005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:31.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:31.792+0000 7f2c81ffb640 1 -- 172.21.15.12:0/941327036 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2c7c0bd0f0 con 0x7f2c84103590 2024-05-09T09:23:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:31 smithi012 ceph-mon[30140]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T09:23:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:31 smithi012 ceph-mon[30140]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T09:23:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:31 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1591430918' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T09:23:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:31 smithi012 ceph-mon[30140]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T09:23:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:31 smithi012 ceph-mon[30140]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:31 smithi012 ceph-mon[30140]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-05-09T09:23:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:31 smithi012 ceph-mon[30140]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-09T09:23:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:31 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2190585372' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T09:23:31.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:31 smithi170 ceph-mon[33632]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T09:23:31.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:31 smithi170 ceph-mon[33632]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T09:23:31.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:31 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1591430918' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T09:23:31.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:31 smithi170 ceph-mon[33632]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T09:23:31.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:31 smithi170 ceph-mon[33632]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:31.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:31 smithi170 ceph-mon[33632]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-05-09T09:23:31.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:31 smithi170 ceph-mon[33632]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-09T09:23:31.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:31 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2190585372' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T09:23:32.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:31 smithi016 ceph-mon[33335]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T09:23:32.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:31 smithi016 ceph-mon[33335]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T09:23:32.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:31 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1591430918' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T09:23:32.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:31 smithi016 ceph-mon[33335]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T09:23:32.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:31 smithi016 ceph-mon[33335]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T09:23:32.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:31 smithi016 ceph-mon[33335]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-05-09T09:23:32.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:31 smithi016 ceph-mon[33335]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-09T09:23:32.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:31 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2190585372' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T09:23:32.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.019+0000 7f2c8a488640 1 -- 172.21.15.12:0/941327036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f2c8402d1c0 con 0x7f2c84103590 2024-05-09T09:23:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.660+0000 7f2c81ffb640 1 -- 172.21.15.12:0/941327036 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f2c7c0bc9f0 con 0x7f2c84103590 2024-05-09T09:23:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.663+0000 7f2c8a488640 1 -- 172.21.15.12:0/941327036 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2c700a2b90 msgr2=0x7f2c700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.663+0000 7f2c8a488640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2c700a2b90 0x7f2c700a5050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2c84069bf0 tx=0x7f2c74005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.663+0000 7f2c8a488640 1 -- 172.21.15.12:0/941327036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c84103590 msgr2=0x7f2c841010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.663+0000 7f2c8a488640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c84103590 0x7f2c841010c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009ca0 tx=0x7f2c7c003b80 comp rx=0 tx=0).stop 2024-05-09T09:23:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.663+0000 7f2c8a488640 1 -- 172.21.15.12:0/941327036 shutdown_connections 2024-05-09T09:23:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.663+0000 7f2c8a488640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2c700a2b90 0x7f2c700a5050 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.663+0000 7f2c8a488640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c84103590 0x7f2c841010c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.663+0000 7f2c8a488640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c84102bd0 0x7f2c84100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.663+0000 7f2c8a488640 1 --2- 172.21.15.12:0/941327036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840ffb60 0x7f2c84102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.663+0000 7f2c8a488640 1 -- 172.21.15.12:0/941327036 >> 172.21.15.12:0/941327036 conn(0x7f2c840fb470 msgr2=0x7f2c84105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.663+0000 7f2c8a488640 1 -- 172.21.15.12:0/941327036 shutdown_connections 2024-05-09T09:23:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.663+0000 7f2c8a488640 1 -- 172.21.15.12:0/941327036 wait complete. 2024-05-09T09:23:32.680 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-09T09:23:32.680 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-09T09:23:32.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.794+0000 7f7943dee640 1 -- 172.21.15.12:0/3775771835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f793c101400 msgr2=0x7f793c101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:32.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.794+0000 7f7943dee640 1 --2- 172.21.15.12:0/3775771835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f793c101400 0x7f793c101880 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7934009860 tx=0x7f793402a340 comp rx=0 tx=0).stop 2024-05-09T09:23:32.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 -- 172.21.15.12:0/3775771835 shutdown_connections 2024-05-09T09:23:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 --2- 172.21.15.12:0/3775771835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f793c101400 0x7f793c101880 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 --2- 172.21.15.12:0/3775771835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c100a60 0x7f793c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 --2- 172.21.15.12:0/3775771835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f793c100100 0x7f793c100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 -- 172.21.15.12:0/3775771835 >> 172.21.15.12:0/3775771835 conn(0x7f793c0fbd20 msgr2=0x7f793c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 -- 172.21.15.12:0/3775771835 shutdown_connections 2024-05-09T09:23:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 -- 172.21.15.12:0/3775771835 wait complete. 2024-05-09T09:23:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 Processor -- start 2024-05-09T09:23:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 -- start start 2024-05-09T09:23:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c100100 0x7f793c199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f793c100a60 0x7f793c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f793c101400 0x7f793c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f793c10f700 con 0x7f793c101400 2024-05-09T09:23:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f793c10f580 con 0x7f793c100100 2024-05-09T09:23:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.795+0000 7f7943dee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f793c10f880 con 0x7f793c100a60 2024-05-09T09:23:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f794238b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f793c101400 0x7f793c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f794238b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f793c101400 0x7f793c1a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55952/0 (socket says 172.21.15.12:55952) 2024-05-09T09:23:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f794238b640 1 -- 172.21.15.12:0/1927765048 learned_addr learned my addr 172.21.15.12:0/1927765048 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f7941389640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f793c100a60 0x7f793c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f7941b8a640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c100100 0x7f793c199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f794238b640 1 -- 172.21.15.12:0/1927765048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f793c100a60 msgr2=0x7f793c19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f794238b640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f793c100a60 0x7f793c19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f794238b640 1 -- 172.21.15.12:0/1927765048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c100100 msgr2=0x7f793c199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f794238b640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c100100 0x7f793c199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f794238b640 1 -- 172.21.15.12:0/1927765048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c1a1940 con 0x7f793c101400 2024-05-09T09:23:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f7941b8a640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c100100 0x7f793c199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:23:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f794238b640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f793c101400 0x7f793c1a12a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7934030580 tx=0x7f7934032740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:32.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f7932ffd640 1 -- 172.21.15.12:0/1927765048 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934037040 con 0x7f793c101400 2024-05-09T09:23:32.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f7943dee640 1 -- 172.21.15.12:0/1927765048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793c1a1bd0 con 0x7f793c101400 2024-05-09T09:23:32.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f7932ffd640 1 -- 172.21.15.12:0/1927765048 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7934030820 con 0x7f793c101400 2024-05-09T09:23:32.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f7932ffd640 1 -- 172.21.15.12:0/1927765048 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934004920 con 0x7f793c101400 2024-05-09T09:23:32.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.796+0000 7f7943dee640 1 -- 172.21.15.12:0/1927765048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f793c1a2110 con 0x7f793c101400 2024-05-09T09:23:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.797+0000 7f7943dee640 1 -- 172.21.15.12:0/1927765048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f793c068aa0 con 0x7f793c101400 2024-05-09T09:23:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.799+0000 7f7932ffd640 1 -- 172.21.15.12:0/1927765048 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7934005170 con 0x7f793c101400 2024-05-09T09:23:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.800+0000 7f7932ffd640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f79280a2b90 0x7f79280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.800+0000 7f7932ffd640 1 -- 172.21.15.12:0/1927765048 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f79340efae0 con 0x7f793c101400 2024-05-09T09:23:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.800+0000 7f7941b8a640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f79280a2b90 0x7f79280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.801+0000 7f7941b8a640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f79280a2b90 0x7f79280a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f793c19b200 tx=0x7f79380095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:32.804+0000 7f7932ffd640 1 -- 172.21.15.12:0/1927765048 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f79340bc7a0 con 0x7f793c101400 2024-05-09T09:23:32.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:32 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2190585372' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T09:23:32.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:32 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] up:active 2024-05-09T09:23:32.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:32 smithi012 ceph-mon[30140]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T09:23:32.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:32 smithi012 ceph-mon[30140]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-05-09T09:23:32.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:32 smithi012 ceph-mon[30140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-05-09T09:23:32.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:32 smithi012 ceph-mon[30140]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-09T09:23:32.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:32 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/941327036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T09:23:32.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:32 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2190585372' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T09:23:32.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:32 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] up:active 2024-05-09T09:23:32.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:32 smithi170 ceph-mon[33632]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T09:23:32.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:32 smithi170 ceph-mon[33632]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-05-09T09:23:32.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:32 smithi170 ceph-mon[33632]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-05-09T09:23:32.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:32 smithi170 ceph-mon[33632]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-09T09:23:32.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:32 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/941327036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T09:23:33.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:32 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2190585372' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T09:23:33.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:32 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] up:active 2024-05-09T09:23:33.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:32 smithi016 ceph-mon[33335]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T09:23:33.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:32 smithi016 ceph-mon[33335]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-05-09T09:23:33.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:32 smithi016 ceph-mon[33335]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-05-09T09:23:33.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:32 smithi016 ceph-mon[33335]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-09T09:23:33.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:32 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/941327036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T09:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.014+0000 7f7943dee640 1 -- 172.21.15.12:0/1927765048 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f793c061d30 con 0x7f79280a2b90 2024-05-09T09:23:33.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.036+0000 7f7932ffd640 1 -- 172.21.15.12:0/1927765048 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f793c061d30 con 0x7f79280a2b90 2024-05-09T09:23:33.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.038+0000 7f7943dee640 1 -- 172.21.15.12:0/1927765048 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f79280a2b90 msgr2=0x7f79280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.038+0000 7f7943dee640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f79280a2b90 0x7f79280a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f793c19b200 tx=0x7f79380095c0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.039+0000 7f7943dee640 1 -- 172.21.15.12:0/1927765048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f793c101400 msgr2=0x7f793c1a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.039+0000 7f7943dee640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f793c101400 0x7f793c1a12a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7934030580 tx=0x7f7934032740 comp rx=0 tx=0).stop 2024-05-09T09:23:33.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.039+0000 7f7943dee640 1 -- 172.21.15.12:0/1927765048 shutdown_connections 2024-05-09T09:23:33.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.039+0000 7f7943dee640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f79280a2b90 0x7f79280a5050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.039+0000 7f7943dee640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f793c101400 0x7f793c1a12a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.039+0000 7f7943dee640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f793c100a60 0x7f793c19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.039+0000 7f7943dee640 1 --2- 172.21.15.12:0/1927765048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c100100 0x7f793c199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.039+0000 7f7943dee640 1 -- 172.21.15.12:0/1927765048 >> 172.21.15.12:0/1927765048 conn(0x7f793c0fbd20 msgr2=0x7f793c0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:33.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.039+0000 7f7943dee640 1 -- 172.21.15.12:0/1927765048 shutdown_connections 2024-05-09T09:23:33.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.039+0000 7f7943dee640 1 -- 172.21.15.12:0/1927765048 wait complete. 2024-05-09T09:23:33.049 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-09T09:23:33.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.161+0000 7fbfd8970640 1 -- 172.21.15.12:0/94086734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfd4101960 msgr2=0x7fbfd4101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.161+0000 7fbfd8970640 1 --2- 172.21.15.12:0/94086734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfd4101960 0x7fbfd4101de0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009900 tx=0x7fbfbc02a410 comp rx=0 tx=0).stop 2024-05-09T09:23:33.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.161+0000 7fbfd8970640 1 -- 172.21.15.12:0/94086734 shutdown_connections 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.161+0000 7fbfd8970640 1 --2- 172.21.15.12:0/94086734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd4102320 0x7fbfd4108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.161+0000 7fbfd8970640 1 --2- 172.21.15.12:0/94086734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfd4101960 0x7fbfd4101de0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.161+0000 7fbfd8970640 1 --2- 172.21.15.12:0/94086734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd4100770 0x7fbfd4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.161+0000 7fbfd8970640 1 -- 172.21.15.12:0/94086734 >> 172.21.15.12:0/94086734 conn(0x7fbfd40fbce0 msgr2=0x7fbfd40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.161+0000 7fbfd8970640 1 -- 172.21.15.12:0/94086734 shutdown_connections 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.161+0000 7fbfd8970640 1 -- 172.21.15.12:0/94086734 wait complete. 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.161+0000 7fbfd8970640 1 Processor -- start 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.161+0000 7fbfd8970640 1 -- start start 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd8970640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd4100770 0x7fbfd4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd8970640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd4101960 0x7fbfd419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd8970640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfd4102320 0x7fbfd41a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd8970640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd410c3d0 con 0x7fbfd4102320 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd8970640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd410c250 con 0x7fbfd4100770 2024-05-09T09:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd8970640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd410c550 con 0x7fbfd4101960 2024-05-09T09:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfd4102320 0x7fbfd41a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfd4102320 0x7fbfd41a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55994/0 (socket says 172.21.15.12:55994) 2024-05-09T09:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd2d9d640 1 -- 172.21.15.12:0/2963256588 learned_addr learned my addr 172.21.15.12:0/2963256588 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfcbfff640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd4101960 0x7fbfd419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd259c640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd4100770 0x7fbfd4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd2d9d640 1 -- 172.21.15.12:0/2963256588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd4101960 msgr2=0x7fbfd419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd2d9d640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd4101960 0x7fbfd419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd2d9d640 1 -- 172.21.15.12:0/2963256588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd4100770 msgr2=0x7fbfd4069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd2d9d640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd4100770 0x7fbfd4069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd2d9d640 1 -- 172.21.15.12:0/2963256588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd41a1a40 con 0x7fbfd4102320 2024-05-09T09:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd259c640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd4100770 0x7fbfd4069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd2d9d640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfd4102320 0x7fbfd41a1250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00eaa0 tx=0x7fbfcc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfcb7fe640 1 -- 172.21.15.12:0/2963256588 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00cdb0 con 0x7fbfd4102320 2024-05-09T09:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd8970640 1 -- 172.21.15.12:0/2963256588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd41a1cd0 con 0x7fbfd4102320 2024-05-09T09:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfcb7fe640 1 -- 172.21.15.12:0/2963256588 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbfcc005020 con 0x7fbfd4102320 2024-05-09T09:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfcb7fe640 1 -- 172.21.15.12:0/2963256588 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc0116b0 con 0x7fbfd4102320 2024-05-09T09:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.162+0000 7fbfd8970640 1 -- 172.21.15.12:0/2963256588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd41a21a0 con 0x7fbfd4102320 2024-05-09T09:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.163+0000 7fbfd8970640 1 -- 172.21.15.12:0/2963256588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd4107b60 con 0x7fbfd4102320 2024-05-09T09:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.165+0000 7fbfcb7fe640 1 -- 172.21.15.12:0/2963256588 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbfcc0051d0 con 0x7fbfd4102320 2024-05-09T09:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.165+0000 7fbfcb7fe640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbfc00a2bc0 0x7fbfc00a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.165+0000 7fbfcb7fe640 1 -- 172.21.15.12:0/2963256588 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fbfcc0cf850 con 0x7fbfd4102320 2024-05-09T09:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.165+0000 7fbfd259c640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbfc00a2bc0 0x7fbfc00a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.166+0000 7fbfd259c640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbfc00a2bc0 0x7fbfc00a5080 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbfd41017c0 tx=0x7fbfc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.169+0000 7fbfcb7fe640 1 -- 172.21.15.12:0/2963256588 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbfcc016030 con 0x7fbfd4102320 2024-05-09T09:23:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.385+0000 7fbfd8970640 1 -- 172.21.15.12:0/2963256588 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbfd4061d00 con 0x7fbfc00a2bc0 2024-05-09T09:23:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.409+0000 7fbfcb7fe640 1 -- 172.21.15.12:0/2963256588 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbfd4061d00 con 0x7fbfc00a2bc0 2024-05-09T09:23:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.411+0000 7fbfd8970640 1 -- 172.21.15.12:0/2963256588 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbfc00a2bc0 msgr2=0x7fbfc00a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.411+0000 7fbfd8970640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbfc00a2bc0 0x7fbfc00a5080 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbfd41017c0 tx=0x7fbfc4005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.411+0000 7fbfd8970640 1 -- 172.21.15.12:0/2963256588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfd4102320 msgr2=0x7fbfd41a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.411+0000 7fbfd8970640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfd4102320 0x7fbfd41a1250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00eaa0 tx=0x7fbfcc00ef60 comp rx=0 tx=0).stop 2024-05-09T09:23:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.411+0000 7fbfd8970640 1 -- 172.21.15.12:0/2963256588 shutdown_connections 2024-05-09T09:23:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.411+0000 7fbfd8970640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbfc00a2bc0 0x7fbfc00a5080 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.411+0000 7fbfd8970640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfd4102320 0x7fbfd41a1250 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.411+0000 7fbfd8970640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd4101960 0x7fbfd419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.411+0000 7fbfd8970640 1 --2- 172.21.15.12:0/2963256588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd4100770 0x7fbfd4069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.411+0000 7fbfd8970640 1 -- 172.21.15.12:0/2963256588 >> 172.21.15.12:0/2963256588 conn(0x7fbfd40fbce0 msgr2=0x7fbfd4104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.411+0000 7fbfd8970640 1 -- 172.21.15.12:0/2963256588 shutdown_connections 2024-05-09T09:23:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.411+0000 7fbfd8970640 1 -- 172.21.15.12:0/2963256588 wait complete. 2024-05-09T09:23:33.420 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:23:33 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:23:33.019+0000 7f03c5367700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T09:23:33.421 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:23:33 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:23:33.019+0000 7f03c5367700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T09:23:33.421 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:23:33 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:23:33.019+0000 7f03c5367700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T09:23:33.421 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:23:33 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:23:33.019+0000 7f03c5367700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T09:23:33.421 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:23:33 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:23:33.019+0000 7f03c5367700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T09:23:33.422 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-09T09:23:33.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.534+0000 7fc6cdf89640 1 -- 172.21.15.12:0/46086127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6c8101890 msgr2=0x7fc6c8101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.534+0000 7fc6cdf89640 1 --2- 172.21.15.12:0/46086127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6c8101890 0x7fc6c8101cb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009900 tx=0x7fc6b802a410 comp rx=0 tx=0).stop 2024-05-09T09:23:33.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.534+0000 7fc6cdf89640 1 -- 172.21.15.12:0/46086127 shutdown_connections 2024-05-09T09:23:33.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.534+0000 7fc6cdf89640 1 --2- 172.21.15.12:0/46086127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c8103440 0x7fc6c8109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.534+0000 7fc6cdf89640 1 --2- 172.21.15.12:0/46086127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6c8102a80 0x7fc6c8102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.534+0000 7fc6cdf89640 1 --2- 172.21.15.12:0/46086127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6c8101890 0x7fc6c8101cb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.534+0000 7fc6cdf89640 1 -- 172.21.15.12:0/46086127 >> 172.21.15.12:0/46086127 conn(0x7fc6c80fce00 msgr2=0x7fc6c80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:33.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.534+0000 7fc6cdf89640 1 -- 172.21.15.12:0/46086127 shutdown_connections 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.534+0000 7fc6cdf89640 1 -- 172.21.15.12:0/46086127 wait complete. 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.535+0000 7fc6cdf89640 1 Processor -- start 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.535+0000 7fc6cdf89640 1 -- start start 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.535+0000 7fc6cdf89640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6c8101890 0x7fc6c8197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.535+0000 7fc6cdf89640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6c8102a80 0x7fc6c8198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.535+0000 7fc6cdf89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c8103440 0x7fc6c819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.535+0000 7fc6cdf89640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c810d4f0 con 0x7fc6c8101890 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.535+0000 7fc6cdf89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c810d370 con 0x7fc6c8103440 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.535+0000 7fc6cdf89640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c810d670 con 0x7fc6c8102a80 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.535+0000 7fc6c77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6c8101890 0x7fc6c8197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.535+0000 7fc6c77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6c8101890 0x7fc6c8197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56020/0 (socket says 172.21.15.12:56020) 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.535+0000 7fc6c77fe640 1 -- 172.21.15.12:0/3800855471 learned_addr learned my addr 172.21.15.12:0/3800855471 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.535+0000 7fc6c6ffd640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6c8102a80 0x7fc6c8198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.535+0000 7fc6c7fff640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c8103440 0x7fc6c819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.536+0000 7fc6c77fe640 1 -- 172.21.15.12:0/3800855471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6c8102a80 msgr2=0x7fc6c8198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.536+0000 7fc6c77fe640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6c8102a80 0x7fc6c8198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.536+0000 7fc6c77fe640 1 -- 172.21.15.12:0/3800855471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c8103440 msgr2=0x7fc6c819f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.536+0000 7fc6c77fe640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c8103440 0x7fc6c819f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.536+0000 7fc6c77fe640 1 -- 172.21.15.12:0/3800855471 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c819f7f0 con 0x7fc6c8101890 2024-05-09T09:23:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.536+0000 7fc6c77fe640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6c8101890 0x7fc6c8197bd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8005ab0 tx=0x7fc6b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.536+0000 7fc6c4ff9640 1 -- 172.21.15.12:0/3800855471 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b803f030 con 0x7fc6c8101890 2024-05-09T09:23:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.536+0000 7fc6c4ff9640 1 -- 172.21.15.12:0/3800855471 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc6b8003ac0 con 0x7fc6c8101890 2024-05-09T09:23:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.536+0000 7fc6cdf89640 1 -- 172.21.15.12:0/3800855471 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c819fa80 con 0x7fc6c8101890 2024-05-09T09:23:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.536+0000 7fc6c4ff9640 1 -- 172.21.15.12:0/3800855471 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8035070 con 0x7fc6c8101890 2024-05-09T09:23:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.536+0000 7fc6cdf89640 1 -- 172.21.15.12:0/3800855471 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c819ff70 con 0x7fc6c8101890 2024-05-09T09:23:33.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.537+0000 7fc6cdf89640 1 -- 172.21.15.12:0/3800855471 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c8108c30 con 0x7fc6c8101890 2024-05-09T09:23:33.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.538+0000 7fc6c4ff9640 1 -- 172.21.15.12:0/3800855471 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc6b8042030 con 0x7fc6c8101890 2024-05-09T09:23:33.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.539+0000 7fc6c4ff9640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc6b00a2b90 0x7fc6b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:33.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.539+0000 7fc6c4ff9640 1 -- 172.21.15.12:0/3800855471 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc6b80ef860 con 0x7fc6c8101890 2024-05-09T09:23:33.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.539+0000 7fc6c6ffd640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc6b00a2b90 0x7fc6b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:33.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.539+0000 7fc6c6ffd640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc6b00a2b90 0x7fc6b00a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc6c81990e0 tx=0x7fc6bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.543+0000 7fc6c4ff9640 1 -- 172.21.15.12:0/3800855471 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc6b80bbed0 con 0x7fc6c8101890 2024-05-09T09:23:33.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 1 op/s 2024-05-09T09:23:33.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/941327036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T09:23:33.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] up:active 2024-05-09T09:23:33.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T09:23:33.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-09T09:23:33.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-05-09T09:23:33.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-09T09:23:33.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:33.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] up:active 2024-05-09T09:23:33.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T09:23:33.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T09:23:33.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: Cluster is now healthy 2024-05-09T09:23:33.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2024-05-09T09:23:33.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-05-09T09:23:33.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:33 smithi012 ceph-mon[30140]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-05-09T09:23:33.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.769+0000 7fc6cdf89640 1 -- 172.21.15.12:0/3800855471 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fc6c8061d00 con 0x7fc6b00a2b90 2024-05-09T09:23:33.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.774+0000 7fc6c4ff9640 1 -- 172.21.15.12:0/3800855471 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc6c8061d00 con 0x7fc6b00a2b90 2024-05-09T09:23:33.774 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3 2024-05-09T09:23:33.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.776+0000 7fc6cdf89640 1 -- 172.21.15.12:0/3800855471 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc6b00a2b90 msgr2=0x7fc6b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.776+0000 7fc6cdf89640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc6b00a2b90 0x7fc6b00a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc6c81990e0 tx=0x7fc6bc005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.776+0000 7fc6cdf89640 1 -- 172.21.15.12:0/3800855471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6c8101890 msgr2=0x7fc6c8197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.776+0000 7fc6cdf89640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6c8101890 0x7fc6c8197bd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8005ab0 tx=0x7fc6b80042f0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.776+0000 7fc6cdf89640 1 -- 172.21.15.12:0/3800855471 shutdown_connections 2024-05-09T09:23:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.776+0000 7fc6cdf89640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc6b00a2b90 0x7fc6b00a5050 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.776+0000 7fc6cdf89640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c8103440 0x7fc6c819f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.776+0000 7fc6cdf89640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6c8102a80 0x7fc6c8198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.776+0000 7fc6cdf89640 1 --2- 172.21.15.12:0/3800855471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6c8101890 0x7fc6c8197bd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.776+0000 7fc6cdf89640 1 -- 172.21.15.12:0/3800855471 >> 172.21.15.12:0/3800855471 conn(0x7fc6c80fce00 msgr2=0x7fc6c8105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.776+0000 7fc6cdf89640 1 -- 172.21.15.12:0/3800855471 shutdown_connections 2024-05-09T09:23:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.776+0000 7fc6cdf89640 1 -- 172.21.15.12:0/3800855471 wait complete. 2024-05-09T09:23:33.787 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-09T09:23:33.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.897+0000 7f53e1d9b640 1 -- 172.21.15.12:0/973715944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc102b10 msgr2=0x7f53dc102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.897+0000 7f53e1d9b640 1 --2- 172.21.15.12:0/973715944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc102b10 0x7f53dc102f90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009930 tx=0x7f53d002a420 comp rx=0 tx=0).stop 2024-05-09T09:23:33.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.897+0000 7f53e1d9b640 1 -- 172.21.15.12:0/973715944 shutdown_connections 2024-05-09T09:23:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.897+0000 7f53e1d9b640 1 --2- 172.21.15.12:0/973715944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53dc1034d0 0x7f53dc109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.897+0000 7f53e1d9b640 1 --2- 172.21.15.12:0/973715944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc102b10 0x7f53dc102f90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.897+0000 7f53e1d9b640 1 --2- 172.21.15.12:0/973715944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53dc0ffb40 0x7f53dc0fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.897+0000 7f53e1d9b640 1 -- 172.21.15.12:0/973715944 >> 172.21.15.12:0/973715944 conn(0x7f53dc0fb470 msgr2=0x7f53dc0fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.897+0000 7f53e1d9b640 1 -- 172.21.15.12:0/973715944 shutdown_connections 2024-05-09T09:23:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.897+0000 7f53e1d9b640 1 -- 172.21.15.12:0/973715944 wait complete. 2024-05-09T09:23:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.898+0000 7f53e1d9b640 1 Processor -- start 2024-05-09T09:23:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.898+0000 7f53e1d9b640 1 -- start start 2024-05-09T09:23:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.898+0000 7f53e1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc0ffb40 0x7f53dc197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.898+0000 7f53e1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53dc102b10 0x7f53dc198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.898+0000 7f53e1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53dc1034d0 0x7f53dc19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.898+0000 7f53e1d9b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc10d500 con 0x7f53dc0ffb40 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.898+0000 7f53e1d9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc10d380 con 0x7f53dc1034d0 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.898+0000 7f53e1d9b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc10d680 con 0x7f53dc102b10 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.898+0000 7f53db7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc0ffb40 0x7f53dc197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.898+0000 7f53db7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc0ffb40 0x7f53dc197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56050/0 (socket says 172.21.15.12:56050) 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.898+0000 7f53db7fe640 1 -- 172.21.15.12:0/699531993 learned_addr learned my addr 172.21.15.12:0/699531993 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.899+0000 7f53daffd640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53dc102b10 0x7f53dc198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.899+0000 7f53dbfff640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53dc1034d0 0x7f53dc19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.899+0000 7f53db7fe640 1 -- 172.21.15.12:0/699531993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53dc102b10 msgr2=0x7f53dc198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.899+0000 7f53db7fe640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53dc102b10 0x7f53dc198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.899+0000 7f53db7fe640 1 -- 172.21.15.12:0/699531993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53dc1034d0 msgr2=0x7f53dc19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.899+0000 7f53db7fe640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53dc1034d0 0x7f53dc19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.899+0000 7f53db7fe640 1 -- 172.21.15.12:0/699531993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc19f830 con 0x7f53dc0ffb40 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.899+0000 7f53db7fe640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc0ffb40 0x7f53dc197c20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00e9c0 tx=0x7f53cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:33.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.899+0000 7f53d8ff9640 1 -- 172.21.15.12:0/699531993 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc00cd80 con 0x7f53dc0ffb40 2024-05-09T09:23:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.899+0000 7f53d8ff9640 1 -- 172.21.15.12:0/699531993 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f53cc0035f0 con 0x7f53dc0ffb40 2024-05-09T09:23:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.899+0000 7f53e1d9b640 1 -- 172.21.15.12:0/699531993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc19fb20 con 0x7f53dc0ffb40 2024-05-09T09:23:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.899+0000 7f53d8ff9640 1 -- 172.21.15.12:0/699531993 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc010690 con 0x7f53dc0ffb40 2024-05-09T09:23:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.899+0000 7f53e1d9b640 1 -- 172.21.15.12:0/699531993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53dc1a0070 con 0x7f53dc0ffb40 2024-05-09T09:23:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.900+0000 7f53e1d9b640 1 -- 172.21.15.12:0/699531993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53dc108d10 con 0x7f53dc0ffb40 2024-05-09T09:23:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.901+0000 7f53d8ff9640 1 -- 172.21.15.12:0/699531993 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f53cc0037a0 con 0x7f53dc0ffb40 2024-05-09T09:23:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.902+0000 7f53d8ff9640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f53c40a2c10 0x7f53c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.903+0000 7f53daffd640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f53c40a2c10 0x7f53c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.903+0000 7f53d8ff9640 1 -- 172.21.15.12:0/699531993 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f53cc0cf7c0 con 0x7f53dc0ffb40 2024-05-09T09:23:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.903+0000 7f53daffd640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f53c40a2c10 0x7f53c40a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f53d0006010 tx=0x7f53d0031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:33.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:33.906+0000 7f53d8ff9640 1 -- 172.21.15.12:0/699531993 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f53cc09c4c0 con 0x7f53dc0ffb40 2024-05-09T09:23:33.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 1 op/s 2024-05-09T09:23:33.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/941327036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T09:23:33.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] up:active 2024-05-09T09:23:33.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T09:23:33.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-09T09:23:33.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-05-09T09:23:33.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-09T09:23:33.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:33.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] up:active 2024-05-09T09:23:33.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T09:23:33.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T09:23:33.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: Cluster is now healthy 2024-05-09T09:23:33.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2024-05-09T09:23:33.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-05-09T09:23:33.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:33 smithi170 ceph-mon[33632]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-05-09T09:23:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 1 op/s 2024-05-09T09:23:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/941327036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T09:23:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] up:active 2024-05-09T09:23:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T09:23:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-09T09:23:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-05-09T09:23:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-09T09:23:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] up:active 2024-05-09T09:23:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T09:23:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T09:23:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: Cluster is now healthy 2024-05-09T09:23:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2024-05-09T09:23:34.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-05-09T09:23:34.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:33 smithi016 ceph-mon[33335]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-05-09T09:23:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.120+0000 7f53e1d9b640 1 -- 172.21.15.12:0/699531993 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f53dc061c70 con 0x7f53c40a2c10 2024-05-09T09:23:34.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.145+0000 7f53d8ff9640 1 -- 172.21.15.12:0/699531993 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f53dc061c70 con 0x7f53c40a2c10 2024-05-09T09:23:34.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.147+0000 7f53e1d9b640 1 -- 172.21.15.12:0/699531993 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f53c40a2c10 msgr2=0x7f53c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.147+0000 7f53e1d9b640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f53c40a2c10 0x7f53c40a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f53d0006010 tx=0x7f53d0031510 comp rx=0 tx=0).stop 2024-05-09T09:23:34.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.147+0000 7f53e1d9b640 1 -- 172.21.15.12:0/699531993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc0ffb40 msgr2=0x7f53dc197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.147+0000 7f53e1d9b640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc0ffb40 0x7f53dc197c20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00e9c0 tx=0x7f53cc00ee80 comp rx=0 tx=0).stop 2024-05-09T09:23:34.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.147+0000 7f53e1d9b640 1 -- 172.21.15.12:0/699531993 shutdown_connections 2024-05-09T09:23:34.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.147+0000 7f53e1d9b640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f53c40a2c10 0x7f53c40a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.147+0000 7f53e1d9b640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53dc1034d0 0x7f53dc19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.147+0000 7f53e1d9b640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53dc102b10 0x7f53dc198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.147+0000 7f53e1d9b640 1 --2- 172.21.15.12:0/699531993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc0ffb40 0x7f53dc197c20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.147+0000 7f53e1d9b640 1 -- 172.21.15.12:0/699531993 >> 172.21.15.12:0/699531993 conn(0x7f53dc0fb470 msgr2=0x7f53dc105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:34.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.147+0000 7f53e1d9b640 1 -- 172.21.15.12:0/699531993 shutdown_connections 2024-05-09T09:23:34.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.147+0000 7f53e1d9b640 1 -- 172.21.15.12:0/699531993 wait complete. 2024-05-09T09:23:34.158 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-09T09:23:34.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.269+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3524621540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e04102310 msgr2=0x7f5e04104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.269+0000 7f5e0b9b0640 1 --2- 172.21.15.12:0/3524621540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e04102310 0x7f5e04104700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009ca0 tx=0x7f5e0002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.269+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3524621540 shutdown_connections 2024-05-09T09:23:34.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.269+0000 7f5e0b9b0640 1 --2- 172.21.15.12:0/3524621540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e04107570 0x7f5e041099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.269+0000 7f5e0b9b0640 1 --2- 172.21.15.12:0/3524621540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e04104c40 0x7f5e04107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.269+0000 7f5e0b9b0640 1 --2- 172.21.15.12:0/3524621540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e04102310 0x7f5e04104700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.269+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3524621540 >> 172.21.15.12:0/3524621540 conn(0x7f5e040fbd20 msgr2=0x7f5e040fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.269+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3524621540 shutdown_connections 2024-05-09T09:23:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.269+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3524621540 wait complete. 2024-05-09T09:23:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.269+0000 7f5e0b9b0640 1 Processor -- start 2024-05-09T09:23:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.269+0000 7f5e0b9b0640 1 -- start start 2024-05-09T09:23:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.269+0000 7f5e0b9b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e04102310 0x7f5e04197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e0b9b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e04104c40 0x7f5e04198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e0b9b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e04107570 0x7f5e0419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e0b9b0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0410d510 con 0x7f5e04107570 2024-05-09T09:23:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e0b9b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0410d390 con 0x7f5e04102310 2024-05-09T09:23:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e0b9b0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0410d690 con 0x7f5e04104c40 2024-05-09T09:23:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e09f4d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e04107570 0x7f5e0419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e08f4b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e04104c40 0x7f5e04198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e09f4d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e04107570 0x7f5e0419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56062/0 (socket says 172.21.15.12:56062) 2024-05-09T09:23:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e09f4d640 1 -- 172.21.15.12:0/3894724153 learned_addr learned my addr 172.21.15.12:0/3894724153 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e08f4b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e04104c40 0x7f5e04198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:58750/0 (socket says 172.21.15.12:58750) 2024-05-09T09:23:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e0974c640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e04102310 0x7f5e04197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e09f4d640 1 -- 172.21.15.12:0/3894724153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e04104c40 msgr2=0x7f5e04198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e09f4d640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e04104c40 0x7f5e04198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e09f4d640 1 -- 172.21.15.12:0/3894724153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e04102310 msgr2=0x7f5e04197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e09f4d640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e04102310 0x7f5e04197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e09f4d640 1 -- 172.21.15.12:0/3894724153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0419f810 con 0x7f5e04107570 2024-05-09T09:23:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e08f4b640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e04104c40 0x7f5e04198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e09f4d640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e04107570 0x7f5e0419f1a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00c970 tx=0x7f5dfc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:34.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5df67fc640 1 -- 172.21.15.12:0/3894724153 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc007bb0 con 0x7f5e04107570 2024-05-09T09:23:34.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5df67fc640 1 -- 172.21.15.12:0/3894724153 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5dfc007d50 con 0x7f5e04107570 2024-05-09T09:23:34.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5df67fc640 1 -- 172.21.15.12:0/3894724153 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc00f450 con 0x7f5e04107570 2024-05-09T09:23:34.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3894724153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0419fb00 con 0x7f5e04107570 2024-05-09T09:23:34.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.270+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3894724153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e041a0000 con 0x7f5e04107570 2024-05-09T09:23:34.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.271+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3894724153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e040fd810 con 0x7f5e04107570 2024-05-09T09:23:34.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.272+0000 7f5df67fc640 1 -- 172.21.15.12:0/3894724153 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5dfc004f30 con 0x7f5e04107570 2024-05-09T09:23:34.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.273+0000 7f5df67fc640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5df00a2b90 0x7f5df00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:34.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.273+0000 7f5e0974c640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5df00a2b90 0x7f5df00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:34.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.273+0000 7f5df67fc640 1 -- 172.21.15.12:0/3894724153 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f5dfc0cefc0 con 0x7f5e04107570 2024-05-09T09:23:34.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.274+0000 7f5e0974c640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5df00a2b90 0x7f5df00a5050 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5e00031be0 tx=0x7f5e00009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:34.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.277+0000 7f5df67fc640 1 -- 172.21.15.12:0/3894724153 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5dfc014030 con 0x7f5e04107570 2024-05-09T09:23:34.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.497+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3894724153 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f5e04061c70 con 0x7f5df00a2b90 2024-05-09T09:23:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.502+0000 7f5df67fc640 1 -- 172.21.15.12:0/3894724153 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5e04061c70 con 0x7f5df00a2b90 2024-05-09T09:23:34.502 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_1/a5484845-0cf6-421f-9523-8fc518c08e7a 2024-05-09T09:23:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.504+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3894724153 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5df00a2b90 msgr2=0x7f5df00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.504+0000 7f5e0b9b0640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5df00a2b90 0x7f5df00a5050 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5e00031be0 tx=0x7f5e00009360 comp rx=0 tx=0).stop 2024-05-09T09:23:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.505+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3894724153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e04107570 msgr2=0x7f5e0419f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.505+0000 7f5e0b9b0640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e04107570 0x7f5e0419f1a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00c970 tx=0x7f5dfc00ce30 comp rx=0 tx=0).stop 2024-05-09T09:23:34.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.505+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3894724153 shutdown_connections 2024-05-09T09:23:34.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.505+0000 7f5e0b9b0640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5df00a2b90 0x7f5df00a5050 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.505+0000 7f5e0b9b0640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e04107570 0x7f5e0419f1a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.505+0000 7f5e0b9b0640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e04104c40 0x7f5e04198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.505+0000 7f5e0b9b0640 1 --2- 172.21.15.12:0/3894724153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e04102310 0x7f5e04197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.505+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3894724153 >> 172.21.15.12:0/3894724153 conn(0x7f5e040fbd20 msgr2=0x7f5e04102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:34.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.505+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3894724153 shutdown_connections 2024-05-09T09:23:34.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.505+0000 7f5e0b9b0640 1 -- 172.21.15.12:0/3894724153 wait complete. 2024-05-09T09:23:34.515 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:23:34.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.624+0000 7f6830d41640 1 -- 172.21.15.12:0/1751936694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f682c075810 msgr2=0x7f682c073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.624+0000 7f6830d41640 1 --2- 172.21.15.12:0/1751936694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f682c075810 0x7f682c073c60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f681c009900 tx=0x7f681c02a480 comp rx=0 tx=0).stop 2024-05-09T09:23:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.624+0000 7f6830d41640 1 -- 172.21.15.12:0/1751936694 shutdown_connections 2024-05-09T09:23:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.624+0000 7f6830d41640 1 --2- 172.21.15.12:0/1751936694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f682c109730 0x7f682c10bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.624+0000 7f6830d41640 1 --2- 172.21.15.12:0/1751936694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f682c074230 0x7f682c074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.624+0000 7f6830d41640 1 --2- 172.21.15.12:0/1751936694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f682c075810 0x7f682c073c60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.624+0000 7f6830d41640 1 -- 172.21.15.12:0/1751936694 >> 172.21.15.12:0/1751936694 conn(0x7f682c0fce10 msgr2=0x7f682c0ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.624+0000 7f6830d41640 1 -- 172.21.15.12:0/1751936694 shutdown_connections 2024-05-09T09:23:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.624+0000 7f6830d41640 1 -- 172.21.15.12:0/1751936694 wait complete. 2024-05-09T09:23:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.624+0000 7f6830d41640 1 Processor -- start 2024-05-09T09:23:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f6830d41640 1 -- start start 2024-05-09T09:23:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f6830d41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f682c074230 0x7f682c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f6830d41640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f682c075810 0x7f682c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f6830d41640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f682c109730 0x7f682c1a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f6830d41640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f682c10f700 con 0x7f682c109730 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f6830d41640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f682c10f580 con 0x7f682c074230 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f6830d41640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f682c10f880 con 0x7f682c075810 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f682ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f682c109730 0x7f682c1a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f682ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f682c109730 0x7f682c1a1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56082/0 (socket says 172.21.15.12:56082) 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f6823fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f682c075810 0x7f682c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f682ad9d640 1 -- 172.21.15.12:0/1816629609 learned_addr learned my addr 172.21.15.12:0/1816629609 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f682a59c640 1 --2- 172.21.15.12:0/1816629609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f682c074230 0x7f682c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f682ad9d640 1 -- 172.21.15.12:0/1816629609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f682c075810 msgr2=0x7f682c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f682ad9d640 1 --2- 172.21.15.12:0/1816629609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f682c075810 0x7f682c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f682ad9d640 1 -- 172.21.15.12:0/1816629609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f682c074230 msgr2=0x7f682c199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f682ad9d640 1 --2- 172.21.15.12:0/1816629609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f682c074230 0x7f682c199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f682ad9d640 1 -- 172.21.15.12:0/1816629609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682c1a19d0 con 0x7f682c109730 2024-05-09T09:23:34.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f682ad9d640 1 --2- 172.21.15.12:0/1816629609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f682c109730 0x7f682c1a1360 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f682400eb10 tx=0x7f682400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:34.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f68237fe640 1 -- 172.21.15.12:0/1816629609 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824004b20 con 0x7f682c109730 2024-05-09T09:23:34.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f68237fe640 1 -- 172.21.15.12:0/1816629609 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6824004cc0 con 0x7f682c109730 2024-05-09T09:23:34.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f6830d41640 1 -- 172.21.15.12:0/1816629609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682c1a1cc0 con 0x7f682c109730 2024-05-09T09:23:34.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.625+0000 7f68237fe640 1 -- 172.21.15.12:0/1816629609 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824010700 con 0x7f682c109730 2024-05-09T09:23:34.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.626+0000 7f6830d41640 1 -- 172.21.15.12:0/1816629609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f682c1a21c0 con 0x7f682c109730 2024-05-09T09:23:34.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.627+0000 7f6830d41640 1 -- 172.21.15.12:0/1816629609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f682c1084f0 con 0x7f682c109730 2024-05-09T09:23:34.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.628+0000 7f68237fe640 1 -- 172.21.15.12:0/1816629609 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6824004e70 con 0x7f682c109730 2024-05-09T09:23:34.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.628+0000 7f68237fe640 1 --2- 172.21.15.12:0/1816629609 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68180a2b90 0x7f68180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:34.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.628+0000 7f68237fe640 1 -- 172.21.15.12:0/1816629609 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f68240cf870 con 0x7f682c109730 2024-05-09T09:23:34.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.628+0000 7f682a59c640 1 --2- 172.21.15.12:0/1816629609 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68180a2b90 0x7f68180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:34.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.629+0000 7f682a59c640 1 --2- 172.21.15.12:0/1816629609 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68180a2b90 0x7f68180a5050 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f681c000c00 tx=0x7f681c031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.633+0000 7f68237fe640 1 -- 172.21.15.12:0/1816629609 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f682409c530 con 0x7f682c109730 2024-05-09T09:23:34.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.857+0000 7f6830d41640 1 -- 172.21.15.12:0/1816629609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f682c02d1c0 con 0x7f682c109730 2024-05-09T09:23:34.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.858+0000 7f68237fe640 1 -- 172.21.15.12:0/1816629609 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11931 (secure 0 0 0) 0x7f682409be30 con 0x7f682c109730 2024-05-09T09:23:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:34.857 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:23:34.596143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[24523,24502,24515,24484,14628]},"id":1}]} 2024-05-09T09:23:34.858 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-05-09T09:23:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.859+0000 7f6830d41640 1 -- 172.21.15.12:0/1816629609 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68180a2b90 msgr2=0x7f68180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.859+0000 7f6830d41640 1 --2- 172.21.15.12:0/1816629609 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68180a2b90 0x7f68180a5050 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f681c000c00 tx=0x7f681c031510 comp rx=0 tx=0).stop 2024-05-09T09:23:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.859+0000 7f6830d41640 1 -- 172.21.15.12:0/1816629609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f682c109730 msgr2=0x7f682c1a1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.859+0000 7f6830d41640 1 --2- 172.21.15.12:0/1816629609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f682c109730 0x7f682c1a1360 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f682400eb10 tx=0x7f682400c2d0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.859+0000 7f6830d41640 1 -- 172.21.15.12:0/1816629609 shutdown_connections 2024-05-09T09:23:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.860+0000 7f6830d41640 1 --2- 172.21.15.12:0/1816629609 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68180a2b90 0x7f68180a5050 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.860+0000 7f6830d41640 1 --2- 172.21.15.12:0/1816629609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f682c109730 0x7f682c1a1360 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.860+0000 7f6830d41640 1 --2- 172.21.15.12:0/1816629609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f682c075810 0x7f682c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.860+0000 7f6830d41640 1 --2- 172.21.15.12:0/1816629609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f682c074230 0x7f682c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.860+0000 7f6830d41640 1 -- 172.21.15.12:0/1816629609 >> 172.21.15.12:0/1816629609 conn(0x7f682c0fce10 msgr2=0x7f682c1056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.860+0000 7f6830d41640 1 -- 172.21.15.12:0/1816629609 shutdown_connections 2024-05-09T09:23:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.860+0000 7f6830d41640 1 -- 172.21.15.12:0/1816629609 wait complete. 2024-05-09T09:23:34.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:34 smithi012 ceph-mon[30140]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:34.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:34 smithi012 ceph-mon[30140]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:34.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:34 smithi012 ceph-mon[30140]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:34.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:34 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] up:active 2024-05-09T09:23:34.870 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:34 smithi012 ceph-mon[30140]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T09:23:34.870 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:34 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:23:34.871 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:23:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.977+0000 7f83bd4d2640 1 -- 172.21.15.12:0/2221010426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b8102a90 msgr2=0x7f83b8102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.977+0000 7f83bd4d2640 1 --2- 172.21.15.12:0/2221010426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b8102a90 0x7f83b8102f10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009900 tx=0x7f83a002a410 comp rx=0 tx=0).stop 2024-05-09T09:23:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.977+0000 7f83bd4d2640 1 -- 172.21.15.12:0/2221010426 shutdown_connections 2024-05-09T09:23:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.977+0000 7f83bd4d2640 1 --2- 172.21.15.12:0/2221010426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b8103450 0x7f83b8109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.977+0000 7f83bd4d2640 1 --2- 172.21.15.12:0/2221010426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b8102a90 0x7f83b8102f10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.977+0000 7f83bd4d2640 1 --2- 172.21.15.12:0/2221010426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83b81018a0 0x7f83b8101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.977+0000 7f83bd4d2640 1 -- 172.21.15.12:0/2221010426 >> 172.21.15.12:0/2221010426 conn(0x7f83b80fce30 msgr2=0x7f83b80ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.977+0000 7f83bd4d2640 1 -- 172.21.15.12:0/2221010426 shutdown_connections 2024-05-09T09:23:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.977+0000 7f83bd4d2640 1 -- 172.21.15.12:0/2221010426 wait complete. 2024-05-09T09:23:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.977+0000 7f83bd4d2640 1 Processor -- start 2024-05-09T09:23:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83bd4d2640 1 -- start start 2024-05-09T09:23:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83bd4d2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83b81018a0 0x7f83b8197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83bd4d2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b8102a90 0x7f83b8198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83bd4d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b8103450 0x7f83b819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83bd4d2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f83b810d500 con 0x7f83b8102a90 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83bd4d2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83b810d380 con 0x7f83b8103450 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83bd4d2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f83b810d680 con 0x7f83b81018a0 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83affff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b8102a90 0x7f83b8198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83affff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b8102a90 0x7f83b8198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56102/0 (socket says 172.21.15.12:56102) 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83affff640 1 -- 172.21.15.12:0/3094347225 learned_addr learned my addr 172.21.15.12:0/3094347225 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83b6ffd640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83b81018a0 0x7f83b8197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83b77fe640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b8103450 0x7f83b819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83affff640 1 -- 172.21.15.12:0/3094347225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83b81018a0 msgr2=0x7f83b8197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83affff640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83b81018a0 0x7f83b8197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83affff640 1 -- 172.21.15.12:0/3094347225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b8103450 msgr2=0x7f83b819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83affff640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b8103450 0x7f83b819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83affff640 1 -- 172.21.15.12:0/3094347225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b819f800 con 0x7f83b8102a90 2024-05-09T09:23:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83affff640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b8102a90 0x7f83b8198120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009230 tx=0x7f83a0031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.978+0000 7f83b4ff9640 1 -- 172.21.15.12:0/3094347225 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a003f030 con 0x7f83b8102a90 2024-05-09T09:23:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.979+0000 7f83b4ff9640 1 -- 172.21.15.12:0/3094347225 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f83a0003ac0 con 0x7f83b8102a90 2024-05-09T09:23:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.979+0000 7f83bd4d2640 1 -- 172.21.15.12:0/3094347225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b819fa90 con 0x7f83b8102a90 2024-05-09T09:23:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.979+0000 7f83b4ff9640 1 -- 172.21.15.12:0/3094347225 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0035070 con 0x7f83b8102a90 2024-05-09T09:23:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.979+0000 7f83bd4d2640 1 -- 172.21.15.12:0/3094347225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b819ff80 con 0x7f83b8102a90 2024-05-09T09:23:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.980+0000 7f83bd4d2640 1 -- 172.21.15.12:0/3094347225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b8108c30 con 0x7f83b8102a90 2024-05-09T09:23:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.981+0000 7f83b4ff9640 1 -- 172.21.15.12:0/3094347225 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f83a0042030 con 0x7f83b8102a90 2024-05-09T09:23:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.981+0000 7f83b4ff9640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f83a40a2b90 0x7f83a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.982+0000 7f83b6ffd640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f83a40a2b90 0x7f83a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.982+0000 7f83b4ff9640 1 -- 172.21.15.12:0/3094347225 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f83a00efc00 con 0x7f83b8102a90 2024-05-09T09:23:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.982+0000 7f83b6ffd640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f83a40a2b90 0x7f83a40a5050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f83b81028f0 tx=0x7f83a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:34.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:34.985+0000 7f83b4ff9640 1 -- 172.21.15.12:0/3094347225 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f83a00bc2d0 con 0x7f83b8102a90 2024-05-09T09:23:34.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:34 smithi170 ceph-mon[33632]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:34.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:34 smithi170 ceph-mon[33632]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:34.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:34 smithi170 ceph-mon[33632]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:34.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:34 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] up:active 2024-05-09T09:23:34.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:34 smithi170 ceph-mon[33632]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T09:23:34.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:34 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:23:35.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:34 smithi016 ceph-mon[33335]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:35.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:34 smithi016 ceph-mon[33335]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:35.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:34 smithi016 ceph-mon[33335]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:35.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:34 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] up:active 2024-05-09T09:23:35.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:34 smithi016 ceph-mon[33335]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T09:23:35.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:34 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:23:35.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.169+0000 7f83bd4d2640 1 -- 172.21.15.12:0/3094347225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83b802d3c0 con 0x7f83b8102a90 2024-05-09T09:23:35.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.171+0000 7f83b4ff9640 1 -- 172.21.15.12:0/3094347225 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24105 (secure 0 0 0) 0x7f83a00c05d0 con 0x7f83b8102a90 2024-05-09T09:23:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:35.171 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:23:14.726332+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:23:35.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.173+0000 7f83bd4d2640 1 -- 172.21.15.12:0/3094347225 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f83a40a2b90 msgr2=0x7f83a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:35.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.173+0000 7f83bd4d2640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f83a40a2b90 0x7f83a40a5050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f83b81028f0 tx=0x7f83a8005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:35.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.173+0000 7f83bd4d2640 1 -- 172.21.15.12:0/3094347225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b8102a90 msgr2=0x7f83b8198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:35.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.173+0000 7f83bd4d2640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b8102a90 0x7f83b8198120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009230 tx=0x7f83a0031ac0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.173+0000 7f83bd4d2640 1 -- 172.21.15.12:0/3094347225 shutdown_connections 2024-05-09T09:23:35.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.173+0000 7f83bd4d2640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f83a40a2b90 0x7f83a40a5050 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.173+0000 7f83bd4d2640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b8103450 0x7f83b819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.173+0000 7f83bd4d2640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b8102a90 0x7f83b8198120 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.173+0000 7f83bd4d2640 1 --2- 172.21.15.12:0/3094347225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83b81018a0 0x7f83b8197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.173+0000 7f83bd4d2640 1 -- 172.21.15.12:0/3094347225 >> 172.21.15.12:0/3094347225 conn(0x7f83b80fce30 msgr2=0x7f83b8105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:35.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.173+0000 7f83bd4d2640 1 -- 172.21.15.12:0/3094347225 shutdown_connections 2024-05-09T09:23:35.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.173+0000 7f83bd4d2640 1 -- 172.21.15.12:0/3094347225 wait complete. 2024-05-09T09:23:35.184 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T09:23:35.184 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T09:23:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.289+0000 7f8676729640 1 -- 172.21.15.12:0/3179663203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8670102310 msgr2=0x7f8670104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.289+0000 7f8676729640 1 --2- 172.21.15.12:0/3179663203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8670102310 0x7f8670104700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8660009900 tx=0x7f866002a410 comp rx=0 tx=0).stop 2024-05-09T09:23:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.289+0000 7f8676729640 1 -- 172.21.15.12:0/3179663203 shutdown_connections 2024-05-09T09:23:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.289+0000 7f8676729640 1 --2- 172.21.15.12:0/3179663203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8670107570 0x7f86701099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.289+0000 7f8676729640 1 --2- 172.21.15.12:0/3179663203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8670104c40 0x7f8670107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.289+0000 7f8676729640 1 --2- 172.21.15.12:0/3179663203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8670102310 0x7f8670104700 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.289+0000 7f8676729640 1 -- 172.21.15.12:0/3179663203 >> 172.21.15.12:0/3179663203 conn(0x7f86700fbd20 msgr2=0x7f86700fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.289+0000 7f8676729640 1 -- 172.21.15.12:0/3179663203 shutdown_connections 2024-05-09T09:23:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.289+0000 7f8676729640 1 -- 172.21.15.12:0/3179663203 wait complete. 2024-05-09T09:23:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.290+0000 7f8676729640 1 Processor -- start 2024-05-09T09:23:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.290+0000 7f8676729640 1 -- start start 2024-05-09T09:23:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.290+0000 7f8676729640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8670102310 0x7f8670199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.290+0000 7f8676729640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8670104c40 0x7f867019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.290+0000 7f8676729640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8670107570 0x7f86701a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.290+0000 7f8676729640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f867010d4f0 con 0x7f8670102310 2024-05-09T09:23:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.290+0000 7f8676729640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f867010d370 con 0x7f8670107570 2024-05-09T09:23:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.290+0000 7f8676729640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f867010d670 con 0x7f8670104c40 2024-05-09T09:23:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.290+0000 7f866ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8670102310 0x7f8670199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.290+0000 7f866ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8670102310 0x7f8670199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56124/0 (socket says 172.21.15.12:56124) 2024-05-09T09:23:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.290+0000 7f866ffff640 1 -- 172.21.15.12:0/1471604163 learned_addr learned my addr 172.21.15.12:0/1471604163 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.291+0000 7f866f7fe640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8670104c40 0x7f867019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.291+0000 7f8674cc6640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8670107570 0x7f86701a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.291+0000 7f866ffff640 1 -- 172.21.15.12:0/1471604163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8670104c40 msgr2=0x7f867019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.291+0000 7f866ffff640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8670104c40 0x7f867019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.291+0000 7f866ffff640 1 -- 172.21.15.12:0/1471604163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8670107570 msgr2=0x7f86701a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.291+0000 7f866ffff640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8670107570 0x7f86701a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.291+0000 7f866ffff640 1 -- 172.21.15.12:0/1471604163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86701a1a30 con 0x7f8670102310 2024-05-09T09:23:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.291+0000 7f866ffff640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8670102310 0x7f8670199e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8660009230 tx=0x7f8660031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.291+0000 7f866d7fa640 1 -- 172.21.15.12:0/1471604163 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866003f030 con 0x7f8670102310 2024-05-09T09:23:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.291+0000 7f866d7fa640 1 -- 172.21.15.12:0/1471604163 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8660003ac0 con 0x7f8670102310 2024-05-09T09:23:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.291+0000 7f8676729640 1 -- 172.21.15.12:0/1471604163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86701a1cc0 con 0x7f8670102310 2024-05-09T09:23:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.291+0000 7f866d7fa640 1 -- 172.21.15.12:0/1471604163 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8660035070 con 0x7f8670102310 2024-05-09T09:23:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.291+0000 7f8676729640 1 -- 172.21.15.12:0/1471604163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86701a21b0 con 0x7f8670102310 2024-05-09T09:23:35.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.292+0000 7f8676729640 1 -- 172.21.15.12:0/1471604163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86701090c0 con 0x7f8670102310 2024-05-09T09:23:35.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.293+0000 7f866d7fa640 1 -- 172.21.15.12:0/1471604163 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8660042030 con 0x7f8670102310 2024-05-09T09:23:35.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.294+0000 7f866d7fa640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86580a2b90 0x7f86580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:35.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.294+0000 7f866f7fe640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86580a2b90 0x7f86580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:35.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.295+0000 7f866d7fa640 1 -- 172.21.15.12:0/1471604163 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f86600ef8b0 con 0x7f8670102310 2024-05-09T09:23:35.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.295+0000 7f866f7fe640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86580a2b90 0x7f86580a5050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f86640095f0 tx=0x7f8664005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:35.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.299+0000 7f866d7fa640 1 -- 172.21.15.12:0/1471604163 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f86600bbf20 con 0x7f8670102310 2024-05-09T09:23:35.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.481+0000 7f8676729640 1 -- 172.21.15.12:0/1471604163 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8670061c70 con 0x7f86580a2b90 2024-05-09T09:23:35.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.491+0000 7f866d7fa640 1 -- 172.21.15.12:0/1471604163 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340364 (secure 0 0 0) 0x7f8670061c70 con 0x7f86580a2b90 2024-05-09T09:23:35.495 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:35.498 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-09T09:23:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.495+0000 7f8676729640 1 -- 172.21.15.12:0/1471604163 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86580a2b90 msgr2=0x7f86580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.495+0000 7f8676729640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86580a2b90 0x7f86580a5050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f86640095f0 tx=0x7f8664005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.495+0000 7f8676729640 1 -- 172.21.15.12:0/1471604163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8670102310 msgr2=0x7f8670199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.495+0000 7f8676729640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8670102310 0x7f8670199e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8660009230 tx=0x7f8660031ac0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.495+0000 7f8676729640 1 -- 172.21.15.12:0/1471604163 shutdown_connections 2024-05-09T09:23:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.495+0000 7f8676729640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86580a2b90 0x7f86580a5050 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.495+0000 7f8676729640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8670107570 0x7f86701a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.495+0000 7f8676729640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8670104c40 0x7f867019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.495+0000 7f8676729640 1 --2- 172.21.15.12:0/1471604163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8670102310 0x7f8670199e10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.495+0000 7f8676729640 1 -- 172.21.15.12:0/1471604163 >> 172.21.15.12:0/1471604163 conn(0x7f86700fbd20 msgr2=0x7f8670101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.495+0000 7f8676729640 1 -- 172.21.15.12:0/1471604163 shutdown_connections 2024-05-09T09:23:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.495+0000 7f8676729640 1 -- 172.21.15.12:0/1471604163 wait complete. 2024-05-09T09:23:35.506 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-05-09T09:23:34.555477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617791,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":143,"num_read_kb":144,"num_write":335,"num_write_kb":3006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387720,"kb_used_data":13568,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554104,"statfs":{"total":1151940427776,"available":1151543402496,"internally_reserved":0,"allocated":13893632,"data_stored":7558349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25013,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":17,"num_read_kb":35,"num_write":109,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003742"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657727+0000","last_change":"2024-05-09T09:23:10.334738+0000","last_active":"2024-05-09T09:23:15.657727+0000","last_peered":"2024-05-09T09:23:15.657727+0000","last_clean":"2024-05-09T09:23:15.657727+0000","last_became_active":"2024-05-09T09:23:10.333406+0000","last_became_peered":"2024-05-09T09:23:10.333406+0000","last_unstale":"2024-05-09T09:23:15.657727+0000","last_undegraded":"2024-05-09T09:23:15.657727+0000","last_fullsized":"2024-05-09T09:23:15.657727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:44.957557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298435+0000","last_change":"2024-05-09T09:23:10.331860+0000","last_active":"2024-05-09T09:23:16.298435+0000","last_peered":"2024-05-09T09:23:16.298435+0000","last_clean":"2024-05-09T09:23:16.298435+0000","last_became_active":"2024-05-09T09:23:10.331396+0000","last_became_peered":"2024-05-09T09:23:10.331396+0000","last_unstale":"2024-05-09T09:23:16.298435+0000","last_undegraded":"2024-05-09T09:23:16.298435+0000","last_fullsized":"2024-05-09T09:23:16.298435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:47.676399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298298+0000","last_change":"2024-05-09T09:23:10.336543+0000","last_active":"2024-05-09T09:23:16.298298+0000","last_peered":"2024-05-09T09:23:16.298298+0000","last_clean":"2024-05-09T09:23:16.298298+0000","last_became_active":"2024-05-09T09:23:10.336189+0000","last_became_peered":"2024-05-09T09:23:10.336189+0000","last_unstale":"2024-05-09T09:23:16.298298+0000","last_undegraded":"2024-05-09T09:23:16.298298+0000","last_fullsized":"2024-05-09T09:23:16.298298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:06.419177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.570095+0000","last_change":"2024-05-09T09:23:10.331739+0000","last_active":"2024-05-09T09:23:15.570095+0000","last_peered":"2024-05-09T09:23:15.570095+0000","last_clean":"2024-05-09T09:23:15.570095+0000","last_became_active":"2024-05-09T09:23:10.331333+0000","last_became_peered":"2024-05-09T09:23:10.331333+0000","last_unstale":"2024-05-09T09:23:15.570095+0000","last_undegraded":"2024-05-09T09:23:15.570095+0000","last_fullsized":"2024-05-09T09:23:15.570095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:04.925697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657434+0000","last_change":"2024-05-09T09:23:10.345294+0000","last_active":"2024-05-09T09:23:15.657434+0000","last_peered":"2024-05-09T09:23:15.657434+0000","last_clean":"2024-05-09T09:23:15.657434+0000","last_became_active":"2024-05-09T09:23:10.344940+0000","last_became_peered":"2024-05-09T09:23:10.344940+0000","last_unstale":"2024-05-09T09:23:15.657434+0000","last_undegraded":"2024-05-09T09:23:15.657434+0000","last_fullsized":"2024-05-09T09:23:15.657434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:41.150284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656543+0000","last_change":"2024-05-09T09:23:10.329143+0000","last_active":"2024-05-09T09:23:15.656543+0000","last_peered":"2024-05-09T09:23:15.656543+0000","last_clean":"2024-05-09T09:23:15.656543+0000","last_became_active":"2024-05-09T09:23:10.328712+0000","last_became_peered":"2024-05-09T09:23:10.328712+0000","last_unstale":"2024-05-09T09:23:15.656543+0000","last_undegraded":"2024-05-09T09:23:15.656543+0000","last_fullsized":"2024-05-09T09:23:15.656543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:10:35.085759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351233+0000","last_change":"2024-05-09T09:23:10.338176+0000","last_active":"2024-05-09T09:23:15.351233+0000","last_peered":"2024-05-09T09:23:15.351233+0000","last_clean":"2024-05-09T09:23:15.351233+0000","last_became_active":"2024-05-09T09:23:10.337845+0000","last_became_peered":"2024-05-09T09:23:10.337845+0000","last_unstale":"2024-05-09T09:23:15.351233+0000","last_undegraded":"2024-05-09T09:23:15.351233+0000","last_fullsized":"2024-05-09T09:23:15.351233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:34:21.783295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656321+0000","last_change":"2024-05-09T09:23:10.342473+0000","last_active":"2024-05-09T09:23:15.656321+0000","last_peered":"2024-05-09T09:23:15.656321+0000","last_clean":"2024-05-09T09:23:15.656321+0000","last_became_active":"2024-05-09T09:23:10.340682+0000","last_became_peered":"2024-05-09T09:23:10.340682+0000","last_unstale":"2024-05-09T09:23:15.656321+0000","last_undegraded":"2024-05-09T09:23:15.656321+0000","last_fullsized":"2024-05-09T09:23:15.656321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:56.730332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.534240+0000","last_change":"2024-05-09T09:23:10.339888+0000","last_active":"2024-05-09T09:23:15.534240+0000","last_peered":"2024-05-09T09:23:15.534240+0000","last_clean":"2024-05-09T09:23:15.534240+0000","last_became_active":"2024-05-09T09:23:10.337849+0000","last_became_peered":"2024-05-09T09:23:10.337849+0000","last_unstale":"2024-05-09T09:23:15.534240+0000","last_undegraded":"2024-05-09T09:23:15.534240+0000","last_fullsized":"2024-05-09T09:23:15.534240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:48.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657034+0000","last_change":"2024-05-09T09:23:10.332395+0000","last_active":"2024-05-09T09:23:15.657034+0000","last_peered":"2024-05-09T09:23:15.657034+0000","last_clean":"2024-05-09T09:23:15.657034+0000","last_became_active":"2024-05-09T09:23:10.332040+0000","last_became_peered":"2024-05-09T09:23:10.332040+0000","last_unstale":"2024-05-09T09:23:15.657034+0000","last_undegraded":"2024-05-09T09:23:15.657034+0000","last_fullsized":"2024-05-09T09:23:15.657034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:04.036871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351804+0000","last_change":"2024-05-09T09:23:10.341148+0000","last_active":"2024-05-09T09:23:15.351804+0000","last_peered":"2024-05-09T09:23:15.351804+0000","last_clean":"2024-05-09T09:23:15.351804+0000","last_became_active":"2024-05-09T09:23:10.340799+0000","last_became_peered":"2024-05-09T09:23:10.340799+0000","last_unstale":"2024-05-09T09:23:15.351804+0000","last_undegraded":"2024-05-09T09:23:15.351804+0000","last_fullsized":"2024-05-09T09:23:15.351804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:31.020659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.570082+0000","last_change":"2024-05-09T09:23:10.336191+0000","last_active":"2024-05-09T09:23:15.570082+0000","last_peered":"2024-05-09T09:23:15.570082+0000","last_clean":"2024-05-09T09:23:15.570082+0000","last_became_active":"2024-05-09T09:23:10.333591+0000","last_became_peered":"2024-05-09T09:23:10.333591+0000","last_unstale":"2024-05-09T09:23:15.570082+0000","last_undegraded":"2024-05-09T09:23:15.570082+0000","last_fullsized":"2024-05-09T09:23:15.570082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:24.836954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656523+0000","last_change":"2024-05-09T09:23:10.314506+0000","last_active":"2024-05-09T09:23:15.656523+0000","last_peered":"2024-05-09T09:23:15.656523+0000","last_clean":"2024-05-09T09:23:15.656523+0000","last_became_active":"2024-05-09T09:23:10.314171+0000","last_became_peered":"2024-05-09T09:23:10.314171+0000","last_unstale":"2024-05-09T09:23:15.656523+0000","last_undegraded":"2024-05-09T09:23:15.656523+0000","last_fullsized":"2024-05-09T09:23:15.656523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:51.088814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.297949+0000","last_change":"2024-05-09T09:23:10.341670+0000","last_active":"2024-05-09T09:23:16.297949+0000","last_peered":"2024-05-09T09:23:16.297949+0000","last_clean":"2024-05-09T09:23:16.297949+0000","last_became_active":"2024-05-09T09:23:10.341155+0000","last_became_peered":"2024-05-09T09:23:10.341155+0000","last_unstale":"2024-05-09T09:23:16.297949+0000","last_undegraded":"2024-05-09T09:23:16.297949+0000","last_fullsized":"2024-05-09T09:23:16.297949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:14:32.734302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298058+0000","last_change":"2024-05-09T09:23:10.333113+0000","last_active":"2024-05-09T09:23:16.298058+0000","last_peered":"2024-05-09T09:23:16.298058+0000","last_clean":"2024-05-09T09:23:16.298058+0000","last_became_active":"2024-05-09T09:23:10.332782+0000","last_became_peered":"2024-05-09T09:23:10.332782+0000","last_unstale":"2024-05-09T09:23:16.298058+0000","last_undegraded":"2024-05-09T09:23:16.298058+0000","last_fullsized":"2024-05-09T09:23:16.298058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:25.555681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.533557+0000","last_change":"2024-05-09T09:23:10.337367+0000","last_active":"2024-05-09T09:23:15.533557+0000","last_peered":"2024-05-09T09:23:15.533557+0000","last_clean":"2024-05-09T09:23:15.533557+0000","last_became_active":"2024-05-09T09:23:10.337000+0000","last_became_peered":"2024-05-09T09:23:10.337000+0000","last_unstale":"2024-05-09T09:23:15.533557+0000","last_undegraded":"2024-05-09T09:23:15.533557+0000","last_fullsized":"2024-05-09T09:23:15.533557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:54.431820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657070+0000","last_change":"2024-05-09T09:23:10.334293+0000","last_active":"2024-05-09T09:23:15.657070+0000","last_peered":"2024-05-09T09:23:15.657070+0000","last_clean":"2024-05-09T09:23:15.657070+0000","last_became_active":"2024-05-09T09:23:10.333938+0000","last_became_peered":"2024-05-09T09:23:10.333938+0000","last_unstale":"2024-05-09T09:23:15.657070+0000","last_undegraded":"2024-05-09T09:23:15.657070+0000","last_fullsized":"2024-05-09T09:23:15.657070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:56:09.048848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351360+0000","last_change":"2024-05-09T09:23:10.332394+0000","last_active":"2024-05-09T09:23:15.351360+0000","last_peered":"2024-05-09T09:23:15.351360+0000","last_clean":"2024-05-09T09:23:15.351360+0000","last_became_active":"2024-05-09T09:23:10.332054+0000","last_became_peered":"2024-05-09T09:23:10.332054+0000","last_unstale":"2024-05-09T09:23:15.351360+0000","last_undegraded":"2024-05-09T09:23:15.351360+0000","last_fullsized":"2024-05-09T09:23:15.351360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:05.628740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351692+0000","last_change":"2024-05-09T09:23:10.336189+0000","last_active":"2024-05-09T09:23:15.351692+0000","last_peered":"2024-05-09T09:23:15.351692+0000","last_clean":"2024-05-09T09:23:15.351692+0000","last_became_active":"2024-05-09T09:23:10.335698+0000","last_became_peered":"2024-05-09T09:23:10.335698+0000","last_unstale":"2024-05-09T09:23:15.351692+0000","last_undegraded":"2024-05-09T09:23:15.351692+0000","last_fullsized":"2024-05-09T09:23:15.351692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:01.403918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351498+0000","last_change":"2024-05-09T09:23:14.372134+0000","last_active":"2024-05-09T09:23:15.351498+0000","last_peered":"2024-05-09T09:23:15.351498+0000","last_clean":"2024-05-09T09:23:15.351498+0000","last_became_active":"2024-05-09T09:23:14.371466+0000","last_became_peered":"2024-05-09T09:23:14.371466+0000","last_unstale":"2024-05-09T09:23:15.351498+0000","last_undegraded":"2024-05-09T09:23:15.351498+0000","last_fullsized":"2024-05-09T09:23:15.351498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:22.048565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'11","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:32.676589+0000","last_change":"2024-05-09T09:23:09.304421+0000","last_active":"2024-05-09T09:23:32.676589+0000","last_peered":"2024-05-09T09:23:32.676589+0000","last_clean":"2024-05-09T09:23:32.676589+0000","last_became_active":"2024-05-09T09:23:09.304109+0000","last_became_peered":"2024-05-09T09:23:09.304109+0000","last_unstale":"2024-05-09T09:23:32.676589+0000","last_undegraded":"2024-05-09T09:23:32.676589+0000","last_fullsized":"2024-05-09T09:23:32.676589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:36.796458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656684+0000","last_change":"2024-05-09T09:23:10.341875+0000","last_active":"2024-05-09T09:23:15.656684+0000","last_peered":"2024-05-09T09:23:15.656684+0000","last_clean":"2024-05-09T09:23:15.656684+0000","last_became_active":"2024-05-09T09:23:10.341385+0000","last_became_peered":"2024-05-09T09:23:10.341385+0000","last_unstale":"2024-05-09T09:23:15.656684+0000","last_undegraded":"2024-05-09T09:23:15.656684+0000","last_fullsized":"2024-05-09T09:23:15.656684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:44:33.939722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569491+0000","last_change":"2024-05-09T09:23:14.371120+0000","last_active":"2024-05-09T09:23:15.569491+0000","last_peered":"2024-05-09T09:23:15.569491+0000","last_clean":"2024-05-09T09:23:15.569491+0000","last_became_active":"2024-05-09T09:23:14.370663+0000","last_became_peered":"2024-05-09T09:23:14.370663+0000","last_unstale":"2024-05-09T09:23:15.569491+0000","last_undegraded":"2024-05-09T09:23:15.569491+0000","last_fullsized":"2024-05-09T09:23:15.569491+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:50:06.553518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657433+0000","last_change":"2024-05-09T09:23:09.304357+0000","last_active":"2024-05-09T09:23:15.657433+0000","last_peered":"2024-05-09T09:23:15.657433+0000","last_clean":"2024-05-09T09:23:15.657433+0000","last_became_active":"2024-05-09T09:23:09.304007+0000","last_became_peered":"2024-05-09T09:23:09.304007+0000","last_unstale":"2024-05-09T09:23:15.657433+0000","last_undegraded":"2024-05-09T09:23:15.657433+0000","last_fullsized":"2024-05-09T09:23:15.657433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:32:46.132287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298413+0000","last_change":"2024-05-09T09:23:10.339533+0000","last_active":"2024-05-09T09:23:16.298413+0000","last_peered":"2024-05-09T09:23:16.298413+0000","last_clean":"2024-05-09T09:23:16.298413+0000","last_became_active":"2024-05-09T09:23:10.337142+0000","last_became_peered":"2024-05-09T09:23:10.337142+0000","last_unstale":"2024-05-09T09:23:16.298413+0000","last_undegraded":"2024-05-09T09:23:16.298413+0000","last_fullsized":"2024-05-09T09:23:16.298413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:11:11.921491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569666+0000","last_change":"2024-05-09T09:23:14.377188+0000","last_active":"2024-05-09T09:23:15.569666+0000","last_peered":"2024-05-09T09:23:15.569666+0000","last_clean":"2024-05-09T09:23:15.569666+0000","last_became_active":"2024-05-09T09:23:14.376247+0000","last_became_peered":"2024-05-09T09:23:14.376247+0000","last_unstale":"2024-05-09T09:23:15.569666+0000","last_undegraded":"2024-05-09T09:23:15.569666+0000","last_fullsized":"2024-05-09T09:23:15.569666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:32:34.054087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:31.666247+0000","last_change":"2024-05-09T09:23:09.300754+0000","last_active":"2024-05-09T09:23:31.666247+0000","last_peered":"2024-05-09T09:23:31.666247+0000","last_clean":"2024-05-09T09:23:31.666247+0000","last_became_active":"2024-05-09T09:23:09.300419+0000","last_became_peered":"2024-05-09T09:23:09.300419+0000","last_unstale":"2024-05-09T09:23:31.666247+0000","last_undegraded":"2024-05-09T09:23:31.666247+0000","last_fullsized":"2024-05-09T09:23:31.666247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:06.205582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.533898+0000","last_change":"2024-05-09T09:23:10.331952+0000","last_active":"2024-05-09T09:23:15.533898+0000","last_peered":"2024-05-09T09:23:15.533898+0000","last_clean":"2024-05-09T09:23:15.533898+0000","last_became_active":"2024-05-09T09:23:10.331515+0000","last_became_peered":"2024-05-09T09:23:10.331515+0000","last_unstale":"2024-05-09T09:23:15.533898+0000","last_undegraded":"2024-05-09T09:23:15.533898+0000","last_fullsized":"2024-05-09T09:23:15.533898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:23.809672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569617+0000","last_change":"2024-05-09T09:23:14.368518+0000","last_active":"2024-05-09T09:23:15.569617+0000","last_peered":"2024-05-09T09:23:15.569617+0000","last_clean":"2024-05-09T09:23:15.569617+0000","last_became_active":"2024-05-09T09:23:14.367995+0000","last_became_peered":"2024-05-09T09:23:14.367995+0000","last_unstale":"2024-05-09T09:23:15.569617+0000","last_undegraded":"2024-05-09T09:23:15.569617+0000","last_fullsized":"2024-05-09T09:23:15.569617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:11.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658412+0000","last_change":"2024-05-09T09:23:09.299656+0000","last_active":"2024-05-09T09:23:15.658412+0000","last_peered":"2024-05-09T09:23:15.658412+0000","last_clean":"2024-05-09T09:23:15.658412+0000","last_became_active":"2024-05-09T09:23:09.299374+0000","last_became_peered":"2024-05-09T09:23:09.299374+0000","last_unstale":"2024-05-09T09:23:15.658412+0000","last_undegraded":"2024-05-09T09:23:15.658412+0000","last_fullsized":"2024-05-09T09:23:15.658412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:05.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658068+0000","last_change":"2024-05-09T09:23:10.341933+0000","last_active":"2024-05-09T09:23:15.658068+0000","last_peered":"2024-05-09T09:23:15.658068+0000","last_clean":"2024-05-09T09:23:15.658068+0000","last_became_active":"2024-05-09T09:23:10.341621+0000","last_became_peered":"2024-05-09T09:23:10.341621+0000","last_unstale":"2024-05-09T09:23:15.658068+0000","last_undegraded":"2024-05-09T09:23:15.658068+0000","last_fullsized":"2024-05-09T09:23:15.658068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:31.938768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351459+0000","last_change":"2024-05-09T09:23:14.364783+0000","last_active":"2024-05-09T09:23:15.351459+0000","last_peered":"2024-05-09T09:23:15.351459+0000","last_clean":"2024-05-09T09:23:15.351459+0000","last_became_active":"2024-05-09T09:23:14.364309+0000","last_became_peered":"2024-05-09T09:23:14.364309+0000","last_unstale":"2024-05-09T09:23:15.351459+0000","last_undegraded":"2024-05-09T09:23:15.351459+0000","last_fullsized":"2024-05-09T09:23:15.351459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:44.252111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.595340+0000","last_change":"2024-05-09T09:23:09.297913+0000","last_active":"2024-05-09T09:23:33.595340+0000","last_peered":"2024-05-09T09:23:33.595340+0000","last_clean":"2024-05-09T09:23:33.595340+0000","last_became_active":"2024-05-09T09:23:09.297612+0000","last_became_peered":"2024-05-09T09:23:09.297612+0000","last_unstale":"2024-05-09T09:23:33.595340+0000","last_undegraded":"2024-05-09T09:23:33.595340+0000","last_fullsized":"2024-05-09T09:23:33.595340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:21.808703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657947+0000","last_change":"2024-05-09T09:23:10.339101+0000","last_active":"2024-05-09T09:23:15.657947+0000","last_peered":"2024-05-09T09:23:15.657947+0000","last_clean":"2024-05-09T09:23:15.657947+0000","last_became_active":"2024-05-09T09:23:10.338453+0000","last_became_peered":"2024-05-09T09:23:10.338453+0000","last_unstale":"2024-05-09T09:23:15.657947+0000","last_undegraded":"2024-05-09T09:23:15.657947+0000","last_fullsized":"2024-05-09T09:23:15.657947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:34:44.933882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351787+0000","last_change":"2024-05-09T09:23:14.371781+0000","last_active":"2024-05-09T09:23:15.351787+0000","last_peered":"2024-05-09T09:23:15.351787+0000","last_clean":"2024-05-09T09:23:15.351787+0000","last_became_active":"2024-05-09T09:23:14.371286+0000","last_became_peered":"2024-05-09T09:23:14.371286+0000","last_unstale":"2024-05-09T09:23:15.351787+0000","last_undegraded":"2024-05-09T09:23:15.351787+0000","last_fullsized":"2024-05-09T09:23:15.351787+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:26:29.185064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:32.677863+0000","last_change":"2024-05-09T09:23:09.303786+0000","last_active":"2024-05-09T09:23:32.677863+0000","last_peered":"2024-05-09T09:23:32.677863+0000","last_clean":"2024-05-09T09:23:32.677863+0000","last_became_active":"2024-05-09T09:23:09.303475+0000","last_became_peered":"2024-05-09T09:23:09.303475+0000","last_unstale":"2024-05-09T09:23:32.677863+0000","last_undegraded":"2024-05-09T09:23:32.677863+0000","last_fullsized":"2024-05-09T09:23:32.677863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:28.833055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657891+0000","last_change":"2024-05-09T09:23:10.344643+0000","last_active":"2024-05-09T09:23:15.657891+0000","last_peered":"2024-05-09T09:23:15.657891+0000","last_clean":"2024-05-09T09:23:15.657891+0000","last_became_active":"2024-05-09T09:23:10.344314+0000","last_became_peered":"2024-05-09T09:23:10.344314+0000","last_unstale":"2024-05-09T09:23:15.657891+0000","last_undegraded":"2024-05-09T09:23:15.657891+0000","last_fullsized":"2024-05-09T09:23:15.657891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:15.147107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657484+0000","last_change":"2024-05-09T09:23:14.361170+0000","last_active":"2024-05-09T09:23:15.657484+0000","last_peered":"2024-05-09T09:23:15.657484+0000","last_clean":"2024-05-09T09:23:15.657484+0000","last_became_active":"2024-05-09T09:23:14.360767+0000","last_became_peered":"2024-05-09T09:23:14.360767+0000","last_unstale":"2024-05-09T09:23:15.657484+0000","last_undegraded":"2024-05-09T09:23:15.657484+0000","last_fullsized":"2024-05-09T09:23:15.657484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:59:21.393148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656335+0000","last_change":"2024-05-09T09:23:09.300757+0000","last_active":"2024-05-09T09:23:15.656335+0000","last_peered":"2024-05-09T09:23:15.656335+0000","last_clean":"2024-05-09T09:23:15.656335+0000","last_became_active":"2024-05-09T09:23:09.300328+0000","last_became_peered":"2024-05-09T09:23:09.300328+0000","last_unstale":"2024-05-09T09:23:15.656335+0000","last_undegraded":"2024-05-09T09:23:15.656335+0000","last_fullsized":"2024-05-09T09:23:15.656335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.553878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657457+0000","last_change":"2024-05-09T09:23:10.336077+0000","last_active":"2024-05-09T09:23:15.657457+0000","last_peered":"2024-05-09T09:23:15.657457+0000","last_clean":"2024-05-09T09:23:15.657457+0000","last_became_active":"2024-05-09T09:23:10.335762+0000","last_became_peered":"2024-05-09T09:23:10.335762+0000","last_unstale":"2024-05-09T09:23:15.657457+0000","last_undegraded":"2024-05-09T09:23:15.657457+0000","last_fullsized":"2024-05-09T09:23:15.657457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:48.063385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298380+0000","last_change":"2024-05-09T09:23:14.371413+0000","last_active":"2024-05-09T09:23:16.298380+0000","last_peered":"2024-05-09T09:23:16.298380+0000","last_clean":"2024-05-09T09:23:16.298380+0000","last_became_active":"2024-05-09T09:23:14.370018+0000","last_became_peered":"2024-05-09T09:23:14.370018+0000","last_unstale":"2024-05-09T09:23:16.298380+0000","last_undegraded":"2024-05-09T09:23:16.298380+0000","last_fullsized":"2024-05-09T09:23:16.298380+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:18.475976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.600388+0000","last_change":"2024-05-09T09:23:09.297328+0000","last_active":"2024-05-09T09:23:33.600388+0000","last_peered":"2024-05-09T09:23:33.600388+0000","last_clean":"2024-05-09T09:23:33.600388+0000","last_became_active":"2024-05-09T09:23:09.297001+0000","last_became_peered":"2024-05-09T09:23:09.297001+0000","last_unstale":"2024-05-09T09:23:33.600388+0000","last_undegraded":"2024-05-09T09:23:33.600388+0000","last_fullsized":"2024-05-09T09:23:33.600388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:19.241362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.350594+0000","last_change":"2024-05-09T09:23:10.338342+0000","last_active":"2024-05-09T09:23:15.350594+0000","last_peered":"2024-05-09T09:23:15.350594+0000","last_clean":"2024-05-09T09:23:15.350594+0000","last_became_active":"2024-05-09T09:23:10.337995+0000","last_became_peered":"2024-05-09T09:23:10.337995+0000","last_unstale":"2024-05-09T09:23:15.350594+0000","last_undegraded":"2024-05-09T09:23:15.350594+0000","last_fullsized":"2024-05-09T09:23:15.350594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.691862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657562+0000","last_change":"2024-05-09T09:23:14.366237+0000","last_active":"2024-05-09T09:23:15.657562+0000","last_peered":"2024-05-09T09:23:15.657562+0000","last_clean":"2024-05-09T09:23:15.657562+0000","last_became_active":"2024-05-09T09:23:14.365809+0000","last_became_peered":"2024-05-09T09:23:14.365809+0000","last_unstale":"2024-05-09T09:23:15.657562+0000","last_undegraded":"2024-05-09T09:23:15.657562+0000","last_fullsized":"2024-05-09T09:23:15.657562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:28.813467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.594788+0000","last_change":"2024-05-09T09:23:09.298658+0000","last_active":"2024-05-09T09:23:33.594788+0000","last_peered":"2024-05-09T09:23:33.594788+0000","last_clean":"2024-05-09T09:23:33.594788+0000","last_became_active":"2024-05-09T09:23:09.298325+0000","last_became_peered":"2024-05-09T09:23:09.298325+0000","last_unstale":"2024-05-09T09:23:33.594788+0000","last_undegraded":"2024-05-09T09:23:33.594788+0000","last_fullsized":"2024-05-09T09:23:33.594788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:22.108202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.534236+0000","last_change":"2024-05-09T09:23:10.326865+0000","last_active":"2024-05-09T09:23:15.534236+0000","last_peered":"2024-05-09T09:23:15.534236+0000","last_clean":"2024-05-09T09:23:15.534236+0000","last_became_active":"2024-05-09T09:23:10.326478+0000","last_became_peered":"2024-05-09T09:23:10.326478+0000","last_unstale":"2024-05-09T09:23:15.534236+0000","last_undegraded":"2024-05-09T09:23:15.534236+0000","last_fullsized":"2024-05-09T09:23:15.534236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:54.521226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.350621+0000","last_change":"2024-05-09T09:23:14.373601+0000","last_active":"2024-05-09T09:23:15.350621+0000","last_peered":"2024-05-09T09:23:15.350621+0000","last_clean":"2024-05-09T09:23:15.350621+0000","last_became_active":"2024-05-09T09:23:14.372764+0000","last_became_peered":"2024-05-09T09:23:14.372764+0000","last_unstale":"2024-05-09T09:23:15.350621+0000","last_undegraded":"2024-05-09T09:23:15.350621+0000","last_fullsized":"2024-05-09T09:23:15.350621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:59.496755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:30.654246+0000","last_change":"2024-05-09T09:23:09.302293+0000","last_active":"2024-05-09T09:23:30.654246+0000","last_peered":"2024-05-09T09:23:30.654246+0000","last_clean":"2024-05-09T09:23:30.654246+0000","last_became_active":"2024-05-09T09:23:09.301216+0000","last_became_peered":"2024-05-09T09:23:09.301216+0000","last_unstale":"2024-05-09T09:23:30.654246+0000","last_undegraded":"2024-05-09T09:23:30.654246+0000","last_fullsized":"2024-05-09T09:23:30.654246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:35.146860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298293+0000","last_change":"2024-05-09T09:23:10.339827+0000","last_active":"2024-05-09T09:23:16.298293+0000","last_peered":"2024-05-09T09:23:16.298293+0000","last_clean":"2024-05-09T09:23:16.298293+0000","last_became_active":"2024-05-09T09:23:10.339415+0000","last_became_peered":"2024-05-09T09:23:10.339415+0000","last_unstale":"2024-05-09T09:23:16.298293+0000","last_undegraded":"2024-05-09T09:23:16.298293+0000","last_fullsized":"2024-05-09T09:23:16.298293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:45.250126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.570035+0000","last_change":"2024-05-09T09:23:14.377001+0000","last_active":"2024-05-09T09:23:15.570035+0000","last_peered":"2024-05-09T09:23:15.570035+0000","last_clean":"2024-05-09T09:23:15.570035+0000","last_became_active":"2024-05-09T09:23:14.376402+0000","last_became_peered":"2024-05-09T09:23:14.376402+0000","last_unstale":"2024-05-09T09:23:15.570035+0000","last_undegraded":"2024-05-09T09:23:15.570035+0000","last_fullsized":"2024-05-09T09:23:15.570035+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:08.142495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:32.678567+0000","last_change":"2024-05-09T09:23:09.299331+0000","last_active":"2024-05-09T09:23:32.678567+0000","last_peered":"2024-05-09T09:23:32.678567+0000","last_clean":"2024-05-09T09:23:32.678567+0000","last_became_active":"2024-05-09T09:23:09.299017+0000","last_became_peered":"2024-05-09T09:23:09.299017+0000","last_unstale":"2024-05-09T09:23:32.678567+0000","last_undegraded":"2024-05-09T09:23:32.678567+0000","last_fullsized":"2024-05-09T09:23:32.678567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:28.328704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.534003+0000","last_change":"2024-05-09T09:23:10.329463+0000","last_active":"2024-05-09T09:23:15.534003+0000","last_peered":"2024-05-09T09:23:15.534003+0000","last_clean":"2024-05-09T09:23:15.534003+0000","last_became_active":"2024-05-09T09:23:10.329119+0000","last_became_peered":"2024-05-09T09:23:10.329119+0000","last_unstale":"2024-05-09T09:23:15.534003+0000","last_undegraded":"2024-05-09T09:23:15.534003+0000","last_fullsized":"2024-05-09T09:23:15.534003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:20.699429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.350656+0000","last_change":"2024-05-09T09:23:14.363362+0000","last_active":"2024-05-09T09:23:15.350656+0000","last_peered":"2024-05-09T09:23:15.350656+0000","last_clean":"2024-05-09T09:23:15.350656+0000","last_became_active":"2024-05-09T09:23:14.362926+0000","last_became_peered":"2024-05-09T09:23:14.362926+0000","last_unstale":"2024-05-09T09:23:15.350656+0000","last_undegraded":"2024-05-09T09:23:15.350656+0000","last_fullsized":"2024-05-09T09:23:15.350656+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:25.339727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'21","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.603772+0000","last_change":"2024-05-09T09:23:09.297814+0000","last_active":"2024-05-09T09:23:33.603772+0000","last_peered":"2024-05-09T09:23:33.603772+0000","last_clean":"2024-05-09T09:23:33.603772+0000","last_became_active":"2024-05-09T09:23:09.297005+0000","last_became_peered":"2024-05-09T09:23:09.297005+0000","last_unstale":"2024-05-09T09:23:33.603772+0000","last_undegraded":"2024-05-09T09:23:33.603772+0000","last_fullsized":"2024-05-09T09:23:33.603772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:39.240217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":24,"num_write":23,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658183+0000","last_change":"2024-05-09T09:23:10.333013+0000","last_active":"2024-05-09T09:23:15.658183+0000","last_peered":"2024-05-09T09:23:15.658183+0000","last_clean":"2024-05-09T09:23:15.658183+0000","last_became_active":"2024-05-09T09:23:10.332666+0000","last_became_peered":"2024-05-09T09:23:10.332666+0000","last_unstale":"2024-05-09T09:23:15.658183+0000","last_undegraded":"2024-05-09T09:23:15.658183+0000","last_fullsized":"2024-05-09T09:23:15.658183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:28:50.306551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658411+0000","last_change":"2024-05-09T09:23:14.367715+0000","last_active":"2024-05-09T09:23:15.658411+0000","last_peered":"2024-05-09T09:23:15.658411+0000","last_clean":"2024-05-09T09:23:15.658411+0000","last_became_active":"2024-05-09T09:23:14.367307+0000","last_became_peered":"2024-05-09T09:23:14.367307+0000","last_unstale":"2024-05-09T09:23:15.658411+0000","last_undegraded":"2024-05-09T09:23:15.658411+0000","last_fullsized":"2024-05-09T09:23:15.658411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:09.075572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.603202+0000","last_change":"2024-05-09T09:23:09.306302+0000","last_active":"2024-05-09T09:23:33.603202+0000","last_peered":"2024-05-09T09:23:33.603202+0000","last_clean":"2024-05-09T09:23:33.603202+0000","last_became_active":"2024-05-09T09:23:09.306074+0000","last_became_peered":"2024-05-09T09:23:09.306074+0000","last_unstale":"2024-05-09T09:23:33.603202+0000","last_undegraded":"2024-05-09T09:23:33.603202+0000","last_fullsized":"2024-05-09T09:23:33.603202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:18.434899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656545+0000","last_change":"2024-05-09T09:23:10.331201+0000","last_active":"2024-05-09T09:23:15.656545+0000","last_peered":"2024-05-09T09:23:15.656545+0000","last_clean":"2024-05-09T09:23:15.656545+0000","last_became_active":"2024-05-09T09:23:10.329830+0000","last_became_peered":"2024-05-09T09:23:10.329830+0000","last_unstale":"2024-05-09T09:23:15.656545+0000","last_undegraded":"2024-05-09T09:23:15.656545+0000","last_fullsized":"2024-05-09T09:23:15.656545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:40.199622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658596+0000","last_change":"2024-05-09T09:23:14.359095+0000","last_active":"2024-05-09T09:23:15.658596+0000","last_peered":"2024-05-09T09:23:15.658596+0000","last_clean":"2024-05-09T09:23:15.658596+0000","last_became_active":"2024-05-09T09:23:14.358717+0000","last_became_peered":"2024-05-09T09:23:14.358717+0000","last_unstale":"2024-05-09T09:23:15.658596+0000","last_undegraded":"2024-05-09T09:23:15.658596+0000","last_fullsized":"2024-05-09T09:23:15.658596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:16:09.874767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:32.678332+0000","last_change":"2024-05-09T09:23:09.305370+0000","last_active":"2024-05-09T09:23:32.678332+0000","last_peered":"2024-05-09T09:23:32.678332+0000","last_clean":"2024-05-09T09:23:32.678332+0000","last_became_active":"2024-05-09T09:23:09.305054+0000","last_became_peered":"2024-05-09T09:23:09.305054+0000","last_unstale":"2024-05-09T09:23:32.678332+0000","last_undegraded":"2024-05-09T09:23:32.678332+0000","last_fullsized":"2024-05-09T09:23:32.678332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:28.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351454+0000","last_change":"2024-05-09T09:23:10.332541+0000","last_active":"2024-05-09T09:23:15.351454+0000","last_peered":"2024-05-09T09:23:15.351454+0000","last_clean":"2024-05-09T09:23:15.351454+0000","last_became_active":"2024-05-09T09:23:10.332155+0000","last_became_peered":"2024-05-09T09:23:10.332155+0000","last_unstale":"2024-05-09T09:23:15.351454+0000","last_undegraded":"2024-05-09T09:23:15.351454+0000","last_fullsized":"2024-05-09T09:23:15.351454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:39.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351216+0000","last_change":"2024-05-09T09:23:14.374499+0000","last_active":"2024-05-09T09:23:15.351216+0000","last_peered":"2024-05-09T09:23:15.351216+0000","last_clean":"2024-05-09T09:23:15.351216+0000","last_became_active":"2024-05-09T09:23:14.370688+0000","last_became_peered":"2024-05-09T09:23:14.370688+0000","last_unstale":"2024-05-09T09:23:15.351216+0000","last_undegraded":"2024-05-09T09:23:15.351216+0000","last_fullsized":"2024-05-09T09:23:15.351216+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:53.367237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569562+0000","last_change":"2024-05-09T09:23:09.297776+0000","last_active":"2024-05-09T09:23:15.569562+0000","last_peered":"2024-05-09T09:23:15.569562+0000","last_clean":"2024-05-09T09:23:15.569562+0000","last_became_active":"2024-05-09T09:23:09.297407+0000","last_became_peered":"2024-05-09T09:23:09.297407+0000","last_unstale":"2024-05-09T09:23:15.569562+0000","last_undegraded":"2024-05-09T09:23:15.569562+0000","last_fullsized":"2024-05-09T09:23:15.569562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:21.762941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351284+0000","last_change":"2024-05-09T09:23:10.339127+0000","last_active":"2024-05-09T09:23:15.351284+0000","last_peered":"2024-05-09T09:23:15.351284+0000","last_clean":"2024-05-09T09:23:15.351284+0000","last_became_active":"2024-05-09T09:23:10.338790+0000","last_became_peered":"2024-05-09T09:23:10.338790+0000","last_unstale":"2024-05-09T09:23:15.351284+0000","last_undegraded":"2024-05-09T09:23:15.351284+0000","last_fullsized":"2024-05-09T09:23:15.351284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:36.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657306+0000","last_change":"2024-05-09T09:23:14.373873+0000","last_active":"2024-05-09T09:23:15.657306+0000","last_peered":"2024-05-09T09:23:15.657306+0000","last_clean":"2024-05-09T09:23:15.657306+0000","last_became_active":"2024-05-09T09:23:14.371453+0000","last_became_peered":"2024-05-09T09:23:14.371453+0000","last_unstale":"2024-05-09T09:23:15.657306+0000","last_undegraded":"2024-05-09T09:23:15.657306+0000","last_fullsized":"2024-05-09T09:23:15.657306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:26.885938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:31.667221+0000","last_change":"2024-05-09T09:23:09.299373+0000","last_active":"2024-05-09T09:23:31.667221+0000","last_peered":"2024-05-09T09:23:31.667221+0000","last_clean":"2024-05-09T09:23:31.667221+0000","last_became_active":"2024-05-09T09:23:09.298926+0000","last_became_peered":"2024-05-09T09:23:09.298926+0000","last_unstale":"2024-05-09T09:23:31.667221+0000","last_undegraded":"2024-05-09T09:23:31.667221+0000","last_fullsized":"2024-05-09T09:23:31.667221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:35.629929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657311+0000","last_change":"2024-05-09T09:23:10.343911+0000","last_active":"2024-05-09T09:23:15.657311+0000","last_peered":"2024-05-09T09:23:15.657311+0000","last_clean":"2024-05-09T09:23:15.657311+0000","last_became_active":"2024-05-09T09:23:10.343568+0000","last_became_peered":"2024-05-09T09:23:10.343568+0000","last_unstale":"2024-05-09T09:23:15.657311+0000","last_undegraded":"2024-05-09T09:23:15.657311+0000","last_fullsized":"2024-05-09T09:23:15.657311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:07.311554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657000+0000","last_change":"2024-05-09T09:23:14.368544+0000","last_active":"2024-05-09T09:23:15.657000+0000","last_peered":"2024-05-09T09:23:15.657000+0000","last_clean":"2024-05-09T09:23:15.657000+0000","last_became_active":"2024-05-09T09:23:14.368140+0000","last_became_peered":"2024-05-09T09:23:14.368140+0000","last_unstale":"2024-05-09T09:23:15.657000+0000","last_undegraded":"2024-05-09T09:23:15.657000+0000","last_fullsized":"2024-05-09T09:23:15.657000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.091768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657887+0000","last_change":"2024-05-09T09:23:09.297612+0000","last_active":"2024-05-09T09:23:15.657887+0000","last_peered":"2024-05-09T09:23:15.657887+0000","last_clean":"2024-05-09T09:23:15.657887+0000","last_became_active":"2024-05-09T09:23:09.297216+0000","last_became_peered":"2024-05-09T09:23:09.297216+0000","last_unstale":"2024-05-09T09:23:15.657887+0000","last_undegraded":"2024-05-09T09:23:15.657887+0000","last_fullsized":"2024-05-09T09:23:15.657887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:09:45.108082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657262+0000","last_change":"2024-05-09T09:23:10.336514+0000","last_active":"2024-05-09T09:23:15.657262+0000","last_peered":"2024-05-09T09:23:15.657262+0000","last_clean":"2024-05-09T09:23:15.657262+0000","last_became_active":"2024-05-09T09:23:10.336173+0000","last_became_peered":"2024-05-09T09:23:10.336173+0000","last_unstale":"2024-05-09T09:23:15.657262+0000","last_undegraded":"2024-05-09T09:23:15.657262+0000","last_fullsized":"2024-05-09T09:23:15.657262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:46:39.036547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657880+0000","last_change":"2024-05-09T09:23:14.366119+0000","last_active":"2024-05-09T09:23:15.657880+0000","last_peered":"2024-05-09T09:23:15.657880+0000","last_clean":"2024-05-09T09:23:15.657880+0000","last_became_active":"2024-05-09T09:23:14.365659+0000","last_became_peered":"2024-05-09T09:23:14.365659+0000","last_unstale":"2024-05-09T09:23:15.657880+0000","last_undegraded":"2024-05-09T09:23:15.657880+0000","last_fullsized":"2024-05-09T09:23:15.657880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:23.312632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351092+0000","last_change":"2024-05-09T09:23:09.297504+0000","last_active":"2024-05-09T09:23:15.351092+0000","last_peered":"2024-05-09T09:23:15.351092+0000","last_clean":"2024-05-09T09:23:15.351092+0000","last_became_active":"2024-05-09T09:23:09.297134+0000","last_became_peered":"2024-05-09T09:23:09.297134+0000","last_unstale":"2024-05-09T09:23:15.351092+0000","last_undegraded":"2024-05-09T09:23:15.351092+0000","last_fullsized":"2024-05-09T09:23:15.351092+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:50:23.900097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656781+0000","last_change":"2024-05-09T09:23:10.339788+0000","last_active":"2024-05-09T09:23:15.656781+0000","last_peered":"2024-05-09T09:23:15.656781+0000","last_clean":"2024-05-09T09:23:15.656781+0000","last_became_active":"2024-05-09T09:23:10.339430+0000","last_became_peered":"2024-05-09T09:23:10.339430+0000","last_unstale":"2024-05-09T09:23:15.656781+0000","last_undegraded":"2024-05-09T09:23:15.656781+0000","last_fullsized":"2024-05-09T09:23:15.656781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:22.467287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657279+0000","last_change":"2024-05-09T09:23:14.368710+0000","last_active":"2024-05-09T09:23:15.657279+0000","last_peered":"2024-05-09T09:23:15.657279+0000","last_clean":"2024-05-09T09:23:15.657279+0000","last_became_active":"2024-05-09T09:23:14.368288+0000","last_became_peered":"2024-05-09T09:23:14.368288+0000","last_unstale":"2024-05-09T09:23:15.657279+0000","last_undegraded":"2024-05-09T09:23:15.657279+0000","last_fullsized":"2024-05-09T09:23:15.657279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:40.941273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351259+0000","last_change":"2024-05-09T09:23:09.301976+0000","last_active":"2024-05-09T09:23:15.351259+0000","last_peered":"2024-05-09T09:23:15.351259+0000","last_clean":"2024-05-09T09:23:15.351259+0000","last_became_active":"2024-05-09T09:23:09.301494+0000","last_became_peered":"2024-05-09T09:23:09.301494+0000","last_unstale":"2024-05-09T09:23:15.351259+0000","last_undegraded":"2024-05-09T09:23:15.351259+0000","last_fullsized":"2024-05-09T09:23:15.351259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.448246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656557+0000","last_change":"2024-05-09T09:23:10.331691+0000","last_active":"2024-05-09T09:23:15.656557+0000","last_peered":"2024-05-09T09:23:15.656557+0000","last_clean":"2024-05-09T09:23:15.656557+0000","last_became_active":"2024-05-09T09:23:10.331332+0000","last_became_peered":"2024-05-09T09:23:10.331332+0000","last_unstale":"2024-05-09T09:23:15.656557+0000","last_undegraded":"2024-05-09T09:23:15.656557+0000","last_fullsized":"2024-05-09T09:23:15.656557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:51.643695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656901+0000","last_change":"2024-05-09T09:23:14.367539+0000","last_active":"2024-05-09T09:23:15.656901+0000","last_peered":"2024-05-09T09:23:15.656901+0000","last_clean":"2024-05-09T09:23:15.656901+0000","last_became_active":"2024-05-09T09:23:14.366806+0000","last_became_peered":"2024-05-09T09:23:14.366806+0000","last_unstale":"2024-05-09T09:23:15.656901+0000","last_undegraded":"2024-05-09T09:23:15.656901+0000","last_fullsized":"2024-05-09T09:23:15.656901+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:58.679095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656853+0000","last_change":"2024-05-09T09:23:10.340849+0000","last_active":"2024-05-09T09:23:15.656853+0000","last_peered":"2024-05-09T09:23:15.656853+0000","last_clean":"2024-05-09T09:23:15.656853+0000","last_became_active":"2024-05-09T09:23:10.340430+0000","last_became_peered":"2024-05-09T09:23:10.340430+0000","last_unstale":"2024-05-09T09:23:15.656853+0000","last_undegraded":"2024-05-09T09:23:15.656853+0000","last_fullsized":"2024-05-09T09:23:15.656853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:33:47.827659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656358+0000","last_change":"2024-05-09T09:23:09.308544+0000","last_active":"2024-05-09T09:23:15.656358+0000","last_peered":"2024-05-09T09:23:15.656358+0000","last_clean":"2024-05-09T09:23:15.656358+0000","last_became_active":"2024-05-09T09:23:09.308195+0000","last_became_peered":"2024-05-09T09:23:09.308195+0000","last_unstale":"2024-05-09T09:23:15.656358+0000","last_undegraded":"2024-05-09T09:23:15.656358+0000","last_fullsized":"2024-05-09T09:23:15.656358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:17:34.884385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351662+0000","last_change":"2024-05-09T09:23:14.365921+0000","last_active":"2024-05-09T09:23:15.351662+0000","last_peered":"2024-05-09T09:23:15.351662+0000","last_clean":"2024-05-09T09:23:15.351662+0000","last_became_active":"2024-05-09T09:23:14.365427+0000","last_became_peered":"2024-05-09T09:23:14.365427+0000","last_unstale":"2024-05-09T09:23:15.351662+0000","last_undegraded":"2024-05-09T09:23:15.351662+0000","last_fullsized":"2024-05-09T09:23:15.351662+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:46.213043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657086+0000","last_change":"2024-05-09T09:23:10.341320+0000","last_active":"2024-05-09T09:23:15.657086+0000","last_peered":"2024-05-09T09:23:15.657086+0000","last_clean":"2024-05-09T09:23:15.657086+0000","last_became_active":"2024-05-09T09:23:10.340868+0000","last_became_peered":"2024-05-09T09:23:10.340868+0000","last_unstale":"2024-05-09T09:23:15.657086+0000","last_undegraded":"2024-05-09T09:23:15.657086+0000","last_fullsized":"2024-05-09T09:23:15.657086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:55.846642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569768+0000","last_change":"2024-05-09T09:23:09.299574+0000","last_active":"2024-05-09T09:23:15.569768+0000","last_peered":"2024-05-09T09:23:15.569768+0000","last_clean":"2024-05-09T09:23:15.569768+0000","last_became_active":"2024-05-09T09:23:09.298878+0000","last_became_peered":"2024-05-09T09:23:09.298878+0000","last_unstale":"2024-05-09T09:23:15.569768+0000","last_undegraded":"2024-05-09T09:23:15.569768+0000","last_fullsized":"2024-05-09T09:23:15.569768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.275907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.350976+0000","last_change":"2024-05-09T09:21:57.576132+0000","last_active":"2024-05-09T09:23:15.350976+0000","last_peered":"2024-05-09T09:23:15.350976+0000","last_clean":"2024-05-09T09:23:15.350976+0000","last_became_active":"2024-05-09T09:21:57.575602+0000","last_became_peered":"2024-05-09T09:21:57.575602+0000","last_unstale":"2024-05-09T09:23:15.350976+0000","last_undegraded":"2024-05-09T09:23:15.350976+0000","last_fullsized":"2024-05-09T09:23:15.350976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_clean_scrub_stamp":"2024-05-09T09:18:31.335063+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:07.639787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658595+0000","last_change":"2024-05-09T09:23:14.371184+0000","last_active":"2024-05-09T09:23:15.658595+0000","last_peered":"2024-05-09T09:23:15.658595+0000","last_clean":"2024-05-09T09:23:15.658595+0000","last_became_active":"2024-05-09T09:23:14.370780+0000","last_became_peered":"2024-05-09T09:23:14.370780+0000","last_unstale":"2024-05-09T09:23:15.658595+0000","last_undegraded":"2024-05-09T09:23:15.658595+0000","last_fullsized":"2024-05-09T09:23:15.658595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:39.361947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656223+0000","last_change":"2024-05-09T09:23:10.330915+0000","last_active":"2024-05-09T09:23:15.656223+0000","last_peered":"2024-05-09T09:23:15.656223+0000","last_clean":"2024-05-09T09:23:15.656223+0000","last_became_active":"2024-05-09T09:23:10.330458+0000","last_became_peered":"2024-05-09T09:23:10.330458+0000","last_unstale":"2024-05-09T09:23:15.656223+0000","last_undegraded":"2024-05-09T09:23:15.656223+0000","last_fullsized":"2024-05-09T09:23:15.656223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:10.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351248+0000","last_change":"2024-05-09T09:23:09.303895+0000","last_active":"2024-05-09T09:23:15.351248+0000","last_peered":"2024-05-09T09:23:15.351248+0000","last_clean":"2024-05-09T09:23:15.351248+0000","last_became_active":"2024-05-09T09:23:09.303555+0000","last_became_peered":"2024-05-09T09:23:09.303555+0000","last_unstale":"2024-05-09T09:23:15.351248+0000","last_undegraded":"2024-05-09T09:23:15.351248+0000","last_fullsized":"2024-05-09T09:23:15.351248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:04.350161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657152+0000","last_change":"2024-05-09T09:23:14.359390+0000","last_active":"2024-05-09T09:23:15.657152+0000","last_peered":"2024-05-09T09:23:15.657152+0000","last_clean":"2024-05-09T09:23:15.657152+0000","last_became_active":"2024-05-09T09:23:14.359000+0000","last_became_peered":"2024-05-09T09:23:14.359000+0000","last_unstale":"2024-05-09T09:23:15.657152+0000","last_undegraded":"2024-05-09T09:23:15.657152+0000","last_fullsized":"2024-05-09T09:23:15.657152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:25.077835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658340+0000","last_change":"2024-05-09T09:23:09.304535+0000","last_active":"2024-05-09T09:23:15.658340+0000","last_peered":"2024-05-09T09:23:15.658340+0000","last_clean":"2024-05-09T09:23:15.658340+0000","last_became_active":"2024-05-09T09:23:09.304166+0000","last_became_peered":"2024-05-09T09:23:09.304166+0000","last_unstale":"2024-05-09T09:23:15.658340+0000","last_undegraded":"2024-05-09T09:23:15.658340+0000","last_fullsized":"2024-05-09T09:23:15.658340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:03:20.281620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657122+0000","last_change":"2024-05-09T09:23:10.335657+0000","last_active":"2024-05-09T09:23:15.657122+0000","last_peered":"2024-05-09T09:23:15.657122+0000","last_clean":"2024-05-09T09:23:15.657122+0000","last_became_active":"2024-05-09T09:23:10.335231+0000","last_became_peered":"2024-05-09T09:23:10.335231+0000","last_unstale":"2024-05-09T09:23:15.657122+0000","last_undegraded":"2024-05-09T09:23:15.657122+0000","last_fullsized":"2024-05-09T09:23:15.657122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:07.990165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656821+0000","last_change":"2024-05-09T09:23:14.366988+0000","last_active":"2024-05-09T09:23:15.656821+0000","last_peered":"2024-05-09T09:23:15.656821+0000","last_clean":"2024-05-09T09:23:15.656821+0000","last_became_active":"2024-05-09T09:23:14.366562+0000","last_became_peered":"2024-05-09T09:23:14.366562+0000","last_unstale":"2024-05-09T09:23:15.656821+0000","last_undegraded":"2024-05-09T09:23:15.656821+0000","last_fullsized":"2024-05-09T09:23:15.656821+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:42.670036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:32.679031+0000","last_change":"2024-05-09T09:23:09.305541+0000","last_active":"2024-05-09T09:23:32.679031+0000","last_peered":"2024-05-09T09:23:32.679031+0000","last_clean":"2024-05-09T09:23:32.679031+0000","last_became_active":"2024-05-09T09:23:09.305267+0000","last_became_peered":"2024-05-09T09:23:09.305267+0000","last_unstale":"2024-05-09T09:23:32.679031+0000","last_undegraded":"2024-05-09T09:23:32.679031+0000","last_fullsized":"2024-05-09T09:23:32.679031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:04.614051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657936+0000","last_change":"2024-05-09T09:23:10.345840+0000","last_active":"2024-05-09T09:23:15.657936+0000","last_peered":"2024-05-09T09:23:15.657936+0000","last_clean":"2024-05-09T09:23:15.657936+0000","last_became_active":"2024-05-09T09:23:10.345487+0000","last_became_peered":"2024-05-09T09:23:10.345487+0000","last_unstale":"2024-05-09T09:23:15.657936+0000","last_undegraded":"2024-05-09T09:23:15.657936+0000","last_fullsized":"2024-05-09T09:23:15.657936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:47.723429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351106+0000","last_change":"2024-05-09T09:23:14.365485+0000","last_active":"2024-05-09T09:23:15.351106+0000","last_peered":"2024-05-09T09:23:15.351106+0000","last_clean":"2024-05-09T09:23:15.351106+0000","last_became_active":"2024-05-09T09:23:14.365018+0000","last_became_peered":"2024-05-09T09:23:14.365018+0000","last_unstale":"2024-05-09T09:23:15.351106+0000","last_undegraded":"2024-05-09T09:23:15.351106+0000","last_fullsized":"2024-05-09T09:23:15.351106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:54:04.712920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.570229+0000","last_change":"2024-05-09T09:23:09.302365+0000","last_active":"2024-05-09T09:23:15.570229+0000","last_peered":"2024-05-09T09:23:15.570229+0000","last_clean":"2024-05-09T09:23:15.570229+0000","last_became_active":"2024-05-09T09:23:09.302008+0000","last_became_peered":"2024-05-09T09:23:09.302008+0000","last_unstale":"2024-05-09T09:23:15.570229+0000","last_undegraded":"2024-05-09T09:23:15.570229+0000","last_fullsized":"2024-05-09T09:23:15.570229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:48.829940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298454+0000","last_change":"2024-05-09T09:23:10.339170+0000","last_active":"2024-05-09T09:23:16.298454+0000","last_peered":"2024-05-09T09:23:16.298454+0000","last_clean":"2024-05-09T09:23:16.298454+0000","last_became_active":"2024-05-09T09:23:10.338829+0000","last_became_peered":"2024-05-09T09:23:10.338829+0000","last_unstale":"2024-05-09T09:23:16.298454+0000","last_undegraded":"2024-05-09T09:23:16.298454+0000","last_fullsized":"2024-05-09T09:23:16.298454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:04.510495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.570060+0000","last_change":"2024-05-09T09:23:14.372622+0000","last_active":"2024-05-09T09:23:15.570060+0000","last_peered":"2024-05-09T09:23:15.570060+0000","last_clean":"2024-05-09T09:23:15.570060+0000","last_became_active":"2024-05-09T09:23:14.371800+0000","last_became_peered":"2024-05-09T09:23:14.371800+0000","last_unstale":"2024-05-09T09:23:15.570060+0000","last_undegraded":"2024-05-09T09:23:15.570060+0000","last_fullsized":"2024-05-09T09:23:15.570060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:02.925505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:30.653830+0000","last_change":"2024-05-09T09:23:09.297418+0000","last_active":"2024-05-09T09:23:30.653830+0000","last_peered":"2024-05-09T09:23:30.653830+0000","last_clean":"2024-05-09T09:23:30.653830+0000","last_became_active":"2024-05-09T09:23:09.296963+0000","last_became_peered":"2024-05-09T09:23:09.296963+0000","last_unstale":"2024-05-09T09:23:30.653830+0000","last_undegraded":"2024-05-09T09:23:30.653830+0000","last_fullsized":"2024-05-09T09:23:30.653830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:54.552857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657327+0000","last_change":"2024-05-09T09:23:10.325882+0000","last_active":"2024-05-09T09:23:15.657327+0000","last_peered":"2024-05-09T09:23:15.657327+0000","last_clean":"2024-05-09T09:23:15.657327+0000","last_became_active":"2024-05-09T09:23:10.325494+0000","last_became_peered":"2024-05-09T09:23:10.325494+0000","last_unstale":"2024-05-09T09:23:15.657327+0000","last_undegraded":"2024-05-09T09:23:15.657327+0000","last_fullsized":"2024-05-09T09:23:15.657327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:27.191310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351324+0000","last_change":"2024-05-09T09:23:14.375572+0000","last_active":"2024-05-09T09:23:15.351324+0000","last_peered":"2024-05-09T09:23:15.351324+0000","last_clean":"2024-05-09T09:23:15.351324+0000","last_became_active":"2024-05-09T09:23:14.374069+0000","last_became_peered":"2024-05-09T09:23:14.374069+0000","last_unstale":"2024-05-09T09:23:15.351324+0000","last_undegraded":"2024-05-09T09:23:15.351324+0000","last_fullsized":"2024-05-09T09:23:15.351324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:44.045012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.599776+0000","last_change":"2024-05-09T09:23:09.304137+0000","last_active":"2024-05-09T09:23:33.599776+0000","last_peered":"2024-05-09T09:23:33.599776+0000","last_clean":"2024-05-09T09:23:33.599776+0000","last_became_active":"2024-05-09T09:23:09.303762+0000","last_became_peered":"2024-05-09T09:23:09.303762+0000","last_unstale":"2024-05-09T09:23:33.599776+0000","last_undegraded":"2024-05-09T09:23:33.599776+0000","last_fullsized":"2024-05-09T09:23:33.599776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:56.811194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657220+0000","last_change":"2024-05-09T09:23:10.331998+0000","last_active":"2024-05-09T09:23:15.657220+0000","last_peered":"2024-05-09T09:23:15.657220+0000","last_clean":"2024-05-09T09:23:15.657220+0000","last_became_active":"2024-05-09T09:23:10.331642+0000","last_became_peered":"2024-05-09T09:23:10.331642+0000","last_unstale":"2024-05-09T09:23:15.657220+0000","last_undegraded":"2024-05-09T09:23:15.657220+0000","last_fullsized":"2024-05-09T09:23:15.657220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:41.569300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569892+0000","last_change":"2024-05-09T09:23:14.377444+0000","last_active":"2024-05-09T09:23:15.569892+0000","last_peered":"2024-05-09T09:23:15.569892+0000","last_clean":"2024-05-09T09:23:15.569892+0000","last_became_active":"2024-05-09T09:23:14.376189+0000","last_became_peered":"2024-05-09T09:23:14.376189+0000","last_unstale":"2024-05-09T09:23:15.569892+0000","last_undegraded":"2024-05-09T09:23:15.569892+0000","last_fullsized":"2024-05-09T09:23:15.569892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:06.973535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:32.678362+0000","last_change":"2024-05-09T09:23:09.304101+0000","last_active":"2024-05-09T09:23:32.678362+0000","last_peered":"2024-05-09T09:23:32.678362+0000","last_clean":"2024-05-09T09:23:32.678362+0000","last_became_active":"2024-05-09T09:23:09.303743+0000","last_became_peered":"2024-05-09T09:23:09.303743+0000","last_unstale":"2024-05-09T09:23:32.678362+0000","last_undegraded":"2024-05-09T09:23:32.678362+0000","last_fullsized":"2024-05-09T09:23:32.678362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:39.339815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657724+0000","last_change":"2024-05-09T09:23:10.345938+0000","last_active":"2024-05-09T09:23:15.657724+0000","last_peered":"2024-05-09T09:23:15.657724+0000","last_clean":"2024-05-09T09:23:15.657724+0000","last_became_active":"2024-05-09T09:23:10.345576+0000","last_became_peered":"2024-05-09T09:23:10.345576+0000","last_unstale":"2024-05-09T09:23:15.657724+0000","last_undegraded":"2024-05-09T09:23:15.657724+0000","last_fullsized":"2024-05-09T09:23:15.657724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:08.571900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569939+0000","last_change":"2024-05-09T09:23:14.371549+0000","last_active":"2024-05-09T09:23:15.569939+0000","last_peered":"2024-05-09T09:23:15.569939+0000","last_clean":"2024-05-09T09:23:15.569939+0000","last_became_active":"2024-05-09T09:23:14.369931+0000","last_became_peered":"2024-05-09T09:23:14.369931+0000","last_unstale":"2024-05-09T09:23:15.569939+0000","last_undegraded":"2024-05-09T09:23:15.569939+0000","last_fullsized":"2024-05-09T09:23:15.569939+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:20.777172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:30.652737+0000","last_change":"2024-05-09T09:23:09.302909+0000","last_active":"2024-05-09T09:23:30.652737+0000","last_peered":"2024-05-09T09:23:30.652737+0000","last_clean":"2024-05-09T09:23:30.652737+0000","last_became_active":"2024-05-09T09:23:09.302570+0000","last_became_peered":"2024-05-09T09:23:09.302570+0000","last_unstale":"2024-05-09T09:23:30.652737+0000","last_undegraded":"2024-05-09T09:23:30.652737+0000","last_fullsized":"2024-05-09T09:23:30.652737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:23.856755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351436+0000","last_change":"2024-05-09T09:23:10.342676+0000","last_active":"2024-05-09T09:23:15.351436+0000","last_peered":"2024-05-09T09:23:15.351436+0000","last_clean":"2024-05-09T09:23:15.351436+0000","last_became_active":"2024-05-09T09:23:10.341906+0000","last_became_peered":"2024-05-09T09:23:10.341906+0000","last_unstale":"2024-05-09T09:23:15.351436+0000","last_undegraded":"2024-05-09T09:23:15.351436+0000","last_fullsized":"2024-05-09T09:23:15.351436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:59:11.137648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.533812+0000","last_change":"2024-05-09T09:23:14.363779+0000","last_active":"2024-05-09T09:23:15.533812+0000","last_peered":"2024-05-09T09:23:15.533812+0000","last_clean":"2024-05-09T09:23:15.533812+0000","last_became_active":"2024-05-09T09:23:14.363307+0000","last_became_peered":"2024-05-09T09:23:14.363307+0000","last_unstale":"2024-05-09T09:23:15.533812+0000","last_undegraded":"2024-05-09T09:23:15.533812+0000","last_fullsized":"2024-05-09T09:23:15.533812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:46.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.596361+0000","last_change":"2024-05-09T09:23:09.298094+0000","last_active":"2024-05-09T09:23:33.596361+0000","last_peered":"2024-05-09T09:23:33.596361+0000","last_clean":"2024-05-09T09:23:33.596361+0000","last_became_active":"2024-05-09T09:23:09.297777+0000","last_became_peered":"2024-05-09T09:23:09.297777+0000","last_unstale":"2024-05-09T09:23:33.596361+0000","last_undegraded":"2024-05-09T09:23:33.596361+0000","last_fullsized":"2024-05-09T09:23:33.596361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:25.099193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351832+0000","last_change":"2024-05-09T09:23:10.340120+0000","last_active":"2024-05-09T09:23:15.351832+0000","last_peered":"2024-05-09T09:23:15.351832+0000","last_clean":"2024-05-09T09:23:15.351832+0000","last_became_active":"2024-05-09T09:23:10.339690+0000","last_became_peered":"2024-05-09T09:23:10.339690+0000","last_unstale":"2024-05-09T09:23:15.351832+0000","last_undegraded":"2024-05-09T09:23:15.351832+0000","last_fullsized":"2024-05-09T09:23:15.351832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:36.783411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298104+0000","last_change":"2024-05-09T09:23:14.377016+0000","last_active":"2024-05-09T09:23:16.298104+0000","last_peered":"2024-05-09T09:23:16.298104+0000","last_clean":"2024-05-09T09:23:16.298104+0000","last_became_active":"2024-05-09T09:23:14.376501+0000","last_became_peered":"2024-05-09T09:23:14.376501+0000","last_unstale":"2024-05-09T09:23:16.298104+0000","last_undegraded":"2024-05-09T09:23:16.298104+0000","last_fullsized":"2024-05-09T09:23:16.298104+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:30.826423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.592779+0000","last_change":"2024-05-09T09:23:09.307592+0000","last_active":"2024-05-09T09:23:33.592779+0000","last_peered":"2024-05-09T09:23:33.592779+0000","last_clean":"2024-05-09T09:23:33.592779+0000","last_became_active":"2024-05-09T09:23:09.307341+0000","last_became_peered":"2024-05-09T09:23:09.307341+0000","last_unstale":"2024-05-09T09:23:33.592779+0000","last_undegraded":"2024-05-09T09:23:33.592779+0000","last_fullsized":"2024-05-09T09:23:33.592779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:21.790980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351436+0000","last_change":"2024-05-09T09:23:10.349217+0000","last_active":"2024-05-09T09:23:15.351436+0000","last_peered":"2024-05-09T09:23:15.351436+0000","last_clean":"2024-05-09T09:23:15.351436+0000","last_became_active":"2024-05-09T09:23:10.348861+0000","last_became_peered":"2024-05-09T09:23:10.348861+0000","last_unstale":"2024-05-09T09:23:15.351436+0000","last_undegraded":"2024-05-09T09:23:15.351436+0000","last_fullsized":"2024-05-09T09:23:15.351436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:51.621045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656698+0000","last_change":"2024-05-09T09:23:14.363773+0000","last_active":"2024-05-09T09:23:15.656698+0000","last_peered":"2024-05-09T09:23:15.656698+0000","last_clean":"2024-05-09T09:23:15.656698+0000","last_became_active":"2024-05-09T09:23:14.363393+0000","last_became_peered":"2024-05-09T09:23:14.363393+0000","last_unstale":"2024-05-09T09:23:15.656698+0000","last_undegraded":"2024-05-09T09:23:15.656698+0000","last_fullsized":"2024-05-09T09:23:15.656698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:57.322985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657297+0000","last_change":"2024-05-09T09:23:09.308723+0000","last_active":"2024-05-09T09:23:15.657297+0000","last_peered":"2024-05-09T09:23:15.657297+0000","last_clean":"2024-05-09T09:23:15.657297+0000","last_became_active":"2024-05-09T09:23:09.308267+0000","last_became_peered":"2024-05-09T09:23:09.308267+0000","last_unstale":"2024-05-09T09:23:15.657297+0000","last_undegraded":"2024-05-09T09:23:15.657297+0000","last_fullsized":"2024-05-09T09:23:15.657297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:09:25.036421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351722+0000","last_change":"2024-05-09T09:23:10.342246+0000","last_active":"2024-05-09T09:23:15.351722+0000","last_peered":"2024-05-09T09:23:15.351722+0000","last_clean":"2024-05-09T09:23:15.351722+0000","last_became_active":"2024-05-09T09:23:10.341927+0000","last_became_peered":"2024-05-09T09:23:10.341927+0000","last_unstale":"2024-05-09T09:23:15.351722+0000","last_undegraded":"2024-05-09T09:23:15.351722+0000","last_fullsized":"2024-05-09T09:23:15.351722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:01.182226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657176+0000","last_change":"2024-05-09T09:23:10.332768+0000","last_active":"2024-05-09T09:23:15.657176+0000","last_peered":"2024-05-09T09:23:15.657176+0000","last_clean":"2024-05-09T09:23:15.657176+0000","last_became_active":"2024-05-09T09:23:10.332441+0000","last_became_peered":"2024-05-09T09:23:10.332441+0000","last_unstale":"2024-05-09T09:23:15.657176+0000","last_undegraded":"2024-05-09T09:23:15.657176+0000","last_fullsized":"2024-05-09T09:23:15.657176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:53.377058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351359+0000","last_change":"2024-05-09T09:23:10.337944+0000","last_active":"2024-05-09T09:23:15.351359+0000","last_peered":"2024-05-09T09:23:15.351359+0000","last_clean":"2024-05-09T09:23:15.351359+0000","last_became_active":"2024-05-09T09:23:10.337593+0000","last_became_peered":"2024-05-09T09:23:10.337593+0000","last_unstale":"2024-05-09T09:23:15.351359+0000","last_undegraded":"2024-05-09T09:23:15.351359+0000","last_fullsized":"2024-05-09T09:23:15.351359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:20.381543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351137+0000","last_change":"2024-05-09T09:23:10.337419+0000","last_active":"2024-05-09T09:23:15.351137+0000","last_peered":"2024-05-09T09:23:15.351137+0000","last_clean":"2024-05-09T09:23:15.351137+0000","last_became_active":"2024-05-09T09:23:10.336718+0000","last_became_peered":"2024-05-09T09:23:10.336718+0000","last_unstale":"2024-05-09T09:23:15.351137+0000","last_undegraded":"2024-05-09T09:23:15.351137+0000","last_fullsized":"2024-05-09T09:23:15.351137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:50.101110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298133+0000","last_change":"2024-05-09T09:23:10.340243+0000","last_active":"2024-05-09T09:23:16.298133+0000","last_peered":"2024-05-09T09:23:16.298133+0000","last_clean":"2024-05-09T09:23:16.298133+0000","last_became_active":"2024-05-09T09:23:10.339919+0000","last_became_peered":"2024-05-09T09:23:10.339919+0000","last_unstale":"2024-05-09T09:23:16.298133+0000","last_undegraded":"2024-05-09T09:23:16.298133+0000","last_fullsized":"2024-05-09T09:23:16.298133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:21.460928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657023+0000","last_change":"2024-05-09T09:23:10.333136+0000","last_active":"2024-05-09T09:23:15.657023+0000","last_peered":"2024-05-09T09:23:15.657023+0000","last_clean":"2024-05-09T09:23:15.657023+0000","last_became_active":"2024-05-09T09:23:10.332799+0000","last_became_peered":"2024-05-09T09:23:10.332799+0000","last_unstale":"2024-05-09T09:23:15.657023+0000","last_undegraded":"2024-05-09T09:23:15.657023+0000","last_fullsized":"2024-05-09T09:23:15.657023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:50.682747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657255+0000","last_change":"2024-05-09T09:23:10.340713+0000","last_active":"2024-05-09T09:23:15.657255+0000","last_peered":"2024-05-09T09:23:15.657255+0000","last_clean":"2024-05-09T09:23:15.657255+0000","last_became_active":"2024-05-09T09:23:10.340363+0000","last_became_peered":"2024-05-09T09:23:10.340363+0000","last_unstale":"2024-05-09T09:23:15.657255+0000","last_undegraded":"2024-05-09T09:23:15.657255+0000","last_fullsized":"2024-05-09T09:23:15.657255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:14:18.009211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351762+0000","last_change":"2024-05-09T09:23:10.341776+0000","last_active":"2024-05-09T09:23:15.351762+0000","last_peered":"2024-05-09T09:23:15.351762+0000","last_clean":"2024-05-09T09:23:15.351762+0000","last_became_active":"2024-05-09T09:23:10.341436+0000","last_became_peered":"2024-05-09T09:23:10.341436+0000","last_unstale":"2024-05-09T09:23:15.351762+0000","last_undegraded":"2024-05-09T09:23:15.351762+0000","last_fullsized":"2024-05-09T09:23:15.351762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:42.178856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658014+0000","last_change":"2024-05-09T09:23:10.331131+0000","last_active":"2024-05-09T09:23:15.658014+0000","last_peered":"2024-05-09T09:23:15.658014+0000","last_clean":"2024-05-09T09:23:15.658014+0000","last_became_active":"2024-05-09T09:23:10.330756+0000","last_became_peered":"2024-05-09T09:23:10.330756+0000","last_unstale":"2024-05-09T09:23:15.658014+0000","last_undegraded":"2024-05-09T09:23:15.658014+0000","last_fullsized":"2024-05-09T09:23:15.658014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:43:27.259375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657130+0000","last_change":"2024-05-09T09:23:10.337117+0000","last_active":"2024-05-09T09:23:15.657130+0000","last_peered":"2024-05-09T09:23:15.657130+0000","last_clean":"2024-05-09T09:23:15.657130+0000","last_became_active":"2024-05-09T09:23:10.336672+0000","last_became_peered":"2024-05-09T09:23:10.336672+0000","last_unstale":"2024-05-09T09:23:15.657130+0000","last_undegraded":"2024-05-09T09:23:15.657130+0000","last_fullsized":"2024-05-09T09:23:15.657130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:59:20.723742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657768+0000","last_change":"2024-05-09T09:23:10.337733+0000","last_active":"2024-05-09T09:23:15.657768+0000","last_peered":"2024-05-09T09:23:15.657768+0000","last_clean":"2024-05-09T09:23:15.657768+0000","last_became_active":"2024-05-09T09:23:10.337343+0000","last_became_peered":"2024-05-09T09:23:10.337343+0000","last_unstale":"2024-05-09T09:23:15.657768+0000","last_undegraded":"2024-05-09T09:23:15.657768+0000","last_fullsized":"2024-05-09T09:23:15.657768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:53.447911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.533886+0000","last_change":"2024-05-09T09:23:10.337485+0000","last_active":"2024-05-09T09:23:15.533886+0000","last_peered":"2024-05-09T09:23:15.533886+0000","last_clean":"2024-05-09T09:23:15.533886+0000","last_became_active":"2024-05-09T09:23:10.337129+0000","last_became_peered":"2024-05-09T09:23:10.337129+0000","last_unstale":"2024-05-09T09:23:15.533886+0000","last_undegraded":"2024-05-09T09:23:15.533886+0000","last_fullsized":"2024-05-09T09:23:15.533886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:20.758552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656565+0000","last_change":"2024-05-09T09:23:10.332271+0000","last_active":"2024-05-09T09:23:15.656565+0000","last_peered":"2024-05-09T09:23:15.656565+0000","last_clean":"2024-05-09T09:23:15.656565+0000","last_became_active":"2024-05-09T09:23:10.331941+0000","last_became_peered":"2024-05-09T09:23:10.331941+0000","last_unstale":"2024-05-09T09:23:15.656565+0000","last_undegraded":"2024-05-09T09:23:15.656565+0000","last_fullsized":"2024-05-09T09:23:15.656565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:27.707554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658611+0000","last_change":"2024-05-09T09:23:10.328965+0000","last_active":"2024-05-09T09:23:15.658611+0000","last_peered":"2024-05-09T09:23:15.658611+0000","last_clean":"2024-05-09T09:23:15.658611+0000","last_became_active":"2024-05-09T09:23:10.328618+0000","last_became_peered":"2024-05-09T09:23:10.328618+0000","last_unstale":"2024-05-09T09:23:15.658611+0000","last_undegraded":"2024-05-09T09:23:15.658611+0000","last_fullsized":"2024-05-09T09:23:15.658611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:16:02.003378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27423,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":17,"num_read_kb":35,"num_write":154,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":79153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":983040,"data_stored":476829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1654784,"data_stored":1092465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":477737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":477567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1581056,"data_stored":1067779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":476829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":480671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":478307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40520,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704632,"statfs":{"total":95995035648,"available":95953543168,"internally_reserved":0,"allocated":1048576,"data_stored":501942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":477259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":987136,"data_stored":476851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1074113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":42443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":25268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:23:35.514 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T09:23:35.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.656+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3200590001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff414100770 msgr2=0x7ff414100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.656+0000 7ff41b5e7640 1 --2- 172.21.15.12:0/3200590001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff414100770 0x7ff414100b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff410009900 tx=0x7ff41002a410 comp rx=0 tx=0).stop 2024-05-09T09:23:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.656+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3200590001 shutdown_connections 2024-05-09T09:23:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.656+0000 7ff41b5e7640 1 --2- 172.21.15.12:0/3200590001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff414102320 0x7ff414108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.656+0000 7ff41b5e7640 1 --2- 172.21.15.12:0/3200590001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff414101960 0x7ff414101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.656+0000 7ff41b5e7640 1 --2- 172.21.15.12:0/3200590001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff414100770 0x7ff414100b90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.656+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3200590001 >> 172.21.15.12:0/3200590001 conn(0x7ff4140fbce0 msgr2=0x7ff4140fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.657+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3200590001 shutdown_connections 2024-05-09T09:23:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.657+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3200590001 wait complete. 2024-05-09T09:23:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.657+0000 7ff41b5e7640 1 Processor -- start 2024-05-09T09:23:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.657+0000 7ff41b5e7640 1 -- start start 2024-05-09T09:23:35.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.657+0000 7ff41b5e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff414100770 0x7ff414069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:35.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff41b5e7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff414101960 0x7ff41419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:35.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff41b5e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff414102320 0x7ff41419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:35.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff41b5e7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff41410c3c0 con 0x7ff414101960 2024-05-09T09:23:35.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff41b5e7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff41410c240 con 0x7ff414100770 2024-05-09T09:23:35.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff41b5e7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff41410c540 con 0x7ff414102320 2024-05-09T09:23:35.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff418b82640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff414101960 0x7ff41419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:35.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff418b82640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff414101960 0x7ff41419c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56142/0 (socket says 172.21.15.12:56142) 2024-05-09T09:23:35.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff418b82640 1 -- 172.21.15.12:0/3978305484 learned_addr learned my addr 172.21.15.12:0/3978305484 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff419383640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff414100770 0x7ff414069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff419b84640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff414102320 0x7ff41419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff418b82640 1 -- 172.21.15.12:0/3978305484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff414102320 msgr2=0x7ff41419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff418b82640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff414102320 0x7ff41419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff418b82640 1 -- 172.21.15.12:0/3978305484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff414100770 msgr2=0x7ff414069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff418b82640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff414100770 0x7ff414069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff418b82640 1 -- 172.21.15.12:0/3978305484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41419f7e0 con 0x7ff414101960 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff418b82640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff414101960 0x7ff41419c8b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff40400e9f0 tx=0x7ff40400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.658+0000 7ff40a7fc640 1 -- 172.21.15.12:0/3978305484 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40400cd80 con 0x7ff414101960 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.659+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3978305484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41419fad0 con 0x7ff414101960 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.659+0000 7ff40a7fc640 1 -- 172.21.15.12:0/3978305484 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff404010900 con 0x7ff414101960 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.659+0000 7ff40a7fc640 1 -- 172.21.15.12:0/3978305484 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4040189e0 con 0x7ff414101960 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.659+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3978305484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4141a0070 con 0x7ff414101960 2024-05-09T09:23:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.660+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3978305484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff414107b60 con 0x7ff414101960 2024-05-09T09:23:35.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.661+0000 7ff40a7fc640 1 -- 172.21.15.12:0/3978305484 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff404010ab0 con 0x7ff414101960 2024-05-09T09:23:35.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.662+0000 7ff40a7fc640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff4000a2c20 0x7ff4000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:35.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.662+0000 7ff40a7fc640 1 -- 172.21.15.12:0/3978305484 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff4040cfba0 con 0x7ff414101960 2024-05-09T09:23:35.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.662+0000 7ff419383640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff4000a2c20 0x7ff4000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:35.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.662+0000 7ff419383640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff4000a2c20 0x7ff4000a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff410006010 tx=0x7ff410031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:35.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.665+0000 7ff40a7fc640 1 -- 172.21.15.12:0/3978305484 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff40409c110 con 0x7ff414101960 2024-05-09T09:23:35.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.848+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3978305484 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff414061c70 con 0x7ff4000a2c20 2024-05-09T09:23:35.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.857+0000 7ff40a7fc640 1 -- 172.21.15.12:0/3978305484 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340364 (secure 0 0 0) 0x7ff414061c70 con 0x7ff4000a2c20 2024-05-09T09:23:35.862 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-09T09:23:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.860+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3978305484 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff4000a2c20 msgr2=0x7ff4000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.860+0000 7ff41b5e7640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff4000a2c20 0x7ff4000a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff410006010 tx=0x7ff410031510 comp rx=0 tx=0).stop 2024-05-09T09:23:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.860+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3978305484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff414101960 msgr2=0x7ff41419c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.860+0000 7ff41b5e7640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff414101960 0x7ff41419c8b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff40400e9f0 tx=0x7ff40400eeb0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.860+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3978305484 shutdown_connections 2024-05-09T09:23:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.860+0000 7ff41b5e7640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff4000a2c20 0x7ff4000a50e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.860+0000 7ff41b5e7640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff414102320 0x7ff41419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.860+0000 7ff41b5e7640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff414101960 0x7ff41419c8b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.861+0000 7ff41b5e7640 1 --2- 172.21.15.12:0/3978305484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff414100770 0x7ff414069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.861+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3978305484 >> 172.21.15.12:0/3978305484 conn(0x7ff4140fbce0 msgr2=0x7ff414104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.861+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3978305484 shutdown_connections 2024-05-09T09:23:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:35.861+0000 7ff41b5e7640 1 -- 172.21.15.12:0/3978305484 wait complete. 2024-05-09T09:23:35.870 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:35 smithi012 ceph-mon[30140]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:35.870 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:35 smithi012 ceph-mon[30140]: pgmap v304: 129 pgs: 129 active+clean; 603 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-05-09T09:23:35.870 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:35 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1816629609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:35.870 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:35 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3094347225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:35.870 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:35 smithi012 ceph-mon[30140]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T09:23:35.871 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-05-09T09:23:34.555477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617791,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":143,"num_read_kb":144,"num_write":335,"num_write_kb":3006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387720,"kb_used_data":13568,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554104,"statfs":{"total":1151940427776,"available":1151543402496,"internally_reserved":0,"allocated":13893632,"data_stored":7558349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25013,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":17,"num_read_kb":35,"num_write":109,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003742"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657727+0000","last_change":"2024-05-09T09:23:10.334738+0000","last_active":"2024-05-09T09:23:15.657727+0000","last_peered":"2024-05-09T09:23:15.657727+0000","last_clean":"2024-05-09T09:23:15.657727+0000","last_became_active":"2024-05-09T09:23:10.333406+0000","last_became_peered":"2024-05-09T09:23:10.333406+0000","last_unstale":"2024-05-09T09:23:15.657727+0000","last_undegraded":"2024-05-09T09:23:15.657727+0000","last_fullsized":"2024-05-09T09:23:15.657727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:44.957557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298435+0000","last_change":"2024-05-09T09:23:10.331860+0000","last_active":"2024-05-09T09:23:16.298435+0000","last_peered":"2024-05-09T09:23:16.298435+0000","last_clean":"2024-05-09T09:23:16.298435+0000","last_became_active":"2024-05-09T09:23:10.331396+0000","last_became_peered":"2024-05-09T09:23:10.331396+0000","last_unstale":"2024-05-09T09:23:16.298435+0000","last_undegraded":"2024-05-09T09:23:16.298435+0000","last_fullsized":"2024-05-09T09:23:16.298435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:47.676399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298298+0000","last_change":"2024-05-09T09:23:10.336543+0000","last_active":"2024-05-09T09:23:16.298298+0000","last_peered":"2024-05-09T09:23:16.298298+0000","last_clean":"2024-05-09T09:23:16.298298+0000","last_became_active":"2024-05-09T09:23:10.336189+0000","last_became_peered":"2024-05-09T09:23:10.336189+0000","last_unstale":"2024-05-09T09:23:16.298298+0000","last_undegraded":"2024-05-09T09:23:16.298298+0000","last_fullsized":"2024-05-09T09:23:16.298298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:06.419177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.570095+0000","last_change":"2024-05-09T09:23:10.331739+0000","last_active":"2024-05-09T09:23:15.570095+0000","last_peered":"2024-05-09T09:23:15.570095+0000","last_clean":"2024-05-09T09:23:15.570095+0000","last_became_active":"2024-05-09T09:23:10.331333+0000","last_became_peered":"2024-05-09T09:23:10.331333+0000","last_unstale":"2024-05-09T09:23:15.570095+0000","last_undegraded":"2024-05-09T09:23:15.570095+0000","last_fullsized":"2024-05-09T09:23:15.570095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:04.925697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657434+0000","last_change":"2024-05-09T09:23:10.345294+0000","last_active":"2024-05-09T09:23:15.657434+0000","last_peered":"2024-05-09T09:23:15.657434+0000","last_clean":"2024-05-09T09:23:15.657434+0000","last_became_active":"2024-05-09T09:23:10.344940+0000","last_became_peered":"2024-05-09T09:23:10.344940+0000","last_unstale":"2024-05-09T09:23:15.657434+0000","last_undegraded":"2024-05-09T09:23:15.657434+0000","last_fullsized":"2024-05-09T09:23:15.657434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:41.150284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656543+0000","last_change":"2024-05-09T09:23:10.329143+0000","last_active":"2024-05-09T09:23:15.656543+0000","last_peered":"2024-05-09T09:23:15.656543+0000","last_clean":"2024-05-09T09:23:15.656543+0000","last_became_active":"2024-05-09T09:23:10.328712+0000","last_became_peered":"2024-05-09T09:23:10.328712+0000","last_unstale":"2024-05-09T09:23:15.656543+0000","last_undegraded":"2024-05-09T09:23:15.656543+0000","last_fullsized":"2024-05-09T09:23:15.656543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:10:35.085759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351233+0000","last_change":"2024-05-09T09:23:10.338176+0000","last_active":"2024-05-09T09:23:15.351233+0000","last_peered":"2024-05-09T09:23:15.351233+0000","last_clean":"2024-05-09T09:23:15.351233+0000","last_became_active":"2024-05-09T09:23:10.337845+0000","last_became_peered":"2024-05-09T09:23:10.337845+0000","last_unstale":"2024-05-09T09:23:15.351233+0000","last_undegraded":"2024-05-09T09:23:15.351233+0000","last_fullsized":"2024-05-09T09:23:15.351233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:34:21.783295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656321+0000","last_change":"2024-05-09T09:23:10.342473+0000","last_active":"2024-05-09T09:23:15.656321+0000","last_peered":"2024-05-09T09:23:15.656321+0000","last_clean":"2024-05-09T09:23:15.656321+0000","last_became_active":"2024-05-09T09:23:10.340682+0000","last_became_peered":"2024-05-09T09:23:10.340682+0000","last_unstale":"2024-05-09T09:23:15.656321+0000","last_undegraded":"2024-05-09T09:23:15.656321+0000","last_fullsized":"2024-05-09T09:23:15.656321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:56.730332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.534240+0000","last_change":"2024-05-09T09:23:10.339888+0000","last_active":"2024-05-09T09:23:15.534240+0000","last_peered":"2024-05-09T09:23:15.534240+0000","last_clean":"2024-05-09T09:23:15.534240+0000","last_became_active":"2024-05-09T09:23:10.337849+0000","last_became_peered":"2024-05-09T09:23:10.337849+0000","last_unstale":"2024-05-09T09:23:15.534240+0000","last_undegraded":"2024-05-09T09:23:15.534240+0000","last_fullsized":"2024-05-09T09:23:15.534240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:48.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657034+0000","last_change":"2024-05-09T09:23:10.332395+0000","last_active":"2024-05-09T09:23:15.657034+0000","last_peered":"2024-05-09T09:23:15.657034+0000","last_clean":"2024-05-09T09:23:15.657034+0000","last_became_active":"2024-05-09T09:23:10.332040+0000","last_became_peered":"2024-05-09T09:23:10.332040+0000","last_unstale":"2024-05-09T09:23:15.657034+0000","last_undegraded":"2024-05-09T09:23:15.657034+0000","last_fullsized":"2024-05-09T09:23:15.657034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:04.036871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351804+0000","last_change":"2024-05-09T09:23:10.341148+0000","last_active":"2024-05-09T09:23:15.351804+0000","last_peered":"2024-05-09T09:23:15.351804+0000","last_clean":"2024-05-09T09:23:15.351804+0000","last_became_active":"2024-05-09T09:23:10.340799+0000","last_became_peered":"2024-05-09T09:23:10.340799+0000","last_unstale":"2024-05-09T09:23:15.351804+0000","last_undegraded":"2024-05-09T09:23:15.351804+0000","last_fullsized":"2024-05-09T09:23:15.351804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:31.020659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.570082+0000","last_change":"2024-05-09T09:23:10.336191+0000","last_active":"2024-05-09T09:23:15.570082+0000","last_peered":"2024-05-09T09:23:15.570082+0000","last_clean":"2024-05-09T09:23:15.570082+0000","last_became_active":"2024-05-09T09:23:10.333591+0000","last_became_peered":"2024-05-09T09:23:10.333591+0000","last_unstale":"2024-05-09T09:23:15.570082+0000","last_undegraded":"2024-05-09T09:23:15.570082+0000","last_fullsized":"2024-05-09T09:23:15.570082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:24.836954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656523+0000","last_change":"2024-05-09T09:23:10.314506+0000","last_active":"2024-05-09T09:23:15.656523+0000","last_peered":"2024-05-09T09:23:15.656523+0000","last_clean":"2024-05-09T09:23:15.656523+0000","last_became_active":"2024-05-09T09:23:10.314171+0000","last_became_peered":"2024-05-09T09:23:10.314171+0000","last_unstale":"2024-05-09T09:23:15.656523+0000","last_undegraded":"2024-05-09T09:23:15.656523+0000","last_fullsized":"2024-05-09T09:23:15.656523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:51.088814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.297949+0000","last_change":"2024-05-09T09:23:10.341670+0000","last_active":"2024-05-09T09:23:16.297949+0000","last_peered":"2024-05-09T09:23:16.297949+0000","last_clean":"2024-05-09T09:23:16.297949+0000","last_became_active":"2024-05-09T09:23:10.341155+0000","last_became_peered":"2024-05-09T09:23:10.341155+0000","last_unstale":"2024-05-09T09:23:16.297949+0000","last_undegraded":"2024-05-09T09:23:16.297949+0000","last_fullsized":"2024-05-09T09:23:16.297949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:14:32.734302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298058+0000","last_change":"2024-05-09T09:23:10.333113+0000","last_active":"2024-05-09T09:23:16.298058+0000","last_peered":"2024-05-09T09:23:16.298058+0000","last_clean":"2024-05-09T09:23:16.298058+0000","last_became_active":"2024-05-09T09:23:10.332782+0000","last_became_peered":"2024-05-09T09:23:10.332782+0000","last_unstale":"2024-05-09T09:23:16.298058+0000","last_undegraded":"2024-05-09T09:23:16.298058+0000","last_fullsized":"2024-05-09T09:23:16.298058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:25.555681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.533557+0000","last_change":"2024-05-09T09:23:10.337367+0000","last_active":"2024-05-09T09:23:15.533557+0000","last_peered":"2024-05-09T09:23:15.533557+0000","last_clean":"2024-05-09T09:23:15.533557+0000","last_became_active":"2024-05-09T09:23:10.337000+0000","last_became_peered":"2024-05-09T09:23:10.337000+0000","last_unstale":"2024-05-09T09:23:15.533557+0000","last_undegraded":"2024-05-09T09:23:15.533557+0000","last_fullsized":"2024-05-09T09:23:15.533557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:54.431820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657070+0000","last_change":"2024-05-09T09:23:10.334293+0000","last_active":"2024-05-09T09:23:15.657070+0000","last_peered":"2024-05-09T09:23:15.657070+0000","last_clean":"2024-05-09T09:23:15.657070+0000","last_became_active":"2024-05-09T09:23:10.333938+0000","last_became_peered":"2024-05-09T09:23:10.333938+0000","last_unstale":"2024-05-09T09:23:15.657070+0000","last_undegraded":"2024-05-09T09:23:15.657070+0000","last_fullsized":"2024-05-09T09:23:15.657070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:56:09.048848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351360+0000","last_change":"2024-05-09T09:23:10.332394+0000","last_active":"2024-05-09T09:23:15.351360+0000","last_peered":"2024-05-09T09:23:15.351360+0000","last_clean":"2024-05-09T09:23:15.351360+0000","last_became_active":"2024-05-09T09:23:10.332054+0000","last_became_peered":"2024-05-09T09:23:10.332054+0000","last_unstale":"2024-05-09T09:23:15.351360+0000","last_undegraded":"2024-05-09T09:23:15.351360+0000","last_fullsized":"2024-05-09T09:23:15.351360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:05.628740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351692+0000","last_change":"2024-05-09T09:23:10.336189+0000","last_active":"2024-05-09T09:23:15.351692+0000","last_peered":"2024-05-09T09:23:15.351692+0000","last_clean":"2024-05-09T09:23:15.351692+0000","last_became_active":"2024-05-09T09:23:10.335698+0000","last_became_peered":"2024-05-09T09:23:10.335698+0000","last_unstale":"2024-05-09T09:23:15.351692+0000","last_undegraded":"2024-05-09T09:23:15.351692+0000","last_fullsized":"2024-05-09T09:23:15.351692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:01.403918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351498+0000","last_change":"2024-05-09T09:23:14.372134+0000","last_active":"2024-05-09T09:23:15.351498+0000","last_peered":"2024-05-09T09:23:15.351498+0000","last_clean":"2024-05-09T09:23:15.351498+0000","last_became_active":"2024-05-09T09:23:14.371466+0000","last_became_peered":"2024-05-09T09:23:14.371466+0000","last_unstale":"2024-05-09T09:23:15.351498+0000","last_undegraded":"2024-05-09T09:23:15.351498+0000","last_fullsized":"2024-05-09T09:23:15.351498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:22.048565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'11","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:32.676589+0000","last_change":"2024-05-09T09:23:09.304421+0000","last_active":"2024-05-09T09:23:32.676589+0000","last_peered":"2024-05-09T09:23:32.676589+0000","last_clean":"2024-05-09T09:23:32.676589+0000","last_became_active":"2024-05-09T09:23:09.304109+0000","last_became_peered":"2024-05-09T09:23:09.304109+0000","last_unstale":"2024-05-09T09:23:32.676589+0000","last_undegraded":"2024-05-09T09:23:32.676589+0000","last_fullsized":"2024-05-09T09:23:32.676589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:36.796458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656684+0000","last_change":"2024-05-09T09:23:10.341875+0000","last_active":"2024-05-09T09:23:15.656684+0000","last_peered":"2024-05-09T09:23:15.656684+0000","last_clean":"2024-05-09T09:23:15.656684+0000","last_became_active":"2024-05-09T09:23:10.341385+0000","last_became_peered":"2024-05-09T09:23:10.341385+0000","last_unstale":"2024-05-09T09:23:15.656684+0000","last_undegraded":"2024-05-09T09:23:15.656684+0000","last_fullsized":"2024-05-09T09:23:15.656684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:44:33.939722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569491+0000","last_change":"2024-05-09T09:23:14.371120+0000","last_active":"2024-05-09T09:23:15.569491+0000","last_peered":"2024-05-09T09:23:15.569491+0000","last_clean":"2024-05-09T09:23:15.569491+0000","last_became_active":"2024-05-09T09:23:14.370663+0000","last_became_peered":"2024-05-09T09:23:14.370663+0000","last_unstale":"2024-05-09T09:23:15.569491+0000","last_undegraded":"2024-05-09T09:23:15.569491+0000","last_fullsized":"2024-05-09T09:23:15.569491+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:50:06.553518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657433+0000","last_change":"2024-05-09T09:23:09.304357+0000","last_active":"2024-05-09T09:23:15.657433+0000","last_peered":"2024-05-09T09:23:15.657433+0000","last_clean":"2024-05-09T09:23:15.657433+0000","last_became_active":"2024-05-09T09:23:09.304007+0000","last_became_peered":"2024-05-09T09:23:09.304007+0000","last_unstale":"2024-05-09T09:23:15.657433+0000","last_undegraded":"2024-05-09T09:23:15.657433+0000","last_fullsized":"2024-05-09T09:23:15.657433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:32:46.132287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298413+0000","last_change":"2024-05-09T09:23:10.339533+0000","last_active":"2024-05-09T09:23:16.298413+0000","last_peered":"2024-05-09T09:23:16.298413+0000","last_clean":"2024-05-09T09:23:16.298413+0000","last_became_active":"2024-05-09T09:23:10.337142+0000","last_became_peered":"2024-05-09T09:23:10.337142+0000","last_unstale":"2024-05-09T09:23:16.298413+0000","last_undegraded":"2024-05-09T09:23:16.298413+0000","last_fullsized":"2024-05-09T09:23:16.298413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:11:11.921491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569666+0000","last_change":"2024-05-09T09:23:14.377188+0000","last_active":"2024-05-09T09:23:15.569666+0000","last_peered":"2024-05-09T09:23:15.569666+0000","last_clean":"2024-05-09T09:23:15.569666+0000","last_became_active":"2024-05-09T09:23:14.376247+0000","last_became_peered":"2024-05-09T09:23:14.376247+0000","last_unstale":"2024-05-09T09:23:15.569666+0000","last_undegraded":"2024-05-09T09:23:15.569666+0000","last_fullsized":"2024-05-09T09:23:15.569666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:32:34.054087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:31.666247+0000","last_change":"2024-05-09T09:23:09.300754+0000","last_active":"2024-05-09T09:23:31.666247+0000","last_peered":"2024-05-09T09:23:31.666247+0000","last_clean":"2024-05-09T09:23:31.666247+0000","last_became_active":"2024-05-09T09:23:09.300419+0000","last_became_peered":"2024-05-09T09:23:09.300419+0000","last_unstale":"2024-05-09T09:23:31.666247+0000","last_undegraded":"2024-05-09T09:23:31.666247+0000","last_fullsized":"2024-05-09T09:23:31.666247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:06.205582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.533898+0000","last_change":"2024-05-09T09:23:10.331952+0000","last_active":"2024-05-09T09:23:15.533898+0000","last_peered":"2024-05-09T09:23:15.533898+0000","last_clean":"2024-05-09T09:23:15.533898+0000","last_became_active":"2024-05-09T09:23:10.331515+0000","last_became_peered":"2024-05-09T09:23:10.331515+0000","last_unstale":"2024-05-09T09:23:15.533898+0000","last_undegraded":"2024-05-09T09:23:15.533898+0000","last_fullsized":"2024-05-09T09:23:15.533898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:23.809672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569617+0000","last_change":"2024-05-09T09:23:14.368518+0000","last_active":"2024-05-09T09:23:15.569617+0000","last_peered":"2024-05-09T09:23:15.569617+0000","last_clean":"2024-05-09T09:23:15.569617+0000","last_became_active":"2024-05-09T09:23:14.367995+0000","last_became_peered":"2024-05-09T09:23:14.367995+0000","last_unstale":"2024-05-09T09:23:15.569617+0000","last_undegraded":"2024-05-09T09:23:15.569617+0000","last_fullsized":"2024-05-09T09:23:15.569617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:11.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658412+0000","last_change":"2024-05-09T09:23:09.299656+0000","last_active":"2024-05-09T09:23:15.658412+0000","last_peered":"2024-05-09T09:23:15.658412+0000","last_clean":"2024-05-09T09:23:15.658412+0000","last_became_active":"2024-05-09T09:23:09.299374+0000","last_became_peered":"2024-05-09T09:23:09.299374+0000","last_unstale":"2024-05-09T09:23:15.658412+0000","last_undegraded":"2024-05-09T09:23:15.658412+0000","last_fullsized":"2024-05-09T09:23:15.658412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:05.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658068+0000","last_change":"2024-05-09T09:23:10.341933+0000","last_active":"2024-05-09T09:23:15.658068+0000","last_peered":"2024-05-09T09:23:15.658068+0000","last_clean":"2024-05-09T09:23:15.658068+0000","last_became_active":"2024-05-09T09:23:10.341621+0000","last_became_peered":"2024-05-09T09:23:10.341621+0000","last_unstale":"2024-05-09T09:23:15.658068+0000","last_undegraded":"2024-05-09T09:23:15.658068+0000","last_fullsized":"2024-05-09T09:23:15.658068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:31.938768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351459+0000","last_change":"2024-05-09T09:23:14.364783+0000","last_active":"2024-05-09T09:23:15.351459+0000","last_peered":"2024-05-09T09:23:15.351459+0000","last_clean":"2024-05-09T09:23:15.351459+0000","last_became_active":"2024-05-09T09:23:14.364309+0000","last_became_peered":"2024-05-09T09:23:14.364309+0000","last_unstale":"2024-05-09T09:23:15.351459+0000","last_undegraded":"2024-05-09T09:23:15.351459+0000","last_fullsized":"2024-05-09T09:23:15.351459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:44.252111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.595340+0000","last_change":"2024-05-09T09:23:09.297913+0000","last_active":"2024-05-09T09:23:33.595340+0000","last_peered":"2024-05-09T09:23:33.595340+0000","last_clean":"2024-05-09T09:23:33.595340+0000","last_became_active":"2024-05-09T09:23:09.297612+0000","last_became_peered":"2024-05-09T09:23:09.297612+0000","last_unstale":"2024-05-09T09:23:33.595340+0000","last_undegraded":"2024-05-09T09:23:33.595340+0000","last_fullsized":"2024-05-09T09:23:33.595340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:21.808703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657947+0000","last_change":"2024-05-09T09:23:10.339101+0000","last_active":"2024-05-09T09:23:15.657947+0000","last_peered":"2024-05-09T09:23:15.657947+0000","last_clean":"2024-05-09T09:23:15.657947+0000","last_became_active":"2024-05-09T09:23:10.338453+0000","last_became_peered":"2024-05-09T09:23:10.338453+0000","last_unstale":"2024-05-09T09:23:15.657947+0000","last_undegraded":"2024-05-09T09:23:15.657947+0000","last_fullsized":"2024-05-09T09:23:15.657947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:34:44.933882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351787+0000","last_change":"2024-05-09T09:23:14.371781+0000","last_active":"2024-05-09T09:23:15.351787+0000","last_peered":"2024-05-09T09:23:15.351787+0000","last_clean":"2024-05-09T09:23:15.351787+0000","last_became_active":"2024-05-09T09:23:14.371286+0000","last_became_peered":"2024-05-09T09:23:14.371286+0000","last_unstale":"2024-05-09T09:23:15.351787+0000","last_undegraded":"2024-05-09T09:23:15.351787+0000","last_fullsized":"2024-05-09T09:23:15.351787+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:26:29.185064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:32.677863+0000","last_change":"2024-05-09T09:23:09.303786+0000","last_active":"2024-05-09T09:23:32.677863+0000","last_peered":"2024-05-09T09:23:32.677863+0000","last_clean":"2024-05-09T09:23:32.677863+0000","last_became_active":"2024-05-09T09:23:09.303475+0000","last_became_peered":"2024-05-09T09:23:09.303475+0000","last_unstale":"2024-05-09T09:23:32.677863+0000","last_undegraded":"2024-05-09T09:23:32.677863+0000","last_fullsized":"2024-05-09T09:23:32.677863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:28.833055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657891+0000","last_change":"2024-05-09T09:23:10.344643+0000","last_active":"2024-05-09T09:23:15.657891+0000","last_peered":"2024-05-09T09:23:15.657891+0000","last_clean":"2024-05-09T09:23:15.657891+0000","last_became_active":"2024-05-09T09:23:10.344314+0000","last_became_peered":"2024-05-09T09:23:10.344314+0000","last_unstale":"2024-05-09T09:23:15.657891+0000","last_undegraded":"2024-05-09T09:23:15.657891+0000","last_fullsized":"2024-05-09T09:23:15.657891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:15.147107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657484+0000","last_change":"2024-05-09T09:23:14.361170+0000","last_active":"2024-05-09T09:23:15.657484+0000","last_peered":"2024-05-09T09:23:15.657484+0000","last_clean":"2024-05-09T09:23:15.657484+0000","last_became_active":"2024-05-09T09:23:14.360767+0000","last_became_peered":"2024-05-09T09:23:14.360767+0000","last_unstale":"2024-05-09T09:23:15.657484+0000","last_undegraded":"2024-05-09T09:23:15.657484+0000","last_fullsized":"2024-05-09T09:23:15.657484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:59:21.393148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656335+0000","last_change":"2024-05-09T09:23:09.300757+0000","last_active":"2024-05-09T09:23:15.656335+0000","last_peered":"2024-05-09T09:23:15.656335+0000","last_clean":"2024-05-09T09:23:15.656335+0000","last_became_active":"2024-05-09T09:23:09.300328+0000","last_became_peered":"2024-05-09T09:23:09.300328+0000","last_unstale":"2024-05-09T09:23:15.656335+0000","last_undegraded":"2024-05-09T09:23:15.656335+0000","last_fullsized":"2024-05-09T09:23:15.656335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.553878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657457+0000","last_change":"2024-05-09T09:23:10.336077+0000","last_active":"2024-05-09T09:23:15.657457+0000","last_peered":"2024-05-09T09:23:15.657457+0000","last_clean":"2024-05-09T09:23:15.657457+0000","last_became_active":"2024-05-09T09:23:10.335762+0000","last_became_peered":"2024-05-09T09:23:10.335762+0000","last_unstale":"2024-05-09T09:23:15.657457+0000","last_undegraded":"2024-05-09T09:23:15.657457+0000","last_fullsized":"2024-05-09T09:23:15.657457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:48.063385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298380+0000","last_change":"2024-05-09T09:23:14.371413+0000","last_active":"2024-05-09T09:23:16.298380+0000","last_peered":"2024-05-09T09:23:16.298380+0000","last_clean":"2024-05-09T09:23:16.298380+0000","last_became_active":"2024-05-09T09:23:14.370018+0000","last_became_peered":"2024-05-09T09:23:14.370018+0000","last_unstale":"2024-05-09T09:23:16.298380+0000","last_undegraded":"2024-05-09T09:23:16.298380+0000","last_fullsized":"2024-05-09T09:23:16.298380+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:18.475976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.600388+0000","last_change":"2024-05-09T09:23:09.297328+0000","last_active":"2024-05-09T09:23:33.600388+0000","last_peered":"2024-05-09T09:23:33.600388+0000","last_clean":"2024-05-09T09:23:33.600388+0000","last_became_active":"2024-05-09T09:23:09.297001+0000","last_became_peered":"2024-05-09T09:23:09.297001+0000","last_unstale":"2024-05-09T09:23:33.600388+0000","last_undegraded":"2024-05-09T09:23:33.600388+0000","last_fullsized":"2024-05-09T09:23:33.600388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:19.241362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.350594+0000","last_change":"2024-05-09T09:23:10.338342+0000","last_active":"2024-05-09T09:23:15.350594+0000","last_peered":"2024-05-09T09:23:15.350594+0000","last_clean":"2024-05-09T09:23:15.350594+0000","last_became_active":"2024-05-09T09:23:10.337995+0000","last_became_peered":"2024-05-09T09:23:10.337995+0000","last_unstale":"2024-05-09T09:23:15.350594+0000","last_undegraded":"2024-05-09T09:23:15.350594+0000","last_fullsized":"2024-05-09T09:23:15.350594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.691862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657562+0000","last_change":"2024-05-09T09:23:14.366237+0000","last_active":"2024-05-09T09:23:15.657562+0000","last_peered":"2024-05-09T09:23:15.657562+0000","last_clean":"2024-05-09T09:23:15.657562+0000","last_became_active":"2024-05-09T09:23:14.365809+0000","last_became_peered":"2024-05-09T09:23:14.365809+0000","last_unstale":"2024-05-09T09:23:15.657562+0000","last_undegraded":"2024-05-09T09:23:15.657562+0000","last_fullsized":"2024-05-09T09:23:15.657562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:28.813467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.594788+0000","last_change":"2024-05-09T09:23:09.298658+0000","last_active":"2024-05-09T09:23:33.594788+0000","last_peered":"2024-05-09T09:23:33.594788+0000","last_clean":"2024-05-09T09:23:33.594788+0000","last_became_active":"2024-05-09T09:23:09.298325+0000","last_became_peered":"2024-05-09T09:23:09.298325+0000","last_unstale":"2024-05-09T09:23:33.594788+0000","last_undegraded":"2024-05-09T09:23:33.594788+0000","last_fullsized":"2024-05-09T09:23:33.594788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:22.108202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.534236+0000","last_change":"2024-05-09T09:23:10.326865+0000","last_active":"2024-05-09T09:23:15.534236+0000","last_peered":"2024-05-09T09:23:15.534236+0000","last_clean":"2024-05-09T09:23:15.534236+0000","last_became_active":"2024-05-09T09:23:10.326478+0000","last_became_peered":"2024-05-09T09:23:10.326478+0000","last_unstale":"2024-05-09T09:23:15.534236+0000","last_undegraded":"2024-05-09T09:23:15.534236+0000","last_fullsized":"2024-05-09T09:23:15.534236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:54.521226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.350621+0000","last_change":"2024-05-09T09:23:14.373601+0000","last_active":"2024-05-09T09:23:15.350621+0000","last_peered":"2024-05-09T09:23:15.350621+0000","last_clean":"2024-05-09T09:23:15.350621+0000","last_became_active":"2024-05-09T09:23:14.372764+0000","last_became_peered":"2024-05-09T09:23:14.372764+0000","last_unstale":"2024-05-09T09:23:15.350621+0000","last_undegraded":"2024-05-09T09:23:15.350621+0000","last_fullsized":"2024-05-09T09:23:15.350621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:59.496755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:30.654246+0000","last_change":"2024-05-09T09:23:09.302293+0000","last_active":"2024-05-09T09:23:30.654246+0000","last_peered":"2024-05-09T09:23:30.654246+0000","last_clean":"2024-05-09T09:23:30.654246+0000","last_became_active":"2024-05-09T09:23:09.301216+0000","last_became_peered":"2024-05-09T09:23:09.301216+0000","last_unstale":"2024-05-09T09:23:30.654246+0000","last_undegraded":"2024-05-09T09:23:30.654246+0000","last_fullsized":"2024-05-09T09:23:30.654246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:35.146860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298293+0000","last_change":"2024-05-09T09:23:10.339827+0000","last_active":"2024-05-09T09:23:16.298293+0000","last_peered":"2024-05-09T09:23:16.298293+0000","last_clean":"2024-05-09T09:23:16.298293+0000","last_became_active":"2024-05-09T09:23:10.339415+0000","last_became_peered":"2024-05-09T09:23:10.339415+0000","last_unstale":"2024-05-09T09:23:16.298293+0000","last_undegraded":"2024-05-09T09:23:16.298293+0000","last_fullsized":"2024-05-09T09:23:16.298293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:45.250126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.570035+0000","last_change":"2024-05-09T09:23:14.377001+0000","last_active":"2024-05-09T09:23:15.570035+0000","last_peered":"2024-05-09T09:23:15.570035+0000","last_clean":"2024-05-09T09:23:15.570035+0000","last_became_active":"2024-05-09T09:23:14.376402+0000","last_became_peered":"2024-05-09T09:23:14.376402+0000","last_unstale":"2024-05-09T09:23:15.570035+0000","last_undegraded":"2024-05-09T09:23:15.570035+0000","last_fullsized":"2024-05-09T09:23:15.570035+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:08.142495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:32.678567+0000","last_change":"2024-05-09T09:23:09.299331+0000","last_active":"2024-05-09T09:23:32.678567+0000","last_peered":"2024-05-09T09:23:32.678567+0000","last_clean":"2024-05-09T09:23:32.678567+0000","last_became_active":"2024-05-09T09:23:09.299017+0000","last_became_peered":"2024-05-09T09:23:09.299017+0000","last_unstale":"2024-05-09T09:23:32.678567+0000","last_undegraded":"2024-05-09T09:23:32.678567+0000","last_fullsized":"2024-05-09T09:23:32.678567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:28.328704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.534003+0000","last_change":"2024-05-09T09:23:10.329463+0000","last_active":"2024-05-09T09:23:15.534003+0000","last_peered":"2024-05-09T09:23:15.534003+0000","last_clean":"2024-05-09T09:23:15.534003+0000","last_became_active":"2024-05-09T09:23:10.329119+0000","last_became_peered":"2024-05-09T09:23:10.329119+0000","last_unstale":"2024-05-09T09:23:15.534003+0000","last_undegraded":"2024-05-09T09:23:15.534003+0000","last_fullsized":"2024-05-09T09:23:15.534003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:20.699429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.350656+0000","last_change":"2024-05-09T09:23:14.363362+0000","last_active":"2024-05-09T09:23:15.350656+0000","last_peered":"2024-05-09T09:23:15.350656+0000","last_clean":"2024-05-09T09:23:15.350656+0000","last_became_active":"2024-05-09T09:23:14.362926+0000","last_became_peered":"2024-05-09T09:23:14.362926+0000","last_unstale":"2024-05-09T09:23:15.350656+0000","last_undegraded":"2024-05-09T09:23:15.350656+0000","last_fullsized":"2024-05-09T09:23:15.350656+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:25.339727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'21","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.603772+0000","last_change":"2024-05-09T09:23:09.297814+0000","last_active":"2024-05-09T09:23:33.603772+0000","last_peered":"2024-05-09T09:23:33.603772+0000","last_clean":"2024-05-09T09:23:33.603772+0000","last_became_active":"2024-05-09T09:23:09.297005+0000","last_became_peered":"2024-05-09T09:23:09.297005+0000","last_unstale":"2024-05-09T09:23:33.603772+0000","last_undegraded":"2024-05-09T09:23:33.603772+0000","last_fullsized":"2024-05-09T09:23:33.603772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:39.240217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":24,"num_write":23,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658183+0000","last_change":"2024-05-09T09:23:10.333013+0000","last_active":"2024-05-09T09:23:15.658183+0000","last_peered":"2024-05-09T09:23:15.658183+0000","last_clean":"2024-05-09T09:23:15.658183+0000","last_became_active":"2024-05-09T09:23:10.332666+0000","last_became_peered":"2024-05-09T09:23:10.332666+0000","last_unstale":"2024-05-09T09:23:15.658183+0000","last_undegraded":"2024-05-09T09:23:15.658183+0000","last_fullsized":"2024-05-09T09:23:15.658183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:28:50.306551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658411+0000","last_change":"2024-05-09T09:23:14.367715+0000","last_active":"2024-05-09T09:23:15.658411+0000","last_peered":"2024-05-09T09:23:15.658411+0000","last_clean":"2024-05-09T09:23:15.658411+0000","last_became_active":"2024-05-09T09:23:14.367307+0000","last_became_peered":"2024-05-09T09:23:14.367307+0000","last_unstale":"2024-05-09T09:23:15.658411+0000","last_undegraded":"2024-05-09T09:23:15.658411+0000","last_fullsized":"2024-05-09T09:23:15.658411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:09.075572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.603202+0000","last_change":"2024-05-09T09:23:09.306302+0000","last_active":"2024-05-09T09:23:33.603202+0000","last_peered":"2024-05-09T09:23:33.603202+0000","last_clean":"2024-05-09T09:23:33.603202+0000","last_became_active":"2024-05-09T09:23:09.306074+0000","last_became_peered":"2024-05-09T09:23:09.306074+0000","last_unstale":"2024-05-09T09:23:33.603202+0000","last_undegraded":"2024-05-09T09:23:33.603202+0000","last_fullsized":"2024-05-09T09:23:33.603202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:18.434899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656545+0000","last_change":"2024-05-09T09:23:10.331201+0000","last_active":"2024-05-09T09:23:15.656545+0000","last_peered":"2024-05-09T09:23:15.656545+0000","last_clean":"2024-05-09T09:23:15.656545+0000","last_became_active":"2024-05-09T09:23:10.329830+0000","last_became_peered":"2024-05-09T09:23:10.329830+0000","last_unstale":"2024-05-09T09:23:15.656545+0000","last_undegraded":"2024-05-09T09:23:15.656545+0000","last_fullsized":"2024-05-09T09:23:15.656545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:40.199622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658596+0000","last_change":"2024-05-09T09:23:14.359095+0000","last_active":"2024-05-09T09:23:15.658596+0000","last_peered":"2024-05-09T09:23:15.658596+0000","last_clean":"2024-05-09T09:23:15.658596+0000","last_became_active":"2024-05-09T09:23:14.358717+0000","last_became_peered":"2024-05-09T09:23:14.358717+0000","last_unstale":"2024-05-09T09:23:15.658596+0000","last_undegraded":"2024-05-09T09:23:15.658596+0000","last_fullsized":"2024-05-09T09:23:15.658596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:16:09.874767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:32.678332+0000","last_change":"2024-05-09T09:23:09.305370+0000","last_active":"2024-05-09T09:23:32.678332+0000","last_peered":"2024-05-09T09:23:32.678332+0000","last_clean":"2024-05-09T09:23:32.678332+0000","last_became_active":"2024-05-09T09:23:09.305054+0000","last_became_peered":"2024-05-09T09:23:09.305054+0000","last_unstale":"2024-05-09T09:23:32.678332+0000","last_undegraded":"2024-05-09T09:23:32.678332+0000","last_fullsized":"2024-05-09T09:23:32.678332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:28.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351454+0000","last_change":"2024-05-09T09:23:10.332541+0000","last_active":"2024-05-09T09:23:15.351454+0000","last_peered":"2024-05-09T09:23:15.351454+0000","last_clean":"2024-05-09T09:23:15.351454+0000","last_became_active":"2024-05-09T09:23:10.332155+0000","last_became_peered":"2024-05-09T09:23:10.332155+0000","last_unstale":"2024-05-09T09:23:15.351454+0000","last_undegraded":"2024-05-09T09:23:15.351454+0000","last_fullsized":"2024-05-09T09:23:15.351454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:39.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351216+0000","last_change":"2024-05-09T09:23:14.374499+0000","last_active":"2024-05-09T09:23:15.351216+0000","last_peered":"2024-05-09T09:23:15.351216+0000","last_clean":"2024-05-09T09:23:15.351216+0000","last_became_active":"2024-05-09T09:23:14.370688+0000","last_became_peered":"2024-05-09T09:23:14.370688+0000","last_unstale":"2024-05-09T09:23:15.351216+0000","last_undegraded":"2024-05-09T09:23:15.351216+0000","last_fullsized":"2024-05-09T09:23:15.351216+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:53.367237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569562+0000","last_change":"2024-05-09T09:23:09.297776+0000","last_active":"2024-05-09T09:23:15.569562+0000","last_peered":"2024-05-09T09:23:15.569562+0000","last_clean":"2024-05-09T09:23:15.569562+0000","last_became_active":"2024-05-09T09:23:09.297407+0000","last_became_peered":"2024-05-09T09:23:09.297407+0000","last_unstale":"2024-05-09T09:23:15.569562+0000","last_undegraded":"2024-05-09T09:23:15.569562+0000","last_fullsized":"2024-05-09T09:23:15.569562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:21.762941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351284+0000","last_change":"2024-05-09T09:23:10.339127+0000","last_active":"2024-05-09T09:23:15.351284+0000","last_peered":"2024-05-09T09:23:15.351284+0000","last_clean":"2024-05-09T09:23:15.351284+0000","last_became_active":"2024-05-09T09:23:10.338790+0000","last_became_peered":"2024-05-09T09:23:10.338790+0000","last_unstale":"2024-05-09T09:23:15.351284+0000","last_undegraded":"2024-05-09T09:23:15.351284+0000","last_fullsized":"2024-05-09T09:23:15.351284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:36.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657306+0000","last_change":"2024-05-09T09:23:14.373873+0000","last_active":"2024-05-09T09:23:15.657306+0000","last_peered":"2024-05-09T09:23:15.657306+0000","last_clean":"2024-05-09T09:23:15.657306+0000","last_became_active":"2024-05-09T09:23:14.371453+0000","last_became_peered":"2024-05-09T09:23:14.371453+0000","last_unstale":"2024-05-09T09:23:15.657306+0000","last_undegraded":"2024-05-09T09:23:15.657306+0000","last_fullsized":"2024-05-09T09:23:15.657306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:26.885938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:31.667221+0000","last_change":"2024-05-09T09:23:09.299373+0000","last_active":"2024-05-09T09:23:31.667221+0000","last_peered":"2024-05-09T09:23:31.667221+0000","last_clean":"2024-05-09T09:23:31.667221+0000","last_became_active":"2024-05-09T09:23:09.298926+0000","last_became_peered":"2024-05-09T09:23:09.298926+0000","last_unstale":"2024-05-09T09:23:31.667221+0000","last_undegraded":"2024-05-09T09:23:31.667221+0000","last_fullsized":"2024-05-09T09:23:31.667221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:35.629929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657311+0000","last_change":"2024-05-09T09:23:10.343911+0000","last_active":"2024-05-09T09:23:15.657311+0000","last_peered":"2024-05-09T09:23:15.657311+0000","last_clean":"2024-05-09T09:23:15.657311+0000","last_became_active":"2024-05-09T09:23:10.343568+0000","last_became_peered":"2024-05-09T09:23:10.343568+0000","last_unstale":"2024-05-09T09:23:15.657311+0000","last_undegraded":"2024-05-09T09:23:15.657311+0000","last_fullsized":"2024-05-09T09:23:15.657311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:07.311554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657000+0000","last_change":"2024-05-09T09:23:14.368544+0000","last_active":"2024-05-09T09:23:15.657000+0000","last_peered":"2024-05-09T09:23:15.657000+0000","last_clean":"2024-05-09T09:23:15.657000+0000","last_became_active":"2024-05-09T09:23:14.368140+0000","last_became_peered":"2024-05-09T09:23:14.368140+0000","last_unstale":"2024-05-09T09:23:15.657000+0000","last_undegraded":"2024-05-09T09:23:15.657000+0000","last_fullsized":"2024-05-09T09:23:15.657000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.091768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657887+0000","last_change":"2024-05-09T09:23:09.297612+0000","last_active":"2024-05-09T09:23:15.657887+0000","last_peered":"2024-05-09T09:23:15.657887+0000","last_clean":"2024-05-09T09:23:15.657887+0000","last_became_active":"2024-05-09T09:23:09.297216+0000","last_became_peered":"2024-05-09T09:23:09.297216+0000","last_unstale":"2024-05-09T09:23:15.657887+0000","last_undegraded":"2024-05-09T09:23:15.657887+0000","last_fullsized":"2024-05-09T09:23:15.657887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:09:45.108082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657262+0000","last_change":"2024-05-09T09:23:10.336514+0000","last_active":"2024-05-09T09:23:15.657262+0000","last_peered":"2024-05-09T09:23:15.657262+0000","last_clean":"2024-05-09T09:23:15.657262+0000","last_became_active":"2024-05-09T09:23:10.336173+0000","last_became_peered":"2024-05-09T09:23:10.336173+0000","last_unstale":"2024-05-09T09:23:15.657262+0000","last_undegraded":"2024-05-09T09:23:15.657262+0000","last_fullsized":"2024-05-09T09:23:15.657262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:46:39.036547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657880+0000","last_change":"2024-05-09T09:23:14.366119+0000","last_active":"2024-05-09T09:23:15.657880+0000","last_peered":"2024-05-09T09:23:15.657880+0000","last_clean":"2024-05-09T09:23:15.657880+0000","last_became_active":"2024-05-09T09:23:14.365659+0000","last_became_peered":"2024-05-09T09:23:14.365659+0000","last_unstale":"2024-05-09T09:23:15.657880+0000","last_undegraded":"2024-05-09T09:23:15.657880+0000","last_fullsized":"2024-05-09T09:23:15.657880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:23.312632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351092+0000","last_change":"2024-05-09T09:23:09.297504+0000","last_active":"2024-05-09T09:23:15.351092+0000","last_peered":"2024-05-09T09:23:15.351092+0000","last_clean":"2024-05-09T09:23:15.351092+0000","last_became_active":"2024-05-09T09:23:09.297134+0000","last_became_peered":"2024-05-09T09:23:09.297134+0000","last_unstale":"2024-05-09T09:23:15.351092+0000","last_undegraded":"2024-05-09T09:23:15.351092+0000","last_fullsized":"2024-05-09T09:23:15.351092+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:50:23.900097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656781+0000","last_change":"2024-05-09T09:23:10.339788+0000","last_active":"2024-05-09T09:23:15.656781+0000","last_peered":"2024-05-09T09:23:15.656781+0000","last_clean":"2024-05-09T09:23:15.656781+0000","last_became_active":"2024-05-09T09:23:10.339430+0000","last_became_peered":"2024-05-09T09:23:10.339430+0000","last_unstale":"2024-05-09T09:23:15.656781+0000","last_undegraded":"2024-05-09T09:23:15.656781+0000","last_fullsized":"2024-05-09T09:23:15.656781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:22.467287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657279+0000","last_change":"2024-05-09T09:23:14.368710+0000","last_active":"2024-05-09T09:23:15.657279+0000","last_peered":"2024-05-09T09:23:15.657279+0000","last_clean":"2024-05-09T09:23:15.657279+0000","last_became_active":"2024-05-09T09:23:14.368288+0000","last_became_peered":"2024-05-09T09:23:14.368288+0000","last_unstale":"2024-05-09T09:23:15.657279+0000","last_undegraded":"2024-05-09T09:23:15.657279+0000","last_fullsized":"2024-05-09T09:23:15.657279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:40.941273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351259+0000","last_change":"2024-05-09T09:23:09.301976+0000","last_active":"2024-05-09T09:23:15.351259+0000","last_peered":"2024-05-09T09:23:15.351259+0000","last_clean":"2024-05-09T09:23:15.351259+0000","last_became_active":"2024-05-09T09:23:09.301494+0000","last_became_peered":"2024-05-09T09:23:09.301494+0000","last_unstale":"2024-05-09T09:23:15.351259+0000","last_undegraded":"2024-05-09T09:23:15.351259+0000","last_fullsized":"2024-05-09T09:23:15.351259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.448246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656557+0000","last_change":"2024-05-09T09:23:10.331691+0000","last_active":"2024-05-09T09:23:15.656557+0000","last_peered":"2024-05-09T09:23:15.656557+0000","last_clean":"2024-05-09T09:23:15.656557+0000","last_became_active":"2024-05-09T09:23:10.331332+0000","last_became_peered":"2024-05-09T09:23:10.331332+0000","last_unstale":"2024-05-09T09:23:15.656557+0000","last_undegraded":"2024-05-09T09:23:15.656557+0000","last_fullsized":"2024-05-09T09:23:15.656557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:51.643695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656901+0000","last_change":"2024-05-09T09:23:14.367539+0000","last_active":"2024-05-09T09:23:15.656901+0000","last_peered":"2024-05-09T09:23:15.656901+0000","last_clean":"2024-05-09T09:23:15.656901+0000","last_became_active":"2024-05-09T09:23:14.366806+0000","last_became_peered":"2024-05-09T09:23:14.366806+0000","last_unstale":"2024-05-09T09:23:15.656901+0000","last_undegraded":"2024-05-09T09:23:15.656901+0000","last_fullsized":"2024-05-09T09:23:15.656901+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:58.679095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656853+0000","last_change":"2024-05-09T09:23:10.340849+0000","last_active":"2024-05-09T09:23:15.656853+0000","last_peered":"2024-05-09T09:23:15.656853+0000","last_clean":"2024-05-09T09:23:15.656853+0000","last_became_active":"2024-05-09T09:23:10.340430+0000","last_became_peered":"2024-05-09T09:23:10.340430+0000","last_unstale":"2024-05-09T09:23:15.656853+0000","last_undegraded":"2024-05-09T09:23:15.656853+0000","last_fullsized":"2024-05-09T09:23:15.656853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:33:47.827659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656358+0000","last_change":"2024-05-09T09:23:09.308544+0000","last_active":"2024-05-09T09:23:15.656358+0000","last_peered":"2024-05-09T09:23:15.656358+0000","last_clean":"2024-05-09T09:23:15.656358+0000","last_became_active":"2024-05-09T09:23:09.308195+0000","last_became_peered":"2024-05-09T09:23:09.308195+0000","last_unstale":"2024-05-09T09:23:15.656358+0000","last_undegraded":"2024-05-09T09:23:15.656358+0000","last_fullsized":"2024-05-09T09:23:15.656358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:17:34.884385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351662+0000","last_change":"2024-05-09T09:23:14.365921+0000","last_active":"2024-05-09T09:23:15.351662+0000","last_peered":"2024-05-09T09:23:15.351662+0000","last_clean":"2024-05-09T09:23:15.351662+0000","last_became_active":"2024-05-09T09:23:14.365427+0000","last_became_peered":"2024-05-09T09:23:14.365427+0000","last_unstale":"2024-05-09T09:23:15.351662+0000","last_undegraded":"2024-05-09T09:23:15.351662+0000","last_fullsized":"2024-05-09T09:23:15.351662+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:46.213043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657086+0000","last_change":"2024-05-09T09:23:10.341320+0000","last_active":"2024-05-09T09:23:15.657086+0000","last_peered":"2024-05-09T09:23:15.657086+0000","last_clean":"2024-05-09T09:23:15.657086+0000","last_became_active":"2024-05-09T09:23:10.340868+0000","last_became_peered":"2024-05-09T09:23:10.340868+0000","last_unstale":"2024-05-09T09:23:15.657086+0000","last_undegraded":"2024-05-09T09:23:15.657086+0000","last_fullsized":"2024-05-09T09:23:15.657086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:55.846642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569768+0000","last_change":"2024-05-09T09:23:09.299574+0000","last_active":"2024-05-09T09:23:15.569768+0000","last_peered":"2024-05-09T09:23:15.569768+0000","last_clean":"2024-05-09T09:23:15.569768+0000","last_became_active":"2024-05-09T09:23:09.298878+0000","last_became_peered":"2024-05-09T09:23:09.298878+0000","last_unstale":"2024-05-09T09:23:15.569768+0000","last_undegraded":"2024-05-09T09:23:15.569768+0000","last_fullsized":"2024-05-09T09:23:15.569768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.275907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.350976+0000","last_change":"2024-05-09T09:21:57.576132+0000","last_active":"2024-05-09T09:23:15.350976+0000","last_peered":"2024-05-09T09:23:15.350976+0000","last_clean":"2024-05-09T09:23:15.350976+0000","last_became_active":"2024-05-09T09:21:57.575602+0000","last_became_peered":"2024-05-09T09:21:57.575602+0000","last_unstale":"2024-05-09T09:23:15.350976+0000","last_undegraded":"2024-05-09T09:23:15.350976+0000","last_fullsized":"2024-05-09T09:23:15.350976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_clean_scrub_stamp":"2024-05-09T09:18:31.335063+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:07.639787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658595+0000","last_change":"2024-05-09T09:23:14.371184+0000","last_active":"2024-05-09T09:23:15.658595+0000","last_peered":"2024-05-09T09:23:15.658595+0000","last_clean":"2024-05-09T09:23:15.658595+0000","last_became_active":"2024-05-09T09:23:14.370780+0000","last_became_peered":"2024-05-09T09:23:14.370780+0000","last_unstale":"2024-05-09T09:23:15.658595+0000","last_undegraded":"2024-05-09T09:23:15.658595+0000","last_fullsized":"2024-05-09T09:23:15.658595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:39.361947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656223+0000","last_change":"2024-05-09T09:23:10.330915+0000","last_active":"2024-05-09T09:23:15.656223+0000","last_peered":"2024-05-09T09:23:15.656223+0000","last_clean":"2024-05-09T09:23:15.656223+0000","last_became_active":"2024-05-09T09:23:10.330458+0000","last_became_peered":"2024-05-09T09:23:10.330458+0000","last_unstale":"2024-05-09T09:23:15.656223+0000","last_undegraded":"2024-05-09T09:23:15.656223+0000","last_fullsized":"2024-05-09T09:23:15.656223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:10.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351248+0000","last_change":"2024-05-09T09:23:09.303895+0000","last_active":"2024-05-09T09:23:15.351248+0000","last_peered":"2024-05-09T09:23:15.351248+0000","last_clean":"2024-05-09T09:23:15.351248+0000","last_became_active":"2024-05-09T09:23:09.303555+0000","last_became_peered":"2024-05-09T09:23:09.303555+0000","last_unstale":"2024-05-09T09:23:15.351248+0000","last_undegraded":"2024-05-09T09:23:15.351248+0000","last_fullsized":"2024-05-09T09:23:15.351248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:04.350161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657152+0000","last_change":"2024-05-09T09:23:14.359390+0000","last_active":"2024-05-09T09:23:15.657152+0000","last_peered":"2024-05-09T09:23:15.657152+0000","last_clean":"2024-05-09T09:23:15.657152+0000","last_became_active":"2024-05-09T09:23:14.359000+0000","last_became_peered":"2024-05-09T09:23:14.359000+0000","last_unstale":"2024-05-09T09:23:15.657152+0000","last_undegraded":"2024-05-09T09:23:15.657152+0000","last_fullsized":"2024-05-09T09:23:15.657152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:25.077835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658340+0000","last_change":"2024-05-09T09:23:09.304535+0000","last_active":"2024-05-09T09:23:15.658340+0000","last_peered":"2024-05-09T09:23:15.658340+0000","last_clean":"2024-05-09T09:23:15.658340+0000","last_became_active":"2024-05-09T09:23:09.304166+0000","last_became_peered":"2024-05-09T09:23:09.304166+0000","last_unstale":"2024-05-09T09:23:15.658340+0000","last_undegraded":"2024-05-09T09:23:15.658340+0000","last_fullsized":"2024-05-09T09:23:15.658340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:03:20.281620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657122+0000","last_change":"2024-05-09T09:23:10.335657+0000","last_active":"2024-05-09T09:23:15.657122+0000","last_peered":"2024-05-09T09:23:15.657122+0000","last_clean":"2024-05-09T09:23:15.657122+0000","last_became_active":"2024-05-09T09:23:10.335231+0000","last_became_peered":"2024-05-09T09:23:10.335231+0000","last_unstale":"2024-05-09T09:23:15.657122+0000","last_undegraded":"2024-05-09T09:23:15.657122+0000","last_fullsized":"2024-05-09T09:23:15.657122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:07.990165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656821+0000","last_change":"2024-05-09T09:23:14.366988+0000","last_active":"2024-05-09T09:23:15.656821+0000","last_peered":"2024-05-09T09:23:15.656821+0000","last_clean":"2024-05-09T09:23:15.656821+0000","last_became_active":"2024-05-09T09:23:14.366562+0000","last_became_peered":"2024-05-09T09:23:14.366562+0000","last_unstale":"2024-05-09T09:23:15.656821+0000","last_undegraded":"2024-05-09T09:23:15.656821+0000","last_fullsized":"2024-05-09T09:23:15.656821+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:42.670036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:32.679031+0000","last_change":"2024-05-09T09:23:09.305541+0000","last_active":"2024-05-09T09:23:32.679031+0000","last_peered":"2024-05-09T09:23:32.679031+0000","last_clean":"2024-05-09T09:23:32.679031+0000","last_became_active":"2024-05-09T09:23:09.305267+0000","last_became_peered":"2024-05-09T09:23:09.305267+0000","last_unstale":"2024-05-09T09:23:32.679031+0000","last_undegraded":"2024-05-09T09:23:32.679031+0000","last_fullsized":"2024-05-09T09:23:32.679031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:04.614051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657936+0000","last_change":"2024-05-09T09:23:10.345840+0000","last_active":"2024-05-09T09:23:15.657936+0000","last_peered":"2024-05-09T09:23:15.657936+0000","last_clean":"2024-05-09T09:23:15.657936+0000","last_became_active":"2024-05-09T09:23:10.345487+0000","last_became_peered":"2024-05-09T09:23:10.345487+0000","last_unstale":"2024-05-09T09:23:15.657936+0000","last_undegraded":"2024-05-09T09:23:15.657936+0000","last_fullsized":"2024-05-09T09:23:15.657936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:47.723429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351106+0000","last_change":"2024-05-09T09:23:14.365485+0000","last_active":"2024-05-09T09:23:15.351106+0000","last_peered":"2024-05-09T09:23:15.351106+0000","last_clean":"2024-05-09T09:23:15.351106+0000","last_became_active":"2024-05-09T09:23:14.365018+0000","last_became_peered":"2024-05-09T09:23:14.365018+0000","last_unstale":"2024-05-09T09:23:15.351106+0000","last_undegraded":"2024-05-09T09:23:15.351106+0000","last_fullsized":"2024-05-09T09:23:15.351106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:54:04.712920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.570229+0000","last_change":"2024-05-09T09:23:09.302365+0000","last_active":"2024-05-09T09:23:15.570229+0000","last_peered":"2024-05-09T09:23:15.570229+0000","last_clean":"2024-05-09T09:23:15.570229+0000","last_became_active":"2024-05-09T09:23:09.302008+0000","last_became_peered":"2024-05-09T09:23:09.302008+0000","last_unstale":"2024-05-09T09:23:15.570229+0000","last_undegraded":"2024-05-09T09:23:15.570229+0000","last_fullsized":"2024-05-09T09:23:15.570229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:48.829940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298454+0000","last_change":"2024-05-09T09:23:10.339170+0000","last_active":"2024-05-09T09:23:16.298454+0000","last_peered":"2024-05-09T09:23:16.298454+0000","last_clean":"2024-05-09T09:23:16.298454+0000","last_became_active":"2024-05-09T09:23:10.338829+0000","last_became_peered":"2024-05-09T09:23:10.338829+0000","last_unstale":"2024-05-09T09:23:16.298454+0000","last_undegraded":"2024-05-09T09:23:16.298454+0000","last_fullsized":"2024-05-09T09:23:16.298454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:04.510495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.570060+0000","last_change":"2024-05-09T09:23:14.372622+0000","last_active":"2024-05-09T09:23:15.570060+0000","last_peered":"2024-05-09T09:23:15.570060+0000","last_clean":"2024-05-09T09:23:15.570060+0000","last_became_active":"2024-05-09T09:23:14.371800+0000","last_became_peered":"2024-05-09T09:23:14.371800+0000","last_unstale":"2024-05-09T09:23:15.570060+0000","last_undegraded":"2024-05-09T09:23:15.570060+0000","last_fullsized":"2024-05-09T09:23:15.570060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:02.925505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:30.653830+0000","last_change":"2024-05-09T09:23:09.297418+0000","last_active":"2024-05-09T09:23:30.653830+0000","last_peered":"2024-05-09T09:23:30.653830+0000","last_clean":"2024-05-09T09:23:30.653830+0000","last_became_active":"2024-05-09T09:23:09.296963+0000","last_became_peered":"2024-05-09T09:23:09.296963+0000","last_unstale":"2024-05-09T09:23:30.653830+0000","last_undegraded":"2024-05-09T09:23:30.653830+0000","last_fullsized":"2024-05-09T09:23:30.653830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:54.552857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657327+0000","last_change":"2024-05-09T09:23:10.325882+0000","last_active":"2024-05-09T09:23:15.657327+0000","last_peered":"2024-05-09T09:23:15.657327+0000","last_clean":"2024-05-09T09:23:15.657327+0000","last_became_active":"2024-05-09T09:23:10.325494+0000","last_became_peered":"2024-05-09T09:23:10.325494+0000","last_unstale":"2024-05-09T09:23:15.657327+0000","last_undegraded":"2024-05-09T09:23:15.657327+0000","last_fullsized":"2024-05-09T09:23:15.657327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:27.191310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351324+0000","last_change":"2024-05-09T09:23:14.375572+0000","last_active":"2024-05-09T09:23:15.351324+0000","last_peered":"2024-05-09T09:23:15.351324+0000","last_clean":"2024-05-09T09:23:15.351324+0000","last_became_active":"2024-05-09T09:23:14.374069+0000","last_became_peered":"2024-05-09T09:23:14.374069+0000","last_unstale":"2024-05-09T09:23:15.351324+0000","last_undegraded":"2024-05-09T09:23:15.351324+0000","last_fullsized":"2024-05-09T09:23:15.351324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:44.045012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.599776+0000","last_change":"2024-05-09T09:23:09.304137+0000","last_active":"2024-05-09T09:23:33.599776+0000","last_peered":"2024-05-09T09:23:33.599776+0000","last_clean":"2024-05-09T09:23:33.599776+0000","last_became_active":"2024-05-09T09:23:09.303762+0000","last_became_peered":"2024-05-09T09:23:09.303762+0000","last_unstale":"2024-05-09T09:23:33.599776+0000","last_undegraded":"2024-05-09T09:23:33.599776+0000","last_fullsized":"2024-05-09T09:23:33.599776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:56.811194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657220+0000","last_change":"2024-05-09T09:23:10.331998+0000","last_active":"2024-05-09T09:23:15.657220+0000","last_peered":"2024-05-09T09:23:15.657220+0000","last_clean":"2024-05-09T09:23:15.657220+0000","last_became_active":"2024-05-09T09:23:10.331642+0000","last_became_peered":"2024-05-09T09:23:10.331642+0000","last_unstale":"2024-05-09T09:23:15.657220+0000","last_undegraded":"2024-05-09T09:23:15.657220+0000","last_fullsized":"2024-05-09T09:23:15.657220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:41.569300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569892+0000","last_change":"2024-05-09T09:23:14.377444+0000","last_active":"2024-05-09T09:23:15.569892+0000","last_peered":"2024-05-09T09:23:15.569892+0000","last_clean":"2024-05-09T09:23:15.569892+0000","last_became_active":"2024-05-09T09:23:14.376189+0000","last_became_peered":"2024-05-09T09:23:14.376189+0000","last_unstale":"2024-05-09T09:23:15.569892+0000","last_undegraded":"2024-05-09T09:23:15.569892+0000","last_fullsized":"2024-05-09T09:23:15.569892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:06.973535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:32.678362+0000","last_change":"2024-05-09T09:23:09.304101+0000","last_active":"2024-05-09T09:23:32.678362+0000","last_peered":"2024-05-09T09:23:32.678362+0000","last_clean":"2024-05-09T09:23:32.678362+0000","last_became_active":"2024-05-09T09:23:09.303743+0000","last_became_peered":"2024-05-09T09:23:09.303743+0000","last_unstale":"2024-05-09T09:23:32.678362+0000","last_undegraded":"2024-05-09T09:23:32.678362+0000","last_fullsized":"2024-05-09T09:23:32.678362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:39.339815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657724+0000","last_change":"2024-05-09T09:23:10.345938+0000","last_active":"2024-05-09T09:23:15.657724+0000","last_peered":"2024-05-09T09:23:15.657724+0000","last_clean":"2024-05-09T09:23:15.657724+0000","last_became_active":"2024-05-09T09:23:10.345576+0000","last_became_peered":"2024-05-09T09:23:10.345576+0000","last_unstale":"2024-05-09T09:23:15.657724+0000","last_undegraded":"2024-05-09T09:23:15.657724+0000","last_fullsized":"2024-05-09T09:23:15.657724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:08.571900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.569939+0000","last_change":"2024-05-09T09:23:14.371549+0000","last_active":"2024-05-09T09:23:15.569939+0000","last_peered":"2024-05-09T09:23:15.569939+0000","last_clean":"2024-05-09T09:23:15.569939+0000","last_became_active":"2024-05-09T09:23:14.369931+0000","last_became_peered":"2024-05-09T09:23:14.369931+0000","last_unstale":"2024-05-09T09:23:15.569939+0000","last_undegraded":"2024-05-09T09:23:15.569939+0000","last_fullsized":"2024-05-09T09:23:15.569939+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:20.777172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:30.652737+0000","last_change":"2024-05-09T09:23:09.302909+0000","last_active":"2024-05-09T09:23:30.652737+0000","last_peered":"2024-05-09T09:23:30.652737+0000","last_clean":"2024-05-09T09:23:30.652737+0000","last_became_active":"2024-05-09T09:23:09.302570+0000","last_became_peered":"2024-05-09T09:23:09.302570+0000","last_unstale":"2024-05-09T09:23:30.652737+0000","last_undegraded":"2024-05-09T09:23:30.652737+0000","last_fullsized":"2024-05-09T09:23:30.652737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:23.856755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351436+0000","last_change":"2024-05-09T09:23:10.342676+0000","last_active":"2024-05-09T09:23:15.351436+0000","last_peered":"2024-05-09T09:23:15.351436+0000","last_clean":"2024-05-09T09:23:15.351436+0000","last_became_active":"2024-05-09T09:23:10.341906+0000","last_became_peered":"2024-05-09T09:23:10.341906+0000","last_unstale":"2024-05-09T09:23:15.351436+0000","last_undegraded":"2024-05-09T09:23:15.351436+0000","last_fullsized":"2024-05-09T09:23:15.351436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:59:11.137648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.533812+0000","last_change":"2024-05-09T09:23:14.363779+0000","last_active":"2024-05-09T09:23:15.533812+0000","last_peered":"2024-05-09T09:23:15.533812+0000","last_clean":"2024-05-09T09:23:15.533812+0000","last_became_active":"2024-05-09T09:23:14.363307+0000","last_became_peered":"2024-05-09T09:23:14.363307+0000","last_unstale":"2024-05-09T09:23:15.533812+0000","last_undegraded":"2024-05-09T09:23:15.533812+0000","last_fullsized":"2024-05-09T09:23:15.533812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:46.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.596361+0000","last_change":"2024-05-09T09:23:09.298094+0000","last_active":"2024-05-09T09:23:33.596361+0000","last_peered":"2024-05-09T09:23:33.596361+0000","last_clean":"2024-05-09T09:23:33.596361+0000","last_became_active":"2024-05-09T09:23:09.297777+0000","last_became_peered":"2024-05-09T09:23:09.297777+0000","last_unstale":"2024-05-09T09:23:33.596361+0000","last_undegraded":"2024-05-09T09:23:33.596361+0000","last_fullsized":"2024-05-09T09:23:33.596361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:25.099193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351832+0000","last_change":"2024-05-09T09:23:10.340120+0000","last_active":"2024-05-09T09:23:15.351832+0000","last_peered":"2024-05-09T09:23:15.351832+0000","last_clean":"2024-05-09T09:23:15.351832+0000","last_became_active":"2024-05-09T09:23:10.339690+0000","last_became_peered":"2024-05-09T09:23:10.339690+0000","last_unstale":"2024-05-09T09:23:15.351832+0000","last_undegraded":"2024-05-09T09:23:15.351832+0000","last_fullsized":"2024-05-09T09:23:15.351832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:36.783411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298104+0000","last_change":"2024-05-09T09:23:14.377016+0000","last_active":"2024-05-09T09:23:16.298104+0000","last_peered":"2024-05-09T09:23:16.298104+0000","last_clean":"2024-05-09T09:23:16.298104+0000","last_became_active":"2024-05-09T09:23:14.376501+0000","last_became_peered":"2024-05-09T09:23:14.376501+0000","last_unstale":"2024-05-09T09:23:16.298104+0000","last_undegraded":"2024-05-09T09:23:16.298104+0000","last_fullsized":"2024-05-09T09:23:16.298104+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:30.826423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:33.592779+0000","last_change":"2024-05-09T09:23:09.307592+0000","last_active":"2024-05-09T09:23:33.592779+0000","last_peered":"2024-05-09T09:23:33.592779+0000","last_clean":"2024-05-09T09:23:33.592779+0000","last_became_active":"2024-05-09T09:23:09.307341+0000","last_became_peered":"2024-05-09T09:23:09.307341+0000","last_unstale":"2024-05-09T09:23:33.592779+0000","last_undegraded":"2024-05-09T09:23:33.592779+0000","last_fullsized":"2024-05-09T09:23:33.592779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:21.790980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351436+0000","last_change":"2024-05-09T09:23:10.349217+0000","last_active":"2024-05-09T09:23:15.351436+0000","last_peered":"2024-05-09T09:23:15.351436+0000","last_clean":"2024-05-09T09:23:15.351436+0000","last_became_active":"2024-05-09T09:23:10.348861+0000","last_became_peered":"2024-05-09T09:23:10.348861+0000","last_unstale":"2024-05-09T09:23:15.351436+0000","last_undegraded":"2024-05-09T09:23:15.351436+0000","last_fullsized":"2024-05-09T09:23:15.351436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:51.621045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656698+0000","last_change":"2024-05-09T09:23:14.363773+0000","last_active":"2024-05-09T09:23:15.656698+0000","last_peered":"2024-05-09T09:23:15.656698+0000","last_clean":"2024-05-09T09:23:15.656698+0000","last_became_active":"2024-05-09T09:23:14.363393+0000","last_became_peered":"2024-05-09T09:23:14.363393+0000","last_unstale":"2024-05-09T09:23:15.656698+0000","last_undegraded":"2024-05-09T09:23:15.656698+0000","last_fullsized":"2024-05-09T09:23:15.656698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:57.322985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657297+0000","last_change":"2024-05-09T09:23:09.308723+0000","last_active":"2024-05-09T09:23:15.657297+0000","last_peered":"2024-05-09T09:23:15.657297+0000","last_clean":"2024-05-09T09:23:15.657297+0000","last_became_active":"2024-05-09T09:23:09.308267+0000","last_became_peered":"2024-05-09T09:23:09.308267+0000","last_unstale":"2024-05-09T09:23:15.657297+0000","last_undegraded":"2024-05-09T09:23:15.657297+0000","last_fullsized":"2024-05-09T09:23:15.657297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:09:25.036421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351722+0000","last_change":"2024-05-09T09:23:10.342246+0000","last_active":"2024-05-09T09:23:15.351722+0000","last_peered":"2024-05-09T09:23:15.351722+0000","last_clean":"2024-05-09T09:23:15.351722+0000","last_became_active":"2024-05-09T09:23:10.341927+0000","last_became_peered":"2024-05-09T09:23:10.341927+0000","last_unstale":"2024-05-09T09:23:15.351722+0000","last_undegraded":"2024-05-09T09:23:15.351722+0000","last_fullsized":"2024-05-09T09:23:15.351722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:01.182226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657176+0000","last_change":"2024-05-09T09:23:10.332768+0000","last_active":"2024-05-09T09:23:15.657176+0000","last_peered":"2024-05-09T09:23:15.657176+0000","last_clean":"2024-05-09T09:23:15.657176+0000","last_became_active":"2024-05-09T09:23:10.332441+0000","last_became_peered":"2024-05-09T09:23:10.332441+0000","last_unstale":"2024-05-09T09:23:15.657176+0000","last_undegraded":"2024-05-09T09:23:15.657176+0000","last_fullsized":"2024-05-09T09:23:15.657176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:53.377058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351359+0000","last_change":"2024-05-09T09:23:10.337944+0000","last_active":"2024-05-09T09:23:15.351359+0000","last_peered":"2024-05-09T09:23:15.351359+0000","last_clean":"2024-05-09T09:23:15.351359+0000","last_became_active":"2024-05-09T09:23:10.337593+0000","last_became_peered":"2024-05-09T09:23:10.337593+0000","last_unstale":"2024-05-09T09:23:15.351359+0000","last_undegraded":"2024-05-09T09:23:15.351359+0000","last_fullsized":"2024-05-09T09:23:15.351359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:20.381543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351137+0000","last_change":"2024-05-09T09:23:10.337419+0000","last_active":"2024-05-09T09:23:15.351137+0000","last_peered":"2024-05-09T09:23:15.351137+0000","last_clean":"2024-05-09T09:23:15.351137+0000","last_became_active":"2024-05-09T09:23:10.336718+0000","last_became_peered":"2024-05-09T09:23:10.336718+0000","last_unstale":"2024-05-09T09:23:15.351137+0000","last_undegraded":"2024-05-09T09:23:15.351137+0000","last_fullsized":"2024-05-09T09:23:15.351137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:50.101110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:16.298133+0000","last_change":"2024-05-09T09:23:10.340243+0000","last_active":"2024-05-09T09:23:16.298133+0000","last_peered":"2024-05-09T09:23:16.298133+0000","last_clean":"2024-05-09T09:23:16.298133+0000","last_became_active":"2024-05-09T09:23:10.339919+0000","last_became_peered":"2024-05-09T09:23:10.339919+0000","last_unstale":"2024-05-09T09:23:16.298133+0000","last_undegraded":"2024-05-09T09:23:16.298133+0000","last_fullsized":"2024-05-09T09:23:16.298133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:21.460928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657023+0000","last_change":"2024-05-09T09:23:10.333136+0000","last_active":"2024-05-09T09:23:15.657023+0000","last_peered":"2024-05-09T09:23:15.657023+0000","last_clean":"2024-05-09T09:23:15.657023+0000","last_became_active":"2024-05-09T09:23:10.332799+0000","last_became_peered":"2024-05-09T09:23:10.332799+0000","last_unstale":"2024-05-09T09:23:15.657023+0000","last_undegraded":"2024-05-09T09:23:15.657023+0000","last_fullsized":"2024-05-09T09:23:15.657023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:50.682747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657255+0000","last_change":"2024-05-09T09:23:10.340713+0000","last_active":"2024-05-09T09:23:15.657255+0000","last_peered":"2024-05-09T09:23:15.657255+0000","last_clean":"2024-05-09T09:23:15.657255+0000","last_became_active":"2024-05-09T09:23:10.340363+0000","last_became_peered":"2024-05-09T09:23:10.340363+0000","last_unstale":"2024-05-09T09:23:15.657255+0000","last_undegraded":"2024-05-09T09:23:15.657255+0000","last_fullsized":"2024-05-09T09:23:15.657255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:14:18.009211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.351762+0000","last_change":"2024-05-09T09:23:10.341776+0000","last_active":"2024-05-09T09:23:15.351762+0000","last_peered":"2024-05-09T09:23:15.351762+0000","last_clean":"2024-05-09T09:23:15.351762+0000","last_became_active":"2024-05-09T09:23:10.341436+0000","last_became_peered":"2024-05-09T09:23:10.341436+0000","last_unstale":"2024-05-09T09:23:15.351762+0000","last_undegraded":"2024-05-09T09:23:15.351762+0000","last_fullsized":"2024-05-09T09:23:15.351762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:42.178856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658014+0000","last_change":"2024-05-09T09:23:10.331131+0000","last_active":"2024-05-09T09:23:15.658014+0000","last_peered":"2024-05-09T09:23:15.658014+0000","last_clean":"2024-05-09T09:23:15.658014+0000","last_became_active":"2024-05-09T09:23:10.330756+0000","last_became_peered":"2024-05-09T09:23:10.330756+0000","last_unstale":"2024-05-09T09:23:15.658014+0000","last_undegraded":"2024-05-09T09:23:15.658014+0000","last_fullsized":"2024-05-09T09:23:15.658014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:43:27.259375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657130+0000","last_change":"2024-05-09T09:23:10.337117+0000","last_active":"2024-05-09T09:23:15.657130+0000","last_peered":"2024-05-09T09:23:15.657130+0000","last_clean":"2024-05-09T09:23:15.657130+0000","last_became_active":"2024-05-09T09:23:10.336672+0000","last_became_peered":"2024-05-09T09:23:10.336672+0000","last_unstale":"2024-05-09T09:23:15.657130+0000","last_undegraded":"2024-05-09T09:23:15.657130+0000","last_fullsized":"2024-05-09T09:23:15.657130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:59:20.723742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.657768+0000","last_change":"2024-05-09T09:23:10.337733+0000","last_active":"2024-05-09T09:23:15.657768+0000","last_peered":"2024-05-09T09:23:15.657768+0000","last_clean":"2024-05-09T09:23:15.657768+0000","last_became_active":"2024-05-09T09:23:10.337343+0000","last_became_peered":"2024-05-09T09:23:10.337343+0000","last_unstale":"2024-05-09T09:23:15.657768+0000","last_undegraded":"2024-05-09T09:23:15.657768+0000","last_fullsized":"2024-05-09T09:23:15.657768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:53.447911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.533886+0000","last_change":"2024-05-09T09:23:10.337485+0000","last_active":"2024-05-09T09:23:15.533886+0000","last_peered":"2024-05-09T09:23:15.533886+0000","last_clean":"2024-05-09T09:23:15.533886+0000","last_became_active":"2024-05-09T09:23:10.337129+0000","last_became_peered":"2024-05-09T09:23:10.337129+0000","last_unstale":"2024-05-09T09:23:15.533886+0000","last_undegraded":"2024-05-09T09:23:15.533886+0000","last_fullsized":"2024-05-09T09:23:15.533886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:20.758552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.656565+0000","last_change":"2024-05-09T09:23:10.332271+0000","last_active":"2024-05-09T09:23:15.656565+0000","last_peered":"2024-05-09T09:23:15.656565+0000","last_clean":"2024-05-09T09:23:15.656565+0000","last_became_active":"2024-05-09T09:23:10.331941+0000","last_became_peered":"2024-05-09T09:23:10.331941+0000","last_unstale":"2024-05-09T09:23:15.656565+0000","last_undegraded":"2024-05-09T09:23:15.656565+0000","last_fullsized":"2024-05-09T09:23:15.656565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:27.707554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T09:23:15.658611+0000","last_change":"2024-05-09T09:23:10.328965+0000","last_active":"2024-05-09T09:23:15.658611+0000","last_peered":"2024-05-09T09:23:15.658611+0000","last_clean":"2024-05-09T09:23:15.658611+0000","last_became_active":"2024-05-09T09:23:10.328618+0000","last_became_peered":"2024-05-09T09:23:10.328618+0000","last_unstale":"2024-05-09T09:23:15.658611+0000","last_undegraded":"2024-05-09T09:23:15.658611+0000","last_fullsized":"2024-05-09T09:23:15.658611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:16:02.003378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27423,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":17,"num_read_kb":35,"num_write":154,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":79153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":983040,"data_stored":476829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1654784,"data_stored":1092465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":477737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":477567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1581056,"data_stored":1067779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":476829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":480671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":478307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40520,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704632,"statfs":{"total":95995035648,"available":95953543168,"internally_reserved":0,"allocated":1048576,"data_stored":501942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":477259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":987136,"data_stored":476851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1074113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":42443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":25268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:23:35.876 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T09:23:35.877 INFO:tasks.cephadm:Setting up client nodes... 2024-05-09T09:23:35.877 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-09T09:23:35.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:35 smithi170 ceph-mon[33632]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:35.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:35 smithi170 ceph-mon[33632]: pgmap v304: 129 pgs: 129 active+clean; 603 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-05-09T09:23:35.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:35 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1816629609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:35.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:35 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3094347225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:35.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:35 smithi170 ceph-mon[33632]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T09:23:35.994 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:23:35 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i[64878]: 2024-05-09T09:23:35.797+0000 7f86049b6700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-09T09:23:35.994 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:23:35 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i[64878]: 2024-05-09T09:23:35.797+0000 7f86049b6700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-09T09:23:35.994 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:23:35 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i[64878]: 2024-05-09T09:23:35.797+0000 7f86049b6700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-09T09:23:35.994 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:23:35 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i[64878]: 2024-05-09T09:23:35.797+0000 7f86049b6700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-09T09:23:36.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:35 smithi016 ceph-mon[33335]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:23:36.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:35 smithi016 ceph-mon[33335]: pgmap v304: 129 pgs: 129 active+clean; 603 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-05-09T09:23:36.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:35 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1816629609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:36.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:35 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3094347225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:36.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:35 smithi016 ceph-mon[33335]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T09:23:36.164 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:36.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:36 smithi012 ceph-mon[30140]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:23:36.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:36 smithi012 ceph-mon[30140]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:23:36.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:36 smithi170 ceph-mon[33632]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:23:36.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:36 smithi170 ceph-mon[33632]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:23:37.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:36 smithi016 ceph-mon[33335]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:23:37.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:36 smithi016 ceph-mon[33335]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:23:37.906 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:37 smithi012 ceph-mon[30140]: pgmap v305: 129 pgs: 129 active+clean; 605 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-05-09T09:23:37.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:37 smithi170 ceph-mon[33632]: pgmap v305: 129 pgs: 129 active+clean; 605 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-05-09T09:23:38.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:37 smithi016 ceph-mon[33335]: pgmap v305: 129 pgs: 129 active+clean; 605 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-05-09T09:23:38.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.026+0000 7ff73b539700 1 -- 172.21.15.12:0/1352473288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7341024a0 msgr2=0x7ff734102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:38.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.026+0000 7ff73b539700 1 --2- 172.21.15.12:0/1352473288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7341024a0 0x7ff734102900 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff7240099c0 tx=0x7ff724009cd0 comp rx=0 tx=0).stop 2024-05-09T09:23:38.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.026+0000 7ff73b539700 1 -- 172.21.15.12:0/1352473288 shutdown_connections 2024-05-09T09:23:38.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.026+0000 7ff73b539700 1 --2- 172.21.15.12:0/1352473288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff734102e40 0x7ff7341096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:38.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.026+0000 7ff73b539700 1 --2- 172.21.15.12:0/1352473288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7341012a0 0x7ff7341016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:38.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.026+0000 7ff73b539700 1 --2- 172.21.15.12:0/1352473288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7341024a0 0x7ff734102900 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:38.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.026+0000 7ff73b539700 1 -- 172.21.15.12:0/1352473288 >> 172.21.15.12:0/1352473288 conn(0x7ff7340fc840 msgr2=0x7ff7340fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:38.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.026+0000 7ff73b539700 1 -- 172.21.15.12:0/1352473288 shutdown_connections 2024-05-09T09:23:38.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.026+0000 7ff73b539700 1 -- 172.21.15.12:0/1352473288 wait complete. 2024-05-09T09:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.026+0000 7ff73b539700 1 Processor -- start 2024-05-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.026+0000 7ff73b539700 1 -- start start 2024-05-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff73b539700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7341012a0 0x7ff7341984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff73b539700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7341024a0 0x7ff734198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff73b539700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff734102e40 0x7ff73419fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff73b539700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff73410ce80 con 0x7ff734102e40 2024-05-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff73b539700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff73410cd00 con 0x7ff7341012a0 2024-05-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff73b539700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff73410d000 con 0x7ff7341024a0 2024-05-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff739ad6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff734102e40 0x7ff73419fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff739ad6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff734102e40 0x7ff73419fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56160/0 (socket says 172.21.15.12:56160) 2024-05-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff739ad6700 1 -- 172.21.15.12:0/12421126 learned_addr learned my addr 172.21.15.12:0/12421126 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff738ad4700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7341024a0 0x7ff734198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff7392d5700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7341012a0 0x7ff7341984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff739ad6700 1 -- 172.21.15.12:0/12421126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7341024a0 msgr2=0x7ff734198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff739ad6700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7341024a0 0x7ff734198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff739ad6700 1 -- 172.21.15.12:0/12421126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7341012a0 msgr2=0x7ff7341984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff739ad6700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7341012a0 0x7ff7341984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff739ad6700 1 -- 172.21.15.12:0/12421126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7341a0150 con 0x7ff734102e40 2024-05-09T09:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff739ad6700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff734102e40 0x7ff73419fab0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff72c007f10 tx=0x7ff72c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.027+0000 7ff72a7fc700 1 -- 172.21.15.12:0/12421126 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c00dd20 con 0x7ff734102e40 2024-05-09T09:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.028+0000 7ff72a7fc700 1 -- 172.21.15.12:0/12421126 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff72c004020 con 0x7ff734102e40 2024-05-09T09:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.028+0000 7ff73b539700 1 -- 172.21.15.12:0/12421126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7341a0440 con 0x7ff734102e40 2024-05-09T09:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.028+0000 7ff72a7fc700 1 -- 172.21.15.12:0/12421126 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c015660 con 0x7ff734102e40 2024-05-09T09:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.028+0000 7ff73b539700 1 -- 172.21.15.12:0/12421126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7341a0980 con 0x7ff734102e40 2024-05-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.029+0000 7ff73b539700 1 -- 172.21.15.12:0/12421126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff734050890 con 0x7ff734102e40 2024-05-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.030+0000 7ff72a7fc700 1 -- 172.21.15.12:0/12421126 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff72c0041d0 con 0x7ff734102e40 2024-05-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.030+0000 7ff72a7fc700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7200a2de0 0x7ff7200a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.031+0000 7ff7392d5700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7200a2de0 0x7ff7200a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.031+0000 7ff72a7fc700 1 -- 172.21.15.12:0/12421126 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff72c0cf5c0 con 0x7ff734102e40 2024-05-09T09:23:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.031+0000 7ff7392d5700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7200a2de0 0x7ff7200a52a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff734102300 tx=0x7ff730007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:38.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.034+0000 7ff72a7fc700 1 -- 172.21.15.12:0/12421126 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff72c010070 con 0x7ff734102e40 2024-05-09T09:23:38.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.325+0000 7ff73b539700 1 -- 172.21.15.12:0/12421126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7ff7341a0f00 con 0x7ff734102e40 2024-05-09T09:23:38.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.327+0000 7ff72a7fc700 1 -- 172.21.15.12:0/12421126 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7ff72c09c120 con 0x7ff734102e40 2024-05-09T09:23:38.829 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-05-09T09:23:38.829 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQAaljxmqkV9ExAATG7wrYgNDW6vED/tSacRoA== 2024-05-09T09:23:38.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.330+0000 7ff73b539700 1 -- 172.21.15.12:0/12421126 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7200a2de0 msgr2=0x7ff7200a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:38.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.330+0000 7ff73b539700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7200a2de0 0x7ff7200a52a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff734102300 tx=0x7ff730007400 comp rx=0 tx=0).stop 2024-05-09T09:23:38.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.330+0000 7ff73b539700 1 -- 172.21.15.12:0/12421126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff734102e40 msgr2=0x7ff73419fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:38.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.330+0000 7ff73b539700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff734102e40 0x7ff73419fab0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff72c007f10 tx=0x7ff72c00d3b0 comp rx=0 tx=0).stop 2024-05-09T09:23:38.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.330+0000 7ff73b539700 1 -- 172.21.15.12:0/12421126 shutdown_connections 2024-05-09T09:23:38.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.330+0000 7ff73b539700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7200a2de0 0x7ff7200a52a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:38.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.330+0000 7ff73b539700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff734102e40 0x7ff73419fab0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:38.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.330+0000 7ff73b539700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7341012a0 0x7ff7341984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:38.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.330+0000 7ff73b539700 1 --2- 172.21.15.12:0/12421126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7341024a0 0x7ff734198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:38.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.330+0000 7ff73b539700 1 -- 172.21.15.12:0/12421126 >> 172.21.15.12:0/12421126 conn(0x7ff7340fc840 msgr2=0x7ff7340fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:38.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.330+0000 7ff73b539700 1 -- 172.21.15.12:0/12421126 shutdown_connections 2024-05-09T09:23:38.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:38.330+0000 7ff73b539700 1 -- 172.21.15.12:0/12421126 wait complete. 2024-05-09T09:23:38.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:38 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/12421126' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T09:23:38.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:38 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/12421126' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T09:23:39.002 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:38 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/12421126' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T09:23:39.003 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:38 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/12421126' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T09:23:39.005 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:38 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/12421126' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T09:23:39.006 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:38 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/12421126' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T09:23:39.239 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:23:39.239 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-09T09:23:39.239 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T09:23:39.281 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T09:23:39.281 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-09T09:23:39.281 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-09T09:23:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:39.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:39 smithi012 ceph-mon[30140]: pgmap v306: 129 pgs: 129 active+clean; 606 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.6 KiB/s wr, 18 op/s 2024-05-09T09:23:39.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:39 smithi170 ceph-mon[33632]: pgmap v306: 129 pgs: 129 active+clean; 606 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.6 KiB/s wr, 18 op/s 2024-05-09T09:23:40.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:39 smithi016 ceph-mon[33335]: pgmap v306: 129 pgs: 129 active+clean; 606 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.6 KiB/s wr, 18 op/s 2024-05-09T09:23:41.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.262+0000 7f7ed0838700 1 -- 172.21.15.12:0/3573552833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ecc101be0 msgr2=0x7f7ecc10e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.262+0000 7f7ed0838700 1 --2- 172.21.15.12:0/3573552833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ecc101be0 0x7f7ecc10e0d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400b0a0 tx=0x7f7ec400b3b0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.263+0000 7f7ed0838700 1 -- 172.21.15.12:0/3573552833 shutdown_connections 2024-05-09T09:23:41.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.263+0000 7f7ed0838700 1 --2- 172.21.15.12:0/3573552833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ecc101be0 0x7f7ecc10e0d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.263+0000 7f7ed0838700 1 --2- 172.21.15.12:0/3573552833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ecc107290 0x7f7ecc107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.263+0000 7f7ed0838700 1 --2- 172.21.15.12:0/3573552833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc101220 0x7f7ecc1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.263+0000 7f7ed0838700 1 -- 172.21.15.12:0/3573552833 >> 172.21.15.12:0/3573552833 conn(0x7f7ecc0781d0 msgr2=0x7f7ecc0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.263+0000 7f7ed0838700 1 -- 172.21.15.12:0/3573552833 shutdown_connections 2024-05-09T09:23:41.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.263+0000 7f7ed0838700 1 -- 172.21.15.12:0/3573552833 wait complete. 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.263+0000 7f7ed0838700 1 Processor -- start 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.263+0000 7f7ed0838700 1 -- start start 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ed0838700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ecc101220 0x7f7ecc19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ed0838700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc101be0 0x7f7ecc19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ed0838700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ecc107290 0x7f7ecc1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ed0838700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1114f0 con 0x7f7ecc101220 2024-05-09T09:23:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ed0838700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc111370 con 0x7f7ecc101be0 2024-05-09T09:23:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ed0838700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc111670 con 0x7f7ecc107290 2024-05-09T09:23:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7eca59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ecc101220 0x7f7ecc19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7eca59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ecc101220 0x7f7ecc19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56180/0 (socket says 172.21.15.12:56180) 2024-05-09T09:23:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7eca59c700 1 -- 172.21.15.12:0/3206456801 learned_addr learned my addr 172.21.15.12:0/3206456801 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ec9d9b700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc101be0 0x7f7ecc19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ecad9d700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ecc107290 0x7f7ecc1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7eca59c700 1 -- 172.21.15.12:0/3206456801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ecc107290 msgr2=0x7f7ecc1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7eca59c700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ecc107290 0x7f7ecc1977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7eca59c700 1 -- 172.21.15.12:0/3206456801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc101be0 msgr2=0x7f7ecc19dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7eca59c700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc101be0 0x7f7ecc19dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7eca59c700 1 -- 172.21.15.12:0/3206456801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc198090 con 0x7f7ecc101220 2024-05-09T09:23:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ec9d9b700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc101be0 0x7f7ecc19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:23:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7eca59c700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ecc101220 0x7f7ecc19d690 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00c9b0 tx=0x7f7ebc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ebb7fe700 1 -- 172.21.15.12:0/3206456801 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc007b00 con 0x7f7ecc101220 2024-05-09T09:23:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ebb7fe700 1 -- 172.21.15.12:0/3206456801 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7ebc007ca0 con 0x7f7ecc101220 2024-05-09T09:23:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ed0838700 1 -- 172.21.15.12:0/3206456801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc198380 con 0x7f7ecc101220 2024-05-09T09:23:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ebb7fe700 1 -- 172.21.15.12:0/3206456801 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc018780 con 0x7f7ecc101220 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.264+0000 7f7ed0838700 1 -- 172.21.15.12:0/3206456801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1a4610 con 0x7f7ecc101220 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.265+0000 7f7ed0838700 1 -- 172.21.15.12:0/3206456801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ecc102940 con 0x7f7ecc101220 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.267+0000 7f7ebb7fe700 1 -- 172.21.15.12:0/3206456801 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7ebc00f450 con 0x7f7ecc101220 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.268+0000 7f7ebb7fe700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7eb40a2cb0 0x7f7eb40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.268+0000 7f7ec9d9b700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7eb40a2cb0 0x7f7eb40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.268+0000 7f7ebb7fe700 1 -- 172.21.15.12:0/3206456801 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7ebc0cf720 con 0x7f7ecc101220 2024-05-09T09:23:41.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.268+0000 7f7ec9d9b700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7eb40a2cb0 0x7f7eb40a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0005fd0 tx=0x7f7ec0005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.271+0000 7f7ebb7fe700 1 -- 172.21.15.12:0/3206456801 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7ebc014030 con 0x7f7ecc101220 2024-05-09T09:23:41.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.539+0000 7f7ed0838700 1 -- 172.21.15.12:0/3206456801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7ecc198c90 con 0x7f7ecc101220 2024-05-09T09:23:41.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.544+0000 7f7ebb7fe700 1 -- 172.21.15.12:0/3206456801 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263199 (secure 0 0 0) 0x7f7ebc09c310 con 0x7f7ecc101220 2024-05-09T09:23:41.551 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.547+0000 7f7ed0838700 1 -- 172.21.15.12:0/3206456801 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7eb40a2cb0 msgr2=0x7f7eb40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.547+0000 7f7ed0838700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7eb40a2cb0 0x7f7eb40a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0005fd0 tx=0x7f7ec0005ee0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.547+0000 7f7ed0838700 1 -- 172.21.15.12:0/3206456801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ecc101220 msgr2=0x7f7ecc19d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.547+0000 7f7ed0838700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ecc101220 0x7f7ecc19d690 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00c9b0 tx=0x7f7ebc00cd70 comp rx=0 tx=0).stop 2024-05-09T09:23:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.547+0000 7f7ed0838700 1 -- 172.21.15.12:0/3206456801 shutdown_connections 2024-05-09T09:23:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.547+0000 7f7ed0838700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7eb40a2cb0 0x7f7eb40a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.547+0000 7f7ed0838700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ecc107290 0x7f7ecc1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.547+0000 7f7ed0838700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ecc101220 0x7f7ecc19d690 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.547+0000 7f7ed0838700 1 --2- 172.21.15.12:0/3206456801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc101be0 0x7f7ecc19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.547+0000 7f7ed0838700 1 -- 172.21.15.12:0/3206456801 >> 172.21.15.12:0/3206456801 conn(0x7f7ecc0781d0 msgr2=0x7f7ecc0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.547+0000 7f7ed0838700 1 -- 172.21.15.12:0/3206456801 shutdown_connections 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:41.547+0000 7f7ed0838700 1 -- 172.21.15.12:0/3206456801 wait complete. 2024-05-09T09:23:41.903 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:41 smithi012 ceph-mon[30140]: pgmap v307: 129 pgs: 129 active+clean; 628 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.6 KiB/s wr, 22 op/s 2024-05-09T09:23:41.903 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:41 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3206456801' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T09:23:41.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:41 smithi170 ceph-mon[33632]: pgmap v307: 129 pgs: 129 active+clean; 628 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.6 KiB/s wr, 22 op/s 2024-05-09T09:23:41.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:41 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3206456801' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T09:23:42.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:41 smithi016 ceph-mon[33335]: pgmap v307: 129 pgs: 129 active+clean; 628 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.6 KiB/s wr, 22 op/s 2024-05-09T09:23:42.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:41 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3206456801' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T09:23:42.210 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3809515815},{"type":"v1","addr":"172.21.15.12:6801","nonce":3809515815}]},"active_addr":"172.21.15.12:6801/3809515815","active_change":"2024-05-09T09:16:04.467042+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.12:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":2129538570}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3115254556}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":278654935}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3709764060}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":699081120}]}]} 2024-05-09T09:23:42.221 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-09T09:23:42.221 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-09T09:23:42.222 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-09T09:23:42.483 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:43.937 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:43 smithi012 ceph-mon[30140]: pgmap v308: 129 pgs: 129 active+clean; 628 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-05-09T09:23:43.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:43 smithi170 ceph-mon[33632]: pgmap v308: 129 pgs: 129 active+clean; 628 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-05-09T09:23:44.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:43 smithi016 ceph-mon[33335]: pgmap v308: 129 pgs: 129 active+clean; 628 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-05-09T09:23:44.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.303+0000 7fde9ff2c700 1 -- 172.21.15.12:0/3746134790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde9806a8c0 msgr2=0x7fde9806ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.303+0000 7fde9ff2c700 1 --2- 172.21.15.12:0/3746134790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde9806a8c0 0x7fde9806ad20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fde88009990 tx=0x7fde88009ca0 comp rx=0 tx=0).stop 2024-05-09T09:23:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.303+0000 7fde9ff2c700 1 -- 172.21.15.12:0/3746134790 shutdown_connections 2024-05-09T09:23:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.303+0000 7fde9ff2c700 1 --2- 172.21.15.12:0/3746134790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde9806b260 0x7fde9810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.303+0000 7fde9ff2c700 1 --2- 172.21.15.12:0/3746134790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde98108390 0x7fde98108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.303+0000 7fde9ff2c700 1 --2- 172.21.15.12:0/3746134790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde9806a8c0 0x7fde9806ad20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.303+0000 7fde9ff2c700 1 -- 172.21.15.12:0/3746134790 >> 172.21.15.12:0/3746134790 conn(0x7fde98078190 msgr2=0x7fde980785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.303+0000 7fde9ff2c700 1 -- 172.21.15.12:0/3746134790 shutdown_connections 2024-05-09T09:23:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.303+0000 7fde9ff2c700 1 -- 172.21.15.12:0/3746134790 wait complete. 2024-05-09T09:23:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.304+0000 7fde9ff2c700 1 Processor -- start 2024-05-09T09:23:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.304+0000 7fde9ff2c700 1 -- start start 2024-05-09T09:23:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.304+0000 7fde9ff2c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde9806a8c0 0x7fde9819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.304+0000 7fde9ff2c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde9806b260 0x7fde9819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.304+0000 7fde9ff2c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde98108390 0x7fde98198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.304+0000 7fde9ff2c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fde98112660 con 0x7fde9806a8c0 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.304+0000 7fde9ff2c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde981124e0 con 0x7fde98108390 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.304+0000 7fde9ff2c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fde981127e0 con 0x7fde9806b260 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.304+0000 7fde9dcc8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde9806a8c0 0x7fde9819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.304+0000 7fde9dcc8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde9806a8c0 0x7fde9819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44884/0 (socket says 172.21.15.12:44884) 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.304+0000 7fde9dcc8700 1 -- 172.21.15.12:0/108250630 learned_addr learned my addr 172.21.15.12:0/108250630 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde9d4c7700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde9806b260 0x7fde9819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde9e4c9700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde98108390 0x7fde98198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde9dcc8700 1 -- 172.21.15.12:0/108250630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde9806b260 msgr2=0x7fde9819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde9dcc8700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde9806b260 0x7fde9819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde9dcc8700 1 -- 172.21.15.12:0/108250630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde98108390 msgr2=0x7fde98198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde9dcc8700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde98108390 0x7fde98198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde9dcc8700 1 -- 172.21.15.12:0/108250630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde98199160 con 0x7fde9806a8c0 2024-05-09T09:23:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde9e4c9700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde98108390 0x7fde98198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:23:44.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde9dcc8700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde9806a8c0 0x7fde9819e840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fde9400ba50 tx=0x7fde9400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:44.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde8effd700 1 -- 172.21.15.12:0/108250630 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94019070 con 0x7fde9806a8c0 2024-05-09T09:23:44.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde9ff2c700 1 -- 172.21.15.12:0/108250630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde98199450 con 0x7fde9806a8c0 2024-05-09T09:23:44.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde8effd700 1 -- 172.21.15.12:0/108250630 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde94014440 con 0x7fde9806a8c0 2024-05-09T09:23:44.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde8effd700 1 -- 172.21.15.12:0/108250630 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde940075d0 con 0x7fde9806a8c0 2024-05-09T09:23:44.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.305+0000 7fde9ff2c700 1 -- 172.21.15.12:0/108250630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde981a5710 con 0x7fde9806a8c0 2024-05-09T09:23:44.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.306+0000 7fde9ff2c700 1 -- 172.21.15.12:0/108250630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde98069010 con 0x7fde9806a8c0 2024-05-09T09:23:44.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.307+0000 7fde8effd700 1 -- 172.21.15.12:0/108250630 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fde9401c070 con 0x7fde9806a8c0 2024-05-09T09:23:44.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.308+0000 7fde8effd700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fde840a2cb0 0x7fde840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:44.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.308+0000 7fde8effd700 1 -- 172.21.15.12:0/108250630 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fde940cecf0 con 0x7fde9806a8c0 2024-05-09T09:23:44.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.308+0000 7fde9d4c7700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fde840a2cb0 0x7fde840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:44.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.309+0000 7fde9d4c7700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fde840a2cb0 0x7fde840a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde88005b40 tx=0x7fde88005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:44.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.311+0000 7fde8effd700 1 -- 172.21.15.12:0/108250630 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fde9409b9c0 con 0x7fde9806a8c0 2024-05-09T09:23:44.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.521+0000 7fde9ff2c700 1 -- 172.21.15.12:0/108250630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde98199f30 con 0x7fde9806a8c0 2024-05-09T09:23:44.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.522+0000 7fde8effd700 1 -- 172.21.15.12:0/108250630 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24105 (secure 0 0 0) 0x7fde9409b2c0 con 0x7fde9806a8c0 2024-05-09T09:23:44.522 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:44.522 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:23:14.726332+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:23:44.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.525+0000 7fde9ff2c700 1 -- 172.21.15.12:0/108250630 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fde840a2cb0 msgr2=0x7fde840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:44.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.525+0000 7fde9ff2c700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fde840a2cb0 0x7fde840a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde88005b40 tx=0x7fde88005a90 comp rx=0 tx=0).stop 2024-05-09T09:23:44.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.525+0000 7fde9ff2c700 1 -- 172.21.15.12:0/108250630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde9806a8c0 msgr2=0x7fde9819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:44.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.525+0000 7fde9ff2c700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde9806a8c0 0x7fde9819e840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fde9400ba50 tx=0x7fde9400be10 comp rx=0 tx=0).stop 2024-05-09T09:23:44.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.525+0000 7fde9ff2c700 1 -- 172.21.15.12:0/108250630 shutdown_connections 2024-05-09T09:23:44.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.525+0000 7fde9ff2c700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fde840a2cb0 0x7fde840a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.525+0000 7fde9ff2c700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde98108390 0x7fde98198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.525+0000 7fde9ff2c700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde9806a8c0 0x7fde9819e840 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.525+0000 7fde9ff2c700 1 --2- 172.21.15.12:0/108250630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde9806b260 0x7fde9819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.525+0000 7fde9ff2c700 1 -- 172.21.15.12:0/108250630 >> 172.21.15.12:0/108250630 conn(0x7fde98078190 msgr2=0x7fde980ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.525+0000 7fde9ff2c700 1 -- 172.21.15.12:0/108250630 shutdown_connections 2024-05-09T09:23:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:44.525+0000 7fde9ff2c700 1 -- 172.21.15.12:0/108250630 wait complete. 2024-05-09T09:23:44.954 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:44 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/108250630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:44.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:44 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/108250630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:45.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:44 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/108250630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:45.179 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-09T09:23:45.179 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-09T09:23:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:45.820 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:45 smithi012 ceph-mon[30140]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 25 op/s 2024-05-09T09:23:45.997 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:45 smithi170 ceph-mon[33632]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 25 op/s 2024-05-09T09:23:46.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:45 smithi016 ceph-mon[33335]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 25 op/s 2024-05-09T09:23:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.286+0000 7fab32caa700 1 -- 172.21.15.12:0/721112523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2c106540 msgr2=0x7fab2c10e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.286+0000 7fab32caa700 1 --2- 172.21.15.12:0/721112523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2c106540 0x7fab2c10e110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b0d0 tx=0x7fab2400b3e0 comp rx=0 tx=0).stop 2024-05-09T09:23:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.286+0000 7fab32caa700 1 -- 172.21.15.12:0/721112523 shutdown_connections 2024-05-09T09:23:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.286+0000 7fab32caa700 1 --2- 172.21.15.12:0/721112523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2c106540 0x7fab2c10e110 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.286+0000 7fab32caa700 1 --2- 172.21.15.12:0/721112523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c1014b0 0x7fab2c101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.286+0000 7fab32caa700 1 --2- 172.21.15.12:0/721112523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab2c101e60 0x7fab2c105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.286+0000 7fab32caa700 1 -- 172.21.15.12:0/721112523 >> 172.21.15.12:0/721112523 conn(0x7fab2c0781d0 msgr2=0x7fab2c0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.286+0000 7fab32caa700 1 -- 172.21.15.12:0/721112523 shutdown_connections 2024-05-09T09:23:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.286+0000 7fab32caa700 1 -- 172.21.15.12:0/721112523 wait complete. 2024-05-09T09:23:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.287+0000 7fab32caa700 1 Processor -- start 2024-05-09T09:23:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.287+0000 7fab32caa700 1 -- start start 2024-05-09T09:23:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.287+0000 7fab32caa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c1014b0 0x7fab2c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.287+0000 7fab32caa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab2c101e60 0x7fab2c199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.287+0000 7fab32caa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2c106540 0x7fab2c1a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.287+0000 7fab32caa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c111520 con 0x7fab2c106540 2024-05-09T09:23:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.287+0000 7fab32caa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1113a0 con 0x7fab2c1014b0 2024-05-09T09:23:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.287+0000 7fab32caa700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1116a0 con 0x7fab2c101e60 2024-05-09T09:23:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab31247700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2c106540 0x7fab2c1a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab31247700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2c106540 0x7fab2c1a49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44904/0 (socket says 172.21.15.12:44904) 2024-05-09T09:23:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab31247700 1 -- 172.21.15.12:0/608356384 learned_addr learned my addr 172.21.15.12:0/608356384 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab30a46700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c1014b0 0x7fab2c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:47.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab2bfff700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab2c101e60 0x7fab2c199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:47.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab31247700 1 -- 172.21.15.12:0/608356384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab2c101e60 msgr2=0x7fab2c199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:47.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab31247700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab2c101e60 0x7fab2c199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:47.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab31247700 1 -- 172.21.15.12:0/608356384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c1014b0 msgr2=0x7fab2c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:47.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab31247700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c1014b0 0x7fab2c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:47.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab31247700 1 -- 172.21.15.12:0/608356384 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c1a4f00 con 0x7fab2c106540 2024-05-09T09:23:47.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab31247700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2c106540 0x7fab2c1a49c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fab24007a00 tx=0x7fab24007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:47.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab29ffb700 1 -- 172.21.15.12:0/608356384 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2401f070 con 0x7fab2c106540 2024-05-09T09:23:47.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab32caa700 1 -- 172.21.15.12:0/608356384 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2c1a50d0 con 0x7fab2c106540 2024-05-09T09:23:47.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab29ffb700 1 -- 172.21.15.12:0/608356384 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fab24003c00 con 0x7fab2c106540 2024-05-09T09:23:47.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.288+0000 7fab29ffb700 1 -- 172.21.15.12:0/608356384 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2401b6a0 con 0x7fab2c106540 2024-05-09T09:23:47.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.289+0000 7fab32caa700 1 -- 172.21.15.12:0/608356384 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2c1a54d0 con 0x7fab2c106540 2024-05-09T09:23:47.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.290+0000 7fab32caa700 1 -- 172.21.15.12:0/608356384 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2c069010 con 0x7fab2c106540 2024-05-09T09:23:47.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.291+0000 7fab29ffb700 1 -- 172.21.15.12:0/608356384 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fab24022070 con 0x7fab2c106540 2024-05-09T09:23:47.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.291+0000 7fab29ffb700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fab140a2cb0 0x7fab140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:47.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.292+0000 7fab29ffb700 1 -- 172.21.15.12:0/608356384 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fab2400f020 con 0x7fab2c106540 2024-05-09T09:23:47.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.292+0000 7fab30a46700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fab140a2cb0 0x7fab140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:47.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.292+0000 7fab30a46700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fab140a2cb0 0x7fab140a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab1c005950 tx=0x7fab1c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:47.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.296+0000 7fab29ffb700 1 -- 172.21.15.12:0/608356384 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fab24029020 con 0x7fab2c106540 2024-05-09T09:23:47.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.502+0000 7fab32caa700 1 -- 172.21.15.12:0/608356384 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab2c02dde0 con 0x7fab2c106540 2024-05-09T09:23:47.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.503+0000 7fab29ffb700 1 -- 172.21.15.12:0/608356384 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24105 (secure 0 0 0) 0x7fab2409d160 con 0x7fab2c106540 2024-05-09T09:23:47.506 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:23:47.506 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:23:14.726332+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:23:47.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.506+0000 7fab32caa700 1 -- 172.21.15.12:0/608356384 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fab140a2cb0 msgr2=0x7fab140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:47.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.506+0000 7fab32caa700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fab140a2cb0 0x7fab140a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab1c005950 tx=0x7fab1c00b410 comp rx=0 tx=0).stop 2024-05-09T09:23:47.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.506+0000 7fab32caa700 1 -- 172.21.15.12:0/608356384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2c106540 msgr2=0x7fab2c1a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:47.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.506+0000 7fab32caa700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2c106540 0x7fab2c1a49c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fab24007a00 tx=0x7fab24007a30 comp rx=0 tx=0).stop 2024-05-09T09:23:47.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.506+0000 7fab32caa700 1 -- 172.21.15.12:0/608356384 shutdown_connections 2024-05-09T09:23:47.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.506+0000 7fab32caa700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fab140a2cb0 0x7fab140a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:47.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.506+0000 7fab32caa700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2c106540 0x7fab2c1a49c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:47.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.506+0000 7fab32caa700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c1014b0 0x7fab2c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:47.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.506+0000 7fab32caa700 1 --2- 172.21.15.12:0/608356384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab2c101e60 0x7fab2c199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:47.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.506+0000 7fab32caa700 1 -- 172.21.15.12:0/608356384 >> 172.21.15.12:0/608356384 conn(0x7fab2c0781d0 msgr2=0x7fab2c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:47.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.506+0000 7fab32caa700 1 -- 172.21.15.12:0/608356384 shutdown_connections 2024-05-09T09:23:47.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:47.506+0000 7fab32caa700 1 -- 172.21.15.12:0/608356384 wait complete. 2024-05-09T09:23:47.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:47 smithi012 ceph-mon[30140]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.6 KiB/s wr, 17 op/s 2024-05-09T09:23:47.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:47 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/608356384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:47.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:47 smithi170 ceph-mon[33632]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.6 KiB/s wr, 17 op/s 2024-05-09T09:23:47.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:47 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/608356384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:48.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:47 smithi016 ceph-mon[33335]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.6 KiB/s wr, 17 op/s 2024-05-09T09:23:48.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:47 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/608356384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:23:48.233 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-09T09:23:48.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-09T09:23:48.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-09T09:23:48.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-09T09:23:48.235 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-09T09:23:48.235 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-09T09:23:48.235 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-09T09:23:48.236 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-09T09:23:48.236 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-09T09:23:48.237 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-09T09:23:48.237 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-09T09:23:48.237 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-09T09:23:48.968 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:48.984 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:49.950 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:49.950 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:49.961 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:49.961 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:49.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:49 smithi012 ceph-mon[30140]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-05-09T09:23:49.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:49 smithi170 ceph-mon[33632]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-05-09T09:23:50.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:49 smithi016 ceph-mon[33335]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-05-09T09:23:51.067 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:51.172 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:23:51.976 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:51 smithi012 ceph-mon[30140]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-05-09T09:23:51.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:51 smithi170 ceph-mon[33632]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-05-09T09:23:52.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:51 smithi016 ceph-mon[33335]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-05-09T09:23:53.835 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:53 smithi012 ceph-mon[30140]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-09T09:23:53.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:53 smithi170 ceph-mon[33632]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-09T09:23:54.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:53 smithi016 ceph-mon[33335]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-09T09:23:55.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:55 smithi012 ceph-mon[30140]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T09:23:55.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:55 smithi170 ceph-mon[33632]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T09:23:56.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:55 smithi016 ceph-mon[33335]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T09:23:56.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.963+0000 7f3a809fc700 1 -- 172.21.15.12:0/2445775638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a7c074550 msgr2=0x7f3a7c074930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:56.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.963+0000 7f3a809fc700 1 --2- 172.21.15.12:0/2445775638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a7c074550 0x7f3a7c074930 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009990 tx=0x7f3a6c009ca0 comp rx=0 tx=0).stop 2024-05-09T09:23:56.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.963+0000 7f3a809fc700 1 -- 172.21.15.12:0/2445775638 shutdown_connections 2024-05-09T09:23:56.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.963+0000 7f3a809fc700 1 --2- 172.21.15.12:0/2445775638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7c10ed50 0x7f3a7c111140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:56.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.963+0000 7f3a809fc700 1 --2- 172.21.15.12:0/2445775638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a7c074550 0x7f3a7c074930 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:56.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.963+0000 7f3a809fc700 1 --2- 172.21.15.12:0/2445775638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a7c074e70 0x7f3a7c10e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:56.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.963+0000 7f3a809fc700 1 -- 172.21.15.12:0/2445775638 >> 172.21.15.12:0/2445775638 conn(0x7f3a7c0794d0 msgr2=0x7f3a7c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:56.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.963+0000 7f3a809fc700 1 -- 172.21.15.12:0/2445775638 shutdown_connections 2024-05-09T09:23:56.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.963+0000 7f3a809fc700 1 -- 172.21.15.12:0/2445775638 wait complete. 2024-05-09T09:23:56.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.964+0000 7f3a809fc700 1 Processor -- start 2024-05-09T09:23:56.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.964+0000 7f3a809fc700 1 -- start start 2024-05-09T09:23:56.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.964+0000 7f3a809fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a7c074550 0x7f3a7c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:56.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.964+0000 7f3a809fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7c074e70 0x7f3a7c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:56.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.964+0000 7f3a809fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a7c10ed50 0x7f3a7c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:56.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.964+0000 7f3a809fc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c114910 con 0x7f3a7c10ed50 2024-05-09T09:23:56.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.964+0000 7f3a809fc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c114790 con 0x7f3a7c074e70 2024-05-09T09:23:56.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.964+0000 7f3a809fc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c114a90 con 0x7f3a7c074550 2024-05-09T09:23:56.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.964+0000 7f3a7ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a7c10ed50 0x7f3a7c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:56.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.964+0000 7f3a79d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7c074e70 0x7f3a7c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.964+0000 7f3a79d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7c074e70 0x7f3a7c19b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:55830/0 (socket says 172.21.15.12:55830) 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.964+0000 7f3a79d9b700 1 -- 172.21.15.12:0/2983452748 learned_addr learned my addr 172.21.15.12:0/2983452748 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.964+0000 7f3a7a59c700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a7c074550 0x7f3a7c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.965+0000 7f3a7ad9d700 1 -- 172.21.15.12:0/2983452748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a7c074550 msgr2=0x7f3a7c19ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.965+0000 7f3a7ad9d700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a7c074550 0x7f3a7c19ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.965+0000 7f3a7ad9d700 1 -- 172.21.15.12:0/2983452748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7c074e70 msgr2=0x7f3a7c19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.965+0000 7f3a7ad9d700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7c074e70 0x7f3a7c19b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.965+0000 7f3a7ad9d700 1 -- 172.21.15.12:0/2983452748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c19fdf0 con 0x7f3a7c10ed50 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.965+0000 7f3a79d9b700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7c074e70 0x7f3a7c19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.965+0000 7f3a7a59c700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a7c074550 0x7f3a7c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.965+0000 7f3a7ad9d700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a7c10ed50 0x7f3a7c19f6b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a74007ee0 tx=0x7f3a7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.965+0000 7f3a6b7fe700 1 -- 172.21.15.12:0/2983452748 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400ddd0 con 0x7f3a7c10ed50 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.965+0000 7f3a6b7fe700 1 -- 172.21.15.12:0/2983452748 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3a74009e20 con 0x7f3a7c10ed50 2024-05-09T09:23:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.965+0000 7f3a809fc700 1 -- 172.21.15.12:0/2983452748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7c1a00e0 con 0x7f3a7c10ed50 2024-05-09T09:23:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.965+0000 7f3a809fc700 1 -- 172.21.15.12:0/2983452748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7c1a7960 con 0x7f3a7c10ed50 2024-05-09T09:23:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.965+0000 7f3a6b7fe700 1 -- 172.21.15.12:0/2983452748 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74015400 con 0x7f3a7c10ed50 2024-05-09T09:23:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.966+0000 7f3a809fc700 1 -- 172.21.15.12:0/2983452748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a7c062a30 con 0x7f3a7c10ed50 2024-05-09T09:23:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.968+0000 7f3a6b7fe700 1 -- 172.21.15.12:0/2983452748 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3a74004ad0 con 0x7f3a7c10ed50 2024-05-09T09:23:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.968+0000 7f3a6b7fe700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f3a640a2cb0 0x7f3a640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.969+0000 7f3a7a59c700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f3a640a2cb0 0x7f3a640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.969+0000 7f3a6b7fe700 1 -- 172.21.15.12:0/2983452748 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f3a740cf9f0 con 0x7f3a7c10ed50 2024-05-09T09:23:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.969+0000 7f3a6b7fe700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847] conn(0x7f3a640ad970 0x7f3a640afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.969+0000 7f3a6b7fe700 1 -- 172.21.15.12:0/2983452748 --> [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a74020e50 con 0x7f3a640ad970 2024-05-09T09:23:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.969+0000 7f3a6b7fe700 1 -- 172.21.15.12:0/2983452748 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f3a740cfe40 con 0x7f3a7c10ed50 2024-05-09T09:23:56.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.969+0000 7f3a7a59c700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f3a640a2cb0 0x7f3a640a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c006010 tx=0x7f3a6c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:56.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.969+0000 7f3a79d9b700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847] conn(0x7f3a640ad970 0x7f3a640afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:56.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.970+0000 7f3a79d9b700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847] conn(0x7f3a640ad970 0x7f3a640afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:56.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.970+0000 7f3a6b7fe700 1 -- 172.21.15.12:0/2983452748 <== osd.8 v2:172.21.15.170:6816/4075084847 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a74020e50 con 0x7f3a640ad970 2024-05-09T09:23:56.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.987+0000 7f3a809fc700 1 -- 172.21.15.12:0/2983452748 --> [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3a7c062e30 con 0x7f3a640ad970 2024-05-09T09:23:56.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.987+0000 7f3a6b7fe700 1 -- 172.21.15.12:0/2983452748 <== osd.8 v2:172.21.15.170:6816/4075084847 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3a7c062e30 con 0x7f3a640ad970 2024-05-09T09:23:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.987+0000 7f3a809fc700 1 -- 172.21.15.12:0/2983452748 >> [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847] conn(0x7f3a640ad970 msgr2=0x7f3a640afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.987+0000 7f3a809fc700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847] conn(0x7f3a640ad970 0x7f3a640afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.987+0000 7f3a809fc700 1 -- 172.21.15.12:0/2983452748 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f3a640a2cb0 msgr2=0x7f3a640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.987+0000 7f3a809fc700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f3a640a2cb0 0x7f3a640a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c006010 tx=0x7f3a6c00b540 comp rx=0 tx=0).stop 2024-05-09T09:23:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.987+0000 7f3a809fc700 1 -- 172.21.15.12:0/2983452748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a7c10ed50 msgr2=0x7f3a7c19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.987+0000 7f3a809fc700 1 --2- 172.21.15.12:0/2983452748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a7c10ed50 0x7f3a7c19f6b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a74007ee0 tx=0x7f3a7400d3b0 comp rx=0 tx=0).stop 2024-05-09T09:23:56.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.987+0000 7f3a7ad9d700 1 -- 172.21.15.12:0/2983452748 reap_dead start 2024-05-09T09:23:56.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.988+0000 7f3a809fc700 1 -- 172.21.15.12:0/2983452748 shutdown_connections 2024-05-09T09:23:56.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.988+0000 7f3a809fc700 1 -- 172.21.15.12:0/2983452748 >> 172.21.15.12:0/2983452748 conn(0x7f3a7c0794d0 msgr2=0x7f3a7c0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:56.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.988+0000 7f3a809fc700 1 -- 172.21.15.12:0/2983452748 shutdown_connections 2024-05-09T09:23:56.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:56.988+0000 7f3a809fc700 1 -- 172.21.15.12:0/2983452748 wait complete. 2024-05-09T09:23:57.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:57 smithi012 ceph-mon[30140]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:23:57.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:57 smithi170 ceph-mon[33632]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:23:58.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:57 smithi016 ceph-mon[33335]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.006+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3910562913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca34113520 msgr2=0x7fca34115910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.006+0000 7fca3a3bf700 1 --2- 172.21.15.12:0/3910562913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca34113520 0x7fca34115910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00b0a0 tx=0x7fca2c00b3b0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.006+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3910562913 shutdown_connections 2024-05-09T09:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.006+0000 7fca3a3bf700 1 --2- 172.21.15.12:0/3910562913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca34113520 0x7fca34115910 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.006+0000 7fca3a3bf700 1 --2- 172.21.15.12:0/3910562913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca34074550 0x7fca34074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.006+0000 7fca3a3bf700 1 --2- 172.21.15.12:0/3910562913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca34074e70 0x7fca34112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.006+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3910562913 >> 172.21.15.12:0/3910562913 conn(0x7fca34079490 msgr2=0x7fca340798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.006+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3910562913 shutdown_connections 2024-05-09T09:23:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.006+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3910562913 wait complete. 2024-05-09T09:23:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca3a3bf700 1 Processor -- start 2024-05-09T09:23:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca3a3bf700 1 -- start start 2024-05-09T09:23:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca3a3bf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca34074550 0x7fca3419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca3a3bf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca34074e70 0x7fca3419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca3a3bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca34113520 0x7fca341a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca3a3bf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fca34118c50 con 0x7fca34074550 2024-05-09T09:23:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca3a3bf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca34118ad0 con 0x7fca34113520 2024-05-09T09:23:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca3a3bf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fca34118dd0 con 0x7fca34074e70 2024-05-09T09:23:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca33fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca34074550 0x7fca3419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca33fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca34074550 0x7fca3419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60740/0 (socket says 172.21.15.12:60740) 2024-05-09T09:23:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca33fff700 1 -- 172.21.15.12:0/3284907277 learned_addr learned my addr 172.21.15.12:0/3284907277 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca3895c700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca34113520 0x7fca341a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca2bfff700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca34074e70 0x7fca3419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca33fff700 1 -- 172.21.15.12:0/3284907277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca34074e70 msgr2=0x7fca3419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca33fff700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca34074e70 0x7fca3419f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca33fff700 1 -- 172.21.15.12:0/3284907277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca34113520 msgr2=0x7fca341a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca33fff700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca34113520 0x7fca341a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.007+0000 7fca33fff700 1 -- 172.21.15.12:0/3284907277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca341a4070 con 0x7fca34074550 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.008+0000 7fca33fff700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca34074550 0x7fca3419f060 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fca2400e9a0 tx=0x7fca2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.008+0000 7fca31ffb700 1 -- 172.21.15.12:0/3284907277 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2400cbd0 con 0x7fca34074550 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.008+0000 7fca31ffb700 1 -- 172.21.15.12:0/3284907277 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fca2400cd70 con 0x7fca34074550 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.008+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3284907277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca341a4360 con 0x7fca34074550 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.008+0000 7fca31ffb700 1 -- 172.21.15.12:0/3284907277 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24018680 con 0x7fca34074550 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.008+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3284907277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca341abbe0 con 0x7fca34074550 2024-05-09T09:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.009+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3284907277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fca34112c90 con 0x7fca34074550 2024-05-09T09:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.010+0000 7fca31ffb700 1 -- 172.21.15.12:0/3284907277 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fca24010430 con 0x7fca34074550 2024-05-09T09:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.011+0000 7fca31ffb700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fca200a2cb0 0x7fca200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.011+0000 7fca2bfff700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fca200a2cb0 0x7fca200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.011+0000 7fca31ffb700 1 -- 172.21.15.12:0/3284907277 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fca240cff70 con 0x7fca34074550 2024-05-09T09:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.011+0000 7fca31ffb700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948] conn(0x7fca200ad970 0x7fca200afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.011+0000 7fca31ffb700 1 -- 172.21.15.12:0/3284907277 --> [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca2401bec0 con 0x7fca200ad970 2024-05-09T09:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.011+0000 7fca31ffb700 1 -- 172.21.15.12:0/3284907277 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fca240d0320 con 0x7fca34074550 2024-05-09T09:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.011+0000 7fca2bfff700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fca200a2cb0 0x7fca200a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca1c005950 tx=0x7fca1c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.011+0000 7fca3895c700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948] conn(0x7fca200ad970 0x7fca200afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.012+0000 7fca3895c700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948] conn(0x7fca200ad970 0x7fca200afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.013+0000 7fca31ffb700 1 -- 172.21.15.12:0/3284907277 <== osd.1 v2:172.21.15.16:6800/2346791948 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fca2401bec0 con 0x7fca200ad970 2024-05-09T09:23:58.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.033+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3284907277 --> [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fca34075c40 con 0x7fca200ad970 2024-05-09T09:23:58.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.034+0000 7fca31ffb700 1 -- 172.21.15.12:0/3284907277 <== osd.1 v2:172.21.15.16:6800/2346791948 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fca34075c40 con 0x7fca200ad970 2024-05-09T09:23:58.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.035+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3284907277 >> [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948] conn(0x7fca200ad970 msgr2=0x7fca200afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:58.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.035+0000 7fca3a3bf700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948] conn(0x7fca200ad970 0x7fca200afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.035+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3284907277 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fca200a2cb0 msgr2=0x7fca200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:58.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.035+0000 7fca3a3bf700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fca200a2cb0 0x7fca200a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca1c005950 tx=0x7fca1c0058e0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.035+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3284907277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca34074550 msgr2=0x7fca3419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:58.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.035+0000 7fca3a3bf700 1 --2- 172.21.15.12:0/3284907277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca34074550 0x7fca3419f060 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fca2400e9a0 tx=0x7fca2400ecb0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.035+0000 7fca3895c700 1 -- 172.21.15.12:0/3284907277 reap_dead start 2024-05-09T09:23:58.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.035+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3284907277 shutdown_connections 2024-05-09T09:23:58.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.035+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3284907277 >> 172.21.15.12:0/3284907277 conn(0x7fca34079490 msgr2=0x7fca34112270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:58.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.035+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3284907277 shutdown_connections 2024-05-09T09:23:58.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:58.035+0000 7fca3a3bf700 1 -- 172.21.15.12:0/3284907277 wait complete. 2024-05-09T09:23:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.903+0000 7f182d859700 1 -- 172.21.15.12:0/1922700199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1828101860 msgr2=0x7f1828101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.903+0000 7f182d859700 1 --2- 172.21.15.12:0/1922700199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1828101860 0x7f1828101c40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f18180099c0 tx=0x7f1818009cd0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.903+0000 7f182d859700 1 -- 172.21.15.12:0/1922700199 shutdown_connections 2024-05-09T09:23:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.903+0000 7f182d859700 1 --2- 172.21.15.12:0/1922700199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18281065b0 0x7f182810e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.903+0000 7f182d859700 1 --2- 172.21.15.12:0/1922700199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1828101860 0x7f1828101c40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.903+0000 7f182d859700 1 --2- 172.21.15.12:0/1922700199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1828102210 0x7f1828105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.903+0000 7f182d859700 1 -- 172.21.15.12:0/1922700199 >> 172.21.15.12:0/1922700199 conn(0x7f18280781d0 msgr2=0x7f18280785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.903+0000 7f182d859700 1 -- 172.21.15.12:0/1922700199 shutdown_connections 2024-05-09T09:23:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.903+0000 7f182d859700 1 -- 172.21.15.12:0/1922700199 wait complete. 2024-05-09T09:23:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.903+0000 7f182d859700 1 Processor -- start 2024-05-09T09:23:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f182d859700 1 -- start start 2024-05-09T09:23:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f182d859700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1828101860 0x7f182819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f182d859700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1828102210 0x7f182819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f182d859700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18281065b0 0x7f18281986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f182d859700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1828111520 con 0x7f1828101860 2024-05-09T09:23:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f182d859700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18281113a0 con 0x7f1828102210 2024-05-09T09:23:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f182d859700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f18281116a0 con 0x7f18281065b0 2024-05-09T09:23:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f1826ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1828101860 0x7f182819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f1826ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1828101860 0x7f182819e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60714/0 (socket says 172.21.15.12:60714) 2024-05-09T09:23:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f1826ffd700 1 -- 172.21.15.12:0/82014469 learned_addr learned my addr 172.21.15.12:0/82014469 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:23:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f181ffff700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1828102210 0x7f182819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f18277fe700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18281065b0 0x7f18281986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f1826ffd700 1 -- 172.21.15.12:0/82014469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18281065b0 msgr2=0x7f18281986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f1826ffd700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18281065b0 0x7f18281986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f1826ffd700 1 -- 172.21.15.12:0/82014469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1828102210 msgr2=0x7f182819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f1826ffd700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1828102210 0x7f182819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f1826ffd700 1 -- 172.21.15.12:0/82014469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1828198fb0 con 0x7f1828101860 2024-05-09T09:23:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f181ffff700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1828102210 0x7f182819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.904+0000 7f1826ffd700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1828101860 0x7f182819e600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1818009fd0 tx=0x7f181800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.905+0000 7f182c857700 1 -- 172.21.15.12:0/82014469 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818027070 con 0x7f1828101860 2024-05-09T09:23:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.905+0000 7f182d859700 1 -- 172.21.15.12:0/82014469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1828199240 con 0x7f1828101860 2024-05-09T09:23:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.905+0000 7f182c857700 1 -- 172.21.15.12:0/82014469 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1818022470 con 0x7f1828101860 2024-05-09T09:23:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.905+0000 7f182c857700 1 -- 172.21.15.12:0/82014469 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181800f6c0 con 0x7f1828101860 2024-05-09T09:23:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.905+0000 7f182d859700 1 -- 172.21.15.12:0/82014469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18281a5580 con 0x7f1828101860 2024-05-09T09:23:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.906+0000 7f182d859700 1 -- 172.21.15.12:0/82014469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1828062a70 con 0x7f1828101860 2024-05-09T09:23:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.907+0000 7f182c857700 1 -- 172.21.15.12:0/82014469 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f181802a070 con 0x7f1828101860 2024-05-09T09:23:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.908+0000 7f182c857700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f18140a2cb0 0x7f18140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.908+0000 7f181ffff700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f18140a2cb0 0x7f18140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.908+0000 7f182c857700 1 -- 172.21.15.12:0/82014469 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f18180d0850 con 0x7f1828101860 2024-05-09T09:23:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.909+0000 7f182c857700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120] conn(0x7f18140ad840 0x7f18140afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:58.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.909+0000 7f182c857700 1 -- 172.21.15.12:0/82014469 --> [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f181800b0b0 con 0x7f18140ad840 2024-05-09T09:23:58.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.909+0000 7f182c857700 1 -- 172.21.15.12:0/82014469 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f18180d0c00 con 0x7f1828101860 2024-05-09T09:23:58.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.909+0000 7f181ffff700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f18140a2cb0 0x7f18140a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1828199e90 tx=0x7f1810005f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:58.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.909+0000 7f18277fe700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120] conn(0x7f18140ad840 0x7f18140afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:58.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.909+0000 7f18277fe700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120] conn(0x7f18140ad840 0x7f18140afc80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:58.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.910+0000 7f182c857700 1 -- 172.21.15.12:0/82014469 <== osd.4 v2:172.21.15.16:6808/587973120 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f181800b0b0 con 0x7f18140ad840 2024-05-09T09:23:58.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.927+0000 7f182d859700 1 -- 172.21.15.12:0/82014469 --> [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1828062e70 con 0x7f18140ad840 2024-05-09T09:23:58.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.927+0000 7f182c857700 1 -- 172.21.15.12:0/82014469 <== osd.4 v2:172.21.15.16:6808/587973120 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1828062e70 con 0x7f18140ad840 2024-05-09T09:23:58.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.928+0000 7f182d859700 1 -- 172.21.15.12:0/82014469 >> [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120] conn(0x7f18140ad840 msgr2=0x7f18140afc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:58.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.928+0000 7f182d859700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120] conn(0x7f18140ad840 0x7f18140afc80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.928+0000 7f182d859700 1 -- 172.21.15.12:0/82014469 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f18140a2cb0 msgr2=0x7f18140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.928+0000 7f182d859700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f18140a2cb0 0x7f18140a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1828199e90 tx=0x7f1810005f30 comp rx=0 tx=0).stop 2024-05-09T09:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.928+0000 7f182d859700 1 -- 172.21.15.12:0/82014469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1828101860 msgr2=0x7f182819e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.928+0000 7f182d859700 1 --2- 172.21.15.12:0/82014469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1828101860 0x7f182819e600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1818009fd0 tx=0x7f181800bf60 comp rx=0 tx=0).stop 2024-05-09T09:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.928+0000 7f18277fe700 1 -- 172.21.15.12:0/82014469 reap_dead start 2024-05-09T09:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.928+0000 7f182d859700 1 -- 172.21.15.12:0/82014469 shutdown_connections 2024-05-09T09:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.928+0000 7f182d859700 1 -- 172.21.15.12:0/82014469 >> 172.21.15.12:0/82014469 conn(0x7f18280781d0 msgr2=0x7f182806ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.928+0000 7f182d859700 1 -- 172.21.15.12:0/82014469 shutdown_connections 2024-05-09T09:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:23:57.928+0000 7f182d859700 1 -- 172.21.15.12:0/82014469 wait complete. 2024-05-09T09:23:58.739 INFO:teuthology.orchestra.run.smithi012.stdout:141733920827 2024-05-09T09:23:58.739 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-09T09:23:59.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:23:59 smithi012 ceph-mon[30140]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:23:59.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:23:59 smithi170 ceph-mon[33632]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:24:00.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:23:59 smithi016 ceph-mon[33335]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:24:01.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:01 smithi012 ceph-mon[30140]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:24:01.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:01 smithi170 ceph-mon[33632]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:24:02.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:01 smithi016 ceph-mon[33335]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:24:02.583 INFO:teuthology.orchestra.run.smithi012.stdout:253403070502 2024-05-09T09:24:02.584 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-09T09:24:02.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.792+0000 7fc03278d700 1 -- 172.21.15.12:0/988717970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c101be0 msgr2=0x7fc02c10e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:02.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.792+0000 7fc03278d700 1 --2- 172.21.15.12:0/988717970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c101be0 0x7fc02c10e0d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc02400b0a0 tx=0x7fc02400b3b0 comp rx=0 tx=0).stop 2024-05-09T09:24:02.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.792+0000 7fc03278d700 1 -- 172.21.15.12:0/988717970 shutdown_connections 2024-05-09T09:24:02.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.792+0000 7fc03278d700 1 --2- 172.21.15.12:0/988717970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c101be0 0x7fc02c10e0d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:02.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.792+0000 7fc03278d700 1 --2- 172.21.15.12:0/988717970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c107290 0x7fc02c107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:02.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.792+0000 7fc03278d700 1 --2- 172.21.15.12:0/988717970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c101220 0x7fc02c1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:02.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.792+0000 7fc03278d700 1 -- 172.21.15.12:0/988717970 >> 172.21.15.12:0/988717970 conn(0x7fc02c0781d0 msgr2=0x7fc02c0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:02.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.792+0000 7fc03278d700 1 -- 172.21.15.12:0/988717970 shutdown_connections 2024-05-09T09:24:02.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.792+0000 7fc03278d700 1 -- 172.21.15.12:0/988717970 wait complete. 2024-05-09T09:24:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.793+0000 7fc03278d700 1 Processor -- start 2024-05-09T09:24:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.793+0000 7fc03278d700 1 -- start start 2024-05-09T09:24:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc03278d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c101220 0x7fc02c073c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc03278d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c101be0 0x7fc02c06ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc03278d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c107290 0x7fc02c06f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc03278d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1114f0 con 0x7fc02c107290 2024-05-09T09:24:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc03278d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c111370 con 0x7fc02c101220 2024-05-09T09:24:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc03278d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c111670 con 0x7fc02c101be0 2024-05-09T09:24:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc030d2a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c107290 0x7fc02c06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc030d2a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c107290 0x7fc02c06f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46332/0 (socket says 172.21.15.12:46332) 2024-05-09T09:24:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc030d2a700 1 -- 172.21.15.12:0/3181328016 learned_addr learned my addr 172.21.15.12:0/3181328016 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc02b7fe700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c101be0 0x7fc02c06ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc02bfff700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c101220 0x7fc02c073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc030d2a700 1 -- 172.21.15.12:0/3181328016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c101be0 msgr2=0x7fc02c06ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc030d2a700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c101be0 0x7fc02c06ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc030d2a700 1 -- 172.21.15.12:0/3181328016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c101220 msgr2=0x7fc02c073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc030d2a700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c101220 0x7fc02c073c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc030d2a700 1 -- 172.21.15.12:0/3181328016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02c06f740 con 0x7fc02c107290 2024-05-09T09:24:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc02b7fe700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c101be0 0x7fc02c06ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.794+0000 7fc030d2a700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c107290 0x7fc02c06f130 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc024006070 tx=0x7fc024004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.795+0000 7fc0297fa700 1 -- 172.21.15.12:0/3181328016 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024021070 con 0x7fc02c107290 2024-05-09T09:24:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.795+0000 7fc03278d700 1 -- 172.21.15.12:0/3181328016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02c06f9d0 con 0x7fc02c107290 2024-05-09T09:24:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.795+0000 7fc0297fa700 1 -- 172.21.15.12:0/3181328016 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc024007e50 con 0x7fc02c107290 2024-05-09T09:24:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.795+0000 7fc03278d700 1 -- 172.21.15.12:0/3181328016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02c1adce0 con 0x7fc02c107290 2024-05-09T09:24:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.795+0000 7fc0297fa700 1 -- 172.21.15.12:0/3181328016 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024004780 con 0x7fc02c107290 2024-05-09T09:24:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.796+0000 7fc03278d700 1 -- 172.21.15.12:0/3181328016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc02c0629b0 con 0x7fc02c107290 2024-05-09T09:24:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.797+0000 7fc0297fa700 1 -- 172.21.15.12:0/3181328016 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc024007980 con 0x7fc02c107290 2024-05-09T09:24:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.798+0000 7fc0297fa700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc0140a2cb0 0x7fc0140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.798+0000 7fc0297fa700 1 -- 172.21.15.12:0/3181328016 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc02400f020 con 0x7fc02c107290 2024-05-09T09:24:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.798+0000 7fc02bfff700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc0140a2cb0 0x7fc0140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.798+0000 7fc0297fa700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273] conn(0x7fc0140ad970 0x7fc0140afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.798+0000 7fc0297fa700 1 -- 172.21.15.12:0/3181328016 --> [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc0140b0480 con 0x7fc0140ad970 2024-05-09T09:24:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.798+0000 7fc0297fa700 1 -- 172.21.15.12:0/3181328016 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc0240d0ae0 con 0x7fc02c107290 2024-05-09T09:24:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.798+0000 7fc02bfff700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc0140a2cb0 0x7fc0140a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009bd0 tx=0x7fc01c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.798+0000 7fc02b7fe700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273] conn(0x7fc0140ad970 0x7fc0140afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.799+0000 7fc02b7fe700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273] conn(0x7fc0140ad970 0x7fc0140afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:02.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.800+0000 7fc0297fa700 1 -- 172.21.15.12:0/3181328016 <== osd.5 v2:172.21.15.170:6808/2002026273 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc0140b0480 con 0x7fc0140ad970 2024-05-09T09:24:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.818+0000 7fc03278d700 1 -- 172.21.15.12:0/3181328016 --> [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc02c062e70 con 0x7fc0140ad970 2024-05-09T09:24:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.819+0000 7fc0297fa700 1 -- 172.21.15.12:0/3181328016 <== osd.5 v2:172.21.15.170:6808/2002026273 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc02c062e70 con 0x7fc0140ad970 2024-05-09T09:24:02.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.820+0000 7fc03278d700 1 -- 172.21.15.12:0/3181328016 >> [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273] conn(0x7fc0140ad970 msgr2=0x7fc0140afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.820+0000 7fc03278d700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273] conn(0x7fc0140ad970 0x7fc0140afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.820+0000 7fc03278d700 1 -- 172.21.15.12:0/3181328016 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc0140a2cb0 msgr2=0x7fc0140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.820+0000 7fc03278d700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc0140a2cb0 0x7fc0140a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009bd0 tx=0x7fc01c009260 comp rx=0 tx=0).stop 2024-05-09T09:24:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.820+0000 7fc03278d700 1 -- 172.21.15.12:0/3181328016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c107290 msgr2=0x7fc02c06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.820+0000 7fc03278d700 1 --2- 172.21.15.12:0/3181328016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02c107290 0x7fc02c06f130 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc024006070 tx=0x7fc024004ce0 comp rx=0 tx=0).stop 2024-05-09T09:24:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.820+0000 7fc030d2a700 1 -- 172.21.15.12:0/3181328016 reap_dead start 2024-05-09T09:24:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.820+0000 7fc03278d700 1 -- 172.21.15.12:0/3181328016 shutdown_connections 2024-05-09T09:24:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.820+0000 7fc03278d700 1 -- 172.21.15.12:0/3181328016 >> 172.21.15.12:0/3181328016 conn(0x7fc02c0781d0 msgr2=0x7fc02c0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.820+0000 7fc03278d700 1 -- 172.21.15.12:0/3181328016 shutdown_connections 2024-05-09T09:24:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:02.820+0000 7fc03278d700 1 -- 172.21.15.12:0/3181328016 wait complete. 2024-05-09T09:24:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.040+0000 7f81c21a9700 1 -- 172.21.15.12:0/2591301789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81bc06a9d0 msgr2=0x7f81bc10f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.040+0000 7f81c21a9700 1 --2- 172.21.15.12:0/2591301789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81bc06a9d0 0x7f81bc10f1e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f81b400b0a0 tx=0x7f81b400b3b0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.040+0000 7f81c21a9700 1 -- 172.21.15.12:0/2591301789 shutdown_connections 2024-05-09T09:24:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.040+0000 7f81c21a9700 1 --2- 172.21.15.12:0/2591301789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81bc06a9d0 0x7f81bc10f1e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.040+0000 7f81c21a9700 1 --2- 172.21.15.12:0/2591301789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81bc1083a0 0x7f81bc108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.040+0000 7f81c21a9700 1 --2- 172.21.15.12:0/2591301789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81bc06a030 0x7f81bc06a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.040+0000 7f81c21a9700 1 -- 172.21.15.12:0/2591301789 >> 172.21.15.12:0/2591301789 conn(0x7f81bc0781d0 msgr2=0x7f81bc0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.040+0000 7f81c21a9700 1 -- 172.21.15.12:0/2591301789 shutdown_connections 2024-05-09T09:24:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.040+0000 7f81c21a9700 1 -- 172.21.15.12:0/2591301789 wait complete. 2024-05-09T09:24:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.040+0000 7f81c21a9700 1 Processor -- start 2024-05-09T09:24:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81c21a9700 1 -- start start 2024-05-09T09:24:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81c21a9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81bc06a030 0x7f81bc19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81c21a9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81bc06a9d0 0x7f81bc19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81c21a9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81bc1083a0 0x7f81bc198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81c21a9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1125e0 con 0x7f81bc06a9d0 2024-05-09T09:24:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81c21a9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc112460 con 0x7f81bc06a030 2024-05-09T09:24:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81c21a9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc112760 con 0x7f81bc1083a0 2024-05-09T09:24:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81baffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81bc06a9d0 0x7f81bc19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81baffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81bc06a9d0 0x7f81bc19ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46338/0 (socket says 172.21.15.12:46338) 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81baffd700 1 -- 172.21.15.12:0/1392168715 learned_addr learned my addr 172.21.15.12:0/1392168715 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81bbfff700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81bc1083a0 0x7f81bc198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81bb7fe700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81bc06a030 0x7f81bc19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81baffd700 1 -- 172.21.15.12:0/1392168715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81bc1083a0 msgr2=0x7f81bc198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81baffd700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81bc1083a0 0x7f81bc198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81baffd700 1 -- 172.21.15.12:0/1392168715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81bc06a030 msgr2=0x7f81bc19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81baffd700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81bc06a030 0x7f81bc19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81baffd700 1 -- 172.21.15.12:0/1392168715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81bc199020 con 0x7f81bc06a9d0 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.041+0000 7f81baffd700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81bc06a9d0 0x7f81bc19ec00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f81b000d950 tx=0x7f81b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.042+0000 7f81b8ff9700 1 -- 172.21.15.12:0/1392168715 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0016070 con 0x7f81bc06a9d0 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.042+0000 7f81b8ff9700 1 -- 172.21.15.12:0/1392168715 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f81b0009e10 con 0x7f81bc06a9d0 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.042+0000 7f81c21a9700 1 -- 172.21.15.12:0/1392168715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81bc199310 con 0x7f81bc06a9d0 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.042+0000 7f81b8ff9700 1 -- 172.21.15.12:0/1392168715 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b000f620 con 0x7f81bc06a9d0 2024-05-09T09:24:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.042+0000 7f81c21a9700 1 -- 172.21.15.12:0/1392168715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81bc1a5520 con 0x7f81bc06a9d0 2024-05-09T09:24:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.043+0000 7f81c21a9700 1 -- 172.21.15.12:0/1392168715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f81bc062a70 con 0x7f81bc06a9d0 2024-05-09T09:24:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.044+0000 7f81b8ff9700 1 -- 172.21.15.12:0/1392168715 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f81b0019070 con 0x7f81bc06a9d0 2024-05-09T09:24:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.044+0000 7f81b8ff9700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f81a40a2cb0 0x7f81a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.045+0000 7f81bb7fe700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f81a40a2cb0 0x7f81a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.045+0000 7f81b8ff9700 1 -- 172.21.15.12:0/1392168715 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f81b00cf000 con 0x7f81bc06a9d0 2024-05-09T09:24:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.045+0000 7f81b8ff9700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859] conn(0x7f81a40ad970 0x7f81a40afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.045+0000 7f81b8ff9700 1 -- 172.21.15.12:0/1392168715 --> [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81b0010e50 con 0x7f81a40ad970 2024-05-09T09:24:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.045+0000 7f81b8ff9700 1 -- 172.21.15.12:0/1392168715 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f81b00d6050 con 0x7f81bc06a9d0 2024-05-09T09:24:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.045+0000 7f81bb7fe700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f81a40a2cb0 0x7f81a40a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005fd0 tx=0x7f81ac005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.045+0000 7f81bbfff700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859] conn(0x7f81a40ad970 0x7f81a40afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.045+0000 7f81bbfff700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859] conn(0x7f81a40ad970 0x7f81a40afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.046+0000 7f81b8ff9700 1 -- 172.21.15.12:0/1392168715 <== osd.2 v2:172.21.15.170:6800/4106675859 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f81b0010e50 con 0x7f81a40ad970 2024-05-09T09:24:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.063+0000 7f81c21a9700 1 -- 172.21.15.12:0/1392168715 --> [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f81bc062f20 con 0x7f81a40ad970 2024-05-09T09:24:03.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.063+0000 7f81b8ff9700 1 -- 172.21.15.12:0/1392168715 <== osd.2 v2:172.21.15.170:6800/4106675859 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f81bc062f20 con 0x7f81a40ad970 2024-05-09T09:24:03.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.064+0000 7f81c21a9700 1 -- 172.21.15.12:0/1392168715 >> [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859] conn(0x7f81a40ad970 msgr2=0x7f81a40afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.064+0000 7f81c21a9700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859] conn(0x7f81a40ad970 0x7f81a40afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.064+0000 7f81c21a9700 1 -- 172.21.15.12:0/1392168715 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f81a40a2cb0 msgr2=0x7f81a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.064+0000 7f81c21a9700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f81a40a2cb0 0x7f81a40a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005fd0 tx=0x7f81ac005ee0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.064+0000 7f81c21a9700 1 -- 172.21.15.12:0/1392168715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81bc06a9d0 msgr2=0x7f81bc19ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.064+0000 7f81c21a9700 1 --2- 172.21.15.12:0/1392168715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81bc06a9d0 0x7f81bc19ec00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f81b000d950 tx=0x7f81b000dc60 comp rx=0 tx=0).stop 2024-05-09T09:24:03.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.064+0000 7f81bbfff700 1 -- 172.21.15.12:0/1392168715 reap_dead start 2024-05-09T09:24:03.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.064+0000 7f81c21a9700 1 -- 172.21.15.12:0/1392168715 shutdown_connections 2024-05-09T09:24:03.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.064+0000 7f81c21a9700 1 -- 172.21.15.12:0/1392168715 >> 172.21.15.12:0/1392168715 conn(0x7f81bc0781d0 msgr2=0x7f81bc0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:03.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.064+0000 7f81c21a9700 1 -- 172.21.15.12:0/1392168715 shutdown_connections 2024-05-09T09:24:03.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.064+0000 7f81c21a9700 1 -- 172.21.15.12:0/1392168715 wait complete. 2024-05-09T09:24:03.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:03 smithi012 ceph-mon[30140]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:24:03.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:03 smithi170 ceph-mon[33632]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:24:04.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:03 smithi016 ceph-mon[33335]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:24:04.239 INFO:teuthology.orchestra.run.smithi012.stdout:77309411398 2024-05-09T09:24:04.240 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-09T09:24:04.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.696+0000 7f94f17fc700 1 -- 172.21.15.12:0/3654519876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ec101e50 msgr2=0x7f94ec105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.696+0000 7f94f17fc700 1 --2- 172.21.15.12:0/3654519876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ec101e50 0x7f94ec105ea0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f94e00099c0 tx=0x7f94e0009cd0 comp rx=0 tx=0).stop 2024-05-09T09:24:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.696+0000 7f94f17fc700 1 -- 172.21.15.12:0/3654519876 shutdown_connections 2024-05-09T09:24:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.696+0000 7f94f17fc700 1 --2- 172.21.15.12:0/3654519876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94ec106580 0x7f94ec10e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.696+0000 7f94f17fc700 1 --2- 172.21.15.12:0/3654519876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec1014a0 0x7f94ec101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.696+0000 7f94f17fc700 1 --2- 172.21.15.12:0/3654519876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ec101e50 0x7f94ec105ea0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.696+0000 7f94f17fc700 1 -- 172.21.15.12:0/3654519876 >> 172.21.15.12:0/3654519876 conn(0x7f94ec0782c0 msgr2=0x7f94ec0786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.696+0000 7f94f17fc700 1 -- 172.21.15.12:0/3654519876 shutdown_connections 2024-05-09T09:24:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.696+0000 7f94f17fc700 1 -- 172.21.15.12:0/3654519876 wait complete. 2024-05-09T09:24:04.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.696+0000 7f94f17fc700 1 Processor -- start 2024-05-09T09:24:04.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.696+0000 7f94f17fc700 1 -- start start 2024-05-09T09:24:04.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94f17fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94ec1014a0 0x7f94ec199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:04.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94f17fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec101e50 0x7f94ec199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:04.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94f17fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ec106580 0x7f94ec1a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:04.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94f17fc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec111500 con 0x7f94ec106580 2024-05-09T09:24:04.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94f17fc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec111380 con 0x7f94ec101e50 2024-05-09T09:24:04.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94f17fc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec111680 con 0x7f94ec1014a0 2024-05-09T09:24:04.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ec106580 0x7f94ec1a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:04.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ec106580 0x7f94ec1a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46342/0 (socket says 172.21.15.12:46342) 2024-05-09T09:24:04.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94eb7fe700 1 -- 172.21.15.12:0/3619195242 learned_addr learned my addr 172.21.15.12:0/3619195242 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:04.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94eaffd700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94ec1014a0 0x7f94ec199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:04.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94ea7fc700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec101e50 0x7f94ec199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94eb7fe700 1 -- 172.21.15.12:0/3619195242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94ec1014a0 msgr2=0x7f94ec199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94eb7fe700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94ec1014a0 0x7f94ec199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94eb7fe700 1 -- 172.21.15.12:0/3619195242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec101e50 msgr2=0x7f94ec199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94eb7fe700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec101e50 0x7f94ec199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.697+0000 7f94eb7fe700 1 -- 172.21.15.12:0/3619195242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec1a4ec0 con 0x7f94ec106580 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.698+0000 7f94eb7fe700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ec106580 0x7f94ec1a4980 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f94e400ada0 tx=0x7f94e400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.698+0000 7f94d3fff700 1 -- 172.21.15.12:0/3619195242 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4017020 con 0x7f94ec106580 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.698+0000 7f94f17fc700 1 -- 172.21.15.12:0/3619195242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ec1a50f0 con 0x7f94ec106580 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.698+0000 7f94d3fff700 1 -- 172.21.15.12:0/3619195242 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f94e400cda0 con 0x7f94ec106580 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.698+0000 7f94d3fff700 1 -- 172.21.15.12:0/3619195242 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e40053a0 con 0x7f94ec106580 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.698+0000 7f94f17fc700 1 -- 172.21.15.12:0/3619195242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ec1a5600 con 0x7f94ec106580 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.699+0000 7f94f17fc700 1 -- 172.21.15.12:0/3619195242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94ec062a30 con 0x7f94ec106580 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.701+0000 7f94d3fff700 1 -- 172.21.15.12:0/3619195242 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f94e400c8d0 con 0x7f94ec106580 2024-05-09T09:24:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.702+0000 7f94d3fff700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f94d40a2cb0 0x7f94d40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.702+0000 7f94eaffd700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f94d40a2cb0 0x7f94d40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.702+0000 7f94d3fff700 1 -- 172.21.15.12:0/3619195242 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f94e40cf390 con 0x7f94ec106580 2024-05-09T09:24:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.702+0000 7f94d3fff700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423] conn(0x7f94d40ad970 0x7f94d40afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.702+0000 7f94d3fff700 1 -- 172.21.15.12:0/3619195242 --> [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94e4019e50 con 0x7f94d40ad970 2024-05-09T09:24:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.702+0000 7f94d3fff700 1 -- 172.21.15.12:0/3619195242 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f94e40cf7e0 con 0x7f94ec106580 2024-05-09T09:24:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.702+0000 7f94eaffd700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f94d40a2cb0 0x7f94d40a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f94dc007be0 tx=0x7f94dc0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.702+0000 7f94ea7fc700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423] conn(0x7f94d40ad970 0x7f94d40afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.703+0000 7f94ea7fc700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423] conn(0x7f94d40ad970 0x7f94d40afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.704+0000 7f94d3fff700 1 -- 172.21.15.12:0/3619195242 <== osd.10 v2:172.21.15.16:6824/1985793423 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94e4019e50 con 0x7f94d40ad970 2024-05-09T09:24:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.721+0000 7f94f17fc700 1 -- 172.21.15.12:0/3619195242 --> [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f94ec062eb0 con 0x7f94d40ad970 2024-05-09T09:24:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.721+0000 7f94d3fff700 1 -- 172.21.15.12:0/3619195242 <== osd.10 v2:172.21.15.16:6824/1985793423 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f94ec062eb0 con 0x7f94d40ad970 2024-05-09T09:24:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.721+0000 7f94f17fc700 1 -- 172.21.15.12:0/3619195242 >> [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423] conn(0x7f94d40ad970 msgr2=0x7f94d40afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.721+0000 7f94f17fc700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423] conn(0x7f94d40ad970 0x7f94d40afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.721+0000 7f94f17fc700 1 -- 172.21.15.12:0/3619195242 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f94d40a2cb0 msgr2=0x7f94d40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.721+0000 7f94f17fc700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f94d40a2cb0 0x7f94d40a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f94dc007be0 tx=0x7f94dc0073d0 comp rx=0 tx=0).stop 2024-05-09T09:24:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.721+0000 7f94f17fc700 1 -- 172.21.15.12:0/3619195242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ec106580 msgr2=0x7f94ec1a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.721+0000 7f94f17fc700 1 --2- 172.21.15.12:0/3619195242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ec106580 0x7f94ec1a4980 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f94e400ada0 tx=0x7f94e400c330 comp rx=0 tx=0).stop 2024-05-09T09:24:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.721+0000 7f94eb7fe700 1 -- 172.21.15.12:0/3619195242 reap_dead start 2024-05-09T09:24:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.722+0000 7f94f17fc700 1 -- 172.21.15.12:0/3619195242 shutdown_connections 2024-05-09T09:24:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.722+0000 7f94f17fc700 1 -- 172.21.15.12:0/3619195242 >> 172.21.15.12:0/3619195242 conn(0x7f94ec0782c0 msgr2=0x7f94ec0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.722+0000 7f94f17fc700 1 -- 172.21.15.12:0/3619195242 shutdown_connections 2024-05-09T09:24:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:03.722+0000 7f94f17fc700 1 -- 172.21.15.12:0/3619195242 wait complete. 2024-05-09T09:24:04.900 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-09T09:24:04.900 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-09T09:24:04.900 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-05-09T09:24:04.900 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-09T09:24:04.900 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-05-09T09:24:04.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-09T09:24:04.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-05-09T09:24:04.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-05-09T09:24:04.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-05-09T09:24:04.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:04 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-09T09:24:04.925 INFO:teuthology.orchestra.run.smithi012.stdout:309237645340 2024-05-09T09:24:04.926 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-09T09:24:04.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-09T09:24:04.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-09T09:24:04.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-05-09T09:24:04.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-09T09:24:04.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-05-09T09:24:04.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-09T09:24:04.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-05-09T09:24:04.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-05-09T09:24:04.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-05-09T09:24:04.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:04 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-09T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:04 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-09T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:04 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-09T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:04 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-05-09T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:04 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-09T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:04 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-05-09T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:04 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-09T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:04 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-05-09T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:04 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-05-09T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:04 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-05-09T09:24:05.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:04 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-09T09:24:05.770 INFO:teuthology.orchestra.run.smithi012.stdout:55834574922 2024-05-09T09:24:05.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-09T09:24:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:05 smithi012 ceph-mon[30140]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T09:24:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-09T09:24:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-09T09:24:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-05-09T09:24:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-09T09:24:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-05-09T09:24:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-09T09:24:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-05-09T09:24:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-05-09T09:24:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-05-09T09:24:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-09T09:24:05.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:05 smithi012 ceph-mon[30140]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T09:24:05.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:05 smithi170 ceph-mon[33632]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T09:24:05.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-09T09:24:05.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-09T09:24:05.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-05-09T09:24:05.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-09T09:24:05.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-05-09T09:24:05.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-09T09:24:05.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-05-09T09:24:05.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-05-09T09:24:05.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-05-09T09:24:05.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-09T09:24:05.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:05 smithi170 ceph-mon[33632]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T09:24:06.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:05 smithi016 ceph-mon[33335]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T09:24:06.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-09T09:24:06.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-09T09:24:06.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-05-09T09:24:06.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-09T09:24:06.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-05-09T09:24:06.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-09T09:24:06.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-05-09T09:24:06.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-05-09T09:24:06.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-05-09T09:24:06.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-09T09:24:06.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:05 smithi016 ceph-mon[33335]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T09:24:06.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:06 smithi012 ceph-mon[30140]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T09:24:07.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:06 smithi016 ceph-mon[33335]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T09:24:07.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:06 smithi170 ceph-mon[33632]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T09:24:07.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:07 smithi012 ceph-mon[30140]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-05-09T09:24:08.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:07 smithi016 ceph-mon[33335]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-05-09T09:24:08.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:07 smithi170 ceph-mon[33632]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-05-09T09:24:09.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.362+0000 7fcf28816700 1 -- 172.21.15.12:0/718677133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2406a9d0 msgr2=0x7fcf2410f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:09.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.362+0000 7fcf28816700 1 --2- 172.21.15.12:0/718677133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2406a9d0 0x7fcf2410f1e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00b0a0 tx=0x7fcf1c00b3b0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.363+0000 7fcf28816700 1 -- 172.21.15.12:0/718677133 shutdown_connections 2024-05-09T09:24:09.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.363+0000 7fcf28816700 1 --2- 172.21.15.12:0/718677133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2406a9d0 0x7fcf2410f1e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.363+0000 7fcf28816700 1 --2- 172.21.15.12:0/718677133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf241083a0 0x7fcf24108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.363+0000 7fcf28816700 1 --2- 172.21.15.12:0/718677133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2406a030 0x7fcf2406a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.363+0000 7fcf28816700 1 -- 172.21.15.12:0/718677133 >> 172.21.15.12:0/718677133 conn(0x7fcf240781d0 msgr2=0x7fcf240785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:09.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.363+0000 7fcf28816700 1 -- 172.21.15.12:0/718677133 shutdown_connections 2024-05-09T09:24:09.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.363+0000 7fcf28816700 1 -- 172.21.15.12:0/718677133 wait complete. 2024-05-09T09:24:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.364+0000 7fcf28816700 1 Processor -- start 2024-05-09T09:24:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.364+0000 7fcf28816700 1 -- start start 2024-05-09T09:24:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.364+0000 7fcf28816700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2406a030 0x7fcf241012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.364+0000 7fcf28816700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2406a9d0 0x7fcf24101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.364+0000 7fcf28816700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf241083a0 0x7fcf24101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.364+0000 7fcf28816700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcf24112640 con 0x7fcf2406a9d0 2024-05-09T09:24:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.364+0000 7fcf28816700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf241124c0 con 0x7fcf241083a0 2024-05-09T09:24:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.364+0000 7fcf28816700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf241127c0 con 0x7fcf2406a030 2024-05-09T09:24:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.364+0000 7fcf2259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2406a030 0x7fcf241012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.364+0000 7fcf2259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2406a030 0x7fcf241012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:59296/0 (socket says 172.21.15.12:59296) 2024-05-09T09:24:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.364+0000 7fcf2259c700 1 -- 172.21.15.12:0/2268157712 learned_addr learned my addr 172.21.15.12:0/2268157712 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.365+0000 7fcf21d9b700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2406a9d0 0x7fcf24101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:09.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.365+0000 7fcf22d9d700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf241083a0 0x7fcf24101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:09.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.365+0000 7fcf21d9b700 1 -- 172.21.15.12:0/2268157712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2406a030 msgr2=0x7fcf241012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:09.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.365+0000 7fcf21d9b700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2406a030 0x7fcf241012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.365+0000 7fcf21d9b700 1 -- 172.21.15.12:0/2268157712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf241083a0 msgr2=0x7fcf24101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:09.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.365+0000 7fcf21d9b700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf241083a0 0x7fcf24101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.365+0000 7fcf21d9b700 1 -- 172.21.15.12:0/2268157712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf241ab460 con 0x7fcf2406a9d0 2024-05-09T09:24:09.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.365+0000 7fcf22d9d700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf241083a0 0x7fcf24101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:09.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.366+0000 7fcf21d9b700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2406a9d0 0x7fcf24101830 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800e9f0 tx=0x7fcf1800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.366+0000 7fcf137fe700 1 -- 172.21.15.12:0/2268157712 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18004220 con 0x7fcf2406a9d0 2024-05-09T09:24:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.366+0000 7fcf137fe700 1 -- 172.21.15.12:0/2268157712 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcf18004d00 con 0x7fcf2406a9d0 2024-05-09T09:24:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.366+0000 7fcf137fe700 1 -- 172.21.15.12:0/2268157712 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18005020 con 0x7fcf2406a9d0 2024-05-09T09:24:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.366+0000 7fcf28816700 1 -- 172.21.15.12:0/2268157712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf241ab6f0 con 0x7fcf2406a9d0 2024-05-09T09:24:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.366+0000 7fcf28816700 1 -- 172.21.15.12:0/2268157712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf241abbe0 con 0x7fcf2406a9d0 2024-05-09T09:24:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.367+0000 7fcf28816700 1 -- 172.21.15.12:0/2268157712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf24062a30 con 0x7fcf2406a9d0 2024-05-09T09:24:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.369+0000 7fcf137fe700 1 -- 172.21.15.12:0/2268157712 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcf18004750 con 0x7fcf2406a9d0 2024-05-09T09:24:09.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.369+0000 7fcf137fe700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcf0c0a2d30 0x7fcf0c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:09.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.370+0000 7fcf137fe700 1 -- 172.21.15.12:0/2268157712 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fcf180cf690 con 0x7fcf2406a9d0 2024-05-09T09:24:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.370+0000 7fcf2259c700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcf0c0a2d30 0x7fcf0c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.370+0000 7fcf137fe700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494] conn(0x7fcf0c0ade90 0x7fcf0c0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.370+0000 7fcf137fe700 1 -- 172.21.15.12:0/2268157712 --> [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf0c0b0a00 con 0x7fcf0c0ade90 2024-05-09T09:24:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.370+0000 7fcf137fe700 1 -- 172.21.15.12:0/2268157712 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fcf180cfae0 con 0x7fcf2406a9d0 2024-05-09T09:24:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.370+0000 7fcf22d9d700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494] conn(0x7fcf0c0ade90 0x7fcf0c0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.371+0000 7fcf2259c700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcf0c0a2d30 0x7fcf0c0a51f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009ca0 tx=0x7fcf14009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:09.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.371+0000 7fcf22d9d700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494] conn(0x7fcf0c0ade90 0x7fcf0c0b0330 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:09.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.372+0000 7fcf137fe700 1 -- 172.21.15.12:0/2268157712 <== osd.6 v2:172.21.15.12:6818/3412258494 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf0c0b0a00 con 0x7fcf0c0ade90 2024-05-09T09:24:09.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.390+0000 7fcf28816700 1 -- 172.21.15.12:0/2268157712 --> [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf24062eb0 con 0x7fcf0c0ade90 2024-05-09T09:24:09.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.391+0000 7fcf137fe700 1 -- 172.21.15.12:0/2268157712 <== osd.6 v2:172.21.15.12:6818/3412258494 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcf24062eb0 con 0x7fcf0c0ade90 2024-05-09T09:24:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.391+0000 7fcf28816700 1 -- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494] conn(0x7fcf0c0ade90 msgr2=0x7fcf0c0b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.391+0000 7fcf28816700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494] conn(0x7fcf0c0ade90 0x7fcf0c0b0330 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.391+0000 7fcf28816700 1 -- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcf0c0a2d30 msgr2=0x7fcf0c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.391+0000 7fcf28816700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcf0c0a2d30 0x7fcf0c0a51f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009ca0 tx=0x7fcf14009260 comp rx=0 tx=0).stop 2024-05-09T09:24:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.391+0000 7fcf28816700 1 -- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2406a9d0 msgr2=0x7fcf24101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.391+0000 7fcf28816700 1 --2- 172.21.15.12:0/2268157712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2406a9d0 0x7fcf24101830 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800e9f0 tx=0x7fcf1800ed00 comp rx=0 tx=0).stop 2024-05-09T09:24:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.391+0000 7fcf22d9d700 1 -- 172.21.15.12:0/2268157712 reap_dead start 2024-05-09T09:24:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.392+0000 7fcf28816700 1 -- 172.21.15.12:0/2268157712 shutdown_connections 2024-05-09T09:24:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.392+0000 7fcf28816700 1 -- 172.21.15.12:0/2268157712 >> 172.21.15.12:0/2268157712 conn(0x7fcf240781d0 msgr2=0x7fcf240ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.392+0000 7fcf28816700 1 -- 172.21.15.12:0/2268157712 shutdown_connections 2024-05-09T09:24:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:09.392+0000 7fcf28816700 1 -- 172.21.15.12:0/2268157712 wait complete. 2024-05-09T09:24:09.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.335+0000 7f09e3370700 1 -- 172.21.15.12:0/3484033574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09dc06a030 msgr2=0x7f09dc06a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:09.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.335+0000 7f09e3370700 1 --2- 172.21.15.12:0/3484033574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09dc06a030 0x7f09dc06a490 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009990 tx=0x7f09cc009ca0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.335+0000 7f09e3370700 1 -- 172.21.15.12:0/3484033574 shutdown_connections 2024-05-09T09:24:09.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.335+0000 7f09e3370700 1 --2- 172.21.15.12:0/3484033574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09dc06a9d0 0x7f09dc10f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.335+0000 7f09e3370700 1 --2- 172.21.15.12:0/3484033574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc1083a0 0x7f09dc108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.335+0000 7f09e3370700 1 --2- 172.21.15.12:0/3484033574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09dc06a030 0x7f09dc06a490 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.335+0000 7f09e3370700 1 -- 172.21.15.12:0/3484033574 >> 172.21.15.12:0/3484033574 conn(0x7f09dc0781d0 msgr2=0x7f09dc0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.335+0000 7f09e3370700 1 -- 172.21.15.12:0/3484033574 shutdown_connections 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.336+0000 7f09e3370700 1 -- 172.21.15.12:0/3484033574 wait complete. 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.336+0000 7f09e3370700 1 Processor -- start 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.336+0000 7f09e3370700 1 -- start start 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.336+0000 7f09e3370700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09dc06a030 0x7f09dc1a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.336+0000 7f09e3370700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc06a9d0 0x7f09dc1a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.336+0000 7f09e3370700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09dc1083a0 0x7f09dc19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.336+0000 7f09e3370700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc1125d0 con 0x7f09dc06a030 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.336+0000 7f09e3370700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc112450 con 0x7f09dc1083a0 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.336+0000 7f09e3370700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc112750 con 0x7f09dc06a9d0 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09e110c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09dc06a030 0x7f09dc1a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09e110c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09dc06a030 0x7f09dc1a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46394/0 (socket says 172.21.15.12:46394) 2024-05-09T09:24:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09e110c700 1 -- 172.21.15.12:0/329250637 learned_addr learned my addr 172.21.15.12:0/329250637 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:09.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09e090b700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc06a9d0 0x7f09dc1a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:09.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09e190d700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09dc1083a0 0x7f09dc19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:09.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09e110c700 1 -- 172.21.15.12:0/329250637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc06a9d0 msgr2=0x7f09dc1a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:09.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09e110c700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc06a9d0 0x7f09dc1a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09e110c700 1 -- 172.21.15.12:0/329250637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09dc1083a0 msgr2=0x7f09dc19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09e110c700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09dc1083a0 0x7f09dc19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09e110c700 1 -- 172.21.15.12:0/329250637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc19b210 con 0x7f09dc06a030 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09e110c700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09dc06a030 0x7f09dc1a08f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f09d8007e20 tx=0x7f09d800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09d27fc700 1 -- 172.21.15.12:0/329250637 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8016020 con 0x7f09dc06a030 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09d27fc700 1 -- 172.21.15.12:0/329250637 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f09d800ad40 con 0x7f09dc06a030 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09d27fc700 1 -- 172.21.15.12:0/329250637 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8011700 con 0x7f09dc06a030 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.337+0000 7f09e3370700 1 -- 172.21.15.12:0/329250637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09dc19b500 con 0x7f09dc06a030 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.338+0000 7f09e3370700 1 -- 172.21.15.12:0/329250637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09dc1a77c0 con 0x7f09dc06a030 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.338+0000 7f09e3370700 1 -- 172.21.15.12:0/329250637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09dc1004f0 con 0x7f09dc06a030 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.340+0000 7f09d27fc700 1 -- 172.21.15.12:0/329250637 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f09d8019020 con 0x7f09dc06a030 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.341+0000 7f09d27fc700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f09c80a2cb0 0x7f09c80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.341+0000 7f09e090b700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f09c80a2cb0 0x7f09c80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.341+0000 7f09d27fc700 1 -- 172.21.15.12:0/329250637 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f09d80ceea0 con 0x7f09dc06a030 2024-05-09T09:24:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.341+0000 7f09d27fc700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834] conn(0x7f09c80ade10 0x7f09c80b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.341+0000 7f09d27fc700 1 -- 172.21.15.12:0/329250637 --> [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09d8008e50 con 0x7f09c80ade10 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.341+0000 7f09d27fc700 1 -- 172.21.15.12:0/329250637 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f09d80d1560 con 0x7f09dc06a030 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.341+0000 7f09e190d700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834] conn(0x7f09c80ade10 0x7f09c80b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.341+0000 7f09e090b700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f09c80a2cb0 0x7f09c80a5170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0098e0 tx=0x7f09cc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.342+0000 7f09e190d700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834] conn(0x7f09c80ade10 0x7f09c80b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.342+0000 7f09d27fc700 1 -- 172.21.15.12:0/329250637 <== osd.9 v2:172.21.15.12:6826/1889064834 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09d8008e50 con 0x7f09c80ade10 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.359+0000 7f09e3370700 1 -- 172.21.15.12:0/329250637 --> [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f09dc103a50 con 0x7f09c80ade10 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.359+0000 7f09d27fc700 1 -- 172.21.15.12:0/329250637 <== osd.9 v2:172.21.15.12:6826/1889064834 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f09dc103a50 con 0x7f09c80ade10 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.360+0000 7f09e3370700 1 -- 172.21.15.12:0/329250637 >> [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834] conn(0x7f09c80ade10 msgr2=0x7f09c80b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.360+0000 7f09e3370700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834] conn(0x7f09c80ade10 0x7f09c80b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.360+0000 7f09e3370700 1 -- 172.21.15.12:0/329250637 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f09c80a2cb0 msgr2=0x7f09c80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.360+0000 7f09e3370700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f09c80a2cb0 0x7f09c80a5170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0098e0 tx=0x7f09cc005960 comp rx=0 tx=0).stop 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.360+0000 7f09e3370700 1 -- 172.21.15.12:0/329250637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09dc06a030 msgr2=0x7f09dc1a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.360+0000 7f09e3370700 1 --2- 172.21.15.12:0/329250637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09dc06a030 0x7f09dc1a08f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f09d8007e20 tx=0x7f09d800a340 comp rx=0 tx=0).stop 2024-05-09T09:24:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.360+0000 7f09e190d700 1 -- 172.21.15.12:0/329250637 reap_dead start 2024-05-09T09:24:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.360+0000 7f09e3370700 1 -- 172.21.15.12:0/329250637 shutdown_connections 2024-05-09T09:24:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.360+0000 7f09e3370700 1 -- 172.21.15.12:0/329250637 >> 172.21.15.12:0/329250637 conn(0x7f09dc0781d0 msgr2=0x7f09dc0ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.360+0000 7f09e3370700 1 -- 172.21.15.12:0/329250637 shutdown_connections 2024-05-09T09:24:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.360+0000 7f09e3370700 1 -- 172.21.15.12:0/329250637 wait complete. 2024-05-09T09:24:09.954 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:09 smithi012 ceph-mon[30140]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 8 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:09.954 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:09 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:24:10.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:09 smithi016 ceph-mon[33335]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 8 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:10.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:09 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:24:10.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.239+0000 7f9c63133700 1 -- 172.21.15.12:0/1510669590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c5c1083a0 msgr2=0x7f9c5c108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.239+0000 7f9c63133700 1 --2- 172.21.15.12:0/1510669590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c5c1083a0 0x7f9c5c108780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009990 tx=0x7f9c50009ca0 comp rx=0 tx=0).stop 2024-05-09T09:24:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.239+0000 7f9c63133700 1 -- 172.21.15.12:0/1510669590 shutdown_connections 2024-05-09T09:24:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.239+0000 7f9c63133700 1 --2- 172.21.15.12:0/1510669590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c06a9d0 0x7f9c5c10f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.239+0000 7f9c63133700 1 --2- 172.21.15.12:0/1510669590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c5c1083a0 0x7f9c5c108780 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.239+0000 7f9c63133700 1 --2- 172.21.15.12:0/1510669590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c5c06a030 0x7f9c5c06a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.239+0000 7f9c63133700 1 -- 172.21.15.12:0/1510669590 >> 172.21.15.12:0/1510669590 conn(0x7f9c5c0781d0 msgr2=0x7f9c5c0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.239+0000 7f9c63133700 1 -- 172.21.15.12:0/1510669590 shutdown_connections 2024-05-09T09:24:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.239+0000 7f9c63133700 1 -- 172.21.15.12:0/1510669590 wait complete. 2024-05-09T09:24:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.239+0000 7f9c63133700 1 Processor -- start 2024-05-09T09:24:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.239+0000 7f9c63133700 1 -- start start 2024-05-09T09:24:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.240+0000 7f9c63133700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c06a030 0x7f9c5c1012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.240+0000 7f9c63133700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c5c06a9d0 0x7f9c5c101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.240+0000 7f9c63133700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c5c1083a0 0x7f9c5c101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.240+0000 7f9c63133700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1125d0 con 0x7f9c5c06a9d0 2024-05-09T09:24:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.240+0000 7f9c63133700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c112450 con 0x7f9c5c06a030 2024-05-09T09:24:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.240+0000 7f9c63133700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c112750 con 0x7f9c5c1083a0 2024-05-09T09:24:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.240+0000 7f9c5bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c5c06a9d0 0x7f9c5c101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.240+0000 7f9c60ecf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c06a030 0x7f9c5c1012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.240+0000 7f9c60ecf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c06a030 0x7f9c5c1012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:56712/0 (socket says 172.21.15.12:56712) 2024-05-09T09:24:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.240+0000 7f9c60ecf700 1 -- 172.21.15.12:0/3182014250 learned_addr learned my addr 172.21.15.12:0/3182014250 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.240+0000 7f9c616d0700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c5c1083a0 0x7f9c5c101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.241+0000 7f9c60ecf700 1 -- 172.21.15.12:0/3182014250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c5c1083a0 msgr2=0x7f9c5c101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.241+0000 7f9c60ecf700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c5c1083a0 0x7f9c5c101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:10.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.241+0000 7f9c60ecf700 1 -- 172.21.15.12:0/3182014250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c5c06a9d0 msgr2=0x7f9c5c101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:10.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.241+0000 7f9c60ecf700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c5c06a9d0 0x7f9c5c101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:10.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.241+0000 7f9c60ecf700 1 -- 172.21.15.12:0/3182014250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c1a7030 con 0x7f9c5c06a030 2024-05-09T09:24:10.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.241+0000 7f9c5bfff700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c5c06a9d0 0x7f9c5c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:10.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.241+0000 7f9c60ecf700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c06a030 0x7f9c5c1012f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9c500057c0 tx=0x7f9c500057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:10.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.241+0000 7f9c59ffb700 1 -- 172.21.15.12:0/3182014250 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50026070 con 0x7f9c5c06a030 2024-05-09T09:24:10.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.241+0000 7f9c59ffb700 1 -- 172.21.15.12:0/3182014250 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9c5000fb40 con 0x7f9c5c06a030 2024-05-09T09:24:10.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.242+0000 7f9c59ffb700 1 -- 172.21.15.12:0/3182014250 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c500173f0 con 0x7f9c5c06a030 2024-05-09T09:24:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.242+0000 7f9c63133700 1 -- 172.21.15.12:0/3182014250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5c1a72c0 con 0x7f9c5c06a030 2024-05-09T09:24:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.243+0000 7f9c63133700 1 -- 172.21.15.12:0/3182014250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5c1a7770 con 0x7f9c5c06a030 2024-05-09T09:24:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.243+0000 7f9c63133700 1 -- 172.21.15.12:0/3182014250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c5c062a30 con 0x7f9c5c06a030 2024-05-09T09:24:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.245+0000 7f9c59ffb700 1 -- 172.21.15.12:0/3182014250 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9c50009d10 con 0x7f9c5c06a030 2024-05-09T09:24:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.246+0000 7f9c59ffb700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9c4c0a2c10 0x7f9c4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.247+0000 7f9c5bfff700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9c4c0a2c10 0x7f9c4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.247+0000 7f9c59ffb700 1 -- 172.21.15.12:0/3182014250 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9c500d0030 con 0x7f9c5c06a030 2024-05-09T09:24:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.247+0000 7f9c59ffb700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762] conn(0x7f9c4c0add70 0x7f9c4c0b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.247+0000 7f9c59ffb700 1 -- 172.21.15.12:0/3182014250 --> [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c5000eea0 con 0x7f9c4c0add70 2024-05-09T09:24:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.247+0000 7f9c59ffb700 1 -- 172.21.15.12:0/3182014250 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f9c500d0470 con 0x7f9c5c06a030 2024-05-09T09:24:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.247+0000 7f9c616d0700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762] conn(0x7f9c4c0add70 0x7f9c4c0b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.247+0000 7f9c5bfff700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9c4c0a2c10 0x7f9c4c0a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0788b0 tx=0x7f9c480073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.248+0000 7f9c616d0700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762] conn(0x7f9c4c0add70 0x7f9c4c0b0210 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.248+0000 7f9c59ffb700 1 -- 172.21.15.12:0/3182014250 <== osd.3 v2:172.21.15.12:6810/2152501762 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9c5000eea0 con 0x7f9c4c0add70 2024-05-09T09:24:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.265+0000 7f9c63133700 1 -- 172.21.15.12:0/3182014250 --> [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c5c062eb0 con 0x7f9c4c0add70 2024-05-09T09:24:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.265+0000 7f9c59ffb700 1 -- 172.21.15.12:0/3182014250 <== osd.3 v2:172.21.15.12:6810/2152501762 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9c5c062eb0 con 0x7f9c4c0add70 2024-05-09T09:24:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.266+0000 7f9c63133700 1 -- 172.21.15.12:0/3182014250 >> [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762] conn(0x7f9c4c0add70 msgr2=0x7f9c4c0b0210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.266+0000 7f9c63133700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762] conn(0x7f9c4c0add70 0x7f9c4c0b0210 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.266+0000 7f9c63133700 1 -- 172.21.15.12:0/3182014250 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9c4c0a2c10 msgr2=0x7f9c4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.266+0000 7f9c63133700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9c4c0a2c10 0x7f9c4c0a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0788b0 tx=0x7f9c480073d0 comp rx=0 tx=0).stop 2024-05-09T09:24:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.266+0000 7f9c63133700 1 -- 172.21.15.12:0/3182014250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c06a030 msgr2=0x7f9c5c1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.266+0000 7f9c63133700 1 --2- 172.21.15.12:0/3182014250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c06a030 0x7f9c5c1012f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9c500057c0 tx=0x7f9c500057f0 comp rx=0 tx=0).stop 2024-05-09T09:24:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.266+0000 7f9c616d0700 1 -- 172.21.15.12:0/3182014250 reap_dead start 2024-05-09T09:24:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.266+0000 7f9c63133700 1 -- 172.21.15.12:0/3182014250 shutdown_connections 2024-05-09T09:24:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.266+0000 7f9c63133700 1 -- 172.21.15.12:0/3182014250 >> 172.21.15.12:0/3182014250 conn(0x7f9c5c0781d0 msgr2=0x7f9c5c0fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.266+0000 7f9c63133700 1 -- 172.21.15.12:0/3182014250 shutdown_connections 2024-05-09T09:24:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:08.266+0000 7f9c63133700 1 -- 172.21.15.12:0/3182014250 wait complete. 2024-05-09T09:24:10.134 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:09 smithi170 ceph-mon[33632]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 8 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:10.135 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:09 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:24:10.698 INFO:teuthology.orchestra.run.smithi012.stdout:107374182465 2024-05-09T09:24:10.698 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-09T09:24:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:163208757302 2024-05-09T09:24:11.108 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-09T09:24:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.939+0000 7f5d0f08a700 1 -- 172.21.15.12:0/3337261748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d08107280 msgr2=0x7f5d0806aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.939+0000 7f5d0f08a700 1 --2- 172.21.15.12:0/3337261748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d08107280 0x7f5d0806aad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009990 tx=0x7f5d04009ca0 comp rx=0 tx=0).stop 2024-05-09T09:24:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.939+0000 7f5d0f08a700 1 -- 172.21.15.12:0/3337261748 shutdown_connections 2024-05-09T09:24:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.939+0000 7f5d0f08a700 1 --2- 172.21.15.12:0/3337261748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08104d90 0x7f5d081051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.939+0000 7f5d0f08a700 1 --2- 172.21.15.12:0/3337261748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d08107280 0x7f5d0806aad0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.939+0000 7f5d0f08a700 1 --2- 172.21.15.12:0/3337261748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d0806b010 0x7f5d0806b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.939+0000 7f5d0f08a700 1 -- 172.21.15.12:0/3337261748 >> 172.21.15.12:0/3337261748 conn(0x7f5d080782c0 msgr2=0x7f5d080786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.939+0000 7f5d0f08a700 1 -- 172.21.15.12:0/3337261748 shutdown_connections 2024-05-09T09:24:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.939+0000 7f5d0f08a700 1 -- 172.21.15.12:0/3337261748 wait complete. 2024-05-09T09:24:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0f08a700 1 Processor -- start 2024-05-09T09:24:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0f08a700 1 -- start start 2024-05-09T09:24:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0f08a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d0806b010 0x7f5d0819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0f08a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08104d90 0x7f5d0819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0f08a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d08107280 0x7f5d081a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0f08a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08118c40 con 0x7f5d0806b010 2024-05-09T09:24:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0f08a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08118ac0 con 0x7f5d08104d90 2024-05-09T09:24:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0f08a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08118dc0 con 0x7f5d08107280 2024-05-09T09:24:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0ce26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d0806b010 0x7f5d0819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0ce26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d0806b010 0x7f5d0819f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56162/0 (socket says 172.21.15.12:56162) 2024-05-09T09:24:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0ce26700 1 -- 172.21.15.12:0/2843866429 learned_addr learned my addr 172.21.15.12:0/2843866429 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0d627700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d08107280 0x7f5d081a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5cfffff700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08104d90 0x7f5d0819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0ce26700 1 -- 172.21.15.12:0/2843866429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d08107280 msgr2=0x7f5d081a3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0ce26700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d08107280 0x7f5d081a3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0ce26700 1 -- 172.21.15.12:0/2843866429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08104d90 msgr2=0x7f5d0819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0ce26700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08104d90 0x7f5d0819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.940+0000 7f5d0ce26700 1 -- 172.21.15.12:0/2843866429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d081a40c0 con 0x7f5d0806b010 2024-05-09T09:24:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.941+0000 7f5d0ce26700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d0806b010 0x7f5d0819f0b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400bff0 tx=0x7f5d04005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.941+0000 7f5cfdffb700 1 -- 172.21.15.12:0/2843866429 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04027070 con 0x7f5d0806b010 2024-05-09T09:24:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.941+0000 7f5d0f08a700 1 -- 172.21.15.12:0/2843866429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d081a4350 con 0x7f5d0806b010 2024-05-09T09:24:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.941+0000 7f5cfdffb700 1 -- 172.21.15.12:0/2843866429 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5d04004040 con 0x7f5d0806b010 2024-05-09T09:24:11.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.941+0000 7f5cfdffb700 1 -- 172.21.15.12:0/2843866429 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04022d00 con 0x7f5d0806b010 2024-05-09T09:24:11.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.941+0000 7f5d0f08a700 1 -- 172.21.15.12:0/2843866429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d081abc30 con 0x7f5d0806b010 2024-05-09T09:24:11.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.942+0000 7f5d0f08a700 1 -- 172.21.15.12:0/2843866429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d081047d0 con 0x7f5d0806b010 2024-05-09T09:24:11.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.943+0000 7f5cfdffb700 1 -- 172.21.15.12:0/2843866429 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5d0402a070 con 0x7f5d0806b010 2024-05-09T09:24:11.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.944+0000 7f5cfdffb700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5cf00a2cb0 0x7f5cf00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:11.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.944+0000 7f5cfdffb700 1 -- 172.21.15.12:0/2843866429 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5d040d06b0 con 0x7f5d0806b010 2024-05-09T09:24:11.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.944+0000 7f5cfffff700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5cf00a2cb0 0x7f5cf00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:11.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.944+0000 7f5cfdffb700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281] conn(0x7f5cf00ade10 0x7f5cf00b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:11.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.944+0000 7f5cfdffb700 1 -- 172.21.15.12:0/2843866429 --> [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d040095d0 con 0x7f5cf00ade10 2024-05-09T09:24:11.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.944+0000 7f5cfdffb700 1 -- 172.21.15.12:0/2843866429 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f5d040d0b00 con 0x7f5d0806b010 2024-05-09T09:24:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.944+0000 7f5d0d627700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281] conn(0x7f5cf00ade10 0x7f5cf00b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.944+0000 7f5cfffff700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5cf00a2cb0 0x7f5cf00a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8005950 tx=0x7f5cf80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.945+0000 7f5d0d627700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281] conn(0x7f5cf00ade10 0x7f5cf00b02b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.945+0000 7f5cfdffb700 1 -- 172.21.15.12:0/2843866429 <== osd.0 v2:172.21.15.12:6802/2958883281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d040095d0 con 0x7f5cf00ade10 2024-05-09T09:24:11.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.963+0000 7f5d0f08a700 1 -- 172.21.15.12:0/2843866429 --> [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5d08101870 con 0x7f5cf00ade10 2024-05-09T09:24:11.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.963+0000 7f5cfdffb700 1 -- 172.21.15.12:0/2843866429 <== osd.0 v2:172.21.15.12:6802/2958883281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5d08101870 con 0x7f5cf00ade10 2024-05-09T09:24:11.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.964+0000 7f5d0f08a700 1 -- 172.21.15.12:0/2843866429 >> [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281] conn(0x7f5cf00ade10 msgr2=0x7f5cf00b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:11.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.964+0000 7f5d0f08a700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281] conn(0x7f5cf00ade10 0x7f5cf00b02b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:11.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.964+0000 7f5d0f08a700 1 -- 172.21.15.12:0/2843866429 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5cf00a2cb0 msgr2=0x7f5cf00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:11.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.964+0000 7f5d0f08a700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5cf00a2cb0 0x7f5cf00a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8005950 tx=0x7f5cf80058e0 comp rx=0 tx=0).stop 2024-05-09T09:24:11.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.964+0000 7f5d0f08a700 1 -- 172.21.15.12:0/2843866429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d0806b010 msgr2=0x7f5d0819f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:11.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.964+0000 7f5d0f08a700 1 --2- 172.21.15.12:0/2843866429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d0806b010 0x7f5d0819f0b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400bff0 tx=0x7f5d04005ef0 comp rx=0 tx=0).stop 2024-05-09T09:24:11.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.964+0000 7f5d0d627700 1 -- 172.21.15.12:0/2843866429 reap_dead start 2024-05-09T09:24:11.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.964+0000 7f5d0f08a700 1 -- 172.21.15.12:0/2843866429 shutdown_connections 2024-05-09T09:24:11.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.964+0000 7f5d0f08a700 1 -- 172.21.15.12:0/2843866429 >> 172.21.15.12:0/2843866429 conn(0x7f5d080782c0 msgr2=0x7f5d08103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:11.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.964+0000 7f5d0f08a700 1 -- 172.21.15.12:0/2843866429 shutdown_connections 2024-05-09T09:24:11.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:11.964+0000 7f5d0f08a700 1 -- 172.21.15.12:0/2843866429 wait complete. 2024-05-09T09:24:12.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:12 smithi012 ceph-mon[30140]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:12.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:12 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:12.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:12 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:12.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:12 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:12.177 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:12 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:12.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:12 smithi170 ceph-mon[33632]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:12.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:12 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:12.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:12 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:12.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:12 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:12.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:12 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:12.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:12 smithi016 ceph-mon[33335]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:12.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:12 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:12.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:12 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:12.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:12 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:12.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:12 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:13.021 INFO:teuthology.orchestra.run.smithi012.stdout:34359738449 2024-05-09T09:24:13.022 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-09T09:24:13.772 INFO:teuthology.orchestra.run.smithi012.stdout:287762808867 2024-05-09T09:24:13.772 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-09T09:24:14.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.407+0000 7ff7acdde700 1 -- 172.21.15.12:0/1245747548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a8102f80 msgr2=0x7ff7a8106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.407+0000 7ff7acdde700 1 --2- 172.21.15.12:0/1245747548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a8102f80 0x7ff7a8106f80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009990 tx=0x7ff79c009ca0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.407+0000 7ff7acdde700 1 -- 172.21.15.12:0/1245747548 shutdown_connections 2024-05-09T09:24:14.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.407+0000 7ff7acdde700 1 --2- 172.21.15.12:0/1245747548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a8107660 0x7ff7a810f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.407+0000 7ff7acdde700 1 --2- 172.21.15.12:0/1245747548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a81025d0 0x7ff7a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.407+0000 7ff7acdde700 1 --2- 172.21.15.12:0/1245747548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a8102f80 0x7ff7a8106f80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.407+0000 7ff7acdde700 1 -- 172.21.15.12:0/1245747548 >> 172.21.15.12:0/1245747548 conn(0x7ff7a8079480 msgr2=0x7ff7a8079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:14.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.407+0000 7ff7acdde700 1 -- 172.21.15.12:0/1245747548 shutdown_connections 2024-05-09T09:24:14.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.408+0000 7ff7acdde700 1 -- 172.21.15.12:0/1245747548 wait complete. 2024-05-09T09:24:14.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.408+0000 7ff7acdde700 1 Processor -- start 2024-05-09T09:24:14.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7acdde700 1 -- start start 2024-05-09T09:24:14.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7acdde700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a81025d0 0x7ff7a819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7acdde700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a8102f80 0x7ff7a819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7acdde700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a8107660 0x7ff7a8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7acdde700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8112600 con 0x7ff7a8102f80 2024-05-09T09:24:14.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7acdde700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8112480 con 0x7ff7a8107660 2024-05-09T09:24:14.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7acdde700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8112780 con 0x7ff7a81025d0 2024-05-09T09:24:14.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7a5d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a8102f80 0x7ff7a819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7a5d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a8102f80 0x7ff7a819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56186/0 (socket says 172.21.15.12:56186) 2024-05-09T09:24:14.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7a5d9b700 1 -- 172.21.15.12:0/3965501840 learned_addr learned my addr 172.21.15.12:0/3965501840 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:14.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7a659c700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a81025d0 0x7ff7a819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7a5d9b700 1 -- 172.21.15.12:0/3965501840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a81025d0 msgr2=0x7ff7a819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7a5d9b700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a81025d0 0x7ff7a819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7a5d9b700 1 -- 172.21.15.12:0/3965501840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a8107660 msgr2=0x7ff7a8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7a6d9d700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a8107660 0x7ff7a8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7a5d9b700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a8107660 0x7ff7a8198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7a5d9b700 1 -- 172.21.15.12:0/3965501840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a81990a0 con 0x7ff7a8102f80 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7a659c700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a81025d0 0x7ff7a819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.409+0000 7ff7a5d9b700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a8102f80 0x7ff7a819ecc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff79c000c00 tx=0x7ff79c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.410+0000 7ff7977fe700 1 -- 172.21.15.12:0/3965501840 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c027070 con 0x7ff7a8102f80 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.410+0000 7ff7acdde700 1 -- 172.21.15.12:0/3965501840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a8199330 con 0x7ff7a8102f80 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.410+0000 7ff7acdde700 1 -- 172.21.15.12:0/3965501840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a81a5650 con 0x7ff7a8102f80 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.410+0000 7ff7977fe700 1 -- 172.21.15.12:0/3965501840 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff79c004100 con 0x7ff7a8102f80 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.410+0000 7ff7977fe700 1 -- 172.21.15.12:0/3965501840 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c022ca0 con 0x7ff7a8102f80 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.411+0000 7ff7acdde700 1 -- 172.21.15.12:0/3965501840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff7a8062a70 con 0x7ff7a8102f80 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.412+0000 7ff7977fe700 1 -- 172.21.15.12:0/3965501840 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff79c02a070 con 0x7ff7a8102f80 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.413+0000 7ff7977fe700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7900a2cb0 0x7ff7900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.413+0000 7ff7977fe700 1 -- 172.21.15.12:0/3965501840 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff79c0d0870 con 0x7ff7a8102f80 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.413+0000 7ff7a659c700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7900a2cb0 0x7ff7900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.413+0000 7ff7977fe700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659] conn(0x7ff7900ade10 0x7ff7900b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.413+0000 7ff7977fe700 1 -- 172.21.15.12:0/3965501840 --> [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff79c00b0b0 con 0x7ff7900ade10 2024-05-09T09:24:14.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.413+0000 7ff7977fe700 1 -- 172.21.15.12:0/3965501840 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7ff79c0d0cc0 con 0x7ff7a8102f80 2024-05-09T09:24:14.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.413+0000 7ff7a659c700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7900a2cb0 0x7ff7900a5170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff798005950 tx=0x7ff7980058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:14.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.413+0000 7ff7a6d9d700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659] conn(0x7ff7900ade10 0x7ff7900b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.414+0000 7ff7a6d9d700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659] conn(0x7ff7900ade10 0x7ff7900b02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:14.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.415+0000 7ff7977fe700 1 -- 172.21.15.12:0/3965501840 <== osd.11 v2:172.21.15.170:6824/3375809659 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff79c00b0b0 con 0x7ff7900ade10 2024-05-09T09:24:14.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.431+0000 7ff7acdde700 1 -- 172.21.15.12:0/3965501840 --> [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff7a8062e70 con 0x7ff7900ade10 2024-05-09T09:24:14.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.432+0000 7ff7977fe700 1 -- 172.21.15.12:0/3965501840 <== osd.11 v2:172.21.15.170:6824/3375809659 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff7a8062e70 con 0x7ff7900ade10 2024-05-09T09:24:14.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.432+0000 7ff7acdde700 1 -- 172.21.15.12:0/3965501840 >> [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659] conn(0x7ff7900ade10 msgr2=0x7ff7900b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.432+0000 7ff7acdde700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659] conn(0x7ff7900ade10 0x7ff7900b02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.432+0000 7ff7acdde700 1 -- 172.21.15.12:0/3965501840 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7900a2cb0 msgr2=0x7ff7900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.432+0000 7ff7acdde700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff7900a2cb0 0x7ff7900a5170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff798005950 tx=0x7ff7980058e0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.432+0000 7ff7acdde700 1 -- 172.21.15.12:0/3965501840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a8102f80 msgr2=0x7ff7a819ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.432+0000 7ff7acdde700 1 --2- 172.21.15.12:0/3965501840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7a8102f80 0x7ff7a819ecc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff79c000c00 tx=0x7ff79c009f90 comp rx=0 tx=0).stop 2024-05-09T09:24:14.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.432+0000 7ff7a6d9d700 1 -- 172.21.15.12:0/3965501840 reap_dead start 2024-05-09T09:24:14.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.432+0000 7ff7acdde700 1 -- 172.21.15.12:0/3965501840 shutdown_connections 2024-05-09T09:24:14.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.432+0000 7ff7acdde700 1 -- 172.21.15.12:0/3965501840 >> 172.21.15.12:0/3965501840 conn(0x7ff7a8079480 msgr2=0x7ff7a8100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:14.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.432+0000 7ff7acdde700 1 -- 172.21.15.12:0/3965501840 shutdown_connections 2024-05-09T09:24:14.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:12.432+0000 7ff7acdde700 1 -- 172.21.15.12:0/3965501840 wait complete. 2024-05-09T09:24:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:14 smithi012 ceph-mon[30140]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:14.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:14 smithi170 ceph-mon[33632]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:14.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:14 smithi016 ceph-mon[33335]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:14.696 INFO:teuthology.orchestra.run.smithi012.stdout:343597383706 2024-05-09T09:24:14.697 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-09T09:24:14.711 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:15.136 INFO:teuthology.orchestra.run.smithi012.stdout:197568495667 2024-05-09T09:24:15.136 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-09T09:24:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.041+0000 7fe9bc521700 1 -- 172.21.15.12:0/66090616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b406ad30 msgr2=0x7fe9b406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.041+0000 7fe9bc521700 1 --2- 172.21.15.12:0/66090616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b406ad30 0x7fe9b406b110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00099c0 tx=0x7fe9b0009cd0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.041+0000 7fe9bc521700 1 -- 172.21.15.12:0/66090616 shutdown_connections 2024-05-09T09:24:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.041+0000 7fe9bc521700 1 --2- 172.21.15.12:0/66090616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b4107680 0x7fe9b410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.041+0000 7fe9bc521700 1 --2- 172.21.15.12:0/66090616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b406ad30 0x7fe9b406b110 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.041+0000 7fe9bc521700 1 --2- 172.21.15.12:0/66090616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9b406b6e0 0x7fe9b4106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.041+0000 7fe9bc521700 1 -- 172.21.15.12:0/66090616 >> 172.21.15.12:0/66090616 conn(0x7fe9b40782c0 msgr2=0x7fe9b40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.041+0000 7fe9bc521700 1 -- 172.21.15.12:0/66090616 shutdown_connections 2024-05-09T09:24:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.041+0000 7fe9bc521700 1 -- 172.21.15.12:0/66090616 wait complete. 2024-05-09T09:24:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9bc521700 1 Processor -- start 2024-05-09T09:24:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9bc521700 1 -- start start 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9bc521700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b406ad30 0x7fe9b419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9bc521700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9b406b6e0 0x7fe9b419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9bc521700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b4107680 0x7fe9b419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9bc521700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b4112620 con 0x7fe9b406ad30 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9bc521700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b41124a0 con 0x7fe9b4107680 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9bc521700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b41127a0 con 0x7fe9b406b6e0 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9ba2bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b406ad30 0x7fe9b419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9ba2bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b406ad30 0x7fe9b419ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56204/0 (socket says 172.21.15.12:56204) 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9ba2bd700 1 -- 172.21.15.12:0/2998532103 learned_addr learned my addr 172.21.15.12:0/2998532103 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9b9abc700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9b406b6e0 0x7fe9b419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9baabe700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b4107680 0x7fe9b419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9ba2bd700 1 -- 172.21.15.12:0/2998532103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9b406b6e0 msgr2=0x7fe9b419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9ba2bd700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9b406b6e0 0x7fe9b419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9ba2bd700 1 -- 172.21.15.12:0/2998532103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b4107680 msgr2=0x7fe9b419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9ba2bd700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9b4107680 0x7fe9b419f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.042+0000 7fe9ba2bd700 1 -- 172.21.15.12:0/2998532103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b419fd50 con 0x7fe9b406ad30 2024-05-09T09:24:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.043+0000 7fe9ba2bd700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b406ad30 0x7fe9b419ad40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00059d0 tx=0x7fe9b0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.043+0000 7fe9ab7fe700 1 -- 172.21.15.12:0/2998532103 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0027070 con 0x7fe9b406ad30 2024-05-09T09:24:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.043+0000 7fe9bc521700 1 -- 172.21.15.12:0/2998532103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b419ffe0 con 0x7fe9b406ad30 2024-05-09T09:24:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.043+0000 7fe9ab7fe700 1 -- 172.21.15.12:0/2998532103 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe9b00040b0 con 0x7fe9b406ad30 2024-05-09T09:24:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.043+0000 7fe9ab7fe700 1 -- 172.21.15.12:0/2998532103 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0022dd0 con 0x7fe9b406ad30 2024-05-09T09:24:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.043+0000 7fe9bc521700 1 -- 172.21.15.12:0/2998532103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b41a78c0 con 0x7fe9b406ad30 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.043+0000 7fe9bc521700 1 -- 172.21.15.12:0/2998532103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9b4062a30 con 0x7fe9b406ad30 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.045+0000 7fe9ab7fe700 1 -- 172.21.15.12:0/2998532103 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe9b002a070 con 0x7fe9b406ad30 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.046+0000 7fe9ab7fe700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe9a00a2cb0 0x7fe9a00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.046+0000 7fe9b9abc700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe9a00a2cb0 0x7fe9a00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.046+0000 7fe9ab7fe700 1 -- 172.21.15.12:0/2998532103 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe9b00d06c0 con 0x7fe9b406ad30 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.046+0000 7fe9ab7fe700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888] conn(0x7fe9a00ade10 0x7fe9a00b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.046+0000 7fe9ab7fe700 1 -- 172.21.15.12:0/2998532103 --> [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9b000ce90 con 0x7fe9a00ade10 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.046+0000 7fe9ab7fe700 1 -- 172.21.15.12:0/2998532103 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe9b00d0b10 con 0x7fe9b406ad30 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.046+0000 7fe9b9abc700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe9a00a2cb0 0x7fe9a00a5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4005950 tx=0x7fe9a40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.046+0000 7fe9baabe700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888] conn(0x7fe9a00ade10 0x7fe9a00b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.047+0000 7fe9baabe700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888] conn(0x7fe9a00ade10 0x7fe9a00b02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.048+0000 7fe9ab7fe700 1 -- 172.21.15.12:0/2998532103 <== osd.7 v2:172.21.15.16:6816/1916732888 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9b000ce90 con 0x7fe9a00ade10 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.065+0000 7fe9bc521700 1 -- 172.21.15.12:0/2998532103 --> [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe9b4062e30 con 0x7fe9a00ade10 2024-05-09T09:24:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.066+0000 7fe9ab7fe700 1 -- 172.21.15.12:0/2998532103 <== osd.7 v2:172.21.15.16:6816/1916732888 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe9b4062e30 con 0x7fe9a00ade10 2024-05-09T09:24:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.066+0000 7fe9bc521700 1 -- 172.21.15.12:0/2998532103 >> [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888] conn(0x7fe9a00ade10 msgr2=0x7fe9a00b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.066+0000 7fe9bc521700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888] conn(0x7fe9a00ade10 0x7fe9a00b02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.066+0000 7fe9bc521700 1 -- 172.21.15.12:0/2998532103 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe9a00a2cb0 msgr2=0x7fe9a00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.066+0000 7fe9bc521700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe9a00a2cb0 0x7fe9a00a5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4005950 tx=0x7fe9a40058e0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.066+0000 7fe9bc521700 1 -- 172.21.15.12:0/2998532103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b406ad30 msgr2=0x7fe9b419ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.066+0000 7fe9bc521700 1 --2- 172.21.15.12:0/2998532103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b406ad30 0x7fe9b419ad40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00059d0 tx=0x7fe9b0005e80 comp rx=0 tx=0).stop 2024-05-09T09:24:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.066+0000 7fe9baabe700 1 -- 172.21.15.12:0/2998532103 reap_dead start 2024-05-09T09:24:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.066+0000 7fe9bc521700 1 -- 172.21.15.12:0/2998532103 shutdown_connections 2024-05-09T09:24:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.066+0000 7fe9bc521700 1 -- 172.21.15.12:0/2998532103 >> 172.21.15.12:0/2998532103 conn(0x7fe9b40782c0 msgr2=0x7fe9b410d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.066+0000 7fe9bc521700 1 -- 172.21.15.12:0/2998532103 shutdown_connections 2024-05-09T09:24:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:14.066+0000 7fe9bc521700 1 -- 172.21.15.12:0/2998532103 wait complete. 2024-05-09T09:24:15.907 INFO:teuthology.orchestra.run.smithi012.stdout:219043332142 2024-05-09T09:24:15.908 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-09T09:24:16.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:16 smithi012 ceph-mon[30140]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 11 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:16.295 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:16 smithi170 ceph-mon[33632]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 11 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:16.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:16 smithi016 ceph-mon[33335]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 11 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:17.037 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:17.046 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:17.047 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:17.052 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:17.856 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:17.856 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:18.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:18 smithi012 ceph-mon[30140]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 10 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:18 smithi170 ceph-mon[33632]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 10 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:18.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:18 smithi016 ceph-mon[33335]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 10 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:20.229 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:20.305 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:20 smithi016 ceph-mon[33335]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:20.348 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:20 smithi012 ceph-mon[30140]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:20 smithi170 ceph-mon[33632]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T09:24:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.719+0000 7f7c87bb7700 1 -- 172.21.15.12:0/303275452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c80102fb0 msgr2=0x7f7c80106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.719+0000 7f7c87bb7700 1 --2- 172.21.15.12:0/303275452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c80102fb0 0x7f7c80106ff0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7c740099c0 tx=0x7f7c74009cd0 comp rx=0 tx=0).stop 2024-05-09T09:24:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.719+0000 7f7c87bb7700 1 -- 172.21.15.12:0/303275452 shutdown_connections 2024-05-09T09:24:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.719+0000 7f7c87bb7700 1 --2- 172.21.15.12:0/303275452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c801076d0 0x7f7c8010f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.719+0000 7f7c87bb7700 1 --2- 172.21.15.12:0/303275452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c80102600 0x7f7c801029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.719+0000 7f7c87bb7700 1 --2- 172.21.15.12:0/303275452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c80102fb0 0x7f7c80106ff0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.719+0000 7f7c87bb7700 1 -- 172.21.15.12:0/303275452 >> 172.21.15.12:0/303275452 conn(0x7f7c800794d0 msgr2=0x7f7c800798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.719+0000 7f7c87bb7700 1 -- 172.21.15.12:0/303275452 shutdown_connections 2024-05-09T09:24:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.719+0000 7f7c87bb7700 1 -- 172.21.15.12:0/303275452 wait complete. 2024-05-09T09:24:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.720+0000 7f7c87bb7700 1 Processor -- start 2024-05-09T09:24:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.720+0000 7f7c87bb7700 1 -- start start 2024-05-09T09:24:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.720+0000 7f7c87bb7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80102600 0x7f7c801a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.720+0000 7f7c87bb7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c80102fb0 0x7f7c801a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.720+0000 7f7c87bb7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c801076d0 0x7f7c8019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.720+0000 7f7c87bb7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7c801126a0 con 0x7f7c80102fb0 2024-05-09T09:24:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.720+0000 7f7c87bb7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c80112520 con 0x7f7c80102600 2024-05-09T09:24:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.720+0000 7f7c87bb7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7c80112820 con 0x7f7c801076d0 2024-05-09T09:24:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c85152700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c80102fb0 0x7f7c801a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c85152700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c80102fb0 0x7f7c801a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39072/0 (socket says 172.21.15.12:39072) 2024-05-09T09:24:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c85152700 1 -- 172.21.15.12:0/2337906188 learned_addr learned my addr 172.21.15.12:0/2337906188 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:21.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c85953700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80102600 0x7f7c801a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c86154700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c801076d0 0x7f7c8019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c85152700 1 -- 172.21.15.12:0/2337906188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c801076d0 msgr2=0x7f7c8019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c85152700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c801076d0 0x7f7c8019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c85152700 1 -- 172.21.15.12:0/2337906188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80102600 msgr2=0x7f7c801a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c85152700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80102600 0x7f7c801a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c85152700 1 -- 172.21.15.12:0/2337906188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8019b410 con 0x7f7c80102fb0 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c86154700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c801076d0 0x7f7c8019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c85152700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c80102fb0 0x7f7c801a0fa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7c74000c00 tx=0x7f7c740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c7effd700 1 -- 172.21.15.12:0/2337906188 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74027070 con 0x7f7c80102fb0 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c87bb7700 1 -- 172.21.15.12:0/2337906188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8019b6a0 con 0x7f7c80102fb0 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c7effd700 1 -- 172.21.15.12:0/2337906188 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7c74003fa0 con 0x7f7c80102fb0 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.721+0000 7f7c7effd700 1 -- 172.21.15.12:0/2337906188 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74022d30 con 0x7f7c80102fb0 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.722+0000 7f7c87bb7700 1 -- 172.21.15.12:0/2337906188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c801a79e0 con 0x7f7c80102fb0 2024-05-09T09:24:21.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.723+0000 7f7c87bb7700 1 -- 172.21.15.12:0/2337906188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c80103ad0 con 0x7f7c80102fb0 2024-05-09T09:24:21.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.724+0000 7f7c7effd700 1 -- 172.21.15.12:0/2337906188 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7c7402a070 con 0x7f7c80102fb0 2024-05-09T09:24:21.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.725+0000 7f7c7effd700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7c680a2d30 0x7f7c680a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:21.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.725+0000 7f7c7effd700 1 -- 172.21.15.12:0/2337906188 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7c740d0190 con 0x7f7c80102fb0 2024-05-09T09:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.726+0000 7f7c85953700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7c680a2d30 0x7f7c680a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.726+0000 7f7c85953700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7c680a2d30 0x7f7c680a51f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7c70003f90 tx=0x7f7c70009180 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:21.729 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:21.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.732+0000 7f7c7effd700 1 -- 172.21.15.12:0/2337906188 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7c74013070 con 0x7f7c80102fb0 2024-05-09T09:24:21.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.952+0000 7f7c87bb7700 1 -- 172.21.15.12:0/2337906188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f7c8019bff0 con 0x7f7c80102fb0 2024-05-09T09:24:21.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.952+0000 7f7c7effd700 1 -- 172.21.15.12:0/2337906188 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7c7409ccc0 con 0x7f7c80102fb0 2024-05-09T09:24:21.952 INFO:teuthology.orchestra.run.smithi012.stdout:309237645344 2024-05-09T09:24:21.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.954+0000 7f7c87bb7700 1 -- 172.21.15.12:0/2337906188 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7c680a2d30 msgr2=0x7f7c680a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:21.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.954+0000 7f7c87bb7700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7c680a2d30 0x7f7c680a51f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7c70003f90 tx=0x7f7c70009180 comp rx=0 tx=0).stop 2024-05-09T09:24:21.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.954+0000 7f7c87bb7700 1 -- 172.21.15.12:0/2337906188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c80102fb0 msgr2=0x7f7c801a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:21.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.954+0000 7f7c87bb7700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c80102fb0 0x7f7c801a0fa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7c74000c00 tx=0x7f7c740047b0 comp rx=0 tx=0).stop 2024-05-09T09:24:21.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.955+0000 7f7c87bb7700 1 -- 172.21.15.12:0/2337906188 shutdown_connections 2024-05-09T09:24:21.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.955+0000 7f7c87bb7700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7c680a2d30 0x7f7c680a51f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:21.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.955+0000 7f7c87bb7700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c801076d0 0x7f7c8019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:21.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.955+0000 7f7c87bb7700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80102600 0x7f7c801a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:21.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.955+0000 7f7c87bb7700 1 --2- 172.21.15.12:0/2337906188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c80102fb0 0x7f7c801a0fa0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:21.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.955+0000 7f7c87bb7700 1 -- 172.21.15.12:0/2337906188 >> 172.21.15.12:0/2337906188 conn(0x7f7c800794d0 msgr2=0x7f7c8010d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:21.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.955+0000 7f7c87bb7700 1 -- 172.21.15.12:0/2337906188 shutdown_connections 2024-05-09T09:24:21.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:21.955+0000 7f7c87bb7700 1 -- 172.21.15.12:0/2337906188 wait complete. 2024-05-09T09:24:22.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:22 smithi012 ceph-mon[30140]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-09T09:24:22.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:22 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2337906188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T09:24:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:22 smithi170 ceph-mon[33632]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-09T09:24:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:22 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2337906188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T09:24:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:22 smithi016 ceph-mon[33335]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-09T09:24:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:22 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2337906188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T09:24:24.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:24 smithi012 ceph-mon[30140]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-09T09:24:24.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:24 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1835610835' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T09:24:24.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:24 smithi170 ceph-mon[33632]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-09T09:24:24.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:24 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1835610835' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T09:24:24.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:24 smithi016 ceph-mon[33335]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-09T09:24:24.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:24 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1835610835' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T09:24:25.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.924+0000 7f16deef1700 1 -- 172.21.15.12:0/3131999696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d806b260 msgr2=0x7f16d810f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:25.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.924+0000 7f16deef1700 1 --2- 172.21.15.12:0/3131999696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d806b260 0x7f16d810f1d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f16d000b0a0 tx=0x7f16d000b3b0 comp rx=0 tx=0).stop 2024-05-09T09:24:25.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.924+0000 7f16deef1700 1 -- 172.21.15.12:0/3131999696 shutdown_connections 2024-05-09T09:24:25.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.924+0000 7f16deef1700 1 --2- 172.21.15.12:0/3131999696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d806b260 0x7f16d810f1d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:25.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.924+0000 7f16deef1700 1 --2- 172.21.15.12:0/3131999696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d8108390 0x7f16d8108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:25.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.924+0000 7f16deef1700 1 --2- 172.21.15.12:0/3131999696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d806a8c0 0x7f16d806ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:25.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.924+0000 7f16deef1700 1 -- 172.21.15.12:0/3131999696 >> 172.21.15.12:0/3131999696 conn(0x7f16d8078190 msgr2=0x7f16d80785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:25.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.924+0000 7f16deef1700 1 -- 172.21.15.12:0/3131999696 shutdown_connections 2024-05-09T09:24:25.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.924+0000 7f16deef1700 1 -- 172.21.15.12:0/3131999696 wait complete. 2024-05-09T09:24:25.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.924+0000 7f16deef1700 1 Processor -- start 2024-05-09T09:24:25.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.924+0000 7f16deef1700 1 -- start start 2024-05-09T09:24:25.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16deef1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d806a8c0 0x7f16d81012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:25.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16deef1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d806b260 0x7f16d8101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:25.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16deef1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d8108390 0x7f16d8101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:25.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16deef1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8112620 con 0x7f16d8108390 2024-05-09T09:24:25.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16deef1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81124a0 con 0x7f16d806a8c0 2024-05-09T09:24:25.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16deef1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81127a0 con 0x7f16d806b260 2024-05-09T09:24:25.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16dd48e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d8108390 0x7f16d8101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:25.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16dd48e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d8108390 0x7f16d8101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39086/0 (socket says 172.21.15.12:39086) 2024-05-09T09:24:25.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16dd48e700 1 -- 172.21.15.12:0/1835610835 learned_addr learned my addr 172.21.15.12:0/1835610835 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:25.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16dcc8d700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d806a8c0 0x7f16d81012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:25.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16d7fff700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d806b260 0x7f16d8101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:25.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16dd48e700 1 -- 172.21.15.12:0/1835610835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d806b260 msgr2=0x7f16d8101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:25.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16dd48e700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d806b260 0x7f16d8101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:25.347 INFO:teuthology.orchestra.run.smithi012.stdout:55834574927 2024-05-09T09:24:25.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16dd48e700 1 -- 172.21.15.12:0/1835610835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d806a8c0 msgr2=0x7f16d81012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:25.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16dd48e700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d806a8c0 0x7f16d81012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:25.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16dd48e700 1 -- 172.21.15.12:0/1835610835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d81a6f90 con 0x7f16d8108390 2024-05-09T09:24:25.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.925+0000 7f16dd48e700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d8108390 0x7f16d8101d60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f16d000b070 tx=0x7f16d0004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:25.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.926+0000 7f16d5ffb700 1 -- 172.21.15.12:0/1835610835 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0021070 con 0x7f16d8108390 2024-05-09T09:24:25.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.926+0000 7f16deef1700 1 -- 172.21.15.12:0/1835610835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d81a7220 con 0x7f16d8108390 2024-05-09T09:24:25.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.926+0000 7f16d5ffb700 1 -- 172.21.15.12:0/1835610835 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f16d0003c00 con 0x7f16d8108390 2024-05-09T09:24:25.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.926+0000 7f16d5ffb700 1 -- 172.21.15.12:0/1835610835 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d001d450 con 0x7f16d8108390 2024-05-09T09:24:25.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.926+0000 7f16deef1700 1 -- 172.21.15.12:0/1835610835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d81a7680 con 0x7f16d8108390 2024-05-09T09:24:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.927+0000 7f16deef1700 1 -- 172.21.15.12:0/1835610835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d8069010 con 0x7f16d8108390 2024-05-09T09:24:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.929+0000 7f16d5ffb700 1 -- 172.21.15.12:0/1835610835 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f16d0004000 con 0x7f16d8108390 2024-05-09T09:24:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.930+0000 7f16d5ffb700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f16c80a2cb0 0x7f16c80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.930+0000 7f16d5ffb700 1 -- 172.21.15.12:0/1835610835 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f16d00d0510 con 0x7f16d8108390 2024-05-09T09:24:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.930+0000 7f16dcc8d700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f16c80a2cb0 0x7f16c80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.930+0000 7f16dcc8d700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f16c80a2cb0 0x7f16c80a5170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009be0 tx=0x7f16cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:22.934+0000 7f16d5ffb700 1 -- 172.21.15.12:0/1835610835 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f16d009d040 con 0x7f16d8108390 2024-05-09T09:24:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.146+0000 7f16deef1700 1 -- 172.21.15.12:0/1835610835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f16d802db80 con 0x7f16d8108390 2024-05-09T09:24:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.146+0000 7f16d5ffb700 1 -- 172.21.15.12:0/1835610835 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f16d009c940 con 0x7f16d8108390 2024-05-09T09:24:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.148+0000 7f16deef1700 1 -- 172.21.15.12:0/1835610835 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f16c80a2cb0 msgr2=0x7f16c80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.148+0000 7f16deef1700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f16c80a2cb0 0x7f16c80a5170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009be0 tx=0x7f16cc009260 comp rx=0 tx=0).stop 2024-05-09T09:24:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.148+0000 7f16deef1700 1 -- 172.21.15.12:0/1835610835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d8108390 msgr2=0x7f16d8101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.148+0000 7f16deef1700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d8108390 0x7f16d8101d60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f16d000b070 tx=0x7f16d0004dc0 comp rx=0 tx=0).stop 2024-05-09T09:24:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.148+0000 7f16deef1700 1 -- 172.21.15.12:0/1835610835 shutdown_connections 2024-05-09T09:24:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.148+0000 7f16deef1700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f16c80a2cb0 0x7f16c80a5170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.148+0000 7f16deef1700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d8108390 0x7f16d8101d60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.148+0000 7f16deef1700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d806a8c0 0x7f16d81012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.148+0000 7f16deef1700 1 --2- 172.21.15.12:0/1835610835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d806b260 0x7f16d8101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.148+0000 7f16deef1700 1 -- 172.21.15.12:0/1835610835 >> 172.21.15.12:0/1835610835 conn(0x7f16d8078190 msgr2=0x7f16d80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.149+0000 7f16deef1700 1 -- 172.21.15.12:0/1835610835 shutdown_connections 2024-05-09T09:24:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:23.149+0000 7f16deef1700 1 -- 172.21.15.12:0/1835610835 wait complete. 2024-05-09T09:24:25.956 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574927 for osd.1 2024-05-09T09:24:25.956 DEBUG:teuthology.parallel:result is None 2024-05-09T09:24:26.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.137+0000 7ff9984b0700 1 -- 172.21.15.12:0/3449139481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff99006a9d0 msgr2=0x7ff99010f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:26.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.137+0000 7ff9984b0700 1 --2- 172.21.15.12:0/3449139481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff99006a9d0 0x7ff99010f1e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff98800b0a0 tx=0x7ff98800b3b0 comp rx=0 tx=0).stop 2024-05-09T09:24:26.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.137+0000 7ff9984b0700 1 -- 172.21.15.12:0/3449139481 shutdown_connections 2024-05-09T09:24:26.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.137+0000 7ff9984b0700 1 --2- 172.21.15.12:0/3449139481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff99006a9d0 0x7ff99010f1e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.137+0000 7ff9984b0700 1 --2- 172.21.15.12:0/3449139481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9901083a0 0x7ff990108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.137+0000 7ff9984b0700 1 --2- 172.21.15.12:0/3449139481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff99006a030 0x7ff99006a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.137+0000 7ff9984b0700 1 -- 172.21.15.12:0/3449139481 >> 172.21.15.12:0/3449139481 conn(0x7ff9900781d0 msgr2=0x7ff9900785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.137+0000 7ff9984b0700 1 -- 172.21.15.12:0/3449139481 shutdown_connections 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.137+0000 7ff9984b0700 1 -- 172.21.15.12:0/3449139481 wait complete. 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.138+0000 7ff9984b0700 1 Processor -- start 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.138+0000 7ff9984b0700 1 -- start start 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.138+0000 7ff9984b0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff99006a030 0x7ff9901012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.138+0000 7ff9984b0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99006a9d0 0x7ff990101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.138+0000 7ff9984b0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9901083a0 0x7ff990101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.138+0000 7ff9984b0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901125f0 con 0x7ff99006a030 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.138+0000 7ff9984b0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff990112470 con 0x7ff99006a9d0 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.138+0000 7ff9984b0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff990112770 con 0x7ff9901083a0 2024-05-09T09:24:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.138+0000 7ff99624c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff99006a030 0x7ff9901012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:26.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.138+0000 7ff99624c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff99006a030 0x7ff9901012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39108/0 (socket says 172.21.15.12:39108) 2024-05-09T09:24:26.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.138+0000 7ff99624c700 1 -- 172.21.15.12:0/3036414664 learned_addr learned my addr 172.21.15.12:0/3036414664 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:26.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.138+0000 7ff995a4b700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99006a9d0 0x7ff990101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:26.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.138+0000 7ff996a4d700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9901083a0 0x7ff990101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:26.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.139+0000 7ff99624c700 1 -- 172.21.15.12:0/3036414664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9901083a0 msgr2=0x7ff990101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:26.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.139+0000 7ff99624c700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9901083a0 0x7ff990101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:26.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.139+0000 7ff99624c700 1 -- 172.21.15.12:0/3036414664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99006a9d0 msgr2=0x7ff990101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:26.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.139+0000 7ff99624c700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99006a9d0 0x7ff990101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:26.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.139+0000 7ff99624c700 1 -- 172.21.15.12:0/3036414664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9901a6f80 con 0x7ff99006a030 2024-05-09T09:24:26.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.139+0000 7ff996a4d700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9901083a0 0x7ff990101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:26.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.139+0000 7ff99624c700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff99006a030 0x7ff9901012f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00e9a0 tx=0x7ff98c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:26.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.139+0000 7ff9877fe700 1 -- 172.21.15.12:0/3036414664 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00cbd0 con 0x7ff99006a030 2024-05-09T09:24:26.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.139+0000 7ff9877fe700 1 -- 172.21.15.12:0/3036414664 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff98c00cd70 con 0x7ff99006a030 2024-05-09T09:24:26.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.139+0000 7ff9877fe700 1 -- 172.21.15.12:0/3036414664 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c018680 con 0x7ff99006a030 2024-05-09T09:24:26.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.139+0000 7ff9984b0700 1 -- 172.21.15.12:0/3036414664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9901a7270 con 0x7ff99006a030 2024-05-09T09:24:26.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.139+0000 7ff9984b0700 1 -- 172.21.15.12:0/3036414664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9901a77b0 con 0x7ff99006a030 2024-05-09T09:24:26.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.140+0000 7ff9984b0700 1 -- 172.21.15.12:0/3036414664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff990069010 con 0x7ff99006a030 2024-05-09T09:24:26.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.142+0000 7ff9877fe700 1 -- 172.21.15.12:0/3036414664 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff98c010430 con 0x7ff99006a030 2024-05-09T09:24:26.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.143+0000 7ff9877fe700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff97c0a2cb0 0x7ff97c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:26.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.143+0000 7ff9877fe700 1 -- 172.21.15.12:0/3036414664 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff98c0cfc80 con 0x7ff99006a030 2024-05-09T09:24:26.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.143+0000 7ff995a4b700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff97c0a2cb0 0x7ff97c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:26.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.144+0000 7ff995a4b700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff97c0a2cb0 0x7ff97c0a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9900787d0 tx=0x7ff98000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:26.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.147+0000 7ff9877fe700 1 -- 172.21.15.12:0/3036414664 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff98c09c6c0 con 0x7ff99006a030 2024-05-09T09:24:26.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.358+0000 7ff9984b0700 1 -- 172.21.15.12:0/3036414664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff990102800 con 0x7ff99006a030 2024-05-09T09:24:26.359 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:26 smithi012 ceph-mon[30140]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:24:26.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.359+0000 7ff9877fe700 1 -- 172.21.15.12:0/3036414664 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff990102800 con 0x7ff99006a030 2024-05-09T09:24:26.359 INFO:teuthology.orchestra.run.smithi012.stdout:77309411403 2024-05-09T09:24:26.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.361+0000 7ff9984b0700 1 -- 172.21.15.12:0/3036414664 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff97c0a2cb0 msgr2=0x7ff97c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:26.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.361+0000 7ff9984b0700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff97c0a2cb0 0x7ff97c0a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9900787d0 tx=0x7ff98000b3c0 comp rx=0 tx=0).stop 2024-05-09T09:24:26.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.361+0000 7ff9984b0700 1 -- 172.21.15.12:0/3036414664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff99006a030 msgr2=0x7ff9901012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:26.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.361+0000 7ff9984b0700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff99006a030 0x7ff9901012f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00e9a0 tx=0x7ff98c00ecb0 comp rx=0 tx=0).stop 2024-05-09T09:24:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.361+0000 7ff9984b0700 1 -- 172.21.15.12:0/3036414664 shutdown_connections 2024-05-09T09:24:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.361+0000 7ff9984b0700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff97c0a2cb0 0x7ff97c0a5170 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.361+0000 7ff9984b0700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9901083a0 0x7ff990101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.361+0000 7ff9984b0700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff99006a030 0x7ff9901012f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.361+0000 7ff9984b0700 1 --2- 172.21.15.12:0/3036414664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99006a9d0 0x7ff990101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.361+0000 7ff9984b0700 1 -- 172.21.15.12:0/3036414664 >> 172.21.15.12:0/3036414664 conn(0x7ff9900781d0 msgr2=0x7ff9900ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.361+0000 7ff9984b0700 1 -- 172.21.15.12:0/3036414664 shutdown_connections 2024-05-09T09:24:26.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:26.361+0000 7ff9984b0700 1 -- 172.21.15.12:0/3036414664 wait complete. 2024-05-09T09:24:26.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:26 smithi170 ceph-mon[33632]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:24:26.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:26 smithi016 ceph-mon[33335]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:24:27.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:27 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3036414664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T09:24:27.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:27 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3036414664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T09:24:27.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:27 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3036414664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T09:24:28.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:28 smithi012 ceph-mon[30140]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:24:28.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:28 smithi170 ceph-mon[33632]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:24:28.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:28 smithi016 ceph-mon[33335]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:24:30.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:30 smithi012 ceph-mon[30140]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T09:24:30.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:30 smithi170 ceph-mon[33632]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T09:24:30.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:30 smithi016 ceph-mon[33335]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T09:24:30.670 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645340 got 309237645344 for osd.10 2024-05-09T09:24:30.671 DEBUG:teuthology.parallel:result is None 2024-05-09T09:24:30.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.884+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/2933716778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fd406b260 msgr2=0x7f2fd410f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:30.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.884+0000 7f2fdc3ad700 1 --2- 172.21.15.12:0/2933716778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fd406b260 0x7f2fd410f1d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00b0a0 tx=0x7f2fcc00b3b0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.884+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/2933716778 shutdown_connections 2024-05-09T09:24:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.884+0000 7f2fdc3ad700 1 --2- 172.21.15.12:0/2933716778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fd406b260 0x7f2fd410f1d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.884+0000 7f2fdc3ad700 1 --2- 172.21.15.12:0/2933716778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fd4108390 0x7f2fd4108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.884+0000 7f2fdc3ad700 1 --2- 172.21.15.12:0/2933716778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd406a8c0 0x7f2fd406ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.884+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/2933716778 >> 172.21.15.12:0/2933716778 conn(0x7f2fd4078190 msgr2=0x7f2fd40785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.885+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/2933716778 shutdown_connections 2024-05-09T09:24:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.885+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/2933716778 wait complete. 2024-05-09T09:24:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.885+0000 7f2fdc3ad700 1 Processor -- start 2024-05-09T09:24:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.885+0000 7f2fdc3ad700 1 -- start start 2024-05-09T09:24:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fdc3ad700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fd406a8c0 0x7f2fd419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fdc3ad700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fd406b260 0x7f2fd419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fdc3ad700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd4108390 0x7f2fd41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fdc3ad700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd41125d0 con 0x7f2fd406b260 2024-05-09T09:24:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fdc3ad700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd4112450 con 0x7f2fd4108390 2024-05-09T09:24:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fdc3ad700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd4112750 con 0x7f2fd406a8c0 2024-05-09T09:24:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fd9948700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fd406b260 0x7f2fd419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fd9948700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fd406b260 0x7f2fd419ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39116/0 (socket says 172.21.15.12:39116) 2024-05-09T09:24:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fd9948700 1 -- 172.21.15.12:0/530951339 learned_addr learned my addr 172.21.15.12:0/530951339 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fda149700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fd406a8c0 0x7f2fd419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fda94a700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd4108390 0x7f2fd41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fd9948700 1 -- 172.21.15.12:0/530951339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fd406a8c0 msgr2=0x7f2fd419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fd9948700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fd406a8c0 0x7f2fd419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fd9948700 1 -- 172.21.15.12:0/530951339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd4108390 msgr2=0x7f2fd41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fd9948700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd4108390 0x7f2fd41988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fd9948700 1 -- 172.21.15.12:0/530951339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd4199010 con 0x7f2fd406b260 2024-05-09T09:24:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.886+0000 7f2fd9948700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fd406b260 0x7f2fd419ec30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400d950 tx=0x7f2fc400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.887+0000 7f2fcb7fe700 1 -- 172.21.15.12:0/530951339 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4016070 con 0x7f2fd406b260 2024-05-09T09:24:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.887+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/530951339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd4199300 con 0x7f2fd406b260 2024-05-09T09:24:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.887+0000 7f2fcb7fe700 1 -- 172.21.15.12:0/530951339 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2fc400fd60 con 0x7f2fd406b260 2024-05-09T09:24:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.887+0000 7f2fcb7fe700 1 -- 172.21.15.12:0/530951339 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400e3b0 con 0x7f2fd406b260 2024-05-09T09:24:30.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.887+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/530951339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd41a55c0 con 0x7f2fd406b260 2024-05-09T09:24:30.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.888+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/530951339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd4069010 con 0x7f2fd406b260 2024-05-09T09:24:30.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.889+0000 7f2fcb7fe700 1 -- 172.21.15.12:0/530951339 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2fc4019070 con 0x7f2fd406b260 2024-05-09T09:24:30.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.890+0000 7f2fcb7fe700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2fc00a2cb0 0x7f2fc00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.890+0000 7f2fda149700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2fc00a2cb0 0x7f2fc00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.890+0000 7f2fcb7fe700 1 -- 172.21.15.12:0/530951339 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2fc40cf080 con 0x7f2fd406b260 2024-05-09T09:24:30.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.890+0000 7f2fda149700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2fc00a2cb0 0x7f2fc00a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2fd41999f0 tx=0x7f2fd000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:30.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:30.894+0000 7f2fcb7fe700 1 -- 172.21.15.12:0/530951339 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2fc409bb20 con 0x7f2fd406b260 2024-05-09T09:24:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.128+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/530951339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2fd402db80 con 0x7f2fd406b260 2024-05-09T09:24:31.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.129+0000 7f2fcb7fe700 1 -- 172.21.15.12:0/530951339 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2fc409b420 con 0x7f2fd406b260 2024-05-09T09:24:31.129 INFO:teuthology.orchestra.run.smithi012.stdout:141733920834 2024-05-09T09:24:31.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.131+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/530951339 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2fc00a2cb0 msgr2=0x7f2fc00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:31.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.131+0000 7f2fdc3ad700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2fc00a2cb0 0x7f2fc00a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2fd41999f0 tx=0x7f2fd000a380 comp rx=0 tx=0).stop 2024-05-09T09:24:31.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.131+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/530951339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fd406b260 msgr2=0x7f2fd419ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:31.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.131+0000 7f2fdc3ad700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fd406b260 0x7f2fd419ec30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400d950 tx=0x7f2fc400dc60 comp rx=0 tx=0).stop 2024-05-09T09:24:31.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.131+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/530951339 shutdown_connections 2024-05-09T09:24:31.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.131+0000 7f2fdc3ad700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2fc00a2cb0 0x7f2fc00a5170 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.131+0000 7f2fdc3ad700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd4108390 0x7f2fd41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.131+0000 7f2fdc3ad700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fd406a8c0 0x7f2fd419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.131+0000 7f2fdc3ad700 1 --2- 172.21.15.12:0/530951339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fd406b260 0x7f2fd419ec30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.131+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/530951339 >> 172.21.15.12:0/530951339 conn(0x7f2fd4078190 msgr2=0x7f2fd40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:31.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.131+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/530951339 shutdown_connections 2024-05-09T09:24:31.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.131+0000 7f2fdc3ad700 1 -- 172.21.15.12:0/530951339 wait complete. 2024-05-09T09:24:31.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.606+0000 7f56676f8700 1 -- 172.21.15.12:0/3309195798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660102600 msgr2=0x7f56601029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:31.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.606+0000 7f56676f8700 1 --2- 172.21.15.12:0/3309195798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660102600 0x7f56601029e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f565c009990 tx=0x7f565c009ca0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.606+0000 7f56676f8700 1 -- 172.21.15.12:0/3309195798 shutdown_connections 2024-05-09T09:24:31.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.606+0000 7f56676f8700 1 --2- 172.21.15.12:0/3309195798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56601076d0 0x7f566010f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.606+0000 7f56676f8700 1 --2- 172.21.15.12:0/3309195798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660102600 0x7f56601029e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.606+0000 7f56676f8700 1 --2- 172.21.15.12:0/3309195798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5660102fb0 0x7f5660106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.606+0000 7f56676f8700 1 -- 172.21.15.12:0/3309195798 >> 172.21.15.12:0/3309195798 conn(0x7f56600794d0 msgr2=0x7f56600798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:31.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.606+0000 7f56676f8700 1 -- 172.21.15.12:0/3309195798 shutdown_connections 2024-05-09T09:24:31.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.606+0000 7f56676f8700 1 -- 172.21.15.12:0/3309195798 wait complete. 2024-05-09T09:24:31.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.607+0000 7f56676f8700 1 Processor -- start 2024-05-09T09:24:31.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.607+0000 7f56676f8700 1 -- start start 2024-05-09T09:24:31.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.607+0000 7f56676f8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5660102600 0x7f5660074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:31.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.607+0000 7f56676f8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660102fb0 0x7f5660074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:31.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.607+0000 7f56676f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56601076d0 0x7f56600750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:31.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.607+0000 7f56676f8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56601126c0 con 0x7f5660102fb0 2024-05-09T09:24:31.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.607+0000 7f56676f8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5660112540 con 0x7f56601076d0 2024-05-09T09:24:31.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.607+0000 7f56676f8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5660112840 con 0x7f5660102600 2024-05-09T09:24:31.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.607+0000 7f5665c95700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56601076d0 0x7f56600750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:31.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.607+0000 7f5665c95700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56601076d0 0x7f56600750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41220/0 (socket says 172.21.15.12:41220) 2024-05-09T09:24:31.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.607+0000 7f5665c95700 1 -- 172.21.15.12:0/1809430098 learned_addr learned my addr 172.21.15.12:0/1809430098 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:31.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.608+0000 7f5665494700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5660102600 0x7f5660074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:31.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.608+0000 7f5665c95700 1 -- 172.21.15.12:0/1809430098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5660102600 msgr2=0x7f5660074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:31.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.608+0000 7f5664c93700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660102fb0 0x7f5660074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:31.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.608+0000 7f5665c95700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5660102600 0x7f5660074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.608+0000 7f5665c95700 1 -- 172.21.15.12:0/1809430098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660102fb0 msgr2=0x7f5660074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:31.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.608+0000 7f5665c95700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660102fb0 0x7f5660074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.608+0000 7f5665c95700 1 -- 172.21.15.12:0/1809430098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56601a70e0 con 0x7f56601076d0 2024-05-09T09:24:31.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.608+0000 7f5664c93700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660102fb0 0x7f5660074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:31.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.609+0000 7f5665c95700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56601076d0 0x7f56600750a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f565800ef30 tx=0x7f565800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.609+0000 7f56567fc700 1 -- 172.21.15.12:0/1809430098 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56580092e0 con 0x7f56601076d0 2024-05-09T09:24:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.609+0000 7f56567fc700 1 -- 172.21.15.12:0/1809430098 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5658004bb0 con 0x7f56601076d0 2024-05-09T09:24:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.609+0000 7f56676f8700 1 -- 172.21.15.12:0/1809430098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56601a7370 con 0x7f56601076d0 2024-05-09T09:24:31.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.609+0000 7f56567fc700 1 -- 172.21.15.12:0/1809430098 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658004280 con 0x7f56601076d0 2024-05-09T09:24:31.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.609+0000 7f56676f8700 1 -- 172.21.15.12:0/1809430098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56601a78b0 con 0x7f56601076d0 2024-05-09T09:24:31.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.610+0000 7f56676f8700 1 -- 172.21.15.12:0/1809430098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5660069010 con 0x7f56601076d0 2024-05-09T09:24:31.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.612+0000 7f56567fc700 1 -- 172.21.15.12:0/1809430098 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f565801b030 con 0x7f56601076d0 2024-05-09T09:24:31.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.613+0000 7f56567fc700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f564c0a2d30 0x7f564c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:31.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.613+0000 7f56567fc700 1 -- 172.21.15.12:0/1809430098 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f56580cefa0 con 0x7f56601076d0 2024-05-09T09:24:31.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.613+0000 7f5665494700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f564c0a2d30 0x7f564c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:31.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.614+0000 7f5665494700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f564c0a2d30 0x7f564c0a51f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f565c0098e0 tx=0x7f565c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:31.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.618+0000 7f56567fc700 1 -- 172.21.15.12:0/1809430098 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5658022020 con 0x7f56601076d0 2024-05-09T09:24:31.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.831+0000 7f56676f8700 1 -- 172.21.15.12:0/1809430098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f566002db80 con 0x7f56601076d0 2024-05-09T09:24:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.832+0000 7f56567fc700 1 -- 172.21.15.12:0/1809430098 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5658010040 con 0x7f56601076d0 2024-05-09T09:24:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:253403070509 2024-05-09T09:24:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.834+0000 7f56676f8700 1 -- 172.21.15.12:0/1809430098 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f564c0a2d30 msgr2=0x7f564c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.834+0000 7f56676f8700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f564c0a2d30 0x7f564c0a51f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f565c0098e0 tx=0x7f565c005960 comp rx=0 tx=0).stop 2024-05-09T09:24:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.834+0000 7f56676f8700 1 -- 172.21.15.12:0/1809430098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56601076d0 msgr2=0x7f56600750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.834+0000 7f56676f8700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56601076d0 0x7f56600750a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f565800ef30 tx=0x7f565800c3a0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.834+0000 7f56676f8700 1 -- 172.21.15.12:0/1809430098 shutdown_connections 2024-05-09T09:24:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.834+0000 7f56676f8700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f564c0a2d30 0x7f564c0a51f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.834+0000 7f56676f8700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56601076d0 0x7f56600750a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.834+0000 7f56676f8700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5660102600 0x7f5660074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.834+0000 7f56676f8700 1 --2- 172.21.15.12:0/1809430098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660102fb0 0x7f5660074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.834+0000 7f56676f8700 1 -- 172.21.15.12:0/1809430098 >> 172.21.15.12:0/1809430098 conn(0x7f56600794d0 msgr2=0x7f5660106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.834+0000 7f56676f8700 1 -- 172.21.15.12:0/1809430098 shutdown_connections 2024-05-09T09:24:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:31.834+0000 7f56676f8700 1 -- 172.21.15.12:0/1809430098 wait complete. 2024-05-09T09:24:32.082 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:32 smithi012 ceph-mon[30140]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-09T09:24:32.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:32 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/530951339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T09:24:32.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:32 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1809430098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T09:24:32.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:32 smithi170 ceph-mon[33632]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-09T09:24:32.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:32 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/530951339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T09:24:32.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:32 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1809430098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T09:24:32.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:32 smithi016 ceph-mon[33335]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-09T09:24:32.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:32 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/530951339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T09:24:32.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:32 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1809430098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T09:24:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.278+0000 7f90d0815700 1 -- 172.21.15.12:0/3466389165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90cc102d40 msgr2=0x7f90cc10f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.278+0000 7f90d0815700 1 --2- 172.21.15.12:0/3466389165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90cc102d40 0x7f90cc10f220 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f90c400b0a0 tx=0x7f90c400b3b0 comp rx=0 tx=0).stop 2024-05-09T09:24:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.278+0000 7f90d0815700 1 -- 172.21.15.12:0/3466389165 shutdown_connections 2024-05-09T09:24:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.278+0000 7f90d0815700 1 --2- 172.21.15.12:0/3466389165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90cc102d40 0x7f90cc10f220 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.278+0000 7f90d0815700 1 --2- 172.21.15.12:0/3466389165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1083e0 0x7f90cc1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.278+0000 7f90d0815700 1 --2- 172.21.15.12:0/3466389165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc1023a0 0x7f90cc102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.278+0000 7f90d0815700 1 -- 172.21.15.12:0/3466389165 >> 172.21.15.12:0/3466389165 conn(0x7f90cc0794d0 msgr2=0x7f90cc0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.278+0000 7f90d0815700 1 -- 172.21.15.12:0/3466389165 shutdown_connections 2024-05-09T09:24:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.278+0000 7f90d0815700 1 -- 172.21.15.12:0/3466389165 wait complete. 2024-05-09T09:24:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90d0815700 1 Processor -- start 2024-05-09T09:24:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90d0815700 1 -- start start 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90d0815700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1023a0 0x7f90cc19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90d0815700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90cc102d40 0x7f90cc19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90d0815700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc1083e0 0x7f90cc198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90d0815700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc112620 con 0x7f90cc102d40 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90d0815700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1124a0 con 0x7f90cc1023a0 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90d0815700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1127a0 con 0x7f90cc1083e0 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90c9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90cc102d40 0x7f90cc19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90c9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90cc102d40 0x7f90cc19ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34704/0 (socket says 172.21.15.12:34704) 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90c9d9b700 1 -- 172.21.15.12:0/3878212299 learned_addr learned my addr 172.21.15.12:0/3878212299 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90cad9d700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc1083e0 0x7f90cc198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90ca59c700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1023a0 0x7f90cc19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90c9d9b700 1 -- 172.21.15.12:0/3878212299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc1083e0 msgr2=0x7f90cc198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90c9d9b700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc1083e0 0x7f90cc198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90c9d9b700 1 -- 172.21.15.12:0/3878212299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1023a0 msgr2=0x7f90cc19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:33.284 INFO:teuthology.orchestra.run.smithi012.stdout:163208757308 2024-05-09T09:24:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90c9d9b700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1023a0 0x7f90cc19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90c9d9b700 1 -- 172.21.15.12:0/3878212299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc199060 con 0x7f90cc102d40 2024-05-09T09:24:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.279+0000 7f90ca59c700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1023a0 0x7f90cc19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.280+0000 7f90c9d9b700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90cc102d40 0x7f90cc19ec30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f90c000e9a0 tx=0x7f90c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.280+0000 7f90bb7fe700 1 -- 172.21.15.12:0/3878212299 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c000cbd0 con 0x7f90cc102d40 2024-05-09T09:24:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.280+0000 7f90bb7fe700 1 -- 172.21.15.12:0/3878212299 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f90c000cd70 con 0x7f90cc102d40 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.280+0000 7f90d0815700 1 -- 172.21.15.12:0/3878212299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90cc199350 con 0x7f90cc102d40 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.280+0000 7f90bb7fe700 1 -- 172.21.15.12:0/3878212299 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0018680 con 0x7f90cc102d40 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.280+0000 7f90d0815700 1 -- 172.21.15.12:0/3878212299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90cc1a55c0 con 0x7f90cc102d40 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.281+0000 7f90d0815700 1 -- 172.21.15.12:0/3878212299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90cc069010 con 0x7f90cc102d40 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.282+0000 7f90bb7fe700 1 -- 172.21.15.12:0/3878212299 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f90c0010430 con 0x7f90cc102d40 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.283+0000 7f90bb7fe700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f90b40a2cb0 0x7f90b40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.283+0000 7f90ca59c700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f90b40a2cb0 0x7f90b40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.283+0000 7f90bb7fe700 1 -- 172.21.15.12:0/3878212299 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f90c00cf8c0 con 0x7f90cc102d40 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.283+0000 7f90ca59c700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f90b40a2cb0 0x7f90b40a5170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f90bc005e10 tx=0x7f90bc005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.287+0000 7f90bb7fe700 1 -- 172.21.15.12:0/3878212299 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f90c009c300 con 0x7f90cc102d40 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.494+0000 7f90d0815700 1 -- 172.21.15.12:0/3878212299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f90cc02db80 con 0x7f90cc102d40 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.494+0000 7f90bb7fe700 1 -- 172.21.15.12:0/3878212299 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f90c009bc00 con 0x7f90cc102d40 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.497+0000 7f90d0815700 1 -- 172.21.15.12:0/3878212299 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f90b40a2cb0 msgr2=0x7f90b40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.497+0000 7f90d0815700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f90b40a2cb0 0x7f90b40a5170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f90bc005e10 tx=0x7f90bc005d60 comp rx=0 tx=0).stop 2024-05-09T09:24:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.497+0000 7f90d0815700 1 -- 172.21.15.12:0/3878212299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90cc102d40 msgr2=0x7f90cc19ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.497+0000 7f90d0815700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90cc102d40 0x7f90cc19ec30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f90c000e9a0 tx=0x7f90c000ecb0 comp rx=0 tx=0).stop 2024-05-09T09:24:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.497+0000 7f90d0815700 1 -- 172.21.15.12:0/3878212299 shutdown_connections 2024-05-09T09:24:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.497+0000 7f90d0815700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f90b40a2cb0 0x7f90b40a5170 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.497+0000 7f90d0815700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc1083e0 0x7f90cc198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.497+0000 7f90d0815700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1023a0 0x7f90cc19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.497+0000 7f90d0815700 1 --2- 172.21.15.12:0/3878212299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90cc102d40 0x7f90cc19ec30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.497+0000 7f90d0815700 1 -- 172.21.15.12:0/3878212299 >> 172.21.15.12:0/3878212299 conn(0x7f90cc0794d0 msgr2=0x7f90cc100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.497+0000 7f90d0815700 1 -- 172.21.15.12:0/3878212299 shutdown_connections 2024-05-09T09:24:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:32.497+0000 7f90d0815700 1 -- 172.21.15.12:0/3878212299 wait complete. 2024-05-09T09:24:33.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:33 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3878212299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T09:24:33.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:33 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3878212299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T09:24:33.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:33 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3878212299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T09:24:33.982 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411398 got 77309411403 for osd.2 2024-05-09T09:24:33.982 DEBUG:teuthology.parallel:result is None 2024-05-09T09:24:34.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:34 smithi012 ceph-mon[30140]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:24:34.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:34 smithi170 ceph-mon[33632]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:24:34.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:34 smithi016 ceph-mon[33335]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:24:34.612 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070502 got 253403070509 for osd.8 2024-05-09T09:24:34.612 DEBUG:teuthology.parallel:result is None 2024-05-09T09:24:35.006 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920827 got 141733920834 for osd.4 2024-05-09T09:24:35.007 DEBUG:teuthology.parallel:result is None 2024-05-09T09:24:35.966 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757302 got 163208757308 for osd.5 2024-05-09T09:24:35.966 DEBUG:teuthology.parallel:result is None 2024-05-09T09:24:36.192 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:36 smithi012 ceph-mon[30140]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:24:36.193 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:36 smithi012 ceph-mon[30140]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-09T09:24:36.193 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:36 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/314862804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T09:24:36.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.641+0000 7f9f026d6700 1 -- 172.21.15.12:0/1503120987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9efc107290 msgr2=0x7f9efc107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:36.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.641+0000 7f9f026d6700 1 --2- 172.21.15.12:0/1503120987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9efc107290 0x7f9efc107670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009990 tx=0x7f9eec009ca0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.641+0000 7f9f026d6700 1 -- 172.21.15.12:0/1503120987 shutdown_connections 2024-05-09T09:24:36.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.641+0000 7f9f026d6700 1 --2- 172.21.15.12:0/1503120987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc101be0 0x7f9efc10e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.641+0000 7f9f026d6700 1 --2- 172.21.15.12:0/1503120987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9efc107290 0x7f9efc107670 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.641+0000 7f9f026d6700 1 --2- 172.21.15.12:0/1503120987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc101220 0x7f9efc1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.641+0000 7f9f026d6700 1 -- 172.21.15.12:0/1503120987 >> 172.21.15.12:0/1503120987 conn(0x7f9efc0781d0 msgr2=0x7f9efc0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:36.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.641+0000 7f9f026d6700 1 -- 172.21.15.12:0/1503120987 shutdown_connections 2024-05-09T09:24:36.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.642+0000 7f9f026d6700 1 -- 172.21.15.12:0/1503120987 wait complete. 2024-05-09T09:24:36.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.642+0000 7f9f026d6700 1 Processor -- start 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.642+0000 7f9f026d6700 1 -- start start 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9f026d6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9efc101220 0x7f9efc102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9f026d6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc101be0 0x7f9efc103440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9f026d6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc107290 0x7f9efc103980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9f026d6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc111510 con 0x7f9efc101220 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9f026d6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc111390 con 0x7f9efc107290 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9f026d6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc111690 con 0x7f9efc101be0 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9efbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9efc101220 0x7f9efc102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9efbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9efc101220 0x7f9efc102f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34724/0 (socket says 172.21.15.12:34724) 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9efbfff700 1 -- 172.21.15.12:0/314862804 learned_addr learned my addr 172.21.15.12:0/314862804 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9efb7fe700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc101be0 0x7f9efc103440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9f00c73700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc107290 0x7f9efc103980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9efbfff700 1 -- 172.21.15.12:0/314862804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc101be0 msgr2=0x7f9efc103440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:36.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9efbfff700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc101be0 0x7f9efc103440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9efbfff700 1 -- 172.21.15.12:0/314862804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc107290 msgr2=0x7f9efc103980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:36.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9efbfff700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc107290 0x7f9efc103980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9efbfff700 1 -- 172.21.15.12:0/314862804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9efc1a6fd0 con 0x7f9efc101220 2024-05-09T09:24:36.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9efbfff700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9efc101220 0x7f9efc102f00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0038b0 tx=0x7f9eec009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:36.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.643+0000 7f9ef97fa700 1 -- 172.21.15.12:0/314862804 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec027070 con 0x7f9efc101220 2024-05-09T09:24:36.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.644+0000 7f9ef97fa700 1 -- 172.21.15.12:0/314862804 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9eec004040 con 0x7f9efc101220 2024-05-09T09:24:36.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.644+0000 7f9ef97fa700 1 -- 172.21.15.12:0/314862804 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec022c80 con 0x7f9efc101220 2024-05-09T09:24:36.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.644+0000 7f9f026d6700 1 -- 172.21.15.12:0/314862804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9efc1a7260 con 0x7f9efc101220 2024-05-09T09:24:36.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.644+0000 7f9f026d6700 1 -- 172.21.15.12:0/314862804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9efc1a7740 con 0x7f9efc101220 2024-05-09T09:24:36.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.644+0000 7f9f026d6700 1 -- 172.21.15.12:0/314862804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9efc069010 con 0x7f9efc101220 2024-05-09T09:24:36.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.646+0000 7f9ef97fa700 1 -- 172.21.15.12:0/314862804 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9eec02a070 con 0x7f9efc101220 2024-05-09T09:24:36.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.647+0000 7f9ef97fa700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9ee80a2c20 0x7f9ee80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:36.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.647+0000 7f9efb7fe700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9ee80a2c20 0x7f9ee80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:36.269 INFO:teuthology.orchestra.run.smithi012.stdout:197568495672 2024-05-09T09:24:36.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.647+0000 7f9ef97fa700 1 -- 172.21.15.12:0/314862804 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9eec0d0580 con 0x7f9efc101220 2024-05-09T09:24:36.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.647+0000 7f9efb7fe700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9ee80a2c20 0x7f9ee80a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0ffa00 tx=0x7f9ee400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:36.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.650+0000 7f9ef97fa700 1 -- 172.21.15.12:0/314862804 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9eec022e20 con 0x7f9efc101220 2024-05-09T09:24:36.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.857+0000 7f9f026d6700 1 -- 172.21.15.12:0/314862804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9efc1a79a0 con 0x7f9efc101220 2024-05-09T09:24:36.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.858+0000 7f9ef97fa700 1 -- 172.21.15.12:0/314862804 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9eec013070 con 0x7f9efc101220 2024-05-09T09:24:36.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.860+0000 7f9f026d6700 1 -- 172.21.15.12:0/314862804 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9ee80a2c20 msgr2=0x7f9ee80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:36.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.860+0000 7f9f026d6700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9ee80a2c20 0x7f9ee80a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0ffa00 tx=0x7f9ee400b3c0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.860+0000 7f9f026d6700 1 -- 172.21.15.12:0/314862804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9efc101220 msgr2=0x7f9efc102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:36.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.860+0000 7f9f026d6700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9efc101220 0x7f9efc102f00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0038b0 tx=0x7f9eec009ee0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.860+0000 7f9f026d6700 1 -- 172.21.15.12:0/314862804 shutdown_connections 2024-05-09T09:24:36.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.860+0000 7f9f026d6700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9ee80a2c20 0x7f9ee80a50e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.860+0000 7f9f026d6700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc107290 0x7f9efc103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.860+0000 7f9f026d6700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9efc101220 0x7f9efc102f00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.860+0000 7f9f026d6700 1 --2- 172.21.15.12:0/314862804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc101be0 0x7f9efc103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.860+0000 7f9f026d6700 1 -- 172.21.15.12:0/314862804 >> 172.21.15.12:0/314862804 conn(0x7f9efc0781d0 msgr2=0x7f9efc100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:36.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.860+0000 7f9f026d6700 1 -- 172.21.15.12:0/314862804 shutdown_connections 2024-05-09T09:24:36.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:35.860+0000 7f9f026d6700 1 -- 172.21.15.12:0/314862804 wait complete. 2024-05-09T09:24:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:36 smithi170 ceph-mon[33632]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:24:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:36 smithi170 ceph-mon[33632]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-09T09:24:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:36 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/314862804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T09:24:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:36 smithi016 ceph-mon[33335]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:24:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:36 smithi016 ceph-mon[33335]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-09T09:24:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:36 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/314862804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T09:24:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.661+0000 7ff28f179700 1 -- 172.21.15.12:0/4159239709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff288101220 msgr2=0x7ff2881016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.661+0000 7ff28f179700 1 --2- 172.21.15.12:0/4159239709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff288101220 0x7ff2881016a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff278009990 tx=0x7ff278009ca0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.661+0000 7ff28f179700 1 -- 172.21.15.12:0/4159239709 shutdown_connections 2024-05-09T09:24:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.661+0000 7ff28f179700 1 --2- 172.21.15.12:0/4159239709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff288101be0 0x7ff28810e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.661+0000 7ff28f179700 1 --2- 172.21.15.12:0/4159239709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288107290 0x7ff288107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.661+0000 7ff28f179700 1 --2- 172.21.15.12:0/4159239709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff288101220 0x7ff2881016a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.661+0000 7ff28f179700 1 -- 172.21.15.12:0/4159239709 >> 172.21.15.12:0/4159239709 conn(0x7ff2880781d0 msgr2=0x7ff2880785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:36.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.661+0000 7ff28f179700 1 -- 172.21.15.12:0/4159239709 shutdown_connections 2024-05-09T09:24:36.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.661+0000 7ff28f179700 1 -- 172.21.15.12:0/4159239709 wait complete. 2024-05-09T09:24:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.662+0000 7ff28f179700 1 Processor -- start 2024-05-09T09:24:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.662+0000 7ff28f179700 1 -- start start 2024-05-09T09:24:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.662+0000 7ff28f179700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff288101220 0x7ff28819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.662+0000 7ff28f179700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff288101be0 0x7ff28819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.662+0000 7ff28f179700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288107290 0x7ff288197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.662+0000 7ff28f179700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff288111570 con 0x7ff288101be0 2024-05-09T09:24:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.662+0000 7ff28f179700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881113f0 con 0x7ff288107290 2024-05-09T09:24:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.662+0000 7ff28f179700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881116f0 con 0x7ff288101220 2024-05-09T09:24:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff27ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff288101be0 0x7ff28819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff27ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff288101be0 0x7ff28819dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34754/0 (socket says 172.21.15.12:34754) 2024-05-09T09:24:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff27ffff700 1 -- 172.21.15.12:0/822271392 learned_addr learned my addr 172.21.15.12:0/822271392 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff28d716700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288107290 0x7ff288197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:36.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff28cf15700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff288101220 0x7ff28819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:36.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff27ffff700 1 -- 172.21.15.12:0/822271392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff288101220 msgr2=0x7ff28819d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:36.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff27ffff700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff288101220 0x7ff28819d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff27ffff700 1 -- 172.21.15.12:0/822271392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288107290 msgr2=0x7ff288197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:36.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff27ffff700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288107290 0x7ff288197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff27ffff700 1 -- 172.21.15.12:0/822271392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff288198030 con 0x7ff288101be0 2024-05-09T09:24:36.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff28d716700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288107290 0x7ff288197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:36.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff27ffff700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff288101be0 0x7ff28819dc10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff27800bff0 tx=0x7ff278005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:36.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff27dffb700 1 -- 172.21.15.12:0/822271392 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278027070 con 0x7ff288101be0 2024-05-09T09:24:36.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff28f179700 1 -- 172.21.15.12:0/822271392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2881982c0 con 0x7ff288101be0 2024-05-09T09:24:36.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff27dffb700 1 -- 172.21.15.12:0/822271392 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff278004140 con 0x7ff288101be0 2024-05-09T09:24:36.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.663+0000 7ff27dffb700 1 -- 172.21.15.12:0/822271392 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27800f460 con 0x7ff288101be0 2024-05-09T09:24:36.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.664+0000 7ff28f179700 1 -- 172.21.15.12:0/822271392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28806a000 con 0x7ff288101be0 2024-05-09T09:24:36.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.664+0000 7ff28f179700 1 -- 172.21.15.12:0/822271392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff288102940 con 0x7ff288101be0 2024-05-09T09:24:36.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.666+0000 7ff27dffb700 1 -- 172.21.15.12:0/822271392 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff27802a070 con 0x7ff288101be0 2024-05-09T09:24:36.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.666+0000 7ff27dffb700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff2700a2c20 0x7ff2700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:36.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.667+0000 7ff28cf15700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff2700a2c20 0x7ff2700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:36.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.667+0000 7ff27dffb700 1 -- 172.21.15.12:0/822271392 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff2780d05a0 con 0x7ff288101be0 2024-05-09T09:24:36.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.667+0000 7ff28cf15700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff2700a2c20 0x7ff2700a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff284006fd0 tx=0x7ff284008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:36.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.670+0000 7ff27dffb700 1 -- 172.21.15.12:0/822271392 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff278013070 con 0x7ff288101be0 2024-05-09T09:24:36.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.878+0000 7ff28f179700 1 -- 172.21.15.12:0/822271392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff288198d80 con 0x7ff288101be0 2024-05-09T09:24:36.878 INFO:teuthology.orchestra.run.smithi012.stdout:219043332146 2024-05-09T09:24:36.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.878+0000 7ff27dffb700 1 -- 172.21.15.12:0/822271392 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff278022470 con 0x7ff288101be0 2024-05-09T09:24:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.881+0000 7ff28f179700 1 -- 172.21.15.12:0/822271392 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff2700a2c20 msgr2=0x7ff2700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.881+0000 7ff28f179700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff2700a2c20 0x7ff2700a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff284006fd0 tx=0x7ff284008040 comp rx=0 tx=0).stop 2024-05-09T09:24:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.881+0000 7ff28f179700 1 -- 172.21.15.12:0/822271392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff288101be0 msgr2=0x7ff28819dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.881+0000 7ff28f179700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff288101be0 0x7ff28819dc10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff27800bff0 tx=0x7ff278005640 comp rx=0 tx=0).stop 2024-05-09T09:24:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.881+0000 7ff28f179700 1 -- 172.21.15.12:0/822271392 shutdown_connections 2024-05-09T09:24:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.881+0000 7ff28f179700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff2700a2c20 0x7ff2700a50e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.881+0000 7ff28f179700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff288107290 0x7ff288197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.881+0000 7ff28f179700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff288101220 0x7ff28819d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.881+0000 7ff28f179700 1 --2- 172.21.15.12:0/822271392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff288101be0 0x7ff28819dc10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.881+0000 7ff28f179700 1 -- 172.21.15.12:0/822271392 >> 172.21.15.12:0/822271392 conn(0x7ff2880781d0 msgr2=0x7ff2880ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.881+0000 7ff28f179700 1 -- 172.21.15.12:0/822271392 shutdown_connections 2024-05-09T09:24:36.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:36.881+0000 7ff28f179700 1 -- 172.21.15.12:0/822271392 wait complete. 2024-05-09T09:24:37.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:37 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/822271392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T09:24:37.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:37 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/822271392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T09:24:37.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:37 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/822271392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T09:24:38.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.070+0000 7f4cc9113700 1 -- 172.21.15.12:0/2172619627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc4101be0 msgr2=0x7f4cc410e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:38.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.070+0000 7f4cc9113700 1 --2- 172.21.15.12:0/2172619627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc4101be0 0x7f4cc410e0d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009990 tx=0x7f4cbc009ca0 comp rx=0 tx=0).stop 2024-05-09T09:24:38.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.070+0000 7f4cc9113700 1 -- 172.21.15.12:0/2172619627 shutdown_connections 2024-05-09T09:24:38.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.070+0000 7f4cc9113700 1 --2- 172.21.15.12:0/2172619627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc4101be0 0x7f4cc410e0d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:38.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.070+0000 7f4cc9113700 1 --2- 172.21.15.12:0/2172619627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cc4107290 0x7f4cc4107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:38.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.070+0000 7f4cc9113700 1 --2- 172.21.15.12:0/2172619627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc4101220 0x7f4cc41016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:38.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.070+0000 7f4cc9113700 1 -- 172.21.15.12:0/2172619627 >> 172.21.15.12:0/2172619627 conn(0x7f4cc40781d0 msgr2=0x7f4cc40785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:38.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.070+0000 7f4cc9113700 1 -- 172.21.15.12:0/2172619627 shutdown_connections 2024-05-09T09:24:38.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.070+0000 7f4cc9113700 1 -- 172.21.15.12:0/2172619627 wait complete. 2024-05-09T09:24:38.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.071+0000 7f4cc9113700 1 Processor -- start 2024-05-09T09:24:38.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.071+0000 7f4cc9113700 1 -- start start 2024-05-09T09:24:38.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.071+0000 7f4cc9113700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc4101220 0x7f4cc419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:38.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.071+0000 7f4cc9113700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cc4101be0 0x7f4cc419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:38.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc9113700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc4107290 0x7f4cc4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:38.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc9113700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4111590 con 0x7f4cc4101220 2024-05-09T09:24:38.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc9113700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4111410 con 0x7f4cc4107290 2024-05-09T09:24:38.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc9113700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4111710 con 0x7f4cc4101be0 2024-05-09T09:24:38.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc4101220 0x7f4cc419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:38.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc4101220 0x7f4cc419e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34772/0 (socket says 172.21.15.12:34772) 2024-05-09T09:24:38.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc2d9d700 1 -- 172.21.15.12:0/2098844561 learned_addr learned my addr 172.21.15.12:0/2098844561 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:38.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc259c700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cc4101be0 0x7f4cc419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:38.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc359e700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc4107290 0x7f4cc4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:38.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc2d9d700 1 -- 172.21.15.12:0/2098844561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cc4101be0 msgr2=0x7f4cc419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc2d9d700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cc4101be0 0x7f4cc419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc2d9d700 1 -- 172.21.15.12:0/2098844561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc4107290 msgr2=0x7f4cc4198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc2d9d700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc4107290 0x7f4cc4198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc2d9d700 1 -- 172.21.15.12:0/2098844561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc4199060 con 0x7f4cc4101220 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc2d9d700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc4101220 0x7f4cc419e740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4007f10 tx=0x7f4cb400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cabfff700 1 -- 172.21.15.12:0/2098844561 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400dbd0 con 0x7f4cc4101220 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc9113700 1 -- 172.21.15.12:0/2098844561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc4199350 con 0x7f4cc4101220 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cabfff700 1 -- 172.21.15.12:0/2098844561 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4cb400dd70 con 0x7f4cc4101220 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cabfff700 1 -- 172.21.15.12:0/2098844561 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb40133f0 con 0x7f4cc4101220 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.072+0000 7f4cc9113700 1 -- 172.21.15.12:0/2098844561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc41a5610 con 0x7f4cc4101220 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.074+0000 7f4cc9113700 1 -- 172.21.15.12:0/2098844561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc4069010 con 0x7f4cc4101220 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.075+0000 7f4cabfff700 1 -- 172.21.15.12:0/2098844561 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4cb401a020 con 0x7f4cc4101220 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.076+0000 7f4cabfff700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4cac0a2c20 0x7f4cac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:38.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.076+0000 7f4cc259c700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4cac0a2c20 0x7f4cac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:38.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.076+0000 7f4cabfff700 1 -- 172.21.15.12:0/2098844561 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4cb40cebd0 con 0x7f4cc4101220 2024-05-09T09:24:38.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.077+0000 7f4cc259c700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4cac0a2c20 0x7f4cac0a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4cc41060b0 tx=0x7f4cb8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:38.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.083+0000 7f4cabfff700 1 -- 172.21.15.12:0/2098844561 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4cb4010070 con 0x7f4cc4101220 2024-05-09T09:24:38.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.298+0000 7f4cc9113700 1 -- 172.21.15.12:0/2098844561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f4cc41a57e0 con 0x7f4cc4101220 2024-05-09T09:24:38.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.299+0000 7f4cabfff700 1 -- 172.21.15.12:0/2098844561 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f4cb409b7d0 con 0x7f4cc4101220 2024-05-09T09:24:38.298 INFO:teuthology.orchestra.run.smithi012.stdout:343597383711 2024-05-09T09:24:38.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.301+0000 7f4cc9113700 1 -- 172.21.15.12:0/2098844561 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4cac0a2c20 msgr2=0x7f4cac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:38.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.301+0000 7f4cc9113700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4cac0a2c20 0x7f4cac0a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4cc41060b0 tx=0x7f4cb8009190 comp rx=0 tx=0).stop 2024-05-09T09:24:38.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.301+0000 7f4cc9113700 1 -- 172.21.15.12:0/2098844561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc4101220 msgr2=0x7f4cc419e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:38.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.301+0000 7f4cc9113700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc4101220 0x7f4cc419e740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4007f10 tx=0x7f4cb400d3b0 comp rx=0 tx=0).stop 2024-05-09T09:24:38.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.301+0000 7f4cc9113700 1 -- 172.21.15.12:0/2098844561 shutdown_connections 2024-05-09T09:24:38.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.301+0000 7f4cc9113700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4cac0a2c20 0x7f4cac0a50e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:38.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.301+0000 7f4cc9113700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc4107290 0x7f4cc4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:38.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.301+0000 7f4cc9113700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc4101220 0x7f4cc419e740 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:38.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.301+0000 7f4cc9113700 1 --2- 172.21.15.12:0/2098844561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cc4101be0 0x7f4cc419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:38.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.301+0000 7f4cc9113700 1 -- 172.21.15.12:0/2098844561 >> 172.21.15.12:0/2098844561 conn(0x7f4cc40781d0 msgr2=0x7f4cc4100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:38.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.301+0000 7f4cc9113700 1 -- 172.21.15.12:0/2098844561 shutdown_connections 2024-05-09T09:24:38.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:38.301+0000 7f4cc9113700 1 -- 172.21.15.12:0/2098844561 wait complete. 2024-05-09T09:24:38.330 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:38 smithi012 ceph-mon[30140]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T09:24:38.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:38 smithi170 ceph-mon[33632]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T09:24:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:38 smithi016 ceph-mon[33335]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T09:24:38.582 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495667 got 197568495672 for osd.6 2024-05-09T09:24:38.583 DEBUG:teuthology.parallel:result is None 2024-05-09T09:24:39.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:39 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2098844561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T09:24:39.253 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332142 got 219043332146 for osd.7 2024-05-09T09:24:39.253 DEBUG:teuthology.parallel:result is None 2024-05-09T09:24:39.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:39 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2098844561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T09:24:39.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:39 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2098844561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T09:24:40.021 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383706 got 343597383711 for osd.11 2024-05-09T09:24:40.021 DEBUG:teuthology.parallel:result is None 2024-05-09T09:24:40.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:40 smithi012 ceph-mon[30140]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T09:24:40.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.468+0000 7f8cb8b29700 1 -- 172.21.15.12:0/3260579369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cb4102fb0 msgr2=0x7f8cb4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:40.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.468+0000 7f8cb8b29700 1 --2- 172.21.15.12:0/3260579369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cb4102fb0 0x7f8cb4106ff0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0099c0 tx=0x7f8c9c009cd0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.468+0000 7f8cb8b29700 1 -- 172.21.15.12:0/3260579369 shutdown_connections 2024-05-09T09:24:40.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.468+0000 7f8cb8b29700 1 --2- 172.21.15.12:0/3260579369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb41076d0 0x7f8cb410f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.468+0000 7f8cb8b29700 1 --2- 172.21.15.12:0/3260579369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cb4102600 0x7f8cb41029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.468+0000 7f8cb8b29700 1 --2- 172.21.15.12:0/3260579369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cb4102fb0 0x7f8cb4106ff0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.468+0000 7f8cb8b29700 1 -- 172.21.15.12:0/3260579369 >> 172.21.15.12:0/3260579369 conn(0x7f8cb40794d0 msgr2=0x7f8cb40798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:40.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.468+0000 7f8cb8b29700 1 -- 172.21.15.12:0/3260579369 shutdown_connections 2024-05-09T09:24:40.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.468+0000 7f8cb8b29700 1 -- 172.21.15.12:0/3260579369 wait complete. 2024-05-09T09:24:40.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.469+0000 7f8cb8b29700 1 Processor -- start 2024-05-09T09:24:40.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.469+0000 7f8cb8b29700 1 -- start start 2024-05-09T09:24:40.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.469+0000 7f8cb8b29700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb4102600 0x7f8cb419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:40.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.469+0000 7f8cb8b29700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cb4102fb0 0x7f8cb419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:40.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.469+0000 7f8cb8b29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cb41076d0 0x7f8cb4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:40.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.469+0000 7f8cb8b29700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb41126a0 con 0x7f8cb41076d0 2024-05-09T09:24:40.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.469+0000 7f8cb8b29700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb4112520 con 0x7f8cb4102600 2024-05-09T09:24:40.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.469+0000 7f8cb8b29700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb4112820 con 0x7f8cb4102fb0 2024-05-09T09:24:40.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cb2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cb41076d0 0x7f8cb4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:40.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cb2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cb41076d0 0x7f8cb4198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34788/0 (socket says 172.21.15.12:34788) 2024-05-09T09:24:40.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cb2d9d700 1 -- 172.21.15.12:0/310338211 learned_addr learned my addr 172.21.15.12:0/310338211 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:40.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cb1d9b700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cb4102fb0 0x7f8cb419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:40.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cb259c700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb4102600 0x7f8cb419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:40.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cb2d9d700 1 -- 172.21.15.12:0/310338211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cb4102fb0 msgr2=0x7f8cb419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:40.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cb2d9d700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cb4102fb0 0x7f8cb419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cb2d9d700 1 -- 172.21.15.12:0/310338211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb4102600 msgr2=0x7f8cb419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:40.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cb2d9d700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb4102600 0x7f8cb419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cb2d9d700 1 -- 172.21.15.12:0/310338211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb41991a0 con 0x7f8cb41076d0 2024-05-09T09:24:40.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cb2d9d700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cb41076d0 0x7f8cb4198a40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00c970 tx=0x7f8cac00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:40.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cab7fe700 1 -- 172.21.15.12:0/310338211 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00ce60 con 0x7f8cb41076d0 2024-05-09T09:24:40.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cab7fe700 1 -- 172.21.15.12:0/310338211 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8cac004830 con 0x7f8cb41076d0 2024-05-09T09:24:40.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cb8b29700 1 -- 172.21.15.12:0/310338211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb4199490 con 0x7f8cb41076d0 2024-05-09T09:24:40.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cab7fe700 1 -- 172.21.15.12:0/310338211 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac005570 con 0x7f8cb41076d0 2024-05-09T09:24:40.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.470+0000 7f8cb8b29700 1 -- 172.21.15.12:0/310338211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb41a5700 con 0x7f8cb41076d0 2024-05-09T09:24:40.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.471+0000 7f8cb8b29700 1 -- 172.21.15.12:0/310338211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb4103ad0 con 0x7f8cb41076d0 2024-05-09T09:24:40.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.473+0000 7f8cab7fe700 1 -- 172.21.15.12:0/310338211 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8cac0049e0 con 0x7f8cb41076d0 2024-05-09T09:24:40.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.473+0000 7f8cab7fe700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8ca00a2c20 0x7f8ca00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:40.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.474+0000 7f8cb259c700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8ca00a2c20 0x7f8ca00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:40.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.474+0000 7f8cab7fe700 1 -- 172.21.15.12:0/310338211 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8cac0ceee0 con 0x7f8cb41076d0 2024-05-09T09:24:40.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.474+0000 7f8cb259c700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8ca00a2c20 0x7f8ca00a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8cb41073b0 tx=0x7f8ca4009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:40.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.478+0000 7f8cab7fe700 1 -- 172.21.15.12:0/310338211 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8cac014030 con 0x7f8cb41076d0 2024-05-09T09:24:40.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:40 smithi170 ceph-mon[33632]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T09:24:40.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:40 smithi016 ceph-mon[33335]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T09:24:40.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.677+0000 7f685ef59700 1 -- 172.21.15.12:0/1063755572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6858101be0 msgr2=0x7f685810e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:40.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.677+0000 7f685ef59700 1 --2- 172.21.15.12:0/1063755572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6858101be0 0x7f685810e0d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f685000b0a0 tx=0x7f685000b3b0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.677+0000 7f685ef59700 1 -- 172.21.15.12:0/1063755572 shutdown_connections 2024-05-09T09:24:40.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.677+0000 7f685ef59700 1 --2- 172.21.15.12:0/1063755572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6858101be0 0x7f685810e0d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.677+0000 7f685ef59700 1 --2- 172.21.15.12:0/1063755572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6858107290 0x7f6858107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.677+0000 7f685ef59700 1 --2- 172.21.15.12:0/1063755572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6858101220 0x7f68581016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.677+0000 7f685ef59700 1 -- 172.21.15.12:0/1063755572 >> 172.21.15.12:0/1063755572 conn(0x7f68580781d0 msgr2=0x7f68580785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:40.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.677+0000 7f685ef59700 1 -- 172.21.15.12:0/1063755572 shutdown_connections 2024-05-09T09:24:40.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.677+0000 7f685ef59700 1 -- 172.21.15.12:0/1063755572 wait complete. 2024-05-09T09:24:40.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.678+0000 7f685ef59700 1 Processor -- start 2024-05-09T09:24:40.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.678+0000 7f685ef59700 1 -- start start 2024-05-09T09:24:40.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.678+0000 7f685ef59700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6858101220 0x7f68581998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.678+0000 7f685ef59700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6858101be0 0x7f6858199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.678+0000 7f685ef59700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6858107290 0x7f68581a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.678+0000 7f685ef59700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f68581114f0 con 0x7f6858107290 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.678+0000 7f685ef59700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6858111370 con 0x7f6858101220 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.678+0000 7f685ef59700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6858111670 con 0x7f6858101be0 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.678+0000 7f685d4f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6858107290 0x7f68581a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.678+0000 7f685d4f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6858107290 0x7f68581a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34808/0 (socket says 172.21.15.12:34808) 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.678+0000 7f685d4f6700 1 -- 172.21.15.12:0/443694231 learned_addr learned my addr 172.21.15.12:0/443694231 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.678+0000 7f685ccf5700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6858101220 0x7f68581998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f6857fff700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6858101be0 0x7f6858199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f685d4f6700 1 -- 172.21.15.12:0/443694231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6858101be0 msgr2=0x7f6858199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f685d4f6700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6858101be0 0x7f6858199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f685d4f6700 1 -- 172.21.15.12:0/443694231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6858101220 msgr2=0x7f68581998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f685d4f6700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6858101220 0x7f68581998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f685d4f6700 1 -- 172.21.15.12:0/443694231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68581a4e70 con 0x7f6858107290 2024-05-09T09:24:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f685ccf5700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6858101220 0x7f68581998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f6857fff700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6858101be0 0x7f6858199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f685d4f6700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6858107290 0x7f68581a4930 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6850006070 tx=0x7f6850004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f6855ffb700 1 -- 172.21.15.12:0/443694231 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850021070 con 0x7f6858107290 2024-05-09T09:24:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f6855ffb700 1 -- 172.21.15.12:0/443694231 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6850007c80 con 0x7f6858107290 2024-05-09T09:24:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f6855ffb700 1 -- 172.21.15.12:0/443694231 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685001d450 con 0x7f6858107290 2024-05-09T09:24:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f685ef59700 1 -- 172.21.15.12:0/443694231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68581a50a0 con 0x7f6858107290 2024-05-09T09:24:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.679+0000 7f685ef59700 1 -- 172.21.15.12:0/443694231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68581a5550 con 0x7f6858107290 2024-05-09T09:24:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.680+0000 7f685ef59700 1 -- 172.21.15.12:0/443694231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6858069010 con 0x7f6858107290 2024-05-09T09:24:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.682+0000 7f6855ffb700 1 -- 172.21.15.12:0/443694231 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68500040b0 con 0x7f6858107290 2024-05-09T09:24:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.682+0000 7f6855ffb700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68480a2c20 0x7f68480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:40.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.682+0000 7f685ccf5700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68480a2c20 0x7f68480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:40.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.683+0000 7f6855ffb700 1 -- 172.21.15.12:0/443694231 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f685000f020 con 0x7f6858107290 2024-05-09T09:24:40.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.683+0000 7f685ccf5700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68480a2c20 0x7f68480a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f684c005e10 tx=0x7f684c005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:40.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.690+0000 7f6855ffb700 1 -- 172.21.15.12:0/443694231 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f685002a020 con 0x7f6858107290 2024-05-09T09:24:40.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.696+0000 7f8cb8b29700 1 -- 172.21.15.12:0/310338211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f8cb402db80 con 0x7f8cb41076d0 2024-05-09T09:24:40.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.696+0000 7f8cab7fe700 1 -- 172.21.15.12:0/310338211 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8cac09bae0 con 0x7f8cb41076d0 2024-05-09T09:24:40.699 INFO:teuthology.orchestra.run.smithi012.stdout:287762808874 2024-05-09T09:24:40.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.699+0000 7f8cb8b29700 1 -- 172.21.15.12:0/310338211 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8ca00a2c20 msgr2=0x7f8ca00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:40.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.699+0000 7f8cb8b29700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8ca00a2c20 0x7f8ca00a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8cb41073b0 tx=0x7f8ca4009040 comp rx=0 tx=0).stop 2024-05-09T09:24:40.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.699+0000 7f8cb8b29700 1 -- 172.21.15.12:0/310338211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cb41076d0 msgr2=0x7f8cb4198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:40.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.699+0000 7f8cb8b29700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cb41076d0 0x7f8cb4198a40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00c970 tx=0x7f8cac00cc80 comp rx=0 tx=0).stop 2024-05-09T09:24:40.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.699+0000 7f8cb8b29700 1 -- 172.21.15.12:0/310338211 shutdown_connections 2024-05-09T09:24:40.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.700+0000 7f8cb8b29700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8ca00a2c20 0x7f8ca00a50e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.700+0000 7f8cb8b29700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cb41076d0 0x7f8cb4198a40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.700+0000 7f8cb8b29700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb4102600 0x7f8cb419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.700+0000 7f8cb8b29700 1 --2- 172.21.15.12:0/310338211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cb4102fb0 0x7f8cb419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.700+0000 7f8cb8b29700 1 -- 172.21.15.12:0/310338211 >> 172.21.15.12:0/310338211 conn(0x7f8cb40794d0 msgr2=0x7f8cb41004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:40.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.700+0000 7f8cb8b29700 1 -- 172.21.15.12:0/310338211 shutdown_connections 2024-05-09T09:24:40.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.700+0000 7f8cb8b29700 1 -- 172.21.15.12:0/310338211 wait complete. 2024-05-09T09:24:40.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.908+0000 7f685ef59700 1 -- 172.21.15.12:0/443694231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f685802db80 con 0x7f6858107290 2024-05-09T09:24:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.908+0000 7f6855ffb700 1 -- 172.21.15.12:0/443694231 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f68500bbe20 con 0x7f6858107290 2024-05-09T09:24:40.908 INFO:teuthology.orchestra.run.smithi012.stdout:107374182471 2024-05-09T09:24:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.910+0000 7f685ef59700 1 -- 172.21.15.12:0/443694231 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68480a2c20 msgr2=0x7f68480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.910+0000 7f685ef59700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68480a2c20 0x7f68480a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f684c005e10 tx=0x7f684c005d60 comp rx=0 tx=0).stop 2024-05-09T09:24:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.910+0000 7f685ef59700 1 -- 172.21.15.12:0/443694231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6858107290 msgr2=0x7f68581a4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.910+0000 7f685ef59700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6858107290 0x7f68581a4930 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6850006070 tx=0x7f6850004e20 comp rx=0 tx=0).stop 2024-05-09T09:24:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.910+0000 7f685ef59700 1 -- 172.21.15.12:0/443694231 shutdown_connections 2024-05-09T09:24:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.910+0000 7f685ef59700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68480a2c20 0x7f68480a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.910+0000 7f685ef59700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6858107290 0x7f68581a4930 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.910+0000 7f685ef59700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6858101220 0x7f68581998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.910+0000 7f685ef59700 1 --2- 172.21.15.12:0/443694231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6858101be0 0x7f6858199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.910+0000 7f685ef59700 1 -- 172.21.15.12:0/443694231 >> 172.21.15.12:0/443694231 conn(0x7f68580781d0 msgr2=0x7f68580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.910+0000 7f685ef59700 1 -- 172.21.15.12:0/443694231 shutdown_connections 2024-05-09T09:24:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:40.910+0000 7f685ef59700 1 -- 172.21.15.12:0/443694231 wait complete. 2024-05-09T09:24:41.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:41 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/310338211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T09:24:41.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:41 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/443694231' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T09:24:41.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:41 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/310338211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T09:24:41.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:41 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/443694231' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T09:24:41.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:41 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/310338211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T09:24:41.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:41 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/443694231' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T09:24:41.573 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808867 got 287762808874 for osd.9 2024-05-09T09:24:41.573 DEBUG:teuthology.parallel:result is None 2024-05-09T09:24:42.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:42 smithi012 ceph-mon[30140]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:24:42.467 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182465 got 107374182471 for osd.3 2024-05-09T09:24:42.468 DEBUG:teuthology.parallel:result is None 2024-05-09T09:24:42.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:42 smithi170 ceph-mon[33632]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:24:42.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:42 smithi016 ceph-mon[33335]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:24:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.701+0000 7faf3c547700 1 -- 172.21.15.12:0/1154713086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf341076d0 msgr2=0x7faf3410f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.701+0000 7faf3c547700 1 --2- 172.21.15.12:0/1154713086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf341076d0 0x7faf3410f2a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00b0d0 tx=0x7faf2c00b3e0 comp rx=0 tx=0).stop 2024-05-09T09:24:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.701+0000 7faf3c547700 1 -- 172.21.15.12:0/1154713086 shutdown_connections 2024-05-09T09:24:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.701+0000 7faf3c547700 1 --2- 172.21.15.12:0/1154713086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf341076d0 0x7faf3410f2a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.701+0000 7faf3c547700 1 --2- 172.21.15.12:0/1154713086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf34102600 0x7faf341029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.701+0000 7faf3c547700 1 --2- 172.21.15.12:0/1154713086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf34102fb0 0x7faf34106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.701+0000 7faf3c547700 1 -- 172.21.15.12:0/1154713086 >> 172.21.15.12:0/1154713086 conn(0x7faf340794d0 msgr2=0x7faf340798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.702+0000 7faf3c547700 1 -- 172.21.15.12:0/1154713086 shutdown_connections 2024-05-09T09:24:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.702+0000 7faf3c547700 1 -- 172.21.15.12:0/1154713086 wait complete. 2024-05-09T09:24:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.702+0000 7faf3c547700 1 Processor -- start 2024-05-09T09:24:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.702+0000 7faf3c547700 1 -- start start 2024-05-09T09:24:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf3c547700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf34102600 0x7faf3419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf3c547700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf34102fb0 0x7faf3419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf3c547700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf341076d0 0x7faf341989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf3c547700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faf341126d0 con 0x7faf34102fb0 2024-05-09T09:24:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf3c547700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf34112550 con 0x7faf34102600 2024-05-09T09:24:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf3c547700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faf34112850 con 0x7faf341076d0 2024-05-09T09:24:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf39ae2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf34102fb0 0x7faf3419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf39ae2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf34102fb0 0x7faf3419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47134/0 (socket says 172.21.15.12:47134) 2024-05-09T09:24:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf39ae2700 1 -- 172.21.15.12:0/3889901645 learned_addr learned my addr 172.21.15.12:0/3889901645 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf3aae4700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf341076d0 0x7faf341989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf3a2e3700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf34102600 0x7faf3419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf39ae2700 1 -- 172.21.15.12:0/3889901645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf341076d0 msgr2=0x7faf341989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf39ae2700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf341076d0 0x7faf341989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf39ae2700 1 -- 172.21.15.12:0/3889901645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf34102600 msgr2=0x7faf3419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf39ae2700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf34102600 0x7faf3419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.703+0000 7faf39ae2700 1 -- 172.21.15.12:0/3889901645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf34199110 con 0x7faf34102fb0 2024-05-09T09:24:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.704+0000 7faf39ae2700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf34102fb0 0x7faf3419ed30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faf2400e970 tx=0x7faf2400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.704+0000 7faf2b7fe700 1 -- 172.21.15.12:0/3889901645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24004a30 con 0x7faf34102fb0 2024-05-09T09:24:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.704+0000 7faf2b7fe700 1 -- 172.21.15.12:0/3889901645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7faf2400ccb0 con 0x7faf34102fb0 2024-05-09T09:24:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.704+0000 7faf3c547700 1 -- 172.21.15.12:0/3889901645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf34199400 con 0x7faf34102fb0 2024-05-09T09:24:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.704+0000 7faf2b7fe700 1 -- 172.21.15.12:0/3889901645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24005680 con 0x7faf34102fb0 2024-05-09T09:24:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.704+0000 7faf3c547700 1 -- 172.21.15.12:0/3889901645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf341a56c0 con 0x7faf34102fb0 2024-05-09T09:24:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.705+0000 7faf3c547700 1 -- 172.21.15.12:0/3889901645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf34069010 con 0x7faf34102fb0 2024-05-09T09:24:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.706+0000 7faf2b7fe700 1 -- 172.21.15.12:0/3889901645 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faf2400c7e0 con 0x7faf34102fb0 2024-05-09T09:24:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.707+0000 7faf2b7fe700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7faf200a2c20 0x7faf200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.707+0000 7faf2b7fe700 1 -- 172.21.15.12:0/3889901645 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7faf240cf750 con 0x7faf34102fb0 2024-05-09T09:24:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.707+0000 7faf3a2e3700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7faf200a2c20 0x7faf200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.707+0000 7faf3a2e3700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7faf200a2c20 0x7faf200a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faf34199af0 tx=0x7faf30009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.710+0000 7faf2b7fe700 1 -- 172.21.15.12:0/3889901645 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faf24053740 con 0x7faf34102fb0 2024-05-09T09:24:42.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.928+0000 7faf3c547700 1 -- 172.21.15.12:0/3889901645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7faf34199ee0 con 0x7faf34102fb0 2024-05-09T09:24:42.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.929+0000 7faf2b7fe700 1 -- 172.21.15.12:0/3889901645 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faf2400cad0 con 0x7faf34102fb0 2024-05-09T09:24:42.928 INFO:teuthology.orchestra.run.smithi012.stdout:34359738455 2024-05-09T09:24:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.931+0000 7faf3c547700 1 -- 172.21.15.12:0/3889901645 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7faf200a2c20 msgr2=0x7faf200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.931+0000 7faf3c547700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7faf200a2c20 0x7faf200a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faf34199af0 tx=0x7faf30009260 comp rx=0 tx=0).stop 2024-05-09T09:24:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.931+0000 7faf3c547700 1 -- 172.21.15.12:0/3889901645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf34102fb0 msgr2=0x7faf3419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.931+0000 7faf3c547700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf34102fb0 0x7faf3419ed30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faf2400e970 tx=0x7faf2400ec80 comp rx=0 tx=0).stop 2024-05-09T09:24:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.931+0000 7faf3c547700 1 -- 172.21.15.12:0/3889901645 shutdown_connections 2024-05-09T09:24:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.931+0000 7faf3c547700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7faf200a2c20 0x7faf200a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.931+0000 7faf3c547700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf341076d0 0x7faf341989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.931+0000 7faf3c547700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf34102600 0x7faf3419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.931+0000 7faf3c547700 1 --2- 172.21.15.12:0/3889901645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf34102fb0 0x7faf3419ed30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.931+0000 7faf3c547700 1 -- 172.21.15.12:0/3889901645 >> 172.21.15.12:0/3889901645 conn(0x7faf340794d0 msgr2=0x7faf34100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.931+0000 7faf3c547700 1 -- 172.21.15.12:0/3889901645 shutdown_connections 2024-05-09T09:24:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:42.931+0000 7faf3c547700 1 -- 172.21.15.12:0/3889901645 wait complete. 2024-05-09T09:24:43.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:43 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3889901645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T09:24:43.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:43 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3889901645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T09:24:43.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:43 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3889901645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T09:24:43.572 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738449 got 34359738455 for osd.0 2024-05-09T09:24:43.572 DEBUG:teuthology.parallel:result is None 2024-05-09T09:24:43.572 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T09:24:43.572 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-09T09:24:43.843 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:44.219 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:44 smithi012 ceph-mon[30140]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:24:44.219 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:44 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:44.219 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:44 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:44.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:44 smithi170 ceph-mon[33632]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:24:44.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:44 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:44.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:44 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:44.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:44 smithi016 ceph-mon[33335]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:24:44.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:44 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:44.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:44 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:45.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:24:45.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:24:45.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:45.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:24:45.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:24:45.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:45.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:24:45.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:24:45.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:24:45.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.638+0000 7fb3733d5700 1 -- 172.21.15.12:0/4232577357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb36c102fb0 msgr2=0x7fb36c106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:45.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.638+0000 7fb3733d5700 1 --2- 172.21.15.12:0/4232577357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb36c102fb0 0x7fb36c106fa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009990 tx=0x7fb35c009ca0 comp rx=0 tx=0).stop 2024-05-09T09:24:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.638+0000 7fb3733d5700 1 -- 172.21.15.12:0/4232577357 shutdown_connections 2024-05-09T09:24:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.638+0000 7fb3733d5700 1 --2- 172.21.15.12:0/4232577357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb36c107680 0x7fb36c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.638+0000 7fb3733d5700 1 --2- 172.21.15.12:0/4232577357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb36c102600 0x7fb36c1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.638+0000 7fb3733d5700 1 --2- 172.21.15.12:0/4232577357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb36c102fb0 0x7fb36c106fa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.638+0000 7fb3733d5700 1 -- 172.21.15.12:0/4232577357 >> 172.21.15.12:0/4232577357 conn(0x7fb36c079490 msgr2=0x7fb36c0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.638+0000 7fb3733d5700 1 -- 172.21.15.12:0/4232577357 shutdown_connections 2024-05-09T09:24:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.638+0000 7fb3733d5700 1 -- 172.21.15.12:0/4232577357 wait complete. 2024-05-09T09:24:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.639+0000 7fb3733d5700 1 Processor -- start 2024-05-09T09:24:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.639+0000 7fb3733d5700 1 -- start start 2024-05-09T09:24:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.639+0000 7fb3733d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb36c102600 0x7fb36c074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.639+0000 7fb3733d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb36c102fb0 0x7fb36c074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.639+0000 7fb3733d5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb36c107680 0x7fb36c0750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.639+0000 7fb3733d5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c112670 con 0x7fb36c102fb0 2024-05-09T09:24:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.639+0000 7fb3733d5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c1124f0 con 0x7fb36c102600 2024-05-09T09:24:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.639+0000 7fb3733d5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c1127f0 con 0x7fb36c107680 2024-05-09T09:24:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb370970700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb36c102fb0 0x7fb36c074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb370970700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb36c102fb0 0x7fb36c074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47158/0 (socket says 172.21.15.12:47158) 2024-05-09T09:24:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb370970700 1 -- 172.21.15.12:0/1816945655 learned_addr learned my addr 172.21.15.12:0/1816945655 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb371171700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb36c102600 0x7fb36c074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:45.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb371972700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb36c107680 0x7fb36c0750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:45.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb370970700 1 -- 172.21.15.12:0/1816945655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb36c107680 msgr2=0x7fb36c0750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:45.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb370970700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb36c107680 0x7fb36c0750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:45.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb370970700 1 -- 172.21.15.12:0/1816945655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb36c102600 msgr2=0x7fb36c074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:45.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb370970700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb36c102600 0x7fb36c074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:45.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb370970700 1 -- 172.21.15.12:0/1816945655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36c1a6fb0 con 0x7fb36c102fb0 2024-05-09T09:24:45.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb370970700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb36c102fb0 0x7fb36c074b60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb35c000c00 tx=0x7fb35c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:45.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb3627fc700 1 -- 172.21.15.12:0/1816945655 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c027070 con 0x7fb36c102fb0 2024-05-09T09:24:45.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb3627fc700 1 -- 172.21.15.12:0/1816945655 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb35c004040 con 0x7fb36c102fb0 2024-05-09T09:24:45.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb3733d5700 1 -- 172.21.15.12:0/1816945655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb36c1a7240 con 0x7fb36c102fb0 2024-05-09T09:24:45.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.640+0000 7fb3627fc700 1 -- 172.21.15.12:0/1816945655 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c022bd0 con 0x7fb36c102fb0 2024-05-09T09:24:45.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.641+0000 7fb3733d5700 1 -- 172.21.15.12:0/1816945655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb36c1a7720 con 0x7fb36c102fb0 2024-05-09T09:24:45.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.641+0000 7fb3733d5700 1 -- 172.21.15.12:0/1816945655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb36c10a8a0 con 0x7fb36c102fb0 2024-05-09T09:24:45.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.643+0000 7fb3627fc700 1 -- 172.21.15.12:0/1816945655 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb35c02a070 con 0x7fb36c102fb0 2024-05-09T09:24:45.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.643+0000 7fb3627fc700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb3580a2c20 0x7fb3580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:45.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.644+0000 7fb371171700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb3580a2c20 0x7fb3580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:45.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.644+0000 7fb3627fc700 1 -- 172.21.15.12:0/1816945655 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb35c0cfe80 con 0x7fb36c102fb0 2024-05-09T09:24:45.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.644+0000 7fb371171700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb3580a2c20 0x7fb3580a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb368004630 tx=0x7fb36800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:45.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.648+0000 7fb3627fc700 1 -- 172.21.15.12:0/1816945655 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb35c013070 con 0x7fb36c102fb0 2024-05-09T09:24:45.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.849+0000 7fb3733d5700 1 -- 172.21.15.12:0/1816945655 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb36c062760 con 0x7fb3580a2c20 2024-05-09T09:24:45.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.859+0000 7fb3627fc700 1 -- 172.21.15.12:0/1816945655 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340526 (secure 0 0 0) 0x7fb36c062760 con 0x7fb3580a2c20 2024-05-09T09:24:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:24:45.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.863+0000 7fb3733d5700 1 -- 172.21.15.12:0/1816945655 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb3580a2c20 msgr2=0x7fb3580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.863+0000 7fb3733d5700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb3580a2c20 0x7fb3580a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb368004630 tx=0x7fb36800b3c0 comp rx=0 tx=0).stop 2024-05-09T09:24:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.863+0000 7fb3733d5700 1 -- 172.21.15.12:0/1816945655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb36c102fb0 msgr2=0x7fb36c074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.863+0000 7fb3733d5700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb36c102fb0 0x7fb36c074b60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb35c000c00 tx=0x7fb35c005ef0 comp rx=0 tx=0).stop 2024-05-09T09:24:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.863+0000 7fb3733d5700 1 -- 172.21.15.12:0/1816945655 shutdown_connections 2024-05-09T09:24:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.863+0000 7fb3733d5700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb3580a2c20 0x7fb3580a50e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.863+0000 7fb3733d5700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb36c107680 0x7fb36c0750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.863+0000 7fb3733d5700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb36c102600 0x7fb36c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.863+0000 7fb3733d5700 1 --2- 172.21.15.12:0/1816945655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb36c102fb0 0x7fb36c074b60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.863+0000 7fb3733d5700 1 -- 172.21.15.12:0/1816945655 >> 172.21.15.12:0/1816945655 conn(0x7fb36c079490 msgr2=0x7fb36c101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.864+0000 7fb3733d5700 1 -- 172.21.15.12:0/1816945655 shutdown_connections 2024-05-09T09:24:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:45.864+0000 7fb3733d5700 1 -- 172.21.15.12:0/1816945655 wait complete. 2024-05-09T09:24:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-09T09:24:46.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:46 smithi012 ceph-mon[30140]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:24:46.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:46 smithi170 ceph-mon[33632]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:24:46.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:46 smithi016 ceph-mon[33335]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:24:46.761 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-05-09T09:24:44.576319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661161,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":862,"num_read_kb":556,"num_write":432,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392512,"kb_used_data":14240,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549312,"statfs":{"total":1151940427776,"available":1151538495488,"internally_reserved":0,"allocated":14581760,"data_stored":7964657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":72,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004065"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.610993+0000","last_change":"2024-05-09T09:23:10.334738+0000","last_active":"2024-05-09T09:24:34.610993+0000","last_peered":"2024-05-09T09:24:34.610993+0000","last_clean":"2024-05-09T09:24:34.610993+0000","last_became_active":"2024-05-09T09:23:10.333406+0000","last_became_peered":"2024-05-09T09:23:10.333406+0000","last_unstale":"2024-05-09T09:24:34.610993+0000","last_undegraded":"2024-05-09T09:24:34.610993+0000","last_fullsized":"2024-05-09T09:24:34.610993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:44.957557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.609671+0000","last_change":"2024-05-09T09:23:10.331860+0000","last_active":"2024-05-09T09:24:34.609671+0000","last_peered":"2024-05-09T09:24:34.609671+0000","last_clean":"2024-05-09T09:24:34.609671+0000","last_became_active":"2024-05-09T09:23:10.331396+0000","last_became_peered":"2024-05-09T09:23:10.331396+0000","last_unstale":"2024-05-09T09:24:34.609671+0000","last_undegraded":"2024-05-09T09:24:34.609671+0000","last_fullsized":"2024-05-09T09:24:34.609671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:47.676399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.611885+0000","last_change":"2024-05-09T09:23:10.336543+0000","last_active":"2024-05-09T09:24:34.611885+0000","last_peered":"2024-05-09T09:24:34.611885+0000","last_clean":"2024-05-09T09:24:34.611885+0000","last_became_active":"2024-05-09T09:23:10.336189+0000","last_became_peered":"2024-05-09T09:23:10.336189+0000","last_unstale":"2024-05-09T09:24:34.611885+0000","last_undegraded":"2024-05-09T09:24:34.611885+0000","last_fullsized":"2024-05-09T09:24:34.611885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:06.419177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.576657+0000","last_change":"2024-05-09T09:23:10.331739+0000","last_active":"2024-05-09T09:24:34.576657+0000","last_peered":"2024-05-09T09:24:34.576657+0000","last_clean":"2024-05-09T09:24:34.576657+0000","last_became_active":"2024-05-09T09:23:10.331333+0000","last_became_peered":"2024-05-09T09:23:10.331333+0000","last_unstale":"2024-05-09T09:24:34.576657+0000","last_undegraded":"2024-05-09T09:24:34.576657+0000","last_fullsized":"2024-05-09T09:24:34.576657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:04.925697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.581434+0000","last_change":"2024-05-09T09:23:10.345294+0000","last_active":"2024-05-09T09:24:34.581434+0000","last_peered":"2024-05-09T09:24:34.581434+0000","last_clean":"2024-05-09T09:24:34.581434+0000","last_became_active":"2024-05-09T09:23:10.344940+0000","last_became_peered":"2024-05-09T09:23:10.344940+0000","last_unstale":"2024-05-09T09:24:34.581434+0000","last_undegraded":"2024-05-09T09:24:34.581434+0000","last_fullsized":"2024-05-09T09:24:34.581434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:41.150284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.579544+0000","last_change":"2024-05-09T09:23:10.329143+0000","last_active":"2024-05-09T09:24:34.579544+0000","last_peered":"2024-05-09T09:24:34.579544+0000","last_clean":"2024-05-09T09:24:34.579544+0000","last_became_active":"2024-05-09T09:23:10.328712+0000","last_became_peered":"2024-05-09T09:23:10.328712+0000","last_unstale":"2024-05-09T09:24:34.579544+0000","last_undegraded":"2024-05-09T09:24:34.579544+0000","last_fullsized":"2024-05-09T09:24:34.579544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:10:35.085759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.593087+0000","last_change":"2024-05-09T09:23:10.338176+0000","last_active":"2024-05-09T09:24:34.593087+0000","last_peered":"2024-05-09T09:24:34.593087+0000","last_clean":"2024-05-09T09:24:34.593087+0000","last_became_active":"2024-05-09T09:23:10.337845+0000","last_became_peered":"2024-05-09T09:23:10.337845+0000","last_unstale":"2024-05-09T09:24:34.593087+0000","last_undegraded":"2024-05-09T09:24:34.593087+0000","last_fullsized":"2024-05-09T09:24:34.593087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:34:21.783295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.604691+0000","last_change":"2024-05-09T09:23:10.342473+0000","last_active":"2024-05-09T09:24:34.604691+0000","last_peered":"2024-05-09T09:24:34.604691+0000","last_clean":"2024-05-09T09:24:34.604691+0000","last_became_active":"2024-05-09T09:23:10.340682+0000","last_became_peered":"2024-05-09T09:23:10.340682+0000","last_unstale":"2024-05-09T09:24:34.604691+0000","last_undegraded":"2024-05-09T09:24:34.604691+0000","last_fullsized":"2024-05-09T09:24:34.604691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:56.730332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.596347+0000","last_change":"2024-05-09T09:23:10.339888+0000","last_active":"2024-05-09T09:24:34.596347+0000","last_peered":"2024-05-09T09:24:34.596347+0000","last_clean":"2024-05-09T09:24:34.596347+0000","last_became_active":"2024-05-09T09:23:10.337849+0000","last_became_peered":"2024-05-09T09:23:10.337849+0000","last_unstale":"2024-05-09T09:24:34.596347+0000","last_undegraded":"2024-05-09T09:24:34.596347+0000","last_fullsized":"2024-05-09T09:24:34.596347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:48.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.586699+0000","last_change":"2024-05-09T09:23:10.332395+0000","last_active":"2024-05-09T09:24:34.586699+0000","last_peered":"2024-05-09T09:24:34.586699+0000","last_clean":"2024-05-09T09:24:34.586699+0000","last_became_active":"2024-05-09T09:23:10.332040+0000","last_became_peered":"2024-05-09T09:23:10.332040+0000","last_unstale":"2024-05-09T09:24:34.586699+0000","last_undegraded":"2024-05-09T09:24:34.586699+0000","last_fullsized":"2024-05-09T09:24:34.586699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:04.036871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.597071+0000","last_change":"2024-05-09T09:23:10.341148+0000","last_active":"2024-05-09T09:24:34.597071+0000","last_peered":"2024-05-09T09:24:34.597071+0000","last_clean":"2024-05-09T09:24:34.597071+0000","last_became_active":"2024-05-09T09:23:10.340799+0000","last_became_peered":"2024-05-09T09:23:10.340799+0000","last_unstale":"2024-05-09T09:24:34.597071+0000","last_undegraded":"2024-05-09T09:24:34.597071+0000","last_fullsized":"2024-05-09T09:24:34.597071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:31.020659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.577617+0000","last_change":"2024-05-09T09:23:10.336191+0000","last_active":"2024-05-09T09:24:34.577617+0000","last_peered":"2024-05-09T09:24:34.577617+0000","last_clean":"2024-05-09T09:24:34.577617+0000","last_became_active":"2024-05-09T09:23:10.333591+0000","last_became_peered":"2024-05-09T09:23:10.333591+0000","last_unstale":"2024-05-09T09:24:34.577617+0000","last_undegraded":"2024-05-09T09:24:34.577617+0000","last_fullsized":"2024-05-09T09:24:34.577617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:24.836954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.599143+0000","last_change":"2024-05-09T09:23:10.314506+0000","last_active":"2024-05-09T09:24:34.599143+0000","last_peered":"2024-05-09T09:24:34.599143+0000","last_clean":"2024-05-09T09:24:34.599143+0000","last_became_active":"2024-05-09T09:23:10.314171+0000","last_became_peered":"2024-05-09T09:23:10.314171+0000","last_unstale":"2024-05-09T09:24:34.599143+0000","last_undegraded":"2024-05-09T09:24:34.599143+0000","last_fullsized":"2024-05-09T09:24:34.599143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:51.088814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.592081+0000","last_change":"2024-05-09T09:23:10.341670+0000","last_active":"2024-05-09T09:24:34.592081+0000","last_peered":"2024-05-09T09:24:34.592081+0000","last_clean":"2024-05-09T09:24:34.592081+0000","last_became_active":"2024-05-09T09:23:10.341155+0000","last_became_peered":"2024-05-09T09:23:10.341155+0000","last_unstale":"2024-05-09T09:24:34.592081+0000","last_undegraded":"2024-05-09T09:24:34.592081+0000","last_fullsized":"2024-05-09T09:24:34.592081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:14:32.734302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.611616+0000","last_change":"2024-05-09T09:24:05.742117+0000","last_active":"2024-05-09T09:24:34.611616+0000","last_peered":"2024-05-09T09:24:34.611616+0000","last_clean":"2024-05-09T09:24:34.611616+0000","last_became_active":"2024-05-09T09:24:05.741655+0000","last_became_peered":"2024-05-09T09:24:05.741655+0000","last_unstale":"2024-05-09T09:24:34.611616+0000","last_undegraded":"2024-05-09T09:24:34.611616+0000","last_fullsized":"2024-05-09T09:24:34.611616+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:11.468527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.594547+0000","last_change":"2024-05-09T09:23:10.337367+0000","last_active":"2024-05-09T09:24:34.594547+0000","last_peered":"2024-05-09T09:24:34.594547+0000","last_clean":"2024-05-09T09:24:34.594547+0000","last_became_active":"2024-05-09T09:23:10.337000+0000","last_became_peered":"2024-05-09T09:23:10.337000+0000","last_unstale":"2024-05-09T09:24:34.594547+0000","last_undegraded":"2024-05-09T09:24:34.594547+0000","last_fullsized":"2024-05-09T09:24:34.594547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:54.431820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.612492+0000","last_change":"2024-05-09T09:23:10.334293+0000","last_active":"2024-05-09T09:24:34.612492+0000","last_peered":"2024-05-09T09:24:34.612492+0000","last_clean":"2024-05-09T09:24:34.612492+0000","last_became_active":"2024-05-09T09:23:10.333938+0000","last_became_peered":"2024-05-09T09:23:10.333938+0000","last_unstale":"2024-05-09T09:24:34.612492+0000","last_undegraded":"2024-05-09T09:24:34.612492+0000","last_fullsized":"2024-05-09T09:24:34.612492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:56:09.048848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.600158+0000","last_change":"2024-05-09T09:23:10.332394+0000","last_active":"2024-05-09T09:24:34.600158+0000","last_peered":"2024-05-09T09:24:34.600158+0000","last_clean":"2024-05-09T09:24:34.600158+0000","last_became_active":"2024-05-09T09:23:10.332054+0000","last_became_peered":"2024-05-09T09:23:10.332054+0000","last_unstale":"2024-05-09T09:24:34.600158+0000","last_undegraded":"2024-05-09T09:24:34.600158+0000","last_fullsized":"2024-05-09T09:24:34.600158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:05.628740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.583441+0000","last_change":"2024-05-09T09:23:10.336189+0000","last_active":"2024-05-09T09:24:34.583441+0000","last_peered":"2024-05-09T09:24:34.583441+0000","last_clean":"2024-05-09T09:24:34.583441+0000","last_became_active":"2024-05-09T09:23:10.335698+0000","last_became_peered":"2024-05-09T09:23:10.335698+0000","last_unstale":"2024-05-09T09:24:34.583441+0000","last_undegraded":"2024-05-09T09:24:34.583441+0000","last_fullsized":"2024-05-09T09:24:34.583441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:01.403918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732540+0000","last_change":"2024-05-09T09:23:14.372134+0000","last_active":"2024-05-09T09:24:05.732540+0000","last_peered":"2024-05-09T09:24:05.732540+0000","last_clean":"2024-05-09T09:24:05.732540+0000","last_became_active":"2024-05-09T09:23:14.371466+0000","last_became_peered":"2024-05-09T09:23:14.371466+0000","last_unstale":"2024-05-09T09:24:05.732540+0000","last_undegraded":"2024-05-09T09:24:05.732540+0000","last_fullsized":"2024-05-09T09:24:05.732540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:22.048565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:41.691814+0000","last_change":"2024-05-09T09:23:09.304421+0000","last_active":"2024-05-09T09:24:41.691814+0000","last_peered":"2024-05-09T09:24:41.691814+0000","last_clean":"2024-05-09T09:24:41.691814+0000","last_became_active":"2024-05-09T09:23:09.304109+0000","last_became_peered":"2024-05-09T09:23:09.304109+0000","last_unstale":"2024-05-09T09:24:41.691814+0000","last_undegraded":"2024-05-09T09:24:41.691814+0000","last_fullsized":"2024-05-09T09:24:41.691814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:36.796458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.603058+0000","last_change":"2024-05-09T09:23:10.341875+0000","last_active":"2024-05-09T09:24:34.603058+0000","last_peered":"2024-05-09T09:24:34.603058+0000","last_clean":"2024-05-09T09:24:34.603058+0000","last_became_active":"2024-05-09T09:23:10.341385+0000","last_became_peered":"2024-05-09T09:23:10.341385+0000","last_unstale":"2024-05-09T09:24:34.603058+0000","last_undegraded":"2024-05-09T09:24:34.603058+0000","last_fullsized":"2024-05-09T09:24:34.603058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:44:33.939722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.731962+0000","last_change":"2024-05-09T09:23:14.371120+0000","last_active":"2024-05-09T09:24:05.731962+0000","last_peered":"2024-05-09T09:24:05.731962+0000","last_clean":"2024-05-09T09:24:05.731962+0000","last_became_active":"2024-05-09T09:23:14.370663+0000","last_became_peered":"2024-05-09T09:23:14.370663+0000","last_unstale":"2024-05-09T09:24:05.731962+0000","last_undegraded":"2024-05-09T09:24:05.731962+0000","last_fullsized":"2024-05-09T09:24:05.731962+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:50:06.553518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.737188+0000","last_change":"2024-05-09T09:23:09.304357+0000","last_active":"2024-05-09T09:24:05.737188+0000","last_peered":"2024-05-09T09:24:05.737188+0000","last_clean":"2024-05-09T09:24:05.737188+0000","last_became_active":"2024-05-09T09:23:09.304007+0000","last_became_peered":"2024-05-09T09:23:09.304007+0000","last_unstale":"2024-05-09T09:24:05.737188+0000","last_undegraded":"2024-05-09T09:24:05.737188+0000","last_fullsized":"2024-05-09T09:24:05.737188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:32:46.132287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.611050+0000","last_change":"2024-05-09T09:23:10.339533+0000","last_active":"2024-05-09T09:24:34.611050+0000","last_peered":"2024-05-09T09:24:34.611050+0000","last_clean":"2024-05-09T09:24:34.611050+0000","last_became_active":"2024-05-09T09:23:10.337142+0000","last_became_peered":"2024-05-09T09:23:10.337142+0000","last_unstale":"2024-05-09T09:24:34.611050+0000","last_undegraded":"2024-05-09T09:24:34.611050+0000","last_fullsized":"2024-05-09T09:24:34.611050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:11:11.921491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.744009+0000","last_change":"2024-05-09T09:24:05.744009+0000","last_active":"2024-05-09T09:24:05.744009+0000","last_peered":"2024-05-09T09:24:05.744009+0000","last_clean":"2024-05-09T09:24:05.744009+0000","last_became_active":"2024-05-09T09:24:05.743611+0000","last_became_peered":"2024-05-09T09:24:05.743611+0000","last_unstale":"2024-05-09T09:24:05.744009+0000","last_undegraded":"2024-05-09T09:24:05.744009+0000","last_fullsized":"2024-05-09T09:24:05.744009+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:55.775740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.744187+0000","last_change":"2024-05-09T09:23:09.300754+0000","last_active":"2024-05-09T09:24:05.744187+0000","last_peered":"2024-05-09T09:24:05.744187+0000","last_clean":"2024-05-09T09:24:05.744187+0000","last_became_active":"2024-05-09T09:23:09.300419+0000","last_became_peered":"2024-05-09T09:23:09.300419+0000","last_unstale":"2024-05-09T09:24:05.744187+0000","last_undegraded":"2024-05-09T09:24:05.744187+0000","last_fullsized":"2024-05-09T09:24:05.744187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:06.205582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.596629+0000","last_change":"2024-05-09T09:23:10.331952+0000","last_active":"2024-05-09T09:24:34.596629+0000","last_peered":"2024-05-09T09:24:34.596629+0000","last_clean":"2024-05-09T09:24:34.596629+0000","last_became_active":"2024-05-09T09:23:10.331515+0000","last_became_peered":"2024-05-09T09:23:10.331515+0000","last_unstale":"2024-05-09T09:24:34.596629+0000","last_undegraded":"2024-05-09T09:24:34.596629+0000","last_fullsized":"2024-05-09T09:24:34.596629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:23.809672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732036+0000","last_change":"2024-05-09T09:23:14.368518+0000","last_active":"2024-05-09T09:24:05.732036+0000","last_peered":"2024-05-09T09:24:05.732036+0000","last_clean":"2024-05-09T09:24:05.732036+0000","last_became_active":"2024-05-09T09:23:14.367995+0000","last_became_peered":"2024-05-09T09:23:14.367995+0000","last_unstale":"2024-05-09T09:24:05.732036+0000","last_undegraded":"2024-05-09T09:24:05.732036+0000","last_fullsized":"2024-05-09T09:24:05.732036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:11.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743850+0000","last_change":"2024-05-09T09:23:09.299656+0000","last_active":"2024-05-09T09:24:05.743850+0000","last_peered":"2024-05-09T09:24:05.743850+0000","last_clean":"2024-05-09T09:24:05.743850+0000","last_became_active":"2024-05-09T09:23:09.299374+0000","last_became_peered":"2024-05-09T09:23:09.299374+0000","last_unstale":"2024-05-09T09:24:05.743850+0000","last_undegraded":"2024-05-09T09:24:05.743850+0000","last_fullsized":"2024-05-09T09:24:05.743850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:05.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.596539+0000","last_change":"2024-05-09T09:23:10.341933+0000","last_active":"2024-05-09T09:24:34.596539+0000","last_peered":"2024-05-09T09:24:34.596539+0000","last_clean":"2024-05-09T09:24:34.596539+0000","last_became_active":"2024-05-09T09:23:10.341621+0000","last_became_peered":"2024-05-09T09:23:10.341621+0000","last_unstale":"2024-05-09T09:24:34.596539+0000","last_undegraded":"2024-05-09T09:24:34.596539+0000","last_fullsized":"2024-05-09T09:24:34.596539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:31.938768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732505+0000","last_change":"2024-05-09T09:23:14.364783+0000","last_active":"2024-05-09T09:24:05.732505+0000","last_peered":"2024-05-09T09:24:05.732505+0000","last_clean":"2024-05-09T09:24:05.732505+0000","last_became_active":"2024-05-09T09:23:14.364309+0000","last_became_peered":"2024-05-09T09:23:14.364309+0000","last_unstale":"2024-05-09T09:24:05.732505+0000","last_undegraded":"2024-05-09T09:24:05.732505+0000","last_fullsized":"2024-05-09T09:24:05.732505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:44.252111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:42.695011+0000","last_change":"2024-05-09T09:23:09.297913+0000","last_active":"2024-05-09T09:24:42.695011+0000","last_peered":"2024-05-09T09:24:42.695011+0000","last_clean":"2024-05-09T09:24:42.695011+0000","last_became_active":"2024-05-09T09:23:09.297612+0000","last_became_peered":"2024-05-09T09:23:09.297612+0000","last_unstale":"2024-05-09T09:24:42.695011+0000","last_undegraded":"2024-05-09T09:24:42.695011+0000","last_fullsized":"2024-05-09T09:24:42.695011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:21.808703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.572712+0000","last_change":"2024-05-09T09:23:10.339101+0000","last_active":"2024-05-09T09:24:34.572712+0000","last_peered":"2024-05-09T09:24:34.572712+0000","last_clean":"2024-05-09T09:24:34.572712+0000","last_became_active":"2024-05-09T09:23:10.338453+0000","last_became_peered":"2024-05-09T09:23:10.338453+0000","last_unstale":"2024-05-09T09:24:34.572712+0000","last_undegraded":"2024-05-09T09:24:34.572712+0000","last_fullsized":"2024-05-09T09:24:34.572712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:34:44.933882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732775+0000","last_change":"2024-05-09T09:23:14.371781+0000","last_active":"2024-05-09T09:24:05.732775+0000","last_peered":"2024-05-09T09:24:05.732775+0000","last_clean":"2024-05-09T09:24:05.732775+0000","last_became_active":"2024-05-09T09:23:14.371286+0000","last_became_peered":"2024-05-09T09:23:14.371286+0000","last_unstale":"2024-05-09T09:24:05.732775+0000","last_undegraded":"2024-05-09T09:24:05.732775+0000","last_fullsized":"2024-05-09T09:24:05.732775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:26:29.185064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.744093+0000","last_change":"2024-05-09T09:23:09.303786+0000","last_active":"2024-05-09T09:24:05.744093+0000","last_peered":"2024-05-09T09:24:05.744093+0000","last_clean":"2024-05-09T09:24:05.744093+0000","last_became_active":"2024-05-09T09:23:09.303475+0000","last_became_peered":"2024-05-09T09:23:09.303475+0000","last_unstale":"2024-05-09T09:24:05.744093+0000","last_undegraded":"2024-05-09T09:24:05.744093+0000","last_fullsized":"2024-05-09T09:24:05.744093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:28.833055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.603291+0000","last_change":"2024-05-09T09:23:10.344643+0000","last_active":"2024-05-09T09:24:34.603291+0000","last_peered":"2024-05-09T09:24:34.603291+0000","last_clean":"2024-05-09T09:24:34.603291+0000","last_became_active":"2024-05-09T09:23:10.344314+0000","last_became_peered":"2024-05-09T09:23:10.344314+0000","last_unstale":"2024-05-09T09:24:34.603291+0000","last_undegraded":"2024-05-09T09:24:34.603291+0000","last_fullsized":"2024-05-09T09:24:34.603291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:15.147107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.739076+0000","last_change":"2024-05-09T09:23:14.361170+0000","last_active":"2024-05-09T09:24:05.739076+0000","last_peered":"2024-05-09T09:24:05.739076+0000","last_clean":"2024-05-09T09:24:05.739076+0000","last_became_active":"2024-05-09T09:23:14.360767+0000","last_became_peered":"2024-05-09T09:23:14.360767+0000","last_unstale":"2024-05-09T09:24:05.739076+0000","last_undegraded":"2024-05-09T09:24:05.739076+0000","last_fullsized":"2024-05-09T09:24:05.739076+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:59:21.393148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743278+0000","last_change":"2024-05-09T09:23:09.300757+0000","last_active":"2024-05-09T09:24:05.743278+0000","last_peered":"2024-05-09T09:24:05.743278+0000","last_clean":"2024-05-09T09:24:05.743278+0000","last_became_active":"2024-05-09T09:23:09.300328+0000","last_became_peered":"2024-05-09T09:23:09.300328+0000","last_unstale":"2024-05-09T09:24:05.743278+0000","last_undegraded":"2024-05-09T09:24:05.743278+0000","last_fullsized":"2024-05-09T09:24:05.743278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.553878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.606729+0000","last_change":"2024-05-09T09:23:10.336077+0000","last_active":"2024-05-09T09:24:34.606729+0000","last_peered":"2024-05-09T09:24:34.606729+0000","last_clean":"2024-05-09T09:24:34.606729+0000","last_became_active":"2024-05-09T09:23:10.335762+0000","last_became_peered":"2024-05-09T09:23:10.335762+0000","last_unstale":"2024-05-09T09:24:34.606729+0000","last_undegraded":"2024-05-09T09:24:34.606729+0000","last_fullsized":"2024-05-09T09:24:34.606729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:48.063385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.734656+0000","last_change":"2024-05-09T09:23:14.371413+0000","last_active":"2024-05-09T09:24:05.734656+0000","last_peered":"2024-05-09T09:24:05.734656+0000","last_clean":"2024-05-09T09:24:05.734656+0000","last_became_active":"2024-05-09T09:23:14.370018+0000","last_became_peered":"2024-05-09T09:23:14.370018+0000","last_unstale":"2024-05-09T09:24:05.734656+0000","last_undegraded":"2024-05-09T09:24:05.734656+0000","last_fullsized":"2024-05-09T09:24:05.734656+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:18.475976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.740718+0000","last_change":"2024-05-09T09:23:09.297328+0000","last_active":"2024-05-09T09:24:05.740718+0000","last_peered":"2024-05-09T09:24:05.740718+0000","last_clean":"2024-05-09T09:24:05.740718+0000","last_became_active":"2024-05-09T09:23:09.297001+0000","last_became_peered":"2024-05-09T09:23:09.297001+0000","last_unstale":"2024-05-09T09:24:05.740718+0000","last_undegraded":"2024-05-09T09:24:05.740718+0000","last_fullsized":"2024-05-09T09:24:05.740718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:19.241362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.583233+0000","last_change":"2024-05-09T09:23:10.338342+0000","last_active":"2024-05-09T09:24:34.583233+0000","last_peered":"2024-05-09T09:24:34.583233+0000","last_clean":"2024-05-09T09:24:34.583233+0000","last_became_active":"2024-05-09T09:23:10.337995+0000","last_became_peered":"2024-05-09T09:23:10.337995+0000","last_unstale":"2024-05-09T09:24:34.583233+0000","last_undegraded":"2024-05-09T09:24:34.583233+0000","last_fullsized":"2024-05-09T09:24:34.583233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.691862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.737440+0000","last_change":"2024-05-09T09:23:14.366237+0000","last_active":"2024-05-09T09:24:05.737440+0000","last_peered":"2024-05-09T09:24:05.737440+0000","last_clean":"2024-05-09T09:24:05.737440+0000","last_became_active":"2024-05-09T09:23:14.365809+0000","last_became_peered":"2024-05-09T09:23:14.365809+0000","last_unstale":"2024-05-09T09:24:05.737440+0000","last_undegraded":"2024-05-09T09:24:05.737440+0000","last_fullsized":"2024-05-09T09:24:05.737440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:28.813467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":172,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:43.695450+0000","last_change":"2024-05-09T09:23:09.298658+0000","last_active":"2024-05-09T09:24:43.695450+0000","last_peered":"2024-05-09T09:24:43.695450+0000","last_clean":"2024-05-09T09:24:43.695450+0000","last_became_active":"2024-05-09T09:23:09.298325+0000","last_became_peered":"2024-05-09T09:23:09.298325+0000","last_unstale":"2024-05-09T09:24:43.695450+0000","last_undegraded":"2024-05-09T09:24:43.695450+0000","last_fullsized":"2024-05-09T09:24:43.695450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:22.108202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":142,"num_read_kb":142,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.600937+0000","last_change":"2024-05-09T09:23:10.326865+0000","last_active":"2024-05-09T09:24:34.600937+0000","last_peered":"2024-05-09T09:24:34.600937+0000","last_clean":"2024-05-09T09:24:34.600937+0000","last_became_active":"2024-05-09T09:23:10.326478+0000","last_became_peered":"2024-05-09T09:23:10.326478+0000","last_unstale":"2024-05-09T09:24:34.600937+0000","last_undegraded":"2024-05-09T09:24:34.600937+0000","last_fullsized":"2024-05-09T09:24:34.600937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:54.521226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.733750+0000","last_change":"2024-05-09T09:23:14.373601+0000","last_active":"2024-05-09T09:24:05.733750+0000","last_peered":"2024-05-09T09:24:05.733750+0000","last_clean":"2024-05-09T09:24:05.733750+0000","last_became_active":"2024-05-09T09:23:14.372764+0000","last_became_peered":"2024-05-09T09:23:14.372764+0000","last_unstale":"2024-05-09T09:24:05.733750+0000","last_undegraded":"2024-05-09T09:24:05.733750+0000","last_fullsized":"2024-05-09T09:24:05.733750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:59.496755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.738453+0000","last_change":"2024-05-09T09:23:09.302293+0000","last_active":"2024-05-09T09:24:05.738453+0000","last_peered":"2024-05-09T09:24:05.738453+0000","last_clean":"2024-05-09T09:24:05.738453+0000","last_became_active":"2024-05-09T09:23:09.301216+0000","last_became_peered":"2024-05-09T09:23:09.301216+0000","last_unstale":"2024-05-09T09:24:05.738453+0000","last_undegraded":"2024-05-09T09:24:05.738453+0000","last_fullsized":"2024-05-09T09:24:05.738453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:35.146860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.599639+0000","last_change":"2024-05-09T09:23:10.339827+0000","last_active":"2024-05-09T09:24:34.599639+0000","last_peered":"2024-05-09T09:24:34.599639+0000","last_clean":"2024-05-09T09:24:34.599639+0000","last_became_active":"2024-05-09T09:23:10.339415+0000","last_became_peered":"2024-05-09T09:23:10.339415+0000","last_unstale":"2024-05-09T09:24:34.599639+0000","last_undegraded":"2024-05-09T09:24:34.599639+0000","last_fullsized":"2024-05-09T09:24:34.599639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:45.250126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732919+0000","last_change":"2024-05-09T09:23:14.377001+0000","last_active":"2024-05-09T09:24:05.732919+0000","last_peered":"2024-05-09T09:24:05.732919+0000","last_clean":"2024-05-09T09:24:05.732919+0000","last_became_active":"2024-05-09T09:23:14.376402+0000","last_became_peered":"2024-05-09T09:23:14.376402+0000","last_unstale":"2024-05-09T09:24:05.732919+0000","last_undegraded":"2024-05-09T09:24:05.732919+0000","last_fullsized":"2024-05-09T09:24:05.732919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:08.142495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.742147+0000","last_change":"2024-05-09T09:23:09.299331+0000","last_active":"2024-05-09T09:24:05.742147+0000","last_peered":"2024-05-09T09:24:05.742147+0000","last_clean":"2024-05-09T09:24:05.742147+0000","last_became_active":"2024-05-09T09:23:09.299017+0000","last_became_peered":"2024-05-09T09:23:09.299017+0000","last_unstale":"2024-05-09T09:24:05.742147+0000","last_undegraded":"2024-05-09T09:24:05.742147+0000","last_fullsized":"2024-05-09T09:24:05.742147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:28.328704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.595671+0000","last_change":"2024-05-09T09:24:05.743586+0000","last_active":"2024-05-09T09:24:34.595671+0000","last_peered":"2024-05-09T09:24:34.595671+0000","last_clean":"2024-05-09T09:24:34.595671+0000","last_became_active":"2024-05-09T09:24:05.742610+0000","last_became_peered":"2024-05-09T09:24:05.742610+0000","last_unstale":"2024-05-09T09:24:34.595671+0000","last_undegraded":"2024-05-09T09:24:34.595671+0000","last_fullsized":"2024-05-09T09:24:34.595671+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:05.586985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.733692+0000","last_change":"2024-05-09T09:23:14.363362+0000","last_active":"2024-05-09T09:24:05.733692+0000","last_peered":"2024-05-09T09:24:05.733692+0000","last_clean":"2024-05-09T09:24:05.733692+0000","last_became_active":"2024-05-09T09:23:14.362926+0000","last_became_peered":"2024-05-09T09:23:14.362926+0000","last_unstale":"2024-05-09T09:24:05.733692+0000","last_undegraded":"2024-05-09T09:24:05.733692+0000","last_fullsized":"2024-05-09T09:24:05.733692+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:25.339727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":213,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:43.695521+0000","last_change":"2024-05-09T09:23:09.297814+0000","last_active":"2024-05-09T09:24:43.695521+0000","last_peered":"2024-05-09T09:24:43.695521+0000","last_clean":"2024-05-09T09:24:43.695521+0000","last_became_active":"2024-05-09T09:23:09.297005+0000","last_became_peered":"2024-05-09T09:23:09.297005+0000","last_unstale":"2024-05-09T09:24:43.695521+0000","last_undegraded":"2024-05-09T09:24:43.695521+0000","last_fullsized":"2024-05-09T09:24:43.695521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:39.240217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":150,"num_read_kb":137,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.581930+0000","last_change":"2024-05-09T09:23:10.333013+0000","last_active":"2024-05-09T09:24:34.581930+0000","last_peered":"2024-05-09T09:24:34.581930+0000","last_clean":"2024-05-09T09:24:34.581930+0000","last_became_active":"2024-05-09T09:23:10.332666+0000","last_became_peered":"2024-05-09T09:23:10.332666+0000","last_unstale":"2024-05-09T09:24:34.581930+0000","last_undegraded":"2024-05-09T09:24:34.581930+0000","last_fullsized":"2024-05-09T09:24:34.581930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:28:50.306551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.742542+0000","last_change":"2024-05-09T09:23:14.367715+0000","last_active":"2024-05-09T09:24:05.742542+0000","last_peered":"2024-05-09T09:24:05.742542+0000","last_clean":"2024-05-09T09:24:05.742542+0000","last_became_active":"2024-05-09T09:23:14.367307+0000","last_became_peered":"2024-05-09T09:23:14.367307+0000","last_unstale":"2024-05-09T09:24:05.742542+0000","last_undegraded":"2024-05-09T09:24:05.742542+0000","last_fullsized":"2024-05-09T09:24:05.742542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:09.075572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732532+0000","last_change":"2024-05-09T09:23:09.306302+0000","last_active":"2024-05-09T09:24:05.732532+0000","last_peered":"2024-05-09T09:24:05.732532+0000","last_clean":"2024-05-09T09:24:05.732532+0000","last_became_active":"2024-05-09T09:23:09.306074+0000","last_became_peered":"2024-05-09T09:23:09.306074+0000","last_unstale":"2024-05-09T09:24:05.732532+0000","last_undegraded":"2024-05-09T09:24:05.732532+0000","last_fullsized":"2024-05-09T09:24:05.732532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:18.434899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.609186+0000","last_change":"2024-05-09T09:23:10.331201+0000","last_active":"2024-05-09T09:24:34.609186+0000","last_peered":"2024-05-09T09:24:34.609186+0000","last_clean":"2024-05-09T09:24:34.609186+0000","last_became_active":"2024-05-09T09:23:10.329830+0000","last_became_peered":"2024-05-09T09:23:10.329830+0000","last_unstale":"2024-05-09T09:24:34.609186+0000","last_undegraded":"2024-05-09T09:24:34.609186+0000","last_fullsized":"2024-05-09T09:24:34.609186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:40.199622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.737803+0000","last_change":"2024-05-09T09:23:14.359095+0000","last_active":"2024-05-09T09:24:05.737803+0000","last_peered":"2024-05-09T09:24:05.737803+0000","last_clean":"2024-05-09T09:24:05.737803+0000","last_became_active":"2024-05-09T09:23:14.358717+0000","last_became_peered":"2024-05-09T09:23:14.358717+0000","last_unstale":"2024-05-09T09:24:05.737803+0000","last_undegraded":"2024-05-09T09:24:05.737803+0000","last_fullsized":"2024-05-09T09:24:05.737803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:16:09.874767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.740939+0000","last_change":"2024-05-09T09:23:09.305370+0000","last_active":"2024-05-09T09:24:05.740939+0000","last_peered":"2024-05-09T09:24:05.740939+0000","last_clean":"2024-05-09T09:24:05.740939+0000","last_became_active":"2024-05-09T09:23:09.305054+0000","last_became_peered":"2024-05-09T09:23:09.305054+0000","last_unstale":"2024-05-09T09:24:05.740939+0000","last_undegraded":"2024-05-09T09:24:05.740939+0000","last_fullsized":"2024-05-09T09:24:05.740939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:28.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.591005+0000","last_change":"2024-05-09T09:23:10.332541+0000","last_active":"2024-05-09T09:24:34.591005+0000","last_peered":"2024-05-09T09:24:34.591005+0000","last_clean":"2024-05-09T09:24:34.591005+0000","last_became_active":"2024-05-09T09:23:10.332155+0000","last_became_peered":"2024-05-09T09:23:10.332155+0000","last_unstale":"2024-05-09T09:24:34.591005+0000","last_undegraded":"2024-05-09T09:24:34.591005+0000","last_fullsized":"2024-05-09T09:24:34.591005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:39.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.734842+0000","last_change":"2024-05-09T09:23:14.374499+0000","last_active":"2024-05-09T09:24:05.734842+0000","last_peered":"2024-05-09T09:24:05.734842+0000","last_clean":"2024-05-09T09:24:05.734842+0000","last_became_active":"2024-05-09T09:23:14.370688+0000","last_became_peered":"2024-05-09T09:23:14.370688+0000","last_unstale":"2024-05-09T09:24:05.734842+0000","last_undegraded":"2024-05-09T09:24:05.734842+0000","last_fullsized":"2024-05-09T09:24:05.734842+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:53.367237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:38.690014+0000","last_change":"2024-05-09T09:23:09.297776+0000","last_active":"2024-05-09T09:24:38.690014+0000","last_peered":"2024-05-09T09:24:38.690014+0000","last_clean":"2024-05-09T09:24:38.690014+0000","last_became_active":"2024-05-09T09:23:09.297407+0000","last_became_peered":"2024-05-09T09:23:09.297407+0000","last_unstale":"2024-05-09T09:24:38.690014+0000","last_undegraded":"2024-05-09T09:24:38.690014+0000","last_fullsized":"2024-05-09T09:24:38.690014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:21.762941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.593679+0000","last_change":"2024-05-09T09:23:10.339127+0000","last_active":"2024-05-09T09:24:34.593679+0000","last_peered":"2024-05-09T09:24:34.593679+0000","last_clean":"2024-05-09T09:24:34.593679+0000","last_became_active":"2024-05-09T09:23:10.338790+0000","last_became_peered":"2024-05-09T09:23:10.338790+0000","last_unstale":"2024-05-09T09:24:34.593679+0000","last_undegraded":"2024-05-09T09:24:34.593679+0000","last_fullsized":"2024-05-09T09:24:34.593679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:36.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743828+0000","last_change":"2024-05-09T09:23:14.373873+0000","last_active":"2024-05-09T09:24:05.743828+0000","last_peered":"2024-05-09T09:24:05.743828+0000","last_clean":"2024-05-09T09:24:05.743828+0000","last_became_active":"2024-05-09T09:23:14.371453+0000","last_became_peered":"2024-05-09T09:23:14.371453+0000","last_unstale":"2024-05-09T09:24:05.743828+0000","last_undegraded":"2024-05-09T09:24:05.743828+0000","last_fullsized":"2024-05-09T09:24:05.743828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:26.885938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.742688+0000","last_change":"2024-05-09T09:23:09.299373+0000","last_active":"2024-05-09T09:24:05.742688+0000","last_peered":"2024-05-09T09:24:05.742688+0000","last_clean":"2024-05-09T09:24:05.742688+0000","last_became_active":"2024-05-09T09:23:09.298926+0000","last_became_peered":"2024-05-09T09:23:09.298926+0000","last_unstale":"2024-05-09T09:24:05.742688+0000","last_undegraded":"2024-05-09T09:24:05.742688+0000","last_fullsized":"2024-05-09T09:24:05.742688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:35.629929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.593074+0000","last_change":"2024-05-09T09:23:10.343911+0000","last_active":"2024-05-09T09:24:34.593074+0000","last_peered":"2024-05-09T09:24:34.593074+0000","last_clean":"2024-05-09T09:24:34.593074+0000","last_became_active":"2024-05-09T09:23:10.343568+0000","last_became_peered":"2024-05-09T09:23:10.343568+0000","last_unstale":"2024-05-09T09:24:34.593074+0000","last_undegraded":"2024-05-09T09:24:34.593074+0000","last_fullsized":"2024-05-09T09:24:34.593074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:07.311554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.740465+0000","last_change":"2024-05-09T09:23:14.368544+0000","last_active":"2024-05-09T09:24:05.740465+0000","last_peered":"2024-05-09T09:24:05.740465+0000","last_clean":"2024-05-09T09:24:05.740465+0000","last_became_active":"2024-05-09T09:23:14.368140+0000","last_became_peered":"2024-05-09T09:23:14.368140+0000","last_unstale":"2024-05-09T09:24:05.740465+0000","last_undegraded":"2024-05-09T09:24:05.740465+0000","last_fullsized":"2024-05-09T09:24:05.740465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.091768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.744278+0000","last_change":"2024-05-09T09:23:09.297612+0000","last_active":"2024-05-09T09:24:05.744278+0000","last_peered":"2024-05-09T09:24:05.744278+0000","last_clean":"2024-05-09T09:24:05.744278+0000","last_became_active":"2024-05-09T09:23:09.297216+0000","last_became_peered":"2024-05-09T09:23:09.297216+0000","last_unstale":"2024-05-09T09:24:05.744278+0000","last_undegraded":"2024-05-09T09:24:05.744278+0000","last_fullsized":"2024-05-09T09:24:05.744278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:09:45.108082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.589108+0000","last_change":"2024-05-09T09:23:10.336514+0000","last_active":"2024-05-09T09:24:34.589108+0000","last_peered":"2024-05-09T09:24:34.589108+0000","last_clean":"2024-05-09T09:24:34.589108+0000","last_became_active":"2024-05-09T09:23:10.336173+0000","last_became_peered":"2024-05-09T09:23:10.336173+0000","last_unstale":"2024-05-09T09:24:34.589108+0000","last_undegraded":"2024-05-09T09:24:34.589108+0000","last_fullsized":"2024-05-09T09:24:34.589108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:46:39.036547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.742861+0000","last_change":"2024-05-09T09:23:14.366119+0000","last_active":"2024-05-09T09:24:05.742861+0000","last_peered":"2024-05-09T09:24:05.742861+0000","last_clean":"2024-05-09T09:24:05.742861+0000","last_became_active":"2024-05-09T09:23:14.365659+0000","last_became_peered":"2024-05-09T09:23:14.365659+0000","last_unstale":"2024-05-09T09:24:05.742861+0000","last_undegraded":"2024-05-09T09:24:05.742861+0000","last_fullsized":"2024-05-09T09:24:05.742861+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:23.312632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.740866+0000","last_change":"2024-05-09T09:24:05.740866+0000","last_active":"2024-05-09T09:24:05.740866+0000","last_peered":"2024-05-09T09:24:05.740866+0000","last_clean":"2024-05-09T09:24:05.740866+0000","last_became_active":"2024-05-09T09:24:05.740418+0000","last_became_peered":"2024-05-09T09:24:05.740418+0000","last_unstale":"2024-05-09T09:24:05.740866+0000","last_undegraded":"2024-05-09T09:24:05.740866+0000","last_fullsized":"2024-05-09T09:24:05.740866+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:45.946040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.580520+0000","last_change":"2024-05-09T09:23:10.339788+0000","last_active":"2024-05-09T09:24:34.580520+0000","last_peered":"2024-05-09T09:24:34.580520+0000","last_clean":"2024-05-09T09:24:34.580520+0000","last_became_active":"2024-05-09T09:23:10.339430+0000","last_became_peered":"2024-05-09T09:23:10.339430+0000","last_unstale":"2024-05-09T09:24:34.580520+0000","last_undegraded":"2024-05-09T09:24:34.580520+0000","last_fullsized":"2024-05-09T09:24:34.580520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:22.467287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743633+0000","last_change":"2024-05-09T09:23:14.368710+0000","last_active":"2024-05-09T09:24:05.743633+0000","last_peered":"2024-05-09T09:24:05.743633+0000","last_clean":"2024-05-09T09:24:05.743633+0000","last_became_active":"2024-05-09T09:23:14.368288+0000","last_became_peered":"2024-05-09T09:23:14.368288+0000","last_unstale":"2024-05-09T09:24:05.743633+0000","last_undegraded":"2024-05-09T09:24:05.743633+0000","last_fullsized":"2024-05-09T09:24:05.743633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:40.941273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":99,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:39.689824+0000","last_change":"2024-05-09T09:23:09.301976+0000","last_active":"2024-05-09T09:24:39.689824+0000","last_peered":"2024-05-09T09:24:39.689824+0000","last_clean":"2024-05-09T09:24:39.689824+0000","last_became_active":"2024-05-09T09:23:09.301494+0000","last_became_peered":"2024-05-09T09:23:09.301494+0000","last_unstale":"2024-05-09T09:24:39.689824+0000","last_undegraded":"2024-05-09T09:24:39.689824+0000","last_fullsized":"2024-05-09T09:24:39.689824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.448246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.577731+0000","last_change":"2024-05-09T09:23:10.331691+0000","last_active":"2024-05-09T09:24:34.577731+0000","last_peered":"2024-05-09T09:24:34.577731+0000","last_clean":"2024-05-09T09:24:34.577731+0000","last_became_active":"2024-05-09T09:23:10.331332+0000","last_became_peered":"2024-05-09T09:23:10.331332+0000","last_unstale":"2024-05-09T09:24:34.577731+0000","last_undegraded":"2024-05-09T09:24:34.577731+0000","last_fullsized":"2024-05-09T09:24:34.577731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:51.643695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743691+0000","last_change":"2024-05-09T09:23:14.367539+0000","last_active":"2024-05-09T09:24:05.743691+0000","last_peered":"2024-05-09T09:24:05.743691+0000","last_clean":"2024-05-09T09:24:05.743691+0000","last_became_active":"2024-05-09T09:23:14.366806+0000","last_became_peered":"2024-05-09T09:23:14.366806+0000","last_unstale":"2024-05-09T09:24:05.743691+0000","last_undegraded":"2024-05-09T09:24:05.743691+0000","last_fullsized":"2024-05-09T09:24:05.743691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:58.679095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.594405+0000","last_change":"2024-05-09T09:23:10.340849+0000","last_active":"2024-05-09T09:24:34.594405+0000","last_peered":"2024-05-09T09:24:34.594405+0000","last_clean":"2024-05-09T09:24:34.594405+0000","last_became_active":"2024-05-09T09:23:10.340430+0000","last_became_peered":"2024-05-09T09:23:10.340430+0000","last_unstale":"2024-05-09T09:24:34.594405+0000","last_undegraded":"2024-05-09T09:24:34.594405+0000","last_fullsized":"2024-05-09T09:24:34.594405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:33:47.827659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":101,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:39.690735+0000","last_change":"2024-05-09T09:23:09.308544+0000","last_active":"2024-05-09T09:24:39.690735+0000","last_peered":"2024-05-09T09:24:39.690735+0000","last_clean":"2024-05-09T09:24:39.690735+0000","last_became_active":"2024-05-09T09:23:09.308195+0000","last_became_peered":"2024-05-09T09:23:09.308195+0000","last_unstale":"2024-05-09T09:24:39.690735+0000","last_undegraded":"2024-05-09T09:24:39.690735+0000","last_fullsized":"2024-05-09T09:24:39.690735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:17:34.884385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.755846+0000","last_change":"2024-05-09T09:24:05.755846+0000","last_active":"2024-05-09T09:24:05.755846+0000","last_peered":"2024-05-09T09:24:05.755846+0000","last_clean":"2024-05-09T09:24:05.755846+0000","last_became_active":"2024-05-09T09:24:05.747246+0000","last_became_peered":"2024-05-09T09:24:05.747246+0000","last_unstale":"2024-05-09T09:24:05.755846+0000","last_undegraded":"2024-05-09T09:24:05.755846+0000","last_fullsized":"2024-05-09T09:24:05.755846+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:23:40.544770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.606229+0000","last_change":"2024-05-09T09:23:10.341320+0000","last_active":"2024-05-09T09:24:34.606229+0000","last_peered":"2024-05-09T09:24:34.606229+0000","last_clean":"2024-05-09T09:24:34.606229+0000","last_became_active":"2024-05-09T09:23:10.340868+0000","last_became_peered":"2024-05-09T09:23:10.340868+0000","last_unstale":"2024-05-09T09:24:34.606229+0000","last_undegraded":"2024-05-09T09:24:34.606229+0000","last_fullsized":"2024-05-09T09:24:34.606229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:55.846642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732460+0000","last_change":"2024-05-09T09:23:09.299574+0000","last_active":"2024-05-09T09:24:05.732460+0000","last_peered":"2024-05-09T09:24:05.732460+0000","last_clean":"2024-05-09T09:24:05.732460+0000","last_became_active":"2024-05-09T09:23:09.298878+0000","last_became_peered":"2024-05-09T09:23:09.298878+0000","last_unstale":"2024-05-09T09:24:05.732460+0000","last_undegraded":"2024-05-09T09:24:05.732460+0000","last_fullsized":"2024-05-09T09:24:05.732460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.275907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.737635+0000","last_change":"2024-05-09T09:21:57.576132+0000","last_active":"2024-05-09T09:24:05.737635+0000","last_peered":"2024-05-09T09:24:05.737635+0000","last_clean":"2024-05-09T09:24:05.737635+0000","last_became_active":"2024-05-09T09:21:57.575602+0000","last_became_peered":"2024-05-09T09:21:57.575602+0000","last_unstale":"2024-05-09T09:24:05.737635+0000","last_undegraded":"2024-05-09T09:24:05.737635+0000","last_fullsized":"2024-05-09T09:24:05.737635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_clean_scrub_stamp":"2024-05-09T09:18:31.335063+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:07.639787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.741624+0000","last_change":"2024-05-09T09:23:14.371184+0000","last_active":"2024-05-09T09:24:05.741624+0000","last_peered":"2024-05-09T09:24:05.741624+0000","last_clean":"2024-05-09T09:24:05.741624+0000","last_became_active":"2024-05-09T09:23:14.370780+0000","last_became_peered":"2024-05-09T09:23:14.370780+0000","last_unstale":"2024-05-09T09:24:05.741624+0000","last_undegraded":"2024-05-09T09:24:05.741624+0000","last_fullsized":"2024-05-09T09:24:05.741624+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:39.361947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.581560+0000","last_change":"2024-05-09T09:23:10.330915+0000","last_active":"2024-05-09T09:24:34.581560+0000","last_peered":"2024-05-09T09:24:34.581560+0000","last_clean":"2024-05-09T09:24:34.581560+0000","last_became_active":"2024-05-09T09:23:10.330458+0000","last_became_peered":"2024-05-09T09:23:10.330458+0000","last_unstale":"2024-05-09T09:24:34.581560+0000","last_undegraded":"2024-05-09T09:24:34.581560+0000","last_fullsized":"2024-05-09T09:24:34.581560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:10.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.738010+0000","last_change":"2024-05-09T09:23:09.303895+0000","last_active":"2024-05-09T09:24:05.738010+0000","last_peered":"2024-05-09T09:24:05.738010+0000","last_clean":"2024-05-09T09:24:05.738010+0000","last_became_active":"2024-05-09T09:23:09.303555+0000","last_became_peered":"2024-05-09T09:23:09.303555+0000","last_unstale":"2024-05-09T09:24:05.738010+0000","last_undegraded":"2024-05-09T09:24:05.738010+0000","last_fullsized":"2024-05-09T09:24:05.738010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:04.350161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.737718+0000","last_change":"2024-05-09T09:23:14.359390+0000","last_active":"2024-05-09T09:24:05.737718+0000","last_peered":"2024-05-09T09:24:05.737718+0000","last_clean":"2024-05-09T09:24:05.737718+0000","last_became_active":"2024-05-09T09:23:14.359000+0000","last_became_peered":"2024-05-09T09:23:14.359000+0000","last_unstale":"2024-05-09T09:24:05.737718+0000","last_undegraded":"2024-05-09T09:24:05.737718+0000","last_fullsized":"2024-05-09T09:24:05.737718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:25.077835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.744023+0000","last_change":"2024-05-09T09:23:09.304535+0000","last_active":"2024-05-09T09:24:05.744023+0000","last_peered":"2024-05-09T09:24:05.744023+0000","last_clean":"2024-05-09T09:24:05.744023+0000","last_became_active":"2024-05-09T09:23:09.304166+0000","last_became_peered":"2024-05-09T09:23:09.304166+0000","last_unstale":"2024-05-09T09:24:05.744023+0000","last_undegraded":"2024-05-09T09:24:05.744023+0000","last_fullsized":"2024-05-09T09:24:05.744023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:03:20.281620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.577760+0000","last_change":"2024-05-09T09:23:10.335657+0000","last_active":"2024-05-09T09:24:34.577760+0000","last_peered":"2024-05-09T09:24:34.577760+0000","last_clean":"2024-05-09T09:24:34.577760+0000","last_became_active":"2024-05-09T09:23:10.335231+0000","last_became_peered":"2024-05-09T09:23:10.335231+0000","last_unstale":"2024-05-09T09:24:34.577760+0000","last_undegraded":"2024-05-09T09:24:34.577760+0000","last_fullsized":"2024-05-09T09:24:34.577760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:07.990165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.736096+0000","last_change":"2024-05-09T09:23:14.366988+0000","last_active":"2024-05-09T09:24:05.736096+0000","last_peered":"2024-05-09T09:24:05.736096+0000","last_clean":"2024-05-09T09:24:05.736096+0000","last_became_active":"2024-05-09T09:23:14.366562+0000","last_became_peered":"2024-05-09T09:23:14.366562+0000","last_unstale":"2024-05-09T09:24:05.736096+0000","last_undegraded":"2024-05-09T09:24:05.736096+0000","last_fullsized":"2024-05-09T09:24:05.736096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:42.670036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'14","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.753460+0000","last_change":"2024-05-09T09:24:05.752748+0000","last_active":"2024-05-09T09:24:05.753460+0000","last_peered":"2024-05-09T09:24:05.753460+0000","last_clean":"2024-05-09T09:24:05.753460+0000","last_became_active":"2024-05-09T09:24:05.743638+0000","last_became_peered":"2024-05-09T09:24:05.743638+0000","last_unstale":"2024-05-09T09:24:05.753460+0000","last_undegraded":"2024-05-09T09:24:05.753460+0000","last_fullsized":"2024-05-09T09:24:05.753460+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:01.834934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.596289+0000","last_change":"2024-05-09T09:23:10.345840+0000","last_active":"2024-05-09T09:24:34.596289+0000","last_peered":"2024-05-09T09:24:34.596289+0000","last_clean":"2024-05-09T09:24:34.596289+0000","last_became_active":"2024-05-09T09:23:10.345487+0000","last_became_peered":"2024-05-09T09:23:10.345487+0000","last_unstale":"2024-05-09T09:24:34.596289+0000","last_undegraded":"2024-05-09T09:24:34.596289+0000","last_fullsized":"2024-05-09T09:24:34.596289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:47.723429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732198+0000","last_change":"2024-05-09T09:23:14.365485+0000","last_active":"2024-05-09T09:24:05.732198+0000","last_peered":"2024-05-09T09:24:05.732198+0000","last_clean":"2024-05-09T09:24:05.732198+0000","last_became_active":"2024-05-09T09:23:14.365018+0000","last_became_peered":"2024-05-09T09:23:14.365018+0000","last_unstale":"2024-05-09T09:24:05.732198+0000","last_undegraded":"2024-05-09T09:24:05.732198+0000","last_fullsized":"2024-05-09T09:24:05.732198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:54:04.712920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732892+0000","last_change":"2024-05-09T09:23:09.302365+0000","last_active":"2024-05-09T09:24:05.732892+0000","last_peered":"2024-05-09T09:24:05.732892+0000","last_clean":"2024-05-09T09:24:05.732892+0000","last_became_active":"2024-05-09T09:23:09.302008+0000","last_became_peered":"2024-05-09T09:23:09.302008+0000","last_unstale":"2024-05-09T09:24:05.732892+0000","last_undegraded":"2024-05-09T09:24:05.732892+0000","last_fullsized":"2024-05-09T09:24:05.732892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:48.829940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.612943+0000","last_change":"2024-05-09T09:23:10.339170+0000","last_active":"2024-05-09T09:24:34.612943+0000","last_peered":"2024-05-09T09:24:34.612943+0000","last_clean":"2024-05-09T09:24:34.612943+0000","last_became_active":"2024-05-09T09:23:10.338829+0000","last_became_peered":"2024-05-09T09:23:10.338829+0000","last_unstale":"2024-05-09T09:24:34.612943+0000","last_undegraded":"2024-05-09T09:24:34.612943+0000","last_fullsized":"2024-05-09T09:24:34.612943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:04.510495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.747227+0000","last_change":"2024-05-09T09:24:05.747227+0000","last_active":"2024-05-09T09:24:05.747227+0000","last_peered":"2024-05-09T09:24:05.747227+0000","last_clean":"2024-05-09T09:24:05.747227+0000","last_became_active":"2024-05-09T09:24:05.746338+0000","last_became_peered":"2024-05-09T09:24:05.746338+0000","last_unstale":"2024-05-09T09:24:05.747227+0000","last_undegraded":"2024-05-09T09:24:05.747227+0000","last_fullsized":"2024-05-09T09:24:05.747227+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:20.899664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.734608+0000","last_change":"2024-05-09T09:23:09.297418+0000","last_active":"2024-05-09T09:24:05.734608+0000","last_peered":"2024-05-09T09:24:05.734608+0000","last_clean":"2024-05-09T09:24:05.734608+0000","last_became_active":"2024-05-09T09:23:09.296963+0000","last_became_peered":"2024-05-09T09:23:09.296963+0000","last_unstale":"2024-05-09T09:24:05.734608+0000","last_undegraded":"2024-05-09T09:24:05.734608+0000","last_fullsized":"2024-05-09T09:24:05.734608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:54.552857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.572545+0000","last_change":"2024-05-09T09:23:10.325882+0000","last_active":"2024-05-09T09:24:34.572545+0000","last_peered":"2024-05-09T09:24:34.572545+0000","last_clean":"2024-05-09T09:24:34.572545+0000","last_became_active":"2024-05-09T09:23:10.325494+0000","last_became_peered":"2024-05-09T09:23:10.325494+0000","last_unstale":"2024-05-09T09:24:34.572545+0000","last_undegraded":"2024-05-09T09:24:34.572545+0000","last_fullsized":"2024-05-09T09:24:34.572545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:27.191310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.735082+0000","last_change":"2024-05-09T09:23:14.375572+0000","last_active":"2024-05-09T09:24:05.735082+0000","last_peered":"2024-05-09T09:24:05.735082+0000","last_clean":"2024-05-09T09:24:05.735082+0000","last_became_active":"2024-05-09T09:23:14.374069+0000","last_became_peered":"2024-05-09T09:23:14.374069+0000","last_unstale":"2024-05-09T09:24:05.735082+0000","last_undegraded":"2024-05-09T09:24:05.735082+0000","last_fullsized":"2024-05-09T09:24:05.735082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:44.045012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.737544+0000","last_change":"2024-05-09T09:23:09.304137+0000","last_active":"2024-05-09T09:24:05.737544+0000","last_peered":"2024-05-09T09:24:05.737544+0000","last_clean":"2024-05-09T09:24:05.737544+0000","last_became_active":"2024-05-09T09:23:09.303762+0000","last_became_peered":"2024-05-09T09:23:09.303762+0000","last_unstale":"2024-05-09T09:24:05.737544+0000","last_undegraded":"2024-05-09T09:24:05.737544+0000","last_fullsized":"2024-05-09T09:24:05.737544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:56.811194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.610846+0000","last_change":"2024-05-09T09:23:10.331998+0000","last_active":"2024-05-09T09:24:34.610846+0000","last_peered":"2024-05-09T09:24:34.610846+0000","last_clean":"2024-05-09T09:24:34.610846+0000","last_became_active":"2024-05-09T09:23:10.331642+0000","last_became_peered":"2024-05-09T09:23:10.331642+0000","last_unstale":"2024-05-09T09:24:34.610846+0000","last_undegraded":"2024-05-09T09:24:34.610846+0000","last_fullsized":"2024-05-09T09:24:34.610846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:41.569300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732445+0000","last_change":"2024-05-09T09:23:14.377444+0000","last_active":"2024-05-09T09:24:05.732445+0000","last_peered":"2024-05-09T09:24:05.732445+0000","last_clean":"2024-05-09T09:24:05.732445+0000","last_became_active":"2024-05-09T09:23:14.376189+0000","last_became_peered":"2024-05-09T09:23:14.376189+0000","last_unstale":"2024-05-09T09:24:05.732445+0000","last_undegraded":"2024-05-09T09:24:05.732445+0000","last_fullsized":"2024-05-09T09:24:05.732445+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:06.973535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732333+0000","last_change":"2024-05-09T09:23:09.304101+0000","last_active":"2024-05-09T09:24:05.732333+0000","last_peered":"2024-05-09T09:24:05.732333+0000","last_clean":"2024-05-09T09:24:05.732333+0000","last_became_active":"2024-05-09T09:23:09.303743+0000","last_became_peered":"2024-05-09T09:23:09.303743+0000","last_unstale":"2024-05-09T09:24:05.732333+0000","last_undegraded":"2024-05-09T09:24:05.732333+0000","last_fullsized":"2024-05-09T09:24:05.732333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:39.339815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.611236+0000","last_change":"2024-05-09T09:23:10.345938+0000","last_active":"2024-05-09T09:24:34.611236+0000","last_peered":"2024-05-09T09:24:34.611236+0000","last_clean":"2024-05-09T09:24:34.611236+0000","last_became_active":"2024-05-09T09:23:10.345576+0000","last_became_peered":"2024-05-09T09:23:10.345576+0000","last_unstale":"2024-05-09T09:24:34.611236+0000","last_undegraded":"2024-05-09T09:24:34.611236+0000","last_fullsized":"2024-05-09T09:24:34.611236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:08.571900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732657+0000","last_change":"2024-05-09T09:23:14.371549+0000","last_active":"2024-05-09T09:24:05.732657+0000","last_peered":"2024-05-09T09:24:05.732657+0000","last_clean":"2024-05-09T09:24:05.732657+0000","last_became_active":"2024-05-09T09:23:14.369931+0000","last_became_peered":"2024-05-09T09:23:14.369931+0000","last_unstale":"2024-05-09T09:24:05.732657+0000","last_undegraded":"2024-05-09T09:24:05.732657+0000","last_fullsized":"2024-05-09T09:24:05.732657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:20.777172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743287+0000","last_change":"2024-05-09T09:23:09.302909+0000","last_active":"2024-05-09T09:24:05.743287+0000","last_peered":"2024-05-09T09:24:05.743287+0000","last_clean":"2024-05-09T09:24:05.743287+0000","last_became_active":"2024-05-09T09:23:09.302570+0000","last_became_peered":"2024-05-09T09:23:09.302570+0000","last_unstale":"2024-05-09T09:24:05.743287+0000","last_undegraded":"2024-05-09T09:24:05.743287+0000","last_fullsized":"2024-05-09T09:24:05.743287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:23.856755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.596848+0000","last_change":"2024-05-09T09:23:10.342676+0000","last_active":"2024-05-09T09:24:34.596848+0000","last_peered":"2024-05-09T09:24:34.596848+0000","last_clean":"2024-05-09T09:24:34.596848+0000","last_became_active":"2024-05-09T09:23:10.341906+0000","last_became_peered":"2024-05-09T09:23:10.341906+0000","last_unstale":"2024-05-09T09:24:34.596848+0000","last_undegraded":"2024-05-09T09:24:34.596848+0000","last_fullsized":"2024-05-09T09:24:34.596848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:59:11.137648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.734734+0000","last_change":"2024-05-09T09:23:14.363779+0000","last_active":"2024-05-09T09:24:05.734734+0000","last_peered":"2024-05-09T09:24:05.734734+0000","last_clean":"2024-05-09T09:24:05.734734+0000","last_became_active":"2024-05-09T09:23:14.363307+0000","last_became_peered":"2024-05-09T09:23:14.363307+0000","last_unstale":"2024-05-09T09:24:05.734734+0000","last_undegraded":"2024-05-09T09:24:05.734734+0000","last_fullsized":"2024-05-09T09:24:05.734734+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:46.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:42.694830+0000","last_change":"2024-05-09T09:23:09.298094+0000","last_active":"2024-05-09T09:24:42.694830+0000","last_peered":"2024-05-09T09:24:42.694830+0000","last_clean":"2024-05-09T09:24:42.694830+0000","last_became_active":"2024-05-09T09:23:09.297777+0000","last_became_peered":"2024-05-09T09:23:09.297777+0000","last_unstale":"2024-05-09T09:24:42.694830+0000","last_undegraded":"2024-05-09T09:24:42.694830+0000","last_fullsized":"2024-05-09T09:24:42.694830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:25.099193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.597200+0000","last_change":"2024-05-09T09:23:10.340120+0000","last_active":"2024-05-09T09:24:34.597200+0000","last_peered":"2024-05-09T09:24:34.597200+0000","last_clean":"2024-05-09T09:24:34.597200+0000","last_became_active":"2024-05-09T09:23:10.339690+0000","last_became_peered":"2024-05-09T09:23:10.339690+0000","last_unstale":"2024-05-09T09:24:34.597200+0000","last_undegraded":"2024-05-09T09:24:34.597200+0000","last_fullsized":"2024-05-09T09:24:34.597200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:36.783411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.744892+0000","last_change":"2024-05-09T09:24:05.744892+0000","last_active":"2024-05-09T09:24:05.744892+0000","last_peered":"2024-05-09T09:24:05.744892+0000","last_clean":"2024-05-09T09:24:05.744892+0000","last_became_active":"2024-05-09T09:24:05.744437+0000","last_became_peered":"2024-05-09T09:24:05.744437+0000","last_unstale":"2024-05-09T09:24:05.744892+0000","last_undegraded":"2024-05-09T09:24:05.744892+0000","last_fullsized":"2024-05-09T09:24:05.744892+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:38:33.709821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.742407+0000","last_change":"2024-05-09T09:23:09.307592+0000","last_active":"2024-05-09T09:24:05.742407+0000","last_peered":"2024-05-09T09:24:05.742407+0000","last_clean":"2024-05-09T09:24:05.742407+0000","last_became_active":"2024-05-09T09:23:09.307341+0000","last_became_peered":"2024-05-09T09:23:09.307341+0000","last_unstale":"2024-05-09T09:24:05.742407+0000","last_undegraded":"2024-05-09T09:24:05.742407+0000","last_fullsized":"2024-05-09T09:24:05.742407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:21.790980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.593713+0000","last_change":"2024-05-09T09:23:10.349217+0000","last_active":"2024-05-09T09:24:34.593713+0000","last_peered":"2024-05-09T09:24:34.593713+0000","last_clean":"2024-05-09T09:24:34.593713+0000","last_became_active":"2024-05-09T09:23:10.348861+0000","last_became_peered":"2024-05-09T09:23:10.348861+0000","last_unstale":"2024-05-09T09:24:34.593713+0000","last_undegraded":"2024-05-09T09:24:34.593713+0000","last_fullsized":"2024-05-09T09:24:34.593713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:51.621045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.739725+0000","last_change":"2024-05-09T09:23:14.363773+0000","last_active":"2024-05-09T09:24:05.739725+0000","last_peered":"2024-05-09T09:24:05.739725+0000","last_clean":"2024-05-09T09:24:05.739725+0000","last_became_active":"2024-05-09T09:23:14.363393+0000","last_became_peered":"2024-05-09T09:23:14.363393+0000","last_unstale":"2024-05-09T09:24:05.739725+0000","last_undegraded":"2024-05-09T09:24:05.739725+0000","last_fullsized":"2024-05-09T09:24:05.739725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:57.322985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743708+0000","last_change":"2024-05-09T09:23:09.308723+0000","last_active":"2024-05-09T09:24:05.743708+0000","last_peered":"2024-05-09T09:24:05.743708+0000","last_clean":"2024-05-09T09:24:05.743708+0000","last_became_active":"2024-05-09T09:23:09.308267+0000","last_became_peered":"2024-05-09T09:23:09.308267+0000","last_unstale":"2024-05-09T09:24:05.743708+0000","last_undegraded":"2024-05-09T09:24:05.743708+0000","last_fullsized":"2024-05-09T09:24:05.743708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:09:25.036421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.588908+0000","last_change":"2024-05-09T09:23:10.342246+0000","last_active":"2024-05-09T09:24:34.588908+0000","last_peered":"2024-05-09T09:24:34.588908+0000","last_clean":"2024-05-09T09:24:34.588908+0000","last_became_active":"2024-05-09T09:23:10.341927+0000","last_became_peered":"2024-05-09T09:23:10.341927+0000","last_unstale":"2024-05-09T09:24:34.588908+0000","last_undegraded":"2024-05-09T09:24:34.588908+0000","last_fullsized":"2024-05-09T09:24:34.588908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:01.182226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.593340+0000","last_change":"2024-05-09T09:23:10.332768+0000","last_active":"2024-05-09T09:24:34.593340+0000","last_peered":"2024-05-09T09:24:34.593340+0000","last_clean":"2024-05-09T09:24:34.593340+0000","last_became_active":"2024-05-09T09:23:10.332441+0000","last_became_peered":"2024-05-09T09:23:10.332441+0000","last_unstale":"2024-05-09T09:24:34.593340+0000","last_undegraded":"2024-05-09T09:24:34.593340+0000","last_fullsized":"2024-05-09T09:24:34.593340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:53.377058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.590074+0000","last_change":"2024-05-09T09:23:10.337944+0000","last_active":"2024-05-09T09:24:34.590074+0000","last_peered":"2024-05-09T09:24:34.590074+0000","last_clean":"2024-05-09T09:24:34.590074+0000","last_became_active":"2024-05-09T09:23:10.337593+0000","last_became_peered":"2024-05-09T09:23:10.337593+0000","last_unstale":"2024-05-09T09:24:34.590074+0000","last_undegraded":"2024-05-09T09:24:34.590074+0000","last_fullsized":"2024-05-09T09:24:34.590074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:20.381543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.610346+0000","last_change":"2024-05-09T09:23:10.337419+0000","last_active":"2024-05-09T09:24:34.610346+0000","last_peered":"2024-05-09T09:24:34.610346+0000","last_clean":"2024-05-09T09:24:34.610346+0000","last_became_active":"2024-05-09T09:23:10.336718+0000","last_became_peered":"2024-05-09T09:23:10.336718+0000","last_unstale":"2024-05-09T09:24:34.610346+0000","last_undegraded":"2024-05-09T09:24:34.610346+0000","last_fullsized":"2024-05-09T09:24:34.610346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:50.101110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.599871+0000","last_change":"2024-05-09T09:23:10.340243+0000","last_active":"2024-05-09T09:24:34.599871+0000","last_peered":"2024-05-09T09:24:34.599871+0000","last_clean":"2024-05-09T09:24:34.599871+0000","last_became_active":"2024-05-09T09:23:10.339919+0000","last_became_peered":"2024-05-09T09:23:10.339919+0000","last_unstale":"2024-05-09T09:24:34.599871+0000","last_undegraded":"2024-05-09T09:24:34.599871+0000","last_fullsized":"2024-05-09T09:24:34.599871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:21.460928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.613410+0000","last_change":"2024-05-09T09:23:10.333136+0000","last_active":"2024-05-09T09:24:34.613410+0000","last_peered":"2024-05-09T09:24:34.613410+0000","last_clean":"2024-05-09T09:24:34.613410+0000","last_became_active":"2024-05-09T09:23:10.332799+0000","last_became_peered":"2024-05-09T09:23:10.332799+0000","last_unstale":"2024-05-09T09:24:34.613410+0000","last_undegraded":"2024-05-09T09:24:34.613410+0000","last_fullsized":"2024-05-09T09:24:34.613410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:50.682747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.587631+0000","last_change":"2024-05-09T09:24:05.745143+0000","last_active":"2024-05-09T09:24:34.587631+0000","last_peered":"2024-05-09T09:24:34.587631+0000","last_clean":"2024-05-09T09:24:34.587631+0000","last_became_active":"2024-05-09T09:24:05.744117+0000","last_became_peered":"2024-05-09T09:24:05.744117+0000","last_unstale":"2024-05-09T09:24:34.587631+0000","last_undegraded":"2024-05-09T09:24:34.587631+0000","last_fullsized":"2024-05-09T09:24:34.587631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:25:09.853820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.581573+0000","last_change":"2024-05-09T09:23:10.341776+0000","last_active":"2024-05-09T09:24:34.581573+0000","last_peered":"2024-05-09T09:24:34.581573+0000","last_clean":"2024-05-09T09:24:34.581573+0000","last_became_active":"2024-05-09T09:23:10.341436+0000","last_became_peered":"2024-05-09T09:23:10.341436+0000","last_unstale":"2024-05-09T09:24:34.581573+0000","last_undegraded":"2024-05-09T09:24:34.581573+0000","last_fullsized":"2024-05-09T09:24:34.581573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:42.178856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.604854+0000","last_change":"2024-05-09T09:24:05.745324+0000","last_active":"2024-05-09T09:24:34.604854+0000","last_peered":"2024-05-09T09:24:34.604854+0000","last_clean":"2024-05-09T09:24:34.604854+0000","last_became_active":"2024-05-09T09:24:05.744899+0000","last_became_peered":"2024-05-09T09:24:05.744899+0000","last_unstale":"2024-05-09T09:24:34.604854+0000","last_undegraded":"2024-05-09T09:24:34.604854+0000","last_fullsized":"2024-05-09T09:24:34.604854+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:59.372367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.602858+0000","last_change":"2024-05-09T09:23:10.337117+0000","last_active":"2024-05-09T09:24:34.602858+0000","last_peered":"2024-05-09T09:24:34.602858+0000","last_clean":"2024-05-09T09:24:34.602858+0000","last_became_active":"2024-05-09T09:23:10.336672+0000","last_became_peered":"2024-05-09T09:23:10.336672+0000","last_unstale":"2024-05-09T09:24:34.602858+0000","last_undegraded":"2024-05-09T09:24:34.602858+0000","last_fullsized":"2024-05-09T09:24:34.602858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:59:20.723742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.605575+0000","last_change":"2024-05-09T09:23:10.337733+0000","last_active":"2024-05-09T09:24:34.605575+0000","last_peered":"2024-05-09T09:24:34.605575+0000","last_clean":"2024-05-09T09:24:34.605575+0000","last_became_active":"2024-05-09T09:23:10.337343+0000","last_became_peered":"2024-05-09T09:23:10.337343+0000","last_unstale":"2024-05-09T09:24:34.605575+0000","last_undegraded":"2024-05-09T09:24:34.605575+0000","last_fullsized":"2024-05-09T09:24:34.605575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:53.447911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.609040+0000","last_change":"2024-05-09T09:23:10.337485+0000","last_active":"2024-05-09T09:24:34.609040+0000","last_peered":"2024-05-09T09:24:34.609040+0000","last_clean":"2024-05-09T09:24:34.609040+0000","last_became_active":"2024-05-09T09:23:10.337129+0000","last_became_peered":"2024-05-09T09:23:10.337129+0000","last_unstale":"2024-05-09T09:24:34.609040+0000","last_undegraded":"2024-05-09T09:24:34.609040+0000","last_fullsized":"2024-05-09T09:24:34.609040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:20.758552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.613415+0000","last_change":"2024-05-09T09:23:10.332271+0000","last_active":"2024-05-09T09:24:34.613415+0000","last_peered":"2024-05-09T09:24:34.613415+0000","last_clean":"2024-05-09T09:24:34.613415+0000","last_became_active":"2024-05-09T09:23:10.331941+0000","last_became_peered":"2024-05-09T09:23:10.331941+0000","last_unstale":"2024-05-09T09:24:34.613415+0000","last_undegraded":"2024-05-09T09:24:34.613415+0000","last_fullsized":"2024-05-09T09:24:34.613415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:27.707554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.590904+0000","last_change":"2024-05-09T09:23:10.328965+0000","last_active":"2024-05-09T09:24:34.590904+0000","last_peered":"2024-05-09T09:24:34.590904+0000","last_clean":"2024-05-09T09:24:34.590904+0000","last_became_active":"2024-05-09T09:23:10.328618+0000","last_became_peered":"2024-05-09T09:23:10.328618+0000","last_unstale":"2024-05-09T09:24:34.590904+0000","last_undegraded":"2024-05-09T09:24:34.590904+0000","last_fullsized":"2024-05-09T09:24:34.590904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:16:02.003378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70559,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":736,"num_read_kb":447,"num_write":249,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383712,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":501047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1156096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":500977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":501355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1097993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":496481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":561116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":497937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40592,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704560,"statfs":{"total":95995035648,"available":95953469440,"internally_reserved":0,"allocated":1122304,"data_stored":561336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":496189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":501087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1093043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:24:46.767 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-09T09:24:47.063 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:47.358 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:47 smithi012 ceph-mon[30140]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:24:47.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:47 smithi170 ceph-mon[33632]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:24:47.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:47 smithi016 ceph-mon[33335]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:24:48.395 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:48 smithi012 ceph-mon[30140]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T09:24:48.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:48 smithi170 ceph-mon[33632]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T09:24:48.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:48 smithi016 ceph-mon[33335]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T09:24:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.947+0000 7f49144eb700 1 -- 172.21.15.12:0/2235062179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f490c101e50 msgr2=0x7f490c105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.947+0000 7f49144eb700 1 --2- 172.21.15.12:0/2235062179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f490c101e50 0x7f490c105ea0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0099c0 tx=0x7f48fc009cd0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.947+0000 7f49144eb700 1 -- 172.21.15.12:0/2235062179 shutdown_connections 2024-05-09T09:24:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.947+0000 7f49144eb700 1 --2- 172.21.15.12:0/2235062179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c106580 0x7f490c10e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.947+0000 7f49144eb700 1 --2- 172.21.15.12:0/2235062179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f490c1014a0 0x7f490c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.947+0000 7f49144eb700 1 --2- 172.21.15.12:0/2235062179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f490c101e50 0x7f490c105ea0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.947+0000 7f49144eb700 1 -- 172.21.15.12:0/2235062179 >> 172.21.15.12:0/2235062179 conn(0x7f490c0782c0 msgr2=0x7f490c0786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.947+0000 7f49144eb700 1 -- 172.21.15.12:0/2235062179 shutdown_connections 2024-05-09T09:24:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.947+0000 7f49144eb700 1 -- 172.21.15.12:0/2235062179 wait complete. 2024-05-09T09:24:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.948+0000 7f49144eb700 1 Processor -- start 2024-05-09T09:24:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.948+0000 7f49144eb700 1 -- start start 2024-05-09T09:24:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.948+0000 7f49144eb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f490c1014a0 0x7f490c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.948+0000 7f49144eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f490c101e50 0x7f490c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.948+0000 7f49144eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c106580 0x7f490c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.948+0000 7f49144eb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f490c112640 con 0x7f490c101e50 2024-05-09T09:24:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.948+0000 7f49144eb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1124c0 con 0x7f490c106580 2024-05-09T09:24:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.948+0000 7f49144eb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1127c0 con 0x7f490c1014a0 2024-05-09T09:24:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.948+0000 7f4911a86700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f490c101e50 0x7f490c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.948+0000 7f4911a86700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f490c101e50 0x7f490c19ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47172/0 (socket says 172.21.15.12:47172) 2024-05-09T09:24:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.948+0000 7f4911a86700 1 -- 172.21.15.12:0/811786151 learned_addr learned my addr 172.21.15.12:0/811786151 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.948+0000 7f4912287700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f490c1014a0 0x7f490c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.948+0000 7f4912a88700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c106580 0x7f490c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.949+0000 7f4911a86700 1 -- 172.21.15.12:0/811786151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f490c1014a0 msgr2=0x7f490c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.949+0000 7f4911a86700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f490c1014a0 0x7f490c19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.949+0000 7f4911a86700 1 -- 172.21.15.12:0/811786151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c106580 msgr2=0x7f490c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.949+0000 7f4911a86700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c106580 0x7f490c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.949+0000 7f4911a86700 1 -- 172.21.15.12:0/811786151 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c1990e0 con 0x7f490c101e50 2024-05-09T09:24:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.949+0000 7f4911a86700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f490c101e50 0x7f490c19ed00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0059d0 tx=0x7f48fc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.949+0000 7f49037fe700 1 -- 172.21.15.12:0/811786151 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc027070 con 0x7f490c101e50 2024-05-09T09:24:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.949+0000 7f49037fe700 1 -- 172.21.15.12:0/811786151 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f48fc0040b0 con 0x7f490c101e50 2024-05-09T09:24:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.949+0000 7f49144eb700 1 -- 172.21.15.12:0/811786151 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c199370 con 0x7f490c101e50 2024-05-09T09:24:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.949+0000 7f49037fe700 1 -- 172.21.15.12:0/811786151 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc022d20 con 0x7f490c101e50 2024-05-09T09:24:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.949+0000 7f49144eb700 1 -- 172.21.15.12:0/811786151 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c1a5690 con 0x7f490c101e50 2024-05-09T09:24:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.950+0000 7f49144eb700 1 -- 172.21.15.12:0/811786151 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f490c102980 con 0x7f490c101e50 2024-05-09T09:24:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.951+0000 7f49037fe700 1 -- 172.21.15.12:0/811786151 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f48fc02a070 con 0x7f490c101e50 2024-05-09T09:24:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.952+0000 7f49037fe700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f48f80a2c20 0x7f48f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.952+0000 7f49037fe700 1 -- 172.21.15.12:0/811786151 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f48fc0d0410 con 0x7f490c101e50 2024-05-09T09:24:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.952+0000 7f4912287700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f48f80a2c20 0x7f48f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.952+0000 7f4912287700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f48f80a2c20 0x7f48f80a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f490c199ac0 tx=0x7f490800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:48.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:48.957+0000 7f49037fe700 1 -- 172.21.15.12:0/811786151 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f48fc05e740 con 0x7f490c101e50 2024-05-09T09:24:49.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.162+0000 7f49144eb700 1 -- 172.21.15.12:0/811786151 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f490c199eb0 con 0x7f48f80a2c20 2024-05-09T09:24:49.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.171+0000 7f49037fe700 1 -- 172.21.15.12:0/811786151 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340527 (secure 0 0 0) 0x7f490c199eb0 con 0x7f48f80a2c20 2024-05-09T09:24:49.175 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:24:49.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.174+0000 7f49144eb700 1 -- 172.21.15.12:0/811786151 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f48f80a2c20 msgr2=0x7f48f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.174+0000 7f49144eb700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f48f80a2c20 0x7f48f80a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f490c199ac0 tx=0x7f490800b410 comp rx=0 tx=0).stop 2024-05-09T09:24:49.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.174+0000 7f49144eb700 1 -- 172.21.15.12:0/811786151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f490c101e50 msgr2=0x7f490c19ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.174+0000 7f49144eb700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f490c101e50 0x7f490c19ed00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0059d0 tx=0x7f48fc0057c0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.174+0000 7f49144eb700 1 -- 172.21.15.12:0/811786151 shutdown_connections 2024-05-09T09:24:49.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.174+0000 7f49144eb700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f48f80a2c20 0x7f48f80a50e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.174+0000 7f49144eb700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c106580 0x7f490c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.174+0000 7f49144eb700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f490c1014a0 0x7f490c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.174+0000 7f49144eb700 1 --2- 172.21.15.12:0/811786151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f490c101e50 0x7f490c19ed00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.174+0000 7f49144eb700 1 -- 172.21.15.12:0/811786151 >> 172.21.15.12:0/811786151 conn(0x7f490c0782c0 msgr2=0x7f490c0ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:49.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.174+0000 7f49144eb700 1 -- 172.21.15.12:0/811786151 shutdown_connections 2024-05-09T09:24:49.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:49.174+0000 7f49144eb700 1 -- 172.21.15.12:0/811786151 wait complete. 2024-05-09T09:24:49.179 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-09T09:24:49.940 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-05-09T09:24:48.577575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661161,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":882,"num_read_kb":566,"num_write":432,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392512,"kb_used_data":14240,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549312,"statfs":{"total":1151940427776,"available":1151538495488,"internally_reserved":0,"allocated":14581760,"data_stored":7964657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":56,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003533"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.610993+0000","last_change":"2024-05-09T09:23:10.334738+0000","last_active":"2024-05-09T09:24:34.610993+0000","last_peered":"2024-05-09T09:24:34.610993+0000","last_clean":"2024-05-09T09:24:34.610993+0000","last_became_active":"2024-05-09T09:23:10.333406+0000","last_became_peered":"2024-05-09T09:23:10.333406+0000","last_unstale":"2024-05-09T09:24:34.610993+0000","last_undegraded":"2024-05-09T09:24:34.610993+0000","last_fullsized":"2024-05-09T09:24:34.610993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:44.957557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.609671+0000","last_change":"2024-05-09T09:23:10.331860+0000","last_active":"2024-05-09T09:24:34.609671+0000","last_peered":"2024-05-09T09:24:34.609671+0000","last_clean":"2024-05-09T09:24:34.609671+0000","last_became_active":"2024-05-09T09:23:10.331396+0000","last_became_peered":"2024-05-09T09:23:10.331396+0000","last_unstale":"2024-05-09T09:24:34.609671+0000","last_undegraded":"2024-05-09T09:24:34.609671+0000","last_fullsized":"2024-05-09T09:24:34.609671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:47.676399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.611885+0000","last_change":"2024-05-09T09:23:10.336543+0000","last_active":"2024-05-09T09:24:34.611885+0000","last_peered":"2024-05-09T09:24:34.611885+0000","last_clean":"2024-05-09T09:24:34.611885+0000","last_became_active":"2024-05-09T09:23:10.336189+0000","last_became_peered":"2024-05-09T09:23:10.336189+0000","last_unstale":"2024-05-09T09:24:34.611885+0000","last_undegraded":"2024-05-09T09:24:34.611885+0000","last_fullsized":"2024-05-09T09:24:34.611885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:06.419177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.576657+0000","last_change":"2024-05-09T09:23:10.331739+0000","last_active":"2024-05-09T09:24:34.576657+0000","last_peered":"2024-05-09T09:24:34.576657+0000","last_clean":"2024-05-09T09:24:34.576657+0000","last_became_active":"2024-05-09T09:23:10.331333+0000","last_became_peered":"2024-05-09T09:23:10.331333+0000","last_unstale":"2024-05-09T09:24:34.576657+0000","last_undegraded":"2024-05-09T09:24:34.576657+0000","last_fullsized":"2024-05-09T09:24:34.576657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:04.925697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.581434+0000","last_change":"2024-05-09T09:23:10.345294+0000","last_active":"2024-05-09T09:24:34.581434+0000","last_peered":"2024-05-09T09:24:34.581434+0000","last_clean":"2024-05-09T09:24:34.581434+0000","last_became_active":"2024-05-09T09:23:10.344940+0000","last_became_peered":"2024-05-09T09:23:10.344940+0000","last_unstale":"2024-05-09T09:24:34.581434+0000","last_undegraded":"2024-05-09T09:24:34.581434+0000","last_fullsized":"2024-05-09T09:24:34.581434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:41.150284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.579544+0000","last_change":"2024-05-09T09:23:10.329143+0000","last_active":"2024-05-09T09:24:34.579544+0000","last_peered":"2024-05-09T09:24:34.579544+0000","last_clean":"2024-05-09T09:24:34.579544+0000","last_became_active":"2024-05-09T09:23:10.328712+0000","last_became_peered":"2024-05-09T09:23:10.328712+0000","last_unstale":"2024-05-09T09:24:34.579544+0000","last_undegraded":"2024-05-09T09:24:34.579544+0000","last_fullsized":"2024-05-09T09:24:34.579544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:10:35.085759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.593087+0000","last_change":"2024-05-09T09:23:10.338176+0000","last_active":"2024-05-09T09:24:34.593087+0000","last_peered":"2024-05-09T09:24:34.593087+0000","last_clean":"2024-05-09T09:24:34.593087+0000","last_became_active":"2024-05-09T09:23:10.337845+0000","last_became_peered":"2024-05-09T09:23:10.337845+0000","last_unstale":"2024-05-09T09:24:34.593087+0000","last_undegraded":"2024-05-09T09:24:34.593087+0000","last_fullsized":"2024-05-09T09:24:34.593087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:34:21.783295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.604691+0000","last_change":"2024-05-09T09:23:10.342473+0000","last_active":"2024-05-09T09:24:34.604691+0000","last_peered":"2024-05-09T09:24:34.604691+0000","last_clean":"2024-05-09T09:24:34.604691+0000","last_became_active":"2024-05-09T09:23:10.340682+0000","last_became_peered":"2024-05-09T09:23:10.340682+0000","last_unstale":"2024-05-09T09:24:34.604691+0000","last_undegraded":"2024-05-09T09:24:34.604691+0000","last_fullsized":"2024-05-09T09:24:34.604691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:56.730332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.596347+0000","last_change":"2024-05-09T09:23:10.339888+0000","last_active":"2024-05-09T09:24:34.596347+0000","last_peered":"2024-05-09T09:24:34.596347+0000","last_clean":"2024-05-09T09:24:34.596347+0000","last_became_active":"2024-05-09T09:23:10.337849+0000","last_became_peered":"2024-05-09T09:23:10.337849+0000","last_unstale":"2024-05-09T09:24:34.596347+0000","last_undegraded":"2024-05-09T09:24:34.596347+0000","last_fullsized":"2024-05-09T09:24:34.596347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:48.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.586699+0000","last_change":"2024-05-09T09:23:10.332395+0000","last_active":"2024-05-09T09:24:34.586699+0000","last_peered":"2024-05-09T09:24:34.586699+0000","last_clean":"2024-05-09T09:24:34.586699+0000","last_became_active":"2024-05-09T09:23:10.332040+0000","last_became_peered":"2024-05-09T09:23:10.332040+0000","last_unstale":"2024-05-09T09:24:34.586699+0000","last_undegraded":"2024-05-09T09:24:34.586699+0000","last_fullsized":"2024-05-09T09:24:34.586699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:04.036871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.597071+0000","last_change":"2024-05-09T09:23:10.341148+0000","last_active":"2024-05-09T09:24:34.597071+0000","last_peered":"2024-05-09T09:24:34.597071+0000","last_clean":"2024-05-09T09:24:34.597071+0000","last_became_active":"2024-05-09T09:23:10.340799+0000","last_became_peered":"2024-05-09T09:23:10.340799+0000","last_unstale":"2024-05-09T09:24:34.597071+0000","last_undegraded":"2024-05-09T09:24:34.597071+0000","last_fullsized":"2024-05-09T09:24:34.597071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:31.020659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.577617+0000","last_change":"2024-05-09T09:23:10.336191+0000","last_active":"2024-05-09T09:24:34.577617+0000","last_peered":"2024-05-09T09:24:34.577617+0000","last_clean":"2024-05-09T09:24:34.577617+0000","last_became_active":"2024-05-09T09:23:10.333591+0000","last_became_peered":"2024-05-09T09:23:10.333591+0000","last_unstale":"2024-05-09T09:24:34.577617+0000","last_undegraded":"2024-05-09T09:24:34.577617+0000","last_fullsized":"2024-05-09T09:24:34.577617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:24.836954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.599143+0000","last_change":"2024-05-09T09:23:10.314506+0000","last_active":"2024-05-09T09:24:34.599143+0000","last_peered":"2024-05-09T09:24:34.599143+0000","last_clean":"2024-05-09T09:24:34.599143+0000","last_became_active":"2024-05-09T09:23:10.314171+0000","last_became_peered":"2024-05-09T09:23:10.314171+0000","last_unstale":"2024-05-09T09:24:34.599143+0000","last_undegraded":"2024-05-09T09:24:34.599143+0000","last_fullsized":"2024-05-09T09:24:34.599143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:51.088814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.592081+0000","last_change":"2024-05-09T09:23:10.341670+0000","last_active":"2024-05-09T09:24:34.592081+0000","last_peered":"2024-05-09T09:24:34.592081+0000","last_clean":"2024-05-09T09:24:34.592081+0000","last_became_active":"2024-05-09T09:23:10.341155+0000","last_became_peered":"2024-05-09T09:23:10.341155+0000","last_unstale":"2024-05-09T09:24:34.592081+0000","last_undegraded":"2024-05-09T09:24:34.592081+0000","last_fullsized":"2024-05-09T09:24:34.592081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:14:32.734302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.611616+0000","last_change":"2024-05-09T09:24:05.742117+0000","last_active":"2024-05-09T09:24:34.611616+0000","last_peered":"2024-05-09T09:24:34.611616+0000","last_clean":"2024-05-09T09:24:34.611616+0000","last_became_active":"2024-05-09T09:24:05.741655+0000","last_became_peered":"2024-05-09T09:24:05.741655+0000","last_unstale":"2024-05-09T09:24:34.611616+0000","last_undegraded":"2024-05-09T09:24:34.611616+0000","last_fullsized":"2024-05-09T09:24:34.611616+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:11.468527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.594547+0000","last_change":"2024-05-09T09:23:10.337367+0000","last_active":"2024-05-09T09:24:34.594547+0000","last_peered":"2024-05-09T09:24:34.594547+0000","last_clean":"2024-05-09T09:24:34.594547+0000","last_became_active":"2024-05-09T09:23:10.337000+0000","last_became_peered":"2024-05-09T09:23:10.337000+0000","last_unstale":"2024-05-09T09:24:34.594547+0000","last_undegraded":"2024-05-09T09:24:34.594547+0000","last_fullsized":"2024-05-09T09:24:34.594547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:54.431820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.612492+0000","last_change":"2024-05-09T09:23:10.334293+0000","last_active":"2024-05-09T09:24:34.612492+0000","last_peered":"2024-05-09T09:24:34.612492+0000","last_clean":"2024-05-09T09:24:34.612492+0000","last_became_active":"2024-05-09T09:23:10.333938+0000","last_became_peered":"2024-05-09T09:23:10.333938+0000","last_unstale":"2024-05-09T09:24:34.612492+0000","last_undegraded":"2024-05-09T09:24:34.612492+0000","last_fullsized":"2024-05-09T09:24:34.612492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:56:09.048848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.600158+0000","last_change":"2024-05-09T09:23:10.332394+0000","last_active":"2024-05-09T09:24:34.600158+0000","last_peered":"2024-05-09T09:24:34.600158+0000","last_clean":"2024-05-09T09:24:34.600158+0000","last_became_active":"2024-05-09T09:23:10.332054+0000","last_became_peered":"2024-05-09T09:23:10.332054+0000","last_unstale":"2024-05-09T09:24:34.600158+0000","last_undegraded":"2024-05-09T09:24:34.600158+0000","last_fullsized":"2024-05-09T09:24:34.600158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:05.628740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.583441+0000","last_change":"2024-05-09T09:23:10.336189+0000","last_active":"2024-05-09T09:24:34.583441+0000","last_peered":"2024-05-09T09:24:34.583441+0000","last_clean":"2024-05-09T09:24:34.583441+0000","last_became_active":"2024-05-09T09:23:10.335698+0000","last_became_peered":"2024-05-09T09:23:10.335698+0000","last_unstale":"2024-05-09T09:24:34.583441+0000","last_undegraded":"2024-05-09T09:24:34.583441+0000","last_fullsized":"2024-05-09T09:24:34.583441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:01.403918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732540+0000","last_change":"2024-05-09T09:23:14.372134+0000","last_active":"2024-05-09T09:24:05.732540+0000","last_peered":"2024-05-09T09:24:05.732540+0000","last_clean":"2024-05-09T09:24:05.732540+0000","last_became_active":"2024-05-09T09:23:14.371466+0000","last_became_peered":"2024-05-09T09:23:14.371466+0000","last_unstale":"2024-05-09T09:24:05.732540+0000","last_undegraded":"2024-05-09T09:24:05.732540+0000","last_fullsized":"2024-05-09T09:24:05.732540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:22.048565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:46.698764+0000","last_change":"2024-05-09T09:23:09.304421+0000","last_active":"2024-05-09T09:24:46.698764+0000","last_peered":"2024-05-09T09:24:46.698764+0000","last_clean":"2024-05-09T09:24:46.698764+0000","last_became_active":"2024-05-09T09:23:09.304109+0000","last_became_peered":"2024-05-09T09:23:09.304109+0000","last_unstale":"2024-05-09T09:24:46.698764+0000","last_undegraded":"2024-05-09T09:24:46.698764+0000","last_fullsized":"2024-05-09T09:24:46.698764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:36.796458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.603058+0000","last_change":"2024-05-09T09:23:10.341875+0000","last_active":"2024-05-09T09:24:34.603058+0000","last_peered":"2024-05-09T09:24:34.603058+0000","last_clean":"2024-05-09T09:24:34.603058+0000","last_became_active":"2024-05-09T09:23:10.341385+0000","last_became_peered":"2024-05-09T09:23:10.341385+0000","last_unstale":"2024-05-09T09:24:34.603058+0000","last_undegraded":"2024-05-09T09:24:34.603058+0000","last_fullsized":"2024-05-09T09:24:34.603058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:44:33.939722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.731962+0000","last_change":"2024-05-09T09:23:14.371120+0000","last_active":"2024-05-09T09:24:05.731962+0000","last_peered":"2024-05-09T09:24:05.731962+0000","last_clean":"2024-05-09T09:24:05.731962+0000","last_became_active":"2024-05-09T09:23:14.370663+0000","last_became_peered":"2024-05-09T09:23:14.370663+0000","last_unstale":"2024-05-09T09:24:05.731962+0000","last_undegraded":"2024-05-09T09:24:05.731962+0000","last_fullsized":"2024-05-09T09:24:05.731962+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:50:06.553518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.737188+0000","last_change":"2024-05-09T09:23:09.304357+0000","last_active":"2024-05-09T09:24:05.737188+0000","last_peered":"2024-05-09T09:24:05.737188+0000","last_clean":"2024-05-09T09:24:05.737188+0000","last_became_active":"2024-05-09T09:23:09.304007+0000","last_became_peered":"2024-05-09T09:23:09.304007+0000","last_unstale":"2024-05-09T09:24:05.737188+0000","last_undegraded":"2024-05-09T09:24:05.737188+0000","last_fullsized":"2024-05-09T09:24:05.737188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:32:46.132287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.611050+0000","last_change":"2024-05-09T09:23:10.339533+0000","last_active":"2024-05-09T09:24:34.611050+0000","last_peered":"2024-05-09T09:24:34.611050+0000","last_clean":"2024-05-09T09:24:34.611050+0000","last_became_active":"2024-05-09T09:23:10.337142+0000","last_became_peered":"2024-05-09T09:23:10.337142+0000","last_unstale":"2024-05-09T09:24:34.611050+0000","last_undegraded":"2024-05-09T09:24:34.611050+0000","last_fullsized":"2024-05-09T09:24:34.611050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:11:11.921491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.744009+0000","last_change":"2024-05-09T09:24:05.744009+0000","last_active":"2024-05-09T09:24:05.744009+0000","last_peered":"2024-05-09T09:24:05.744009+0000","last_clean":"2024-05-09T09:24:05.744009+0000","last_became_active":"2024-05-09T09:24:05.743611+0000","last_became_peered":"2024-05-09T09:24:05.743611+0000","last_unstale":"2024-05-09T09:24:05.744009+0000","last_undegraded":"2024-05-09T09:24:05.744009+0000","last_fullsized":"2024-05-09T09:24:05.744009+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:55.775740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.744187+0000","last_change":"2024-05-09T09:23:09.300754+0000","last_active":"2024-05-09T09:24:05.744187+0000","last_peered":"2024-05-09T09:24:05.744187+0000","last_clean":"2024-05-09T09:24:05.744187+0000","last_became_active":"2024-05-09T09:23:09.300419+0000","last_became_peered":"2024-05-09T09:23:09.300419+0000","last_unstale":"2024-05-09T09:24:05.744187+0000","last_undegraded":"2024-05-09T09:24:05.744187+0000","last_fullsized":"2024-05-09T09:24:05.744187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:06.205582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.596629+0000","last_change":"2024-05-09T09:23:10.331952+0000","last_active":"2024-05-09T09:24:34.596629+0000","last_peered":"2024-05-09T09:24:34.596629+0000","last_clean":"2024-05-09T09:24:34.596629+0000","last_became_active":"2024-05-09T09:23:10.331515+0000","last_became_peered":"2024-05-09T09:23:10.331515+0000","last_unstale":"2024-05-09T09:24:34.596629+0000","last_undegraded":"2024-05-09T09:24:34.596629+0000","last_fullsized":"2024-05-09T09:24:34.596629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:23.809672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732036+0000","last_change":"2024-05-09T09:23:14.368518+0000","last_active":"2024-05-09T09:24:05.732036+0000","last_peered":"2024-05-09T09:24:05.732036+0000","last_clean":"2024-05-09T09:24:05.732036+0000","last_became_active":"2024-05-09T09:23:14.367995+0000","last_became_peered":"2024-05-09T09:23:14.367995+0000","last_unstale":"2024-05-09T09:24:05.732036+0000","last_undegraded":"2024-05-09T09:24:05.732036+0000","last_fullsized":"2024-05-09T09:24:05.732036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:11.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743850+0000","last_change":"2024-05-09T09:23:09.299656+0000","last_active":"2024-05-09T09:24:05.743850+0000","last_peered":"2024-05-09T09:24:05.743850+0000","last_clean":"2024-05-09T09:24:05.743850+0000","last_became_active":"2024-05-09T09:23:09.299374+0000","last_became_peered":"2024-05-09T09:23:09.299374+0000","last_unstale":"2024-05-09T09:24:05.743850+0000","last_undegraded":"2024-05-09T09:24:05.743850+0000","last_fullsized":"2024-05-09T09:24:05.743850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:05.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.596539+0000","last_change":"2024-05-09T09:23:10.341933+0000","last_active":"2024-05-09T09:24:34.596539+0000","last_peered":"2024-05-09T09:24:34.596539+0000","last_clean":"2024-05-09T09:24:34.596539+0000","last_became_active":"2024-05-09T09:23:10.341621+0000","last_became_peered":"2024-05-09T09:23:10.341621+0000","last_unstale":"2024-05-09T09:24:34.596539+0000","last_undegraded":"2024-05-09T09:24:34.596539+0000","last_fullsized":"2024-05-09T09:24:34.596539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:31.938768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732505+0000","last_change":"2024-05-09T09:23:14.364783+0000","last_active":"2024-05-09T09:24:05.732505+0000","last_peered":"2024-05-09T09:24:05.732505+0000","last_clean":"2024-05-09T09:24:05.732505+0000","last_became_active":"2024-05-09T09:23:14.364309+0000","last_became_peered":"2024-05-09T09:23:14.364309+0000","last_unstale":"2024-05-09T09:24:05.732505+0000","last_undegraded":"2024-05-09T09:24:05.732505+0000","last_fullsized":"2024-05-09T09:24:05.732505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:44.252111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:42.695011+0000","last_change":"2024-05-09T09:23:09.297913+0000","last_active":"2024-05-09T09:24:42.695011+0000","last_peered":"2024-05-09T09:24:42.695011+0000","last_clean":"2024-05-09T09:24:42.695011+0000","last_became_active":"2024-05-09T09:23:09.297612+0000","last_became_peered":"2024-05-09T09:23:09.297612+0000","last_unstale":"2024-05-09T09:24:42.695011+0000","last_undegraded":"2024-05-09T09:24:42.695011+0000","last_fullsized":"2024-05-09T09:24:42.695011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:21.808703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.572712+0000","last_change":"2024-05-09T09:23:10.339101+0000","last_active":"2024-05-09T09:24:34.572712+0000","last_peered":"2024-05-09T09:24:34.572712+0000","last_clean":"2024-05-09T09:24:34.572712+0000","last_became_active":"2024-05-09T09:23:10.338453+0000","last_became_peered":"2024-05-09T09:23:10.338453+0000","last_unstale":"2024-05-09T09:24:34.572712+0000","last_undegraded":"2024-05-09T09:24:34.572712+0000","last_fullsized":"2024-05-09T09:24:34.572712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:34:44.933882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732775+0000","last_change":"2024-05-09T09:23:14.371781+0000","last_active":"2024-05-09T09:24:05.732775+0000","last_peered":"2024-05-09T09:24:05.732775+0000","last_clean":"2024-05-09T09:24:05.732775+0000","last_became_active":"2024-05-09T09:23:14.371286+0000","last_became_peered":"2024-05-09T09:23:14.371286+0000","last_unstale":"2024-05-09T09:24:05.732775+0000","last_undegraded":"2024-05-09T09:24:05.732775+0000","last_fullsized":"2024-05-09T09:24:05.732775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:26:29.185064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.744093+0000","last_change":"2024-05-09T09:23:09.303786+0000","last_active":"2024-05-09T09:24:05.744093+0000","last_peered":"2024-05-09T09:24:05.744093+0000","last_clean":"2024-05-09T09:24:05.744093+0000","last_became_active":"2024-05-09T09:23:09.303475+0000","last_became_peered":"2024-05-09T09:23:09.303475+0000","last_unstale":"2024-05-09T09:24:05.744093+0000","last_undegraded":"2024-05-09T09:24:05.744093+0000","last_fullsized":"2024-05-09T09:24:05.744093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:28.833055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.603291+0000","last_change":"2024-05-09T09:23:10.344643+0000","last_active":"2024-05-09T09:24:34.603291+0000","last_peered":"2024-05-09T09:24:34.603291+0000","last_clean":"2024-05-09T09:24:34.603291+0000","last_became_active":"2024-05-09T09:23:10.344314+0000","last_became_peered":"2024-05-09T09:23:10.344314+0000","last_unstale":"2024-05-09T09:24:34.603291+0000","last_undegraded":"2024-05-09T09:24:34.603291+0000","last_fullsized":"2024-05-09T09:24:34.603291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:15.147107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.739076+0000","last_change":"2024-05-09T09:23:14.361170+0000","last_active":"2024-05-09T09:24:05.739076+0000","last_peered":"2024-05-09T09:24:05.739076+0000","last_clean":"2024-05-09T09:24:05.739076+0000","last_became_active":"2024-05-09T09:23:14.360767+0000","last_became_peered":"2024-05-09T09:23:14.360767+0000","last_unstale":"2024-05-09T09:24:05.739076+0000","last_undegraded":"2024-05-09T09:24:05.739076+0000","last_fullsized":"2024-05-09T09:24:05.739076+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:59:21.393148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743278+0000","last_change":"2024-05-09T09:23:09.300757+0000","last_active":"2024-05-09T09:24:05.743278+0000","last_peered":"2024-05-09T09:24:05.743278+0000","last_clean":"2024-05-09T09:24:05.743278+0000","last_became_active":"2024-05-09T09:23:09.300328+0000","last_became_peered":"2024-05-09T09:23:09.300328+0000","last_unstale":"2024-05-09T09:24:05.743278+0000","last_undegraded":"2024-05-09T09:24:05.743278+0000","last_fullsized":"2024-05-09T09:24:05.743278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.553878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.606729+0000","last_change":"2024-05-09T09:23:10.336077+0000","last_active":"2024-05-09T09:24:34.606729+0000","last_peered":"2024-05-09T09:24:34.606729+0000","last_clean":"2024-05-09T09:24:34.606729+0000","last_became_active":"2024-05-09T09:23:10.335762+0000","last_became_peered":"2024-05-09T09:23:10.335762+0000","last_unstale":"2024-05-09T09:24:34.606729+0000","last_undegraded":"2024-05-09T09:24:34.606729+0000","last_fullsized":"2024-05-09T09:24:34.606729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:48.063385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.734656+0000","last_change":"2024-05-09T09:23:14.371413+0000","last_active":"2024-05-09T09:24:05.734656+0000","last_peered":"2024-05-09T09:24:05.734656+0000","last_clean":"2024-05-09T09:24:05.734656+0000","last_became_active":"2024-05-09T09:23:14.370018+0000","last_became_peered":"2024-05-09T09:23:14.370018+0000","last_unstale":"2024-05-09T09:24:05.734656+0000","last_undegraded":"2024-05-09T09:24:05.734656+0000","last_fullsized":"2024-05-09T09:24:05.734656+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:18.475976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.740718+0000","last_change":"2024-05-09T09:23:09.297328+0000","last_active":"2024-05-09T09:24:05.740718+0000","last_peered":"2024-05-09T09:24:05.740718+0000","last_clean":"2024-05-09T09:24:05.740718+0000","last_became_active":"2024-05-09T09:23:09.297001+0000","last_became_peered":"2024-05-09T09:23:09.297001+0000","last_unstale":"2024-05-09T09:24:05.740718+0000","last_undegraded":"2024-05-09T09:24:05.740718+0000","last_fullsized":"2024-05-09T09:24:05.740718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:19.241362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.583233+0000","last_change":"2024-05-09T09:23:10.338342+0000","last_active":"2024-05-09T09:24:34.583233+0000","last_peered":"2024-05-09T09:24:34.583233+0000","last_clean":"2024-05-09T09:24:34.583233+0000","last_became_active":"2024-05-09T09:23:10.337995+0000","last_became_peered":"2024-05-09T09:23:10.337995+0000","last_unstale":"2024-05-09T09:24:34.583233+0000","last_undegraded":"2024-05-09T09:24:34.583233+0000","last_fullsized":"2024-05-09T09:24:34.583233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.691862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.737440+0000","last_change":"2024-05-09T09:23:14.366237+0000","last_active":"2024-05-09T09:24:05.737440+0000","last_peered":"2024-05-09T09:24:05.737440+0000","last_clean":"2024-05-09T09:24:05.737440+0000","last_became_active":"2024-05-09T09:23:14.365809+0000","last_became_peered":"2024-05-09T09:23:14.365809+0000","last_unstale":"2024-05-09T09:24:05.737440+0000","last_undegraded":"2024-05-09T09:24:05.737440+0000","last_fullsized":"2024-05-09T09:24:05.737440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:28.813467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":172,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:43.695450+0000","last_change":"2024-05-09T09:23:09.298658+0000","last_active":"2024-05-09T09:24:43.695450+0000","last_peered":"2024-05-09T09:24:43.695450+0000","last_clean":"2024-05-09T09:24:43.695450+0000","last_became_active":"2024-05-09T09:23:09.298325+0000","last_became_peered":"2024-05-09T09:23:09.298325+0000","last_unstale":"2024-05-09T09:24:43.695450+0000","last_undegraded":"2024-05-09T09:24:43.695450+0000","last_fullsized":"2024-05-09T09:24:43.695450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:22.108202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":142,"num_read_kb":142,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.600937+0000","last_change":"2024-05-09T09:23:10.326865+0000","last_active":"2024-05-09T09:24:34.600937+0000","last_peered":"2024-05-09T09:24:34.600937+0000","last_clean":"2024-05-09T09:24:34.600937+0000","last_became_active":"2024-05-09T09:23:10.326478+0000","last_became_peered":"2024-05-09T09:23:10.326478+0000","last_unstale":"2024-05-09T09:24:34.600937+0000","last_undegraded":"2024-05-09T09:24:34.600937+0000","last_fullsized":"2024-05-09T09:24:34.600937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:54.521226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.733750+0000","last_change":"2024-05-09T09:23:14.373601+0000","last_active":"2024-05-09T09:24:05.733750+0000","last_peered":"2024-05-09T09:24:05.733750+0000","last_clean":"2024-05-09T09:24:05.733750+0000","last_became_active":"2024-05-09T09:23:14.372764+0000","last_became_peered":"2024-05-09T09:23:14.372764+0000","last_unstale":"2024-05-09T09:24:05.733750+0000","last_undegraded":"2024-05-09T09:24:05.733750+0000","last_fullsized":"2024-05-09T09:24:05.733750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:59.496755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.738453+0000","last_change":"2024-05-09T09:23:09.302293+0000","last_active":"2024-05-09T09:24:05.738453+0000","last_peered":"2024-05-09T09:24:05.738453+0000","last_clean":"2024-05-09T09:24:05.738453+0000","last_became_active":"2024-05-09T09:23:09.301216+0000","last_became_peered":"2024-05-09T09:23:09.301216+0000","last_unstale":"2024-05-09T09:24:05.738453+0000","last_undegraded":"2024-05-09T09:24:05.738453+0000","last_fullsized":"2024-05-09T09:24:05.738453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:35.146860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.599639+0000","last_change":"2024-05-09T09:23:10.339827+0000","last_active":"2024-05-09T09:24:34.599639+0000","last_peered":"2024-05-09T09:24:34.599639+0000","last_clean":"2024-05-09T09:24:34.599639+0000","last_became_active":"2024-05-09T09:23:10.339415+0000","last_became_peered":"2024-05-09T09:23:10.339415+0000","last_unstale":"2024-05-09T09:24:34.599639+0000","last_undegraded":"2024-05-09T09:24:34.599639+0000","last_fullsized":"2024-05-09T09:24:34.599639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:45.250126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732919+0000","last_change":"2024-05-09T09:23:14.377001+0000","last_active":"2024-05-09T09:24:05.732919+0000","last_peered":"2024-05-09T09:24:05.732919+0000","last_clean":"2024-05-09T09:24:05.732919+0000","last_became_active":"2024-05-09T09:23:14.376402+0000","last_became_peered":"2024-05-09T09:23:14.376402+0000","last_unstale":"2024-05-09T09:24:05.732919+0000","last_undegraded":"2024-05-09T09:24:05.732919+0000","last_fullsized":"2024-05-09T09:24:05.732919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:08.142495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.742147+0000","last_change":"2024-05-09T09:23:09.299331+0000","last_active":"2024-05-09T09:24:05.742147+0000","last_peered":"2024-05-09T09:24:05.742147+0000","last_clean":"2024-05-09T09:24:05.742147+0000","last_became_active":"2024-05-09T09:23:09.299017+0000","last_became_peered":"2024-05-09T09:23:09.299017+0000","last_unstale":"2024-05-09T09:24:05.742147+0000","last_undegraded":"2024-05-09T09:24:05.742147+0000","last_fullsized":"2024-05-09T09:24:05.742147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:28.328704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.595671+0000","last_change":"2024-05-09T09:24:05.743586+0000","last_active":"2024-05-09T09:24:34.595671+0000","last_peered":"2024-05-09T09:24:34.595671+0000","last_clean":"2024-05-09T09:24:34.595671+0000","last_became_active":"2024-05-09T09:24:05.742610+0000","last_became_peered":"2024-05-09T09:24:05.742610+0000","last_unstale":"2024-05-09T09:24:34.595671+0000","last_undegraded":"2024-05-09T09:24:34.595671+0000","last_fullsized":"2024-05-09T09:24:34.595671+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:05.586985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.733692+0000","last_change":"2024-05-09T09:23:14.363362+0000","last_active":"2024-05-09T09:24:05.733692+0000","last_peered":"2024-05-09T09:24:05.733692+0000","last_clean":"2024-05-09T09:24:05.733692+0000","last_became_active":"2024-05-09T09:23:14.362926+0000","last_became_peered":"2024-05-09T09:23:14.362926+0000","last_unstale":"2024-05-09T09:24:05.733692+0000","last_undegraded":"2024-05-09T09:24:05.733692+0000","last_fullsized":"2024-05-09T09:24:05.733692+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:25.339727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":213,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:43.695521+0000","last_change":"2024-05-09T09:23:09.297814+0000","last_active":"2024-05-09T09:24:43.695521+0000","last_peered":"2024-05-09T09:24:43.695521+0000","last_clean":"2024-05-09T09:24:43.695521+0000","last_became_active":"2024-05-09T09:23:09.297005+0000","last_became_peered":"2024-05-09T09:23:09.297005+0000","last_unstale":"2024-05-09T09:24:43.695521+0000","last_undegraded":"2024-05-09T09:24:43.695521+0000","last_fullsized":"2024-05-09T09:24:43.695521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:39.240217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":150,"num_read_kb":137,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.581930+0000","last_change":"2024-05-09T09:23:10.333013+0000","last_active":"2024-05-09T09:24:34.581930+0000","last_peered":"2024-05-09T09:24:34.581930+0000","last_clean":"2024-05-09T09:24:34.581930+0000","last_became_active":"2024-05-09T09:23:10.332666+0000","last_became_peered":"2024-05-09T09:23:10.332666+0000","last_unstale":"2024-05-09T09:24:34.581930+0000","last_undegraded":"2024-05-09T09:24:34.581930+0000","last_fullsized":"2024-05-09T09:24:34.581930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:28:50.306551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.742542+0000","last_change":"2024-05-09T09:23:14.367715+0000","last_active":"2024-05-09T09:24:05.742542+0000","last_peered":"2024-05-09T09:24:05.742542+0000","last_clean":"2024-05-09T09:24:05.742542+0000","last_became_active":"2024-05-09T09:23:14.367307+0000","last_became_peered":"2024-05-09T09:23:14.367307+0000","last_unstale":"2024-05-09T09:24:05.742542+0000","last_undegraded":"2024-05-09T09:24:05.742542+0000","last_fullsized":"2024-05-09T09:24:05.742542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:09.075572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732532+0000","last_change":"2024-05-09T09:23:09.306302+0000","last_active":"2024-05-09T09:24:05.732532+0000","last_peered":"2024-05-09T09:24:05.732532+0000","last_clean":"2024-05-09T09:24:05.732532+0000","last_became_active":"2024-05-09T09:23:09.306074+0000","last_became_peered":"2024-05-09T09:23:09.306074+0000","last_unstale":"2024-05-09T09:24:05.732532+0000","last_undegraded":"2024-05-09T09:24:05.732532+0000","last_fullsized":"2024-05-09T09:24:05.732532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:18.434899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.609186+0000","last_change":"2024-05-09T09:23:10.331201+0000","last_active":"2024-05-09T09:24:34.609186+0000","last_peered":"2024-05-09T09:24:34.609186+0000","last_clean":"2024-05-09T09:24:34.609186+0000","last_became_active":"2024-05-09T09:23:10.329830+0000","last_became_peered":"2024-05-09T09:23:10.329830+0000","last_unstale":"2024-05-09T09:24:34.609186+0000","last_undegraded":"2024-05-09T09:24:34.609186+0000","last_fullsized":"2024-05-09T09:24:34.609186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:40.199622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.737803+0000","last_change":"2024-05-09T09:23:14.359095+0000","last_active":"2024-05-09T09:24:05.737803+0000","last_peered":"2024-05-09T09:24:05.737803+0000","last_clean":"2024-05-09T09:24:05.737803+0000","last_became_active":"2024-05-09T09:23:14.358717+0000","last_became_peered":"2024-05-09T09:23:14.358717+0000","last_unstale":"2024-05-09T09:24:05.737803+0000","last_undegraded":"2024-05-09T09:24:05.737803+0000","last_fullsized":"2024-05-09T09:24:05.737803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:16:09.874767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.740939+0000","last_change":"2024-05-09T09:23:09.305370+0000","last_active":"2024-05-09T09:24:05.740939+0000","last_peered":"2024-05-09T09:24:05.740939+0000","last_clean":"2024-05-09T09:24:05.740939+0000","last_became_active":"2024-05-09T09:23:09.305054+0000","last_became_peered":"2024-05-09T09:23:09.305054+0000","last_unstale":"2024-05-09T09:24:05.740939+0000","last_undegraded":"2024-05-09T09:24:05.740939+0000","last_fullsized":"2024-05-09T09:24:05.740939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:28.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.591005+0000","last_change":"2024-05-09T09:23:10.332541+0000","last_active":"2024-05-09T09:24:34.591005+0000","last_peered":"2024-05-09T09:24:34.591005+0000","last_clean":"2024-05-09T09:24:34.591005+0000","last_became_active":"2024-05-09T09:23:10.332155+0000","last_became_peered":"2024-05-09T09:23:10.332155+0000","last_unstale":"2024-05-09T09:24:34.591005+0000","last_undegraded":"2024-05-09T09:24:34.591005+0000","last_fullsized":"2024-05-09T09:24:34.591005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:39.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.734842+0000","last_change":"2024-05-09T09:23:14.374499+0000","last_active":"2024-05-09T09:24:05.734842+0000","last_peered":"2024-05-09T09:24:05.734842+0000","last_clean":"2024-05-09T09:24:05.734842+0000","last_became_active":"2024-05-09T09:23:14.370688+0000","last_became_peered":"2024-05-09T09:23:14.370688+0000","last_unstale":"2024-05-09T09:24:05.734842+0000","last_undegraded":"2024-05-09T09:24:05.734842+0000","last_fullsized":"2024-05-09T09:24:05.734842+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:53.367237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:43.696394+0000","last_change":"2024-05-09T09:23:09.297776+0000","last_active":"2024-05-09T09:24:43.696394+0000","last_peered":"2024-05-09T09:24:43.696394+0000","last_clean":"2024-05-09T09:24:43.696394+0000","last_became_active":"2024-05-09T09:23:09.297407+0000","last_became_peered":"2024-05-09T09:23:09.297407+0000","last_unstale":"2024-05-09T09:24:43.696394+0000","last_undegraded":"2024-05-09T09:24:43.696394+0000","last_fullsized":"2024-05-09T09:24:43.696394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:21.762941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.593679+0000","last_change":"2024-05-09T09:23:10.339127+0000","last_active":"2024-05-09T09:24:34.593679+0000","last_peered":"2024-05-09T09:24:34.593679+0000","last_clean":"2024-05-09T09:24:34.593679+0000","last_became_active":"2024-05-09T09:23:10.338790+0000","last_became_peered":"2024-05-09T09:23:10.338790+0000","last_unstale":"2024-05-09T09:24:34.593679+0000","last_undegraded":"2024-05-09T09:24:34.593679+0000","last_fullsized":"2024-05-09T09:24:34.593679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:36.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743828+0000","last_change":"2024-05-09T09:23:14.373873+0000","last_active":"2024-05-09T09:24:05.743828+0000","last_peered":"2024-05-09T09:24:05.743828+0000","last_clean":"2024-05-09T09:24:05.743828+0000","last_became_active":"2024-05-09T09:23:14.371453+0000","last_became_peered":"2024-05-09T09:23:14.371453+0000","last_unstale":"2024-05-09T09:24:05.743828+0000","last_undegraded":"2024-05-09T09:24:05.743828+0000","last_fullsized":"2024-05-09T09:24:05.743828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:26.885938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.742688+0000","last_change":"2024-05-09T09:23:09.299373+0000","last_active":"2024-05-09T09:24:05.742688+0000","last_peered":"2024-05-09T09:24:05.742688+0000","last_clean":"2024-05-09T09:24:05.742688+0000","last_became_active":"2024-05-09T09:23:09.298926+0000","last_became_peered":"2024-05-09T09:23:09.298926+0000","last_unstale":"2024-05-09T09:24:05.742688+0000","last_undegraded":"2024-05-09T09:24:05.742688+0000","last_fullsized":"2024-05-09T09:24:05.742688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:35.629929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.593074+0000","last_change":"2024-05-09T09:23:10.343911+0000","last_active":"2024-05-09T09:24:34.593074+0000","last_peered":"2024-05-09T09:24:34.593074+0000","last_clean":"2024-05-09T09:24:34.593074+0000","last_became_active":"2024-05-09T09:23:10.343568+0000","last_became_peered":"2024-05-09T09:23:10.343568+0000","last_unstale":"2024-05-09T09:24:34.593074+0000","last_undegraded":"2024-05-09T09:24:34.593074+0000","last_fullsized":"2024-05-09T09:24:34.593074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:07.311554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.740465+0000","last_change":"2024-05-09T09:23:14.368544+0000","last_active":"2024-05-09T09:24:05.740465+0000","last_peered":"2024-05-09T09:24:05.740465+0000","last_clean":"2024-05-09T09:24:05.740465+0000","last_became_active":"2024-05-09T09:23:14.368140+0000","last_became_peered":"2024-05-09T09:23:14.368140+0000","last_unstale":"2024-05-09T09:24:05.740465+0000","last_undegraded":"2024-05-09T09:24:05.740465+0000","last_fullsized":"2024-05-09T09:24:05.740465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.091768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.744278+0000","last_change":"2024-05-09T09:23:09.297612+0000","last_active":"2024-05-09T09:24:05.744278+0000","last_peered":"2024-05-09T09:24:05.744278+0000","last_clean":"2024-05-09T09:24:05.744278+0000","last_became_active":"2024-05-09T09:23:09.297216+0000","last_became_peered":"2024-05-09T09:23:09.297216+0000","last_unstale":"2024-05-09T09:24:05.744278+0000","last_undegraded":"2024-05-09T09:24:05.744278+0000","last_fullsized":"2024-05-09T09:24:05.744278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:09:45.108082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.589108+0000","last_change":"2024-05-09T09:23:10.336514+0000","last_active":"2024-05-09T09:24:34.589108+0000","last_peered":"2024-05-09T09:24:34.589108+0000","last_clean":"2024-05-09T09:24:34.589108+0000","last_became_active":"2024-05-09T09:23:10.336173+0000","last_became_peered":"2024-05-09T09:23:10.336173+0000","last_unstale":"2024-05-09T09:24:34.589108+0000","last_undegraded":"2024-05-09T09:24:34.589108+0000","last_fullsized":"2024-05-09T09:24:34.589108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:46:39.036547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.742861+0000","last_change":"2024-05-09T09:23:14.366119+0000","last_active":"2024-05-09T09:24:05.742861+0000","last_peered":"2024-05-09T09:24:05.742861+0000","last_clean":"2024-05-09T09:24:05.742861+0000","last_became_active":"2024-05-09T09:23:14.365659+0000","last_became_peered":"2024-05-09T09:23:14.365659+0000","last_unstale":"2024-05-09T09:24:05.742861+0000","last_undegraded":"2024-05-09T09:24:05.742861+0000","last_fullsized":"2024-05-09T09:24:05.742861+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:23.312632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.740866+0000","last_change":"2024-05-09T09:24:05.740866+0000","last_active":"2024-05-09T09:24:05.740866+0000","last_peered":"2024-05-09T09:24:05.740866+0000","last_clean":"2024-05-09T09:24:05.740866+0000","last_became_active":"2024-05-09T09:24:05.740418+0000","last_became_peered":"2024-05-09T09:24:05.740418+0000","last_unstale":"2024-05-09T09:24:05.740866+0000","last_undegraded":"2024-05-09T09:24:05.740866+0000","last_fullsized":"2024-05-09T09:24:05.740866+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:45.946040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.580520+0000","last_change":"2024-05-09T09:23:10.339788+0000","last_active":"2024-05-09T09:24:34.580520+0000","last_peered":"2024-05-09T09:24:34.580520+0000","last_clean":"2024-05-09T09:24:34.580520+0000","last_became_active":"2024-05-09T09:23:10.339430+0000","last_became_peered":"2024-05-09T09:23:10.339430+0000","last_unstale":"2024-05-09T09:24:34.580520+0000","last_undegraded":"2024-05-09T09:24:34.580520+0000","last_fullsized":"2024-05-09T09:24:34.580520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:22.467287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743633+0000","last_change":"2024-05-09T09:23:14.368710+0000","last_active":"2024-05-09T09:24:05.743633+0000","last_peered":"2024-05-09T09:24:05.743633+0000","last_clean":"2024-05-09T09:24:05.743633+0000","last_became_active":"2024-05-09T09:23:14.368288+0000","last_became_peered":"2024-05-09T09:23:14.368288+0000","last_unstale":"2024-05-09T09:24:05.743633+0000","last_undegraded":"2024-05-09T09:24:05.743633+0000","last_fullsized":"2024-05-09T09:24:05.743633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:40.941273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:44.696722+0000","last_change":"2024-05-09T09:23:09.301976+0000","last_active":"2024-05-09T09:24:44.696722+0000","last_peered":"2024-05-09T09:24:44.696722+0000","last_clean":"2024-05-09T09:24:44.696722+0000","last_became_active":"2024-05-09T09:23:09.301494+0000","last_became_peered":"2024-05-09T09:23:09.301494+0000","last_unstale":"2024-05-09T09:24:44.696722+0000","last_undegraded":"2024-05-09T09:24:44.696722+0000","last_fullsized":"2024-05-09T09:24:44.696722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.448246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.577731+0000","last_change":"2024-05-09T09:23:10.331691+0000","last_active":"2024-05-09T09:24:34.577731+0000","last_peered":"2024-05-09T09:24:34.577731+0000","last_clean":"2024-05-09T09:24:34.577731+0000","last_became_active":"2024-05-09T09:23:10.331332+0000","last_became_peered":"2024-05-09T09:23:10.331332+0000","last_unstale":"2024-05-09T09:24:34.577731+0000","last_undegraded":"2024-05-09T09:24:34.577731+0000","last_fullsized":"2024-05-09T09:24:34.577731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:51.643695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743691+0000","last_change":"2024-05-09T09:23:14.367539+0000","last_active":"2024-05-09T09:24:05.743691+0000","last_peered":"2024-05-09T09:24:05.743691+0000","last_clean":"2024-05-09T09:24:05.743691+0000","last_became_active":"2024-05-09T09:23:14.366806+0000","last_became_peered":"2024-05-09T09:23:14.366806+0000","last_unstale":"2024-05-09T09:24:05.743691+0000","last_undegraded":"2024-05-09T09:24:05.743691+0000","last_fullsized":"2024-05-09T09:24:05.743691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:58.679095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.594405+0000","last_change":"2024-05-09T09:23:10.340849+0000","last_active":"2024-05-09T09:24:34.594405+0000","last_peered":"2024-05-09T09:24:34.594405+0000","last_clean":"2024-05-09T09:24:34.594405+0000","last_became_active":"2024-05-09T09:23:10.340430+0000","last_became_peered":"2024-05-09T09:23:10.340430+0000","last_unstale":"2024-05-09T09:24:34.594405+0000","last_undegraded":"2024-05-09T09:24:34.594405+0000","last_fullsized":"2024-05-09T09:24:34.594405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:33:47.827659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":106,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:44.697107+0000","last_change":"2024-05-09T09:23:09.308544+0000","last_active":"2024-05-09T09:24:44.697107+0000","last_peered":"2024-05-09T09:24:44.697107+0000","last_clean":"2024-05-09T09:24:44.697107+0000","last_became_active":"2024-05-09T09:23:09.308195+0000","last_became_peered":"2024-05-09T09:23:09.308195+0000","last_unstale":"2024-05-09T09:24:44.697107+0000","last_undegraded":"2024-05-09T09:24:44.697107+0000","last_fullsized":"2024-05-09T09:24:44.697107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:17:34.884385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.755846+0000","last_change":"2024-05-09T09:24:05.755846+0000","last_active":"2024-05-09T09:24:05.755846+0000","last_peered":"2024-05-09T09:24:05.755846+0000","last_clean":"2024-05-09T09:24:05.755846+0000","last_became_active":"2024-05-09T09:24:05.747246+0000","last_became_peered":"2024-05-09T09:24:05.747246+0000","last_unstale":"2024-05-09T09:24:05.755846+0000","last_undegraded":"2024-05-09T09:24:05.755846+0000","last_fullsized":"2024-05-09T09:24:05.755846+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:23:40.544770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.606229+0000","last_change":"2024-05-09T09:23:10.341320+0000","last_active":"2024-05-09T09:24:34.606229+0000","last_peered":"2024-05-09T09:24:34.606229+0000","last_clean":"2024-05-09T09:24:34.606229+0000","last_became_active":"2024-05-09T09:23:10.340868+0000","last_became_peered":"2024-05-09T09:23:10.340868+0000","last_unstale":"2024-05-09T09:24:34.606229+0000","last_undegraded":"2024-05-09T09:24:34.606229+0000","last_fullsized":"2024-05-09T09:24:34.606229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:55.846642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732460+0000","last_change":"2024-05-09T09:23:09.299574+0000","last_active":"2024-05-09T09:24:05.732460+0000","last_peered":"2024-05-09T09:24:05.732460+0000","last_clean":"2024-05-09T09:24:05.732460+0000","last_became_active":"2024-05-09T09:23:09.298878+0000","last_became_peered":"2024-05-09T09:23:09.298878+0000","last_unstale":"2024-05-09T09:24:05.732460+0000","last_undegraded":"2024-05-09T09:24:05.732460+0000","last_fullsized":"2024-05-09T09:24:05.732460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.275907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.737635+0000","last_change":"2024-05-09T09:21:57.576132+0000","last_active":"2024-05-09T09:24:05.737635+0000","last_peered":"2024-05-09T09:24:05.737635+0000","last_clean":"2024-05-09T09:24:05.737635+0000","last_became_active":"2024-05-09T09:21:57.575602+0000","last_became_peered":"2024-05-09T09:21:57.575602+0000","last_unstale":"2024-05-09T09:24:05.737635+0000","last_undegraded":"2024-05-09T09:24:05.737635+0000","last_fullsized":"2024-05-09T09:24:05.737635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_clean_scrub_stamp":"2024-05-09T09:18:31.335063+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:07.639787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.741624+0000","last_change":"2024-05-09T09:23:14.371184+0000","last_active":"2024-05-09T09:24:05.741624+0000","last_peered":"2024-05-09T09:24:05.741624+0000","last_clean":"2024-05-09T09:24:05.741624+0000","last_became_active":"2024-05-09T09:23:14.370780+0000","last_became_peered":"2024-05-09T09:23:14.370780+0000","last_unstale":"2024-05-09T09:24:05.741624+0000","last_undegraded":"2024-05-09T09:24:05.741624+0000","last_fullsized":"2024-05-09T09:24:05.741624+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:39.361947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.581560+0000","last_change":"2024-05-09T09:23:10.330915+0000","last_active":"2024-05-09T09:24:34.581560+0000","last_peered":"2024-05-09T09:24:34.581560+0000","last_clean":"2024-05-09T09:24:34.581560+0000","last_became_active":"2024-05-09T09:23:10.330458+0000","last_became_peered":"2024-05-09T09:23:10.330458+0000","last_unstale":"2024-05-09T09:24:34.581560+0000","last_undegraded":"2024-05-09T09:24:34.581560+0000","last_fullsized":"2024-05-09T09:24:34.581560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:10.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.738010+0000","last_change":"2024-05-09T09:23:09.303895+0000","last_active":"2024-05-09T09:24:05.738010+0000","last_peered":"2024-05-09T09:24:05.738010+0000","last_clean":"2024-05-09T09:24:05.738010+0000","last_became_active":"2024-05-09T09:23:09.303555+0000","last_became_peered":"2024-05-09T09:23:09.303555+0000","last_unstale":"2024-05-09T09:24:05.738010+0000","last_undegraded":"2024-05-09T09:24:05.738010+0000","last_fullsized":"2024-05-09T09:24:05.738010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:04.350161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.737718+0000","last_change":"2024-05-09T09:23:14.359390+0000","last_active":"2024-05-09T09:24:05.737718+0000","last_peered":"2024-05-09T09:24:05.737718+0000","last_clean":"2024-05-09T09:24:05.737718+0000","last_became_active":"2024-05-09T09:23:14.359000+0000","last_became_peered":"2024-05-09T09:23:14.359000+0000","last_unstale":"2024-05-09T09:24:05.737718+0000","last_undegraded":"2024-05-09T09:24:05.737718+0000","last_fullsized":"2024-05-09T09:24:05.737718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:25.077835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.744023+0000","last_change":"2024-05-09T09:23:09.304535+0000","last_active":"2024-05-09T09:24:05.744023+0000","last_peered":"2024-05-09T09:24:05.744023+0000","last_clean":"2024-05-09T09:24:05.744023+0000","last_became_active":"2024-05-09T09:23:09.304166+0000","last_became_peered":"2024-05-09T09:23:09.304166+0000","last_unstale":"2024-05-09T09:24:05.744023+0000","last_undegraded":"2024-05-09T09:24:05.744023+0000","last_fullsized":"2024-05-09T09:24:05.744023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:03:20.281620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.577760+0000","last_change":"2024-05-09T09:23:10.335657+0000","last_active":"2024-05-09T09:24:34.577760+0000","last_peered":"2024-05-09T09:24:34.577760+0000","last_clean":"2024-05-09T09:24:34.577760+0000","last_became_active":"2024-05-09T09:23:10.335231+0000","last_became_peered":"2024-05-09T09:23:10.335231+0000","last_unstale":"2024-05-09T09:24:34.577760+0000","last_undegraded":"2024-05-09T09:24:34.577760+0000","last_fullsized":"2024-05-09T09:24:34.577760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:07.990165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.736096+0000","last_change":"2024-05-09T09:23:14.366988+0000","last_active":"2024-05-09T09:24:05.736096+0000","last_peered":"2024-05-09T09:24:05.736096+0000","last_clean":"2024-05-09T09:24:05.736096+0000","last_became_active":"2024-05-09T09:23:14.366562+0000","last_became_peered":"2024-05-09T09:23:14.366562+0000","last_unstale":"2024-05-09T09:24:05.736096+0000","last_undegraded":"2024-05-09T09:24:05.736096+0000","last_fullsized":"2024-05-09T09:24:05.736096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:42.670036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'14","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.753460+0000","last_change":"2024-05-09T09:24:05.752748+0000","last_active":"2024-05-09T09:24:05.753460+0000","last_peered":"2024-05-09T09:24:05.753460+0000","last_clean":"2024-05-09T09:24:05.753460+0000","last_became_active":"2024-05-09T09:24:05.743638+0000","last_became_peered":"2024-05-09T09:24:05.743638+0000","last_unstale":"2024-05-09T09:24:05.753460+0000","last_undegraded":"2024-05-09T09:24:05.753460+0000","last_fullsized":"2024-05-09T09:24:05.753460+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:01.834934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.596289+0000","last_change":"2024-05-09T09:23:10.345840+0000","last_active":"2024-05-09T09:24:34.596289+0000","last_peered":"2024-05-09T09:24:34.596289+0000","last_clean":"2024-05-09T09:24:34.596289+0000","last_became_active":"2024-05-09T09:23:10.345487+0000","last_became_peered":"2024-05-09T09:23:10.345487+0000","last_unstale":"2024-05-09T09:24:34.596289+0000","last_undegraded":"2024-05-09T09:24:34.596289+0000","last_fullsized":"2024-05-09T09:24:34.596289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:47.723429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732198+0000","last_change":"2024-05-09T09:23:14.365485+0000","last_active":"2024-05-09T09:24:05.732198+0000","last_peered":"2024-05-09T09:24:05.732198+0000","last_clean":"2024-05-09T09:24:05.732198+0000","last_became_active":"2024-05-09T09:23:14.365018+0000","last_became_peered":"2024-05-09T09:23:14.365018+0000","last_unstale":"2024-05-09T09:24:05.732198+0000","last_undegraded":"2024-05-09T09:24:05.732198+0000","last_fullsized":"2024-05-09T09:24:05.732198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:54:04.712920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732892+0000","last_change":"2024-05-09T09:23:09.302365+0000","last_active":"2024-05-09T09:24:05.732892+0000","last_peered":"2024-05-09T09:24:05.732892+0000","last_clean":"2024-05-09T09:24:05.732892+0000","last_became_active":"2024-05-09T09:23:09.302008+0000","last_became_peered":"2024-05-09T09:23:09.302008+0000","last_unstale":"2024-05-09T09:24:05.732892+0000","last_undegraded":"2024-05-09T09:24:05.732892+0000","last_fullsized":"2024-05-09T09:24:05.732892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:48.829940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.612943+0000","last_change":"2024-05-09T09:23:10.339170+0000","last_active":"2024-05-09T09:24:34.612943+0000","last_peered":"2024-05-09T09:24:34.612943+0000","last_clean":"2024-05-09T09:24:34.612943+0000","last_became_active":"2024-05-09T09:23:10.338829+0000","last_became_peered":"2024-05-09T09:23:10.338829+0000","last_unstale":"2024-05-09T09:24:34.612943+0000","last_undegraded":"2024-05-09T09:24:34.612943+0000","last_fullsized":"2024-05-09T09:24:34.612943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:04.510495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.747227+0000","last_change":"2024-05-09T09:24:05.747227+0000","last_active":"2024-05-09T09:24:05.747227+0000","last_peered":"2024-05-09T09:24:05.747227+0000","last_clean":"2024-05-09T09:24:05.747227+0000","last_became_active":"2024-05-09T09:24:05.746338+0000","last_became_peered":"2024-05-09T09:24:05.746338+0000","last_unstale":"2024-05-09T09:24:05.747227+0000","last_undegraded":"2024-05-09T09:24:05.747227+0000","last_fullsized":"2024-05-09T09:24:05.747227+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:20.899664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.734608+0000","last_change":"2024-05-09T09:23:09.297418+0000","last_active":"2024-05-09T09:24:05.734608+0000","last_peered":"2024-05-09T09:24:05.734608+0000","last_clean":"2024-05-09T09:24:05.734608+0000","last_became_active":"2024-05-09T09:23:09.296963+0000","last_became_peered":"2024-05-09T09:23:09.296963+0000","last_unstale":"2024-05-09T09:24:05.734608+0000","last_undegraded":"2024-05-09T09:24:05.734608+0000","last_fullsized":"2024-05-09T09:24:05.734608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:54.552857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.572545+0000","last_change":"2024-05-09T09:23:10.325882+0000","last_active":"2024-05-09T09:24:34.572545+0000","last_peered":"2024-05-09T09:24:34.572545+0000","last_clean":"2024-05-09T09:24:34.572545+0000","last_became_active":"2024-05-09T09:23:10.325494+0000","last_became_peered":"2024-05-09T09:23:10.325494+0000","last_unstale":"2024-05-09T09:24:34.572545+0000","last_undegraded":"2024-05-09T09:24:34.572545+0000","last_fullsized":"2024-05-09T09:24:34.572545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:27.191310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.735082+0000","last_change":"2024-05-09T09:23:14.375572+0000","last_active":"2024-05-09T09:24:05.735082+0000","last_peered":"2024-05-09T09:24:05.735082+0000","last_clean":"2024-05-09T09:24:05.735082+0000","last_became_active":"2024-05-09T09:23:14.374069+0000","last_became_peered":"2024-05-09T09:23:14.374069+0000","last_unstale":"2024-05-09T09:24:05.735082+0000","last_undegraded":"2024-05-09T09:24:05.735082+0000","last_fullsized":"2024-05-09T09:24:05.735082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:44.045012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.737544+0000","last_change":"2024-05-09T09:23:09.304137+0000","last_active":"2024-05-09T09:24:05.737544+0000","last_peered":"2024-05-09T09:24:05.737544+0000","last_clean":"2024-05-09T09:24:05.737544+0000","last_became_active":"2024-05-09T09:23:09.303762+0000","last_became_peered":"2024-05-09T09:23:09.303762+0000","last_unstale":"2024-05-09T09:24:05.737544+0000","last_undegraded":"2024-05-09T09:24:05.737544+0000","last_fullsized":"2024-05-09T09:24:05.737544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:56.811194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.610846+0000","last_change":"2024-05-09T09:23:10.331998+0000","last_active":"2024-05-09T09:24:34.610846+0000","last_peered":"2024-05-09T09:24:34.610846+0000","last_clean":"2024-05-09T09:24:34.610846+0000","last_became_active":"2024-05-09T09:23:10.331642+0000","last_became_peered":"2024-05-09T09:23:10.331642+0000","last_unstale":"2024-05-09T09:24:34.610846+0000","last_undegraded":"2024-05-09T09:24:34.610846+0000","last_fullsized":"2024-05-09T09:24:34.610846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:41.569300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732445+0000","last_change":"2024-05-09T09:23:14.377444+0000","last_active":"2024-05-09T09:24:05.732445+0000","last_peered":"2024-05-09T09:24:05.732445+0000","last_clean":"2024-05-09T09:24:05.732445+0000","last_became_active":"2024-05-09T09:23:14.376189+0000","last_became_peered":"2024-05-09T09:23:14.376189+0000","last_unstale":"2024-05-09T09:24:05.732445+0000","last_undegraded":"2024-05-09T09:24:05.732445+0000","last_fullsized":"2024-05-09T09:24:05.732445+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:06.973535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732333+0000","last_change":"2024-05-09T09:23:09.304101+0000","last_active":"2024-05-09T09:24:05.732333+0000","last_peered":"2024-05-09T09:24:05.732333+0000","last_clean":"2024-05-09T09:24:05.732333+0000","last_became_active":"2024-05-09T09:23:09.303743+0000","last_became_peered":"2024-05-09T09:23:09.303743+0000","last_unstale":"2024-05-09T09:24:05.732333+0000","last_undegraded":"2024-05-09T09:24:05.732333+0000","last_fullsized":"2024-05-09T09:24:05.732333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:39.339815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.611236+0000","last_change":"2024-05-09T09:23:10.345938+0000","last_active":"2024-05-09T09:24:34.611236+0000","last_peered":"2024-05-09T09:24:34.611236+0000","last_clean":"2024-05-09T09:24:34.611236+0000","last_became_active":"2024-05-09T09:23:10.345576+0000","last_became_peered":"2024-05-09T09:23:10.345576+0000","last_unstale":"2024-05-09T09:24:34.611236+0000","last_undegraded":"2024-05-09T09:24:34.611236+0000","last_fullsized":"2024-05-09T09:24:34.611236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:08.571900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.732657+0000","last_change":"2024-05-09T09:23:14.371549+0000","last_active":"2024-05-09T09:24:05.732657+0000","last_peered":"2024-05-09T09:24:05.732657+0000","last_clean":"2024-05-09T09:24:05.732657+0000","last_became_active":"2024-05-09T09:23:14.369931+0000","last_became_peered":"2024-05-09T09:23:14.369931+0000","last_unstale":"2024-05-09T09:24:05.732657+0000","last_undegraded":"2024-05-09T09:24:05.732657+0000","last_fullsized":"2024-05-09T09:24:05.732657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:20.777172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743287+0000","last_change":"2024-05-09T09:23:09.302909+0000","last_active":"2024-05-09T09:24:05.743287+0000","last_peered":"2024-05-09T09:24:05.743287+0000","last_clean":"2024-05-09T09:24:05.743287+0000","last_became_active":"2024-05-09T09:23:09.302570+0000","last_became_peered":"2024-05-09T09:23:09.302570+0000","last_unstale":"2024-05-09T09:24:05.743287+0000","last_undegraded":"2024-05-09T09:24:05.743287+0000","last_fullsized":"2024-05-09T09:24:05.743287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:23.856755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.596848+0000","last_change":"2024-05-09T09:23:10.342676+0000","last_active":"2024-05-09T09:24:34.596848+0000","last_peered":"2024-05-09T09:24:34.596848+0000","last_clean":"2024-05-09T09:24:34.596848+0000","last_became_active":"2024-05-09T09:23:10.341906+0000","last_became_peered":"2024-05-09T09:23:10.341906+0000","last_unstale":"2024-05-09T09:24:34.596848+0000","last_undegraded":"2024-05-09T09:24:34.596848+0000","last_fullsized":"2024-05-09T09:24:34.596848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:59:11.137648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.734734+0000","last_change":"2024-05-09T09:23:14.363779+0000","last_active":"2024-05-09T09:24:05.734734+0000","last_peered":"2024-05-09T09:24:05.734734+0000","last_clean":"2024-05-09T09:24:05.734734+0000","last_became_active":"2024-05-09T09:23:14.363307+0000","last_became_peered":"2024-05-09T09:23:14.363307+0000","last_unstale":"2024-05-09T09:24:05.734734+0000","last_undegraded":"2024-05-09T09:24:05.734734+0000","last_fullsized":"2024-05-09T09:24:05.734734+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:46.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:42.694830+0000","last_change":"2024-05-09T09:23:09.298094+0000","last_active":"2024-05-09T09:24:42.694830+0000","last_peered":"2024-05-09T09:24:42.694830+0000","last_clean":"2024-05-09T09:24:42.694830+0000","last_became_active":"2024-05-09T09:23:09.297777+0000","last_became_peered":"2024-05-09T09:23:09.297777+0000","last_unstale":"2024-05-09T09:24:42.694830+0000","last_undegraded":"2024-05-09T09:24:42.694830+0000","last_fullsized":"2024-05-09T09:24:42.694830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:25.099193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.597200+0000","last_change":"2024-05-09T09:23:10.340120+0000","last_active":"2024-05-09T09:24:34.597200+0000","last_peered":"2024-05-09T09:24:34.597200+0000","last_clean":"2024-05-09T09:24:34.597200+0000","last_became_active":"2024-05-09T09:23:10.339690+0000","last_became_peered":"2024-05-09T09:23:10.339690+0000","last_unstale":"2024-05-09T09:24:34.597200+0000","last_undegraded":"2024-05-09T09:24:34.597200+0000","last_fullsized":"2024-05-09T09:24:34.597200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:36.783411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.744892+0000","last_change":"2024-05-09T09:24:05.744892+0000","last_active":"2024-05-09T09:24:05.744892+0000","last_peered":"2024-05-09T09:24:05.744892+0000","last_clean":"2024-05-09T09:24:05.744892+0000","last_became_active":"2024-05-09T09:24:05.744437+0000","last_became_peered":"2024-05-09T09:24:05.744437+0000","last_unstale":"2024-05-09T09:24:05.744892+0000","last_undegraded":"2024-05-09T09:24:05.744892+0000","last_fullsized":"2024-05-09T09:24:05.744892+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:38:33.709821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.742407+0000","last_change":"2024-05-09T09:23:09.307592+0000","last_active":"2024-05-09T09:24:05.742407+0000","last_peered":"2024-05-09T09:24:05.742407+0000","last_clean":"2024-05-09T09:24:05.742407+0000","last_became_active":"2024-05-09T09:23:09.307341+0000","last_became_peered":"2024-05-09T09:23:09.307341+0000","last_unstale":"2024-05-09T09:24:05.742407+0000","last_undegraded":"2024-05-09T09:24:05.742407+0000","last_fullsized":"2024-05-09T09:24:05.742407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:21.790980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.593713+0000","last_change":"2024-05-09T09:23:10.349217+0000","last_active":"2024-05-09T09:24:34.593713+0000","last_peered":"2024-05-09T09:24:34.593713+0000","last_clean":"2024-05-09T09:24:34.593713+0000","last_became_active":"2024-05-09T09:23:10.348861+0000","last_became_peered":"2024-05-09T09:23:10.348861+0000","last_unstale":"2024-05-09T09:24:34.593713+0000","last_undegraded":"2024-05-09T09:24:34.593713+0000","last_fullsized":"2024-05-09T09:24:34.593713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:51.621045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.739725+0000","last_change":"2024-05-09T09:23:14.363773+0000","last_active":"2024-05-09T09:24:05.739725+0000","last_peered":"2024-05-09T09:24:05.739725+0000","last_clean":"2024-05-09T09:24:05.739725+0000","last_became_active":"2024-05-09T09:23:14.363393+0000","last_became_peered":"2024-05-09T09:23:14.363393+0000","last_unstale":"2024-05-09T09:24:05.739725+0000","last_undegraded":"2024-05-09T09:24:05.739725+0000","last_fullsized":"2024-05-09T09:24:05.739725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:57.322985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:05.743708+0000","last_change":"2024-05-09T09:23:09.308723+0000","last_active":"2024-05-09T09:24:05.743708+0000","last_peered":"2024-05-09T09:24:05.743708+0000","last_clean":"2024-05-09T09:24:05.743708+0000","last_became_active":"2024-05-09T09:23:09.308267+0000","last_became_peered":"2024-05-09T09:23:09.308267+0000","last_unstale":"2024-05-09T09:24:05.743708+0000","last_undegraded":"2024-05-09T09:24:05.743708+0000","last_fullsized":"2024-05-09T09:24:05.743708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:09:25.036421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.588908+0000","last_change":"2024-05-09T09:23:10.342246+0000","last_active":"2024-05-09T09:24:34.588908+0000","last_peered":"2024-05-09T09:24:34.588908+0000","last_clean":"2024-05-09T09:24:34.588908+0000","last_became_active":"2024-05-09T09:23:10.341927+0000","last_became_peered":"2024-05-09T09:23:10.341927+0000","last_unstale":"2024-05-09T09:24:34.588908+0000","last_undegraded":"2024-05-09T09:24:34.588908+0000","last_fullsized":"2024-05-09T09:24:34.588908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:01.182226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.593340+0000","last_change":"2024-05-09T09:23:10.332768+0000","last_active":"2024-05-09T09:24:34.593340+0000","last_peered":"2024-05-09T09:24:34.593340+0000","last_clean":"2024-05-09T09:24:34.593340+0000","last_became_active":"2024-05-09T09:23:10.332441+0000","last_became_peered":"2024-05-09T09:23:10.332441+0000","last_unstale":"2024-05-09T09:24:34.593340+0000","last_undegraded":"2024-05-09T09:24:34.593340+0000","last_fullsized":"2024-05-09T09:24:34.593340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:53.377058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.590074+0000","last_change":"2024-05-09T09:23:10.337944+0000","last_active":"2024-05-09T09:24:34.590074+0000","last_peered":"2024-05-09T09:24:34.590074+0000","last_clean":"2024-05-09T09:24:34.590074+0000","last_became_active":"2024-05-09T09:23:10.337593+0000","last_became_peered":"2024-05-09T09:23:10.337593+0000","last_unstale":"2024-05-09T09:24:34.590074+0000","last_undegraded":"2024-05-09T09:24:34.590074+0000","last_fullsized":"2024-05-09T09:24:34.590074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:20.381543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.610346+0000","last_change":"2024-05-09T09:23:10.337419+0000","last_active":"2024-05-09T09:24:34.610346+0000","last_peered":"2024-05-09T09:24:34.610346+0000","last_clean":"2024-05-09T09:24:34.610346+0000","last_became_active":"2024-05-09T09:23:10.336718+0000","last_became_peered":"2024-05-09T09:23:10.336718+0000","last_unstale":"2024-05-09T09:24:34.610346+0000","last_undegraded":"2024-05-09T09:24:34.610346+0000","last_fullsized":"2024-05-09T09:24:34.610346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:50.101110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.599871+0000","last_change":"2024-05-09T09:23:10.340243+0000","last_active":"2024-05-09T09:24:34.599871+0000","last_peered":"2024-05-09T09:24:34.599871+0000","last_clean":"2024-05-09T09:24:34.599871+0000","last_became_active":"2024-05-09T09:23:10.339919+0000","last_became_peered":"2024-05-09T09:23:10.339919+0000","last_unstale":"2024-05-09T09:24:34.599871+0000","last_undegraded":"2024-05-09T09:24:34.599871+0000","last_fullsized":"2024-05-09T09:24:34.599871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:21.460928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.613410+0000","last_change":"2024-05-09T09:23:10.333136+0000","last_active":"2024-05-09T09:24:34.613410+0000","last_peered":"2024-05-09T09:24:34.613410+0000","last_clean":"2024-05-09T09:24:34.613410+0000","last_became_active":"2024-05-09T09:23:10.332799+0000","last_became_peered":"2024-05-09T09:23:10.332799+0000","last_unstale":"2024-05-09T09:24:34.613410+0000","last_undegraded":"2024-05-09T09:24:34.613410+0000","last_fullsized":"2024-05-09T09:24:34.613410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:50.682747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.587631+0000","last_change":"2024-05-09T09:24:05.745143+0000","last_active":"2024-05-09T09:24:34.587631+0000","last_peered":"2024-05-09T09:24:34.587631+0000","last_clean":"2024-05-09T09:24:34.587631+0000","last_became_active":"2024-05-09T09:24:05.744117+0000","last_became_peered":"2024-05-09T09:24:05.744117+0000","last_unstale":"2024-05-09T09:24:34.587631+0000","last_undegraded":"2024-05-09T09:24:34.587631+0000","last_fullsized":"2024-05-09T09:24:34.587631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:25:09.853820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.581573+0000","last_change":"2024-05-09T09:23:10.341776+0000","last_active":"2024-05-09T09:24:34.581573+0000","last_peered":"2024-05-09T09:24:34.581573+0000","last_clean":"2024-05-09T09:24:34.581573+0000","last_became_active":"2024-05-09T09:23:10.341436+0000","last_became_peered":"2024-05-09T09:23:10.341436+0000","last_unstale":"2024-05-09T09:24:34.581573+0000","last_undegraded":"2024-05-09T09:24:34.581573+0000","last_fullsized":"2024-05-09T09:24:34.581573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:42.178856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.604854+0000","last_change":"2024-05-09T09:24:05.745324+0000","last_active":"2024-05-09T09:24:34.604854+0000","last_peered":"2024-05-09T09:24:34.604854+0000","last_clean":"2024-05-09T09:24:34.604854+0000","last_became_active":"2024-05-09T09:24:05.744899+0000","last_became_peered":"2024-05-09T09:24:05.744899+0000","last_unstale":"2024-05-09T09:24:34.604854+0000","last_undegraded":"2024-05-09T09:24:34.604854+0000","last_fullsized":"2024-05-09T09:24:34.604854+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:59.372367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.602858+0000","last_change":"2024-05-09T09:23:10.337117+0000","last_active":"2024-05-09T09:24:34.602858+0000","last_peered":"2024-05-09T09:24:34.602858+0000","last_clean":"2024-05-09T09:24:34.602858+0000","last_became_active":"2024-05-09T09:23:10.336672+0000","last_became_peered":"2024-05-09T09:23:10.336672+0000","last_unstale":"2024-05-09T09:24:34.602858+0000","last_undegraded":"2024-05-09T09:24:34.602858+0000","last_fullsized":"2024-05-09T09:24:34.602858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:59:20.723742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.605575+0000","last_change":"2024-05-09T09:23:10.337733+0000","last_active":"2024-05-09T09:24:34.605575+0000","last_peered":"2024-05-09T09:24:34.605575+0000","last_clean":"2024-05-09T09:24:34.605575+0000","last_became_active":"2024-05-09T09:23:10.337343+0000","last_became_peered":"2024-05-09T09:23:10.337343+0000","last_unstale":"2024-05-09T09:24:34.605575+0000","last_undegraded":"2024-05-09T09:24:34.605575+0000","last_fullsized":"2024-05-09T09:24:34.605575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:53.447911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.609040+0000","last_change":"2024-05-09T09:23:10.337485+0000","last_active":"2024-05-09T09:24:34.609040+0000","last_peered":"2024-05-09T09:24:34.609040+0000","last_clean":"2024-05-09T09:24:34.609040+0000","last_became_active":"2024-05-09T09:23:10.337129+0000","last_became_peered":"2024-05-09T09:23:10.337129+0000","last_unstale":"2024-05-09T09:24:34.609040+0000","last_undegraded":"2024-05-09T09:24:34.609040+0000","last_fullsized":"2024-05-09T09:24:34.609040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:20.758552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.613415+0000","last_change":"2024-05-09T09:23:10.332271+0000","last_active":"2024-05-09T09:24:34.613415+0000","last_peered":"2024-05-09T09:24:34.613415+0000","last_clean":"2024-05-09T09:24:34.613415+0000","last_became_active":"2024-05-09T09:23:10.331941+0000","last_became_peered":"2024-05-09T09:23:10.331941+0000","last_unstale":"2024-05-09T09:24:34.613415+0000","last_undegraded":"2024-05-09T09:24:34.613415+0000","last_fullsized":"2024-05-09T09:24:34.613415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:27.707554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T09:24:34.590904+0000","last_change":"2024-05-09T09:23:10.328965+0000","last_active":"2024-05-09T09:24:34.590904+0000","last_peered":"2024-05-09T09:24:34.590904+0000","last_clean":"2024-05-09T09:24:34.590904+0000","last_became_active":"2024-05-09T09:23:10.328618+0000","last_became_peered":"2024-05-09T09:23:10.328618+0000","last_unstale":"2024-05-09T09:24:34.590904+0000","last_undegraded":"2024-05-09T09:24:34.590904+0000","last_fullsized":"2024-05-09T09:24:34.590904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:16:02.003378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70559,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":756,"num_read_kb":457,"num_write":249,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":501047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1156096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":500977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":501355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1097993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":496481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":561116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":497937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40592,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704560,"statfs":{"total":95995035648,"available":95953469440,"internally_reserved":0,"allocated":1122304,"data_stored":561336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":496189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":501087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1093043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:24:49.945 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T09:24:49.946 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T09:24:49.946 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-09T09:24:49.946 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-09T09:24:50.200 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:50.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:50 smithi012 ceph-mon[30140]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:24:50.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:50 smithi012 ceph-mon[30140]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:24:50.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:50 smithi170 ceph-mon[33632]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:24:50.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:50 smithi170 ceph-mon[33632]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:24:50.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:50 smithi016 ceph-mon[33335]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:24:50.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:50 smithi016 ceph-mon[33335]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:24:51.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.840+0000 7f50030e6700 1 -- 172.21.15.12:0/2895278216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ffc1083a0 msgr2=0x7f4ffc108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:51.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.840+0000 7f50030e6700 1 --2- 172.21.15.12:0/2895278216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ffc1083a0 0x7f4ffc108780 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00b0a0 tx=0x7f4fec00b3b0 comp rx=0 tx=0).stop 2024-05-09T09:24:51.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.840+0000 7f50030e6700 1 -- 172.21.15.12:0/2895278216 shutdown_connections 2024-05-09T09:24:51.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.840+0000 7f50030e6700 1 --2- 172.21.15.12:0/2895278216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ffc06a9d0 0x7f4ffc10f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:51.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.840+0000 7f50030e6700 1 --2- 172.21.15.12:0/2895278216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ffc1083a0 0x7f4ffc108780 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:51.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.840+0000 7f50030e6700 1 --2- 172.21.15.12:0/2895278216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ffc06a030 0x7f4ffc06a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:51.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.840+0000 7f50030e6700 1 -- 172.21.15.12:0/2895278216 >> 172.21.15.12:0/2895278216 conn(0x7f4ffc0781d0 msgr2=0x7f4ffc0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.840+0000 7f50030e6700 1 -- 172.21.15.12:0/2895278216 shutdown_connections 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.840+0000 7f50030e6700 1 -- 172.21.15.12:0/2895278216 wait complete. 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.841+0000 7f50030e6700 1 Processor -- start 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.841+0000 7f50030e6700 1 -- start start 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.841+0000 7f50030e6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ffc06a030 0x7f4ffc19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.841+0000 7f50030e6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ffc06a9d0 0x7f4ffc19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.841+0000 7f50030e6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ffc1083a0 0x7f4ffc1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.841+0000 7f50030e6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc1125d0 con 0x7f4ffc06a9d0 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.841+0000 7f50030e6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc112450 con 0x7f4ffc1083a0 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.841+0000 7f50030e6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc112750 con 0x7f4ffc06a030 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.841+0000 7f4ffbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ffc06a9d0 0x7f4ffc19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.841+0000 7f4ffbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ffc06a9d0 0x7f4ffc19ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49992/0 (socket says 172.21.15.12:49992) 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.841+0000 7f4ffbfff700 1 -- 172.21.15.12:0/1479375265 learned_addr learned my addr 172.21.15.12:0/1479375265 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.841+0000 7f5001683700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ffc1083a0 0x7f4ffc1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.841+0000 7f5000e82700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ffc06a030 0x7f4ffc19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.842+0000 7f4ffbfff700 1 -- 172.21.15.12:0/1479375265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ffc06a030 msgr2=0x7f4ffc19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.842+0000 7f4ffbfff700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ffc06a030 0x7f4ffc19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.842+0000 7f4ffbfff700 1 -- 172.21.15.12:0/1479375265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ffc1083a0 msgr2=0x7f4ffc1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.842+0000 7f4ffbfff700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ffc1083a0 0x7f4ffc1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.842+0000 7f4ffbfff700 1 -- 172.21.15.12:0/1479375265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ffc199050 con 0x7f4ffc06a9d0 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.842+0000 7f4ffbfff700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ffc06a9d0 0x7f4ffc19ec20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c920 tx=0x7f4fe400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.842+0000 7f4ff9ffb700 1 -- 172.21.15.12:0/1479375265 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4007b00 con 0x7f4ffc06a9d0 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.842+0000 7f4ff9ffb700 1 -- 172.21.15.12:0/1479375265 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4fe4004d00 con 0x7f4ffc06a9d0 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.842+0000 7f50030e6700 1 -- 172.21.15.12:0/1479375265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ffc199340 con 0x7f4ffc06a9d0 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.842+0000 7f4ff9ffb700 1 -- 172.21.15.12:0/1479375265 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe40055e0 con 0x7f4ffc06a9d0 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.842+0000 7f50030e6700 1 -- 172.21.15.12:0/1479375265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ffc1a55b0 con 0x7f4ffc06a9d0 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.843+0000 7f50030e6700 1 -- 172.21.15.12:0/1479375265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ffc069010 con 0x7f4ffc06a9d0 2024-05-09T09:24:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.844+0000 7f4ff9ffb700 1 -- 172.21.15.12:0/1479375265 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4fe4004750 con 0x7f4ffc06a9d0 2024-05-09T09:24:51.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.845+0000 7f4ff9ffb700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4fe80a2c20 0x7f4fe80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:51.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.845+0000 7f5000e82700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4fe80a2c20 0x7f4fe80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:51.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.845+0000 7f4ff9ffb700 1 -- 172.21.15.12:0/1479375265 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4fe40cefb0 con 0x7f4ffc06a9d0 2024-05-09T09:24:51.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.845+0000 7f5000e82700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4fe80a2c20 0x7f4fe80a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4fec006130 tx=0x7f4fec0060a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:51.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:51.849+0000 7f4ff9ffb700 1 -- 172.21.15.12:0/1479375265 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4fe4014030 con 0x7f4ffc06a9d0 2024-05-09T09:24:52.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.102+0000 7f50030e6700 1 -- 172.21.15.12:0/1479375265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4ffc02db80 con 0x7f4ffc06a9d0 2024-05-09T09:24:52.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.102+0000 7f4ff9ffb700 1 -- 172.21.15.12:0/1479375265 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f4fe409bb50 con 0x7f4ffc06a9d0 2024-05-09T09:24:52.102 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:24:52.102 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T09:24:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.104+0000 7f50030e6700 1 -- 172.21.15.12:0/1479375265 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4fe80a2c20 msgr2=0x7f4fe80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.104+0000 7f50030e6700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4fe80a2c20 0x7f4fe80a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4fec006130 tx=0x7f4fec0060a0 comp rx=0 tx=0).stop 2024-05-09T09:24:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.104+0000 7f50030e6700 1 -- 172.21.15.12:0/1479375265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ffc06a9d0 msgr2=0x7f4ffc19ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.104+0000 7f50030e6700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ffc06a9d0 0x7f4ffc19ec20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c920 tx=0x7f4fe400cc30 comp rx=0 tx=0).stop 2024-05-09T09:24:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.104+0000 7f50030e6700 1 -- 172.21.15.12:0/1479375265 shutdown_connections 2024-05-09T09:24:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.104+0000 7f50030e6700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4fe80a2c20 0x7f4fe80a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.104+0000 7f50030e6700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ffc1083a0 0x7f4ffc1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.104+0000 7f50030e6700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ffc06a030 0x7f4ffc19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.104+0000 7f50030e6700 1 --2- 172.21.15.12:0/1479375265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ffc06a9d0 0x7f4ffc19ec20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.104+0000 7f50030e6700 1 -- 172.21.15.12:0/1479375265 >> 172.21.15.12:0/1479375265 conn(0x7f4ffc0781d0 msgr2=0x7f4ffc0fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:52.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.104+0000 7f50030e6700 1 -- 172.21.15.12:0/1479375265 shutdown_connections 2024-05-09T09:24:52.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.104+0000 7f50030e6700 1 -- 172.21.15.12:0/1479375265 wait complete. 2024-05-09T09:24:52.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:52 smithi012 ceph-mon[30140]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:24:52.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:52 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1479375265' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T09:24:52.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:52 smithi170 ceph-mon[33632]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:24:52.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:52 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1479375265' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T09:24:52.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:52 smithi016 ceph-mon[33335]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:24:52.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:52 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1479375265' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T09:24:52.802 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-09T09:24:52.802 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:24:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.912+0000 7f6db9914640 1 -- 172.21.15.12:0/4216843421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6db4107570 msgr2=0x7f6db41099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.912+0000 7f6db9914640 1 --2- 172.21.15.12:0/4216843421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6db4107570 0x7f6db41099c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009ca0 tx=0x7f6dac02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:24:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.912+0000 7f6db9914640 1 -- 172.21.15.12:0/4216843421 shutdown_connections 2024-05-09T09:24:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.912+0000 7f6db9914640 1 --2- 172.21.15.12:0/4216843421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6db4107570 0x7f6db41099c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.912+0000 7f6db9914640 1 --2- 172.21.15.12:0/4216843421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db4104c40 0x7f6db4107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.912+0000 7f6db9914640 1 --2- 172.21.15.12:0/4216843421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db4102310 0x7f6db4104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.912+0000 7f6db9914640 1 -- 172.21.15.12:0/4216843421 >> 172.21.15.12:0/4216843421 conn(0x7f6db40fbd20 msgr2=0x7f6db40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.912+0000 7f6db9914640 1 -- 172.21.15.12:0/4216843421 shutdown_connections 2024-05-09T09:24:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.912+0000 7f6db9914640 1 -- 172.21.15.12:0/4216843421 wait complete. 2024-05-09T09:24:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.913+0000 7f6db9914640 1 Processor -- start 2024-05-09T09:24:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.913+0000 7f6db9914640 1 -- start start 2024-05-09T09:24:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.913+0000 7f6db9914640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db4102310 0x7f6db4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.913+0000 7f6db9914640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db4104c40 0x7f6db41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.913+0000 7f6db9914640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6db4107570 0x7f6db419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.913+0000 7f6db9914640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6db410d510 con 0x7f6db4107570 2024-05-09T09:24:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.913+0000 7f6db9914640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6db410d390 con 0x7f6db4104c40 2024-05-09T09:24:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.913+0000 7f6db9914640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6db410d690 con 0x7f6db4102310 2024-05-09T09:24:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.913+0000 7f6db37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6db4107570 0x7f6db419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.913+0000 7f6db37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6db4107570 0x7f6db419f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50006/0 (socket says 172.21.15.12:50006) 2024-05-09T09:24:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6db37fe640 1 -- 172.21.15.12:0/1462492185 learned_addr learned my addr 172.21.15.12:0/1462492185 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:52.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6db2ffd640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db4102310 0x7f6db4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:52.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6daa7fc640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db4104c40 0x7f6db41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:52.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6db37fe640 1 -- 172.21.15.12:0/1462492185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db4102310 msgr2=0x7f6db4197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:52.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6db37fe640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db4102310 0x7f6db4197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:52.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6db37fe640 1 -- 172.21.15.12:0/1462492185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db4104c40 msgr2=0x7f6db41980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:52.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6db37fe640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db4104c40 0x7f6db41980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:52.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6db37fe640 1 -- 172.21.15.12:0/1462492185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db419f7a0 con 0x7f6db4107570 2024-05-09T09:24:52.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6db37fe640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6db4107570 0x7f6db419f130 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6dac004ad0 tx=0x7f6dac02f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:52.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6db8912640 1 -- 172.21.15.12:0/1462492185 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac003cf0 con 0x7f6db4107570 2024-05-09T09:24:52.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6db9914640 1 -- 172.21.15.12:0/1462492185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db419fa30 con 0x7f6db4107570 2024-05-09T09:24:52.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6db8912640 1 -- 172.21.15.12:0/1462492185 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6dac004510 con 0x7f6db4107570 2024-05-09T09:24:52.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6db8912640 1 -- 172.21.15.12:0/1462492185 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac02fd10 con 0x7f6db4107570 2024-05-09T09:24:52.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.914+0000 7f6db9914640 1 -- 172.21.15.12:0/1462492185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db419ffd0 con 0x7f6db4107570 2024-05-09T09:24:52.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.915+0000 7f6db9914640 1 -- 172.21.15.12:0/1462492185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db40fd810 con 0x7f6db4107570 2024-05-09T09:24:52.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.916+0000 7f6db8912640 1 -- 172.21.15.12:0/1462492185 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6dac004040 con 0x7f6db4107570 2024-05-09T09:24:52.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.917+0000 7f6db8912640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6da00a2b90 0x7f6da00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:52.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.917+0000 7f6db2ffd640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6da00a2b90 0x7f6da00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:52.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.917+0000 7f6db8912640 1 -- 172.21.15.12:0/1462492185 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6dac0f0c70 con 0x7f6db4107570 2024-05-09T09:24:52.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.918+0000 7f6db2ffd640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6da00a2b90 0x7f6da00a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6da4005fd0 tx=0x7f6da400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:52.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:52.920+0000 7f6db8912640 1 -- 172.21.15.12:0/1462492185 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6dac004330 con 0x7f6db4107570 2024-05-09T09:24:53.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.107+0000 7f6db9914640 1 -- 172.21.15.12:0/1462492185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6db4062430 con 0x7f6db4107570 2024-05-09T09:24:53.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.108+0000 7f6db8912640 1 -- 172.21.15.12:0/1462492185 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24575 (secure 0 0 0) 0x7f6dac0bd870 con 0x7f6db4107570 2024-05-09T09:24:53.108 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:24:53.109 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":91,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:24:05.727743+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:24:53.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.111+0000 7f6db9914640 1 -- 172.21.15.12:0/1462492185 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6da00a2b90 msgr2=0x7f6da00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.111+0000 7f6db9914640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6da00a2b90 0x7f6da00a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6da4005fd0 tx=0x7f6da400b390 comp rx=0 tx=0).stop 2024-05-09T09:24:53.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.111+0000 7f6db9914640 1 -- 172.21.15.12:0/1462492185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6db4107570 msgr2=0x7f6db419f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.111+0000 7f6db9914640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6db4107570 0x7f6db419f130 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6dac004ad0 tx=0x7f6dac02f450 comp rx=0 tx=0).stop 2024-05-09T09:24:53.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.111+0000 7f6db9914640 1 -- 172.21.15.12:0/1462492185 shutdown_connections 2024-05-09T09:24:53.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.111+0000 7f6db9914640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6da00a2b90 0x7f6da00a5050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.111+0000 7f6db9914640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6db4107570 0x7f6db419f130 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.111+0000 7f6db9914640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db4104c40 0x7f6db41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.111+0000 7f6db9914640 1 --2- 172.21.15.12:0/1462492185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db4102310 0x7f6db4197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.111+0000 7f6db9914640 1 -- 172.21.15.12:0/1462492185 >> 172.21.15.12:0/1462492185 conn(0x7f6db40fbd20 msgr2=0x7f6db4102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:53.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.111+0000 7f6db9914640 1 -- 172.21.15.12:0/1462492185 shutdown_connections 2024-05-09T09:24:53.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.111+0000 7f6db9914640 1 -- 172.21.15.12:0/1462492185 wait complete. 2024-05-09T09:24:53.123 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:23:08.167107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:23:09.065670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:23:12.773640+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:24:53.123 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:24:53.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.233+0000 7fa04733b640 1 -- 172.21.15.12:0/834838704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa040102310 msgr2=0x7fa040104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.233+0000 7fa04733b640 1 --2- 172.21.15.12:0/834838704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa040102310 0x7fa040104700 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa038009900 tx=0x7fa03802a410 comp rx=0 tx=0).stop 2024-05-09T09:24:53.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.233+0000 7fa04733b640 1 -- 172.21.15.12:0/834838704 shutdown_connections 2024-05-09T09:24:53.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.233+0000 7fa04733b640 1 --2- 172.21.15.12:0/834838704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa040107570 0x7fa0401099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.233+0000 7fa04733b640 1 --2- 172.21.15.12:0/834838704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa040104c40 0x7fa040107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.233+0000 7fa04733b640 1 --2- 172.21.15.12:0/834838704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa040102310 0x7fa040104700 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.233+0000 7fa04733b640 1 -- 172.21.15.12:0/834838704 >> 172.21.15.12:0/834838704 conn(0x7fa0400fbd20 msgr2=0x7fa0400fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:53.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.233+0000 7fa04733b640 1 -- 172.21.15.12:0/834838704 shutdown_connections 2024-05-09T09:24:53.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.233+0000 7fa04733b640 1 -- 172.21.15.12:0/834838704 wait complete. 2024-05-09T09:24:53.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.233+0000 7fa04733b640 1 Processor -- start 2024-05-09T09:24:53.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.233+0000 7fa04733b640 1 -- start start 2024-05-09T09:24:53.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa04733b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa040102310 0x7fa040197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:53.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa04733b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa040104c40 0x7fa040198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:53.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa04733b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa040107570 0x7fa04019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:53.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa04733b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa04010d510 con 0x7fa040107570 2024-05-09T09:24:53.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa04733b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa04010d390 con 0x7fa040102310 2024-05-09T09:24:53.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa04733b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa04010d690 con 0x7fa040104c40 2024-05-09T09:24:53.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa0458d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa040107570 0x7fa04019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:53.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa0458d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa040107570 0x7fa04019f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50018/0 (socket says 172.21.15.12:50018) 2024-05-09T09:24:53.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa0458d8640 1 -- 172.21.15.12:0/871005046 learned_addr learned my addr 172.21.15.12:0/871005046 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:53.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa0448d6640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa040104c40 0x7fa040198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:53.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa0450d7640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa040102310 0x7fa040197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:53.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa0458d8640 1 -- 172.21.15.12:0/871005046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa040104c40 msgr2=0x7fa040198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa0458d8640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa040104c40 0x7fa040198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa0458d8640 1 -- 172.21.15.12:0/871005046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa040102310 msgr2=0x7fa040197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa0458d8640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa040102310 0x7fa040197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa0458d8640 1 -- 172.21.15.12:0/871005046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04019f840 con 0x7fa040107570 2024-05-09T09:24:53.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa0450d7640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa040102310 0x7fa040197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:53.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa0458d8640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa040107570 0x7fa04019f1a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa03000cad0 tx=0x7fa03000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:53.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa03e7fc640 1 -- 172.21.15.12:0/871005046 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03000ae20 con 0x7fa040107570 2024-05-09T09:24:53.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa03e7fc640 1 -- 172.21.15.12:0/871005046 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa030005020 con 0x7fa040107570 2024-05-09T09:24:53.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa03e7fc640 1 -- 172.21.15.12:0/871005046 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03000f690 con 0x7fa040107570 2024-05-09T09:24:53.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.234+0000 7fa04733b640 1 -- 172.21.15.12:0/871005046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04019fb30 con 0x7fa040107570 2024-05-09T09:24:53.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.235+0000 7fa04733b640 1 -- 172.21.15.12:0/871005046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0401a0030 con 0x7fa040107570 2024-05-09T09:24:53.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.236+0000 7fa04733b640 1 -- 172.21.15.12:0/871005046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0400fd810 con 0x7fa040107570 2024-05-09T09:24:53.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.237+0000 7fa03e7fc640 1 -- 172.21.15.12:0/871005046 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa0300051d0 con 0x7fa040107570 2024-05-09T09:24:53.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.237+0000 7fa03e7fc640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa0280a2b90 0x7fa0280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:53.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.238+0000 7fa03e7fc640 1 -- 172.21.15.12:0/871005046 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa0300cf880 con 0x7fa040107570 2024-05-09T09:24:53.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.238+0000 7fa0450d7640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa0280a2b90 0x7fa0280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:53.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.238+0000 7fa0450d7640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa0280a2b90 0x7fa0280a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa0380096c0 tx=0x7fa038009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:53.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.241+0000 7fa03e7fc640 1 -- 172.21.15.12:0/871005046 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa030014030 con 0x7fa040107570 2024-05-09T09:24:53.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.428+0000 7fa04733b640 1 -- 172.21.15.12:0/871005046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa04002d410 con 0x7fa040107570 2024-05-09T09:24:53.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.429+0000 7fa03e7fc640 1 -- 172.21.15.12:0/871005046 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fa03009c480 con 0x7fa040107570 2024-05-09T09:24:53.428 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:24:53.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.431+0000 7fa04733b640 1 -- 172.21.15.12:0/871005046 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa0280a2b90 msgr2=0x7fa0280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.431+0000 7fa04733b640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa0280a2b90 0x7fa0280a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa0380096c0 tx=0x7fa038009650 comp rx=0 tx=0).stop 2024-05-09T09:24:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.431+0000 7fa04733b640 1 -- 172.21.15.12:0/871005046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa040107570 msgr2=0x7fa04019f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.431+0000 7fa04733b640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa040107570 0x7fa04019f1a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa03000cad0 tx=0x7fa03000cf90 comp rx=0 tx=0).stop 2024-05-09T09:24:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.431+0000 7fa04733b640 1 -- 172.21.15.12:0/871005046 shutdown_connections 2024-05-09T09:24:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.431+0000 7fa04733b640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa0280a2b90 0x7fa0280a5050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.431+0000 7fa04733b640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa040107570 0x7fa04019f1a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.431+0000 7fa04733b640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa040104c40 0x7fa040198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.431+0000 7fa04733b640 1 --2- 172.21.15.12:0/871005046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa040102310 0x7fa040197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.431+0000 7fa04733b640 1 -- 172.21.15.12:0/871005046 >> 172.21.15.12:0/871005046 conn(0x7fa0400fbd20 msgr2=0x7fa040102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.431+0000 7fa04733b640 1 -- 172.21.15.12:0/871005046 shutdown_connections 2024-05-09T09:24:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.431+0000 7fa04733b640 1 -- 172.21.15.12:0/871005046 wait complete. 2024-05-09T09:24:53.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:53 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1462492185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:24:53.442 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:24:53.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:53 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1462492185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:24:53.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:53 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1462492185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:24:53.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.550+0000 7f266c5ce640 1 -- 172.21.15.12:0/60143614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2664103430 msgr2=0x7f2664109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.550+0000 7f266c5ce640 1 --2- 172.21.15.12:0/60143614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2664103430 0x7f2664109cc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f265c009ca0 tx=0x7f265c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.550+0000 7f266c5ce640 1 -- 172.21.15.12:0/60143614 shutdown_connections 2024-05-09T09:24:53.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.550+0000 7f266c5ce640 1 --2- 172.21.15.12:0/60143614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2664103430 0x7f2664109cc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.550+0000 7f266c5ce640 1 --2- 172.21.15.12:0/60143614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2664102a70 0x7f2664102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.550+0000 7f266c5ce640 1 --2- 172.21.15.12:0/60143614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2664101880 0x7f2664101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.550+0000 7f266c5ce640 1 -- 172.21.15.12:0/60143614 >> 172.21.15.12:0/60143614 conn(0x7f26640fce10 msgr2=0x7f26640ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:53.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f266c5ce640 1 -- 172.21.15.12:0/60143614 shutdown_connections 2024-05-09T09:24:53.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f266c5ce640 1 -- 172.21.15.12:0/60143614 wait complete. 2024-05-09T09:24:53.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f266c5ce640 1 Processor -- start 2024-05-09T09:24:53.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f266c5ce640 1 -- start start 2024-05-09T09:24:53.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f266c5ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2664101880 0x7f2664199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:53.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f266c5ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2664102a70 0x7f266419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:53.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f266c5ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2664103430 0x7f26641a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:53.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f266c5ce640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f266410d4e0 con 0x7f2664101880 2024-05-09T09:24:53.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f266c5ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f266410d360 con 0x7f2664102a70 2024-05-09T09:24:53.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f266c5ce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f266410d660 con 0x7f2664103430 2024-05-09T09:24:53.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f266a36a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2664101880 0x7f2664199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:53.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f266a36a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2664101880 0x7f2664199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50036/0 (socket says 172.21.15.12:50036) 2024-05-09T09:24:53.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f266a36a640 1 -- 172.21.15.12:0/2745322674 learned_addr learned my addr 172.21.15.12:0/2745322674 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:53.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.551+0000 7f2669b69640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2664102a70 0x7f266419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:53.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.552+0000 7f266ab6b640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2664103430 0x7f26641a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:53.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.552+0000 7f266a36a640 1 -- 172.21.15.12:0/2745322674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2664103430 msgr2=0x7f26641a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.552+0000 7f266a36a640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2664103430 0x7f26641a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.552+0000 7f266a36a640 1 -- 172.21.15.12:0/2745322674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2664102a70 msgr2=0x7f266419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.552+0000 7f266a36a640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2664102a70 0x7f266419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.552+0000 7f266a36a640 1 -- 172.21.15.12:0/2745322674 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26641a1980 con 0x7f2664101880 2024-05-09T09:24:53.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.552+0000 7f2669b69640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2664102a70 0x7f266419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:53.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.552+0000 7f266a36a640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2664101880 0x7f2664199d60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f266000d950 tx=0x7f266000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:53.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.552+0000 7f265b7fe640 1 -- 172.21.15.12:0/2745322674 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660017070 con 0x7f2664101880 2024-05-09T09:24:53.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.552+0000 7f265b7fe640 1 -- 172.21.15.12:0/2745322674 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2660009c30 con 0x7f2664101880 2024-05-09T09:24:53.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.552+0000 7f266c5ce640 1 -- 172.21.15.12:0/2745322674 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26641a1c70 con 0x7f2664101880 2024-05-09T09:24:53.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.552+0000 7f265b7fe640 1 -- 172.21.15.12:0/2745322674 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266000f620 con 0x7f2664101880 2024-05-09T09:24:53.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.552+0000 7f266c5ce640 1 -- 172.21.15.12:0/2745322674 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26641a2170 con 0x7f2664101880 2024-05-09T09:24:53.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.553+0000 7f266c5ce640 1 -- 172.21.15.12:0/2745322674 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26641a2340 con 0x7f2664101880 2024-05-09T09:24:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.554+0000 7f265b7fe640 1 -- 172.21.15.12:0/2745322674 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f266001a070 con 0x7f2664101880 2024-05-09T09:24:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.555+0000 7f265b7fe640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f26500a2b90 0x7f26500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.555+0000 7f2669b69640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f26500a2b90 0x7f26500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.555+0000 7f265b7fe640 1 -- 172.21.15.12:0/2745322674 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f26600cef80 con 0x7f2664101880 2024-05-09T09:24:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.555+0000 7f2669b69640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f26500a2b90 0x7f26500a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f26540058c0 tx=0x7f2654005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:53.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.559+0000 7f265b7fe640 1 -- 172.21.15.12:0/2745322674 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2660009de0 con 0x7f2664101880 2024-05-09T09:24:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.744+0000 7f266c5ce640 1 -- 172.21.15.12:0/2745322674 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f26641a26d0 con 0x7f2664101880 2024-05-09T09:24:53.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.745+0000 7f265b7fe640 1 -- 172.21.15.12:0/2745322674 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f266009bb20 con 0x7f2664101880 2024-05-09T09:24:53.745 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:24:53.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.747+0000 7f266c5ce640 1 -- 172.21.15.12:0/2745322674 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f26500a2b90 msgr2=0x7f26500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.747+0000 7f266c5ce640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f26500a2b90 0x7f26500a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f26540058c0 tx=0x7f2654005850 comp rx=0 tx=0).stop 2024-05-09T09:24:53.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.747+0000 7f266c5ce640 1 -- 172.21.15.12:0/2745322674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2664101880 msgr2=0x7f2664199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.747+0000 7f266c5ce640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2664101880 0x7f2664199d60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f266000d950 tx=0x7f266000de10 comp rx=0 tx=0).stop 2024-05-09T09:24:53.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.747+0000 7f266c5ce640 1 -- 172.21.15.12:0/2745322674 shutdown_connections 2024-05-09T09:24:53.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.747+0000 7f266c5ce640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f26500a2b90 0x7f26500a5050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.747+0000 7f266c5ce640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2664103430 0x7f26641a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.747+0000 7f266c5ce640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2664102a70 0x7f266419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.747+0000 7f266c5ce640 1 --2- 172.21.15.12:0/2745322674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2664101880 0x7f2664199d60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.747+0000 7f266c5ce640 1 -- 172.21.15.12:0/2745322674 >> 172.21.15.12:0/2745322674 conn(0x7f26640fce10 msgr2=0x7f2664073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:53.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.747+0000 7f266c5ce640 1 -- 172.21.15.12:0/2745322674 shutdown_connections 2024-05-09T09:24:53.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.747+0000 7f266c5ce640 1 -- 172.21.15.12:0/2745322674 wait complete. 2024-05-09T09:24:53.758 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:24:53.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.867+0000 7fa88c963640 1 -- 172.21.15.12:0/4056058157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8881013d0 msgr2=0x7fa888108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.867+0000 7fa88c963640 1 --2- 172.21.15.12:0/4056058157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8881013d0 0x7fa888108c70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa880009930 tx=0x7fa88002a420 comp rx=0 tx=0).stop 2024-05-09T09:24:53.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.867+0000 7fa88c963640 1 -- 172.21.15.12:0/4056058157 shutdown_connections 2024-05-09T09:24:53.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.867+0000 7fa88c963640 1 --2- 172.21.15.12:0/4056058157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8881013d0 0x7fa888108c70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.867+0000 7fa88c963640 1 --2- 172.21.15.12:0/4056058157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa888100a10 0x7fa888100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.867+0000 7fa88c963640 1 --2- 172.21.15.12:0/4056058157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa888101d70 0x7fa888102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.867+0000 7fa88c963640 1 -- 172.21.15.12:0/4056058157 >> 172.21.15.12:0/4056058157 conn(0x7fa8880fb460 msgr2=0x7fa8880fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.867+0000 7fa88c963640 1 -- 172.21.15.12:0/4056058157 shutdown_connections 2024-05-09T09:24:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.867+0000 7fa88c963640 1 -- 172.21.15.12:0/4056058157 wait complete. 2024-05-09T09:24:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.868+0000 7fa88c963640 1 Processor -- start 2024-05-09T09:24:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.868+0000 7fa88c963640 1 -- start start 2024-05-09T09:24:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.868+0000 7fa88c963640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa888100a10 0x7fa888196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.868+0000 7fa88c963640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8881013d0 0x7fa888196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.868+0000 7fa88c963640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa888101d70 0x7fa88819e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.868+0000 7fa88c963640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa88810c410 con 0x7fa8881013d0 2024-05-09T09:24:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.868+0000 7fa88c963640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa88810c290 con 0x7fa888101d70 2024-05-09T09:24:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.868+0000 7fa88c963640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa88810c590 con 0x7fa888100a10 2024-05-09T09:24:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.868+0000 7fa885d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8881013d0 0x7fa888196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa885d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8881013d0 0x7fa888196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50054/0 (socket says 172.21.15.12:50054) 2024-05-09T09:24:53.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa885d9b640 1 -- 172.21.15.12:0/3199069511 learned_addr learned my addr 172.21.15.12:0/3199069511 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:53.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa88659c640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa888100a10 0x7fa888196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:53.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa886d9d640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa888101d70 0x7fa88819e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:53.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa885d9b640 1 -- 172.21.15.12:0/3199069511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa888100a10 msgr2=0x7fa888196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa885d9b640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa888100a10 0x7fa888196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa885d9b640 1 -- 172.21.15.12:0/3199069511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa888101d70 msgr2=0x7fa88819e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:53.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa885d9b640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa888101d70 0x7fa88819e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:53.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa885d9b640 1 -- 172.21.15.12:0/3199069511 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88819e6a0 con 0x7fa8881013d0 2024-05-09T09:24:53.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa885d9b640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8881013d0 0x7fa888196fc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009900 tx=0x7fa87c009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:53.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa8777fe640 1 -- 172.21.15.12:0/3199069511 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c00d030 con 0x7fa8881013d0 2024-05-09T09:24:53.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa88c963640 1 -- 172.21.15.12:0/3199069511 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8880ff960 con 0x7fa8881013d0 2024-05-09T09:24:53.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa8777fe640 1 -- 172.21.15.12:0/3199069511 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa87c00d1d0 con 0x7fa8881013d0 2024-05-09T09:24:53.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa8777fe640 1 -- 172.21.15.12:0/3199069511 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c005260 con 0x7fa8881013d0 2024-05-09T09:24:53.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.869+0000 7fa88c963640 1 -- 172.21.15.12:0/3199069511 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8880ffeb0 con 0x7fa8881013d0 2024-05-09T09:24:53.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.870+0000 7fa88c963640 1 -- 172.21.15.12:0/3199069511 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa888068aa0 con 0x7fa8881013d0 2024-05-09T09:24:53.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.871+0000 7fa8777fe640 1 -- 172.21.15.12:0/3199069511 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa87c005400 con 0x7fa8881013d0 2024-05-09T09:24:53.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.872+0000 7fa8777fe640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa8700a2b90 0x7fa8700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:53.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.872+0000 7fa88659c640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa8700a2b90 0x7fa8700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:53.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.872+0000 7fa8777fe640 1 -- 172.21.15.12:0/3199069511 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa87c0d0b20 con 0x7fa8881013d0 2024-05-09T09:24:53.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.873+0000 7fa88659c640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa8700a2b90 0x7fa8700a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa888069bf0 tx=0x7fa878009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:53.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:53.877+0000 7fa8777fe640 1 -- 172.21.15.12:0/3199069511 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa87c09d720 con 0x7fa8881013d0 2024-05-09T09:24:54.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.061+0000 7fa88c963640 1 -- 172.21.15.12:0/3199069511 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa88819e870 con 0x7fa8881013d0 2024-05-09T09:24:54.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.061+0000 7fa8777fe640 1 -- 172.21.15.12:0/3199069511 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fa87c09d020 con 0x7fa8881013d0 2024-05-09T09:24:54.061 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T09:24:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.063+0000 7fa88c963640 1 -- 172.21.15.12:0/3199069511 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa8700a2b90 msgr2=0x7fa8700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.063+0000 7fa88c963640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa8700a2b90 0x7fa8700a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa888069bf0 tx=0x7fa878009680 comp rx=0 tx=0).stop 2024-05-09T09:24:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.063+0000 7fa88c963640 1 -- 172.21.15.12:0/3199069511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8881013d0 msgr2=0x7fa888196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.063+0000 7fa88c963640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8881013d0 0x7fa888196fc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009900 tx=0x7fa87c009dc0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.063+0000 7fa88c963640 1 -- 172.21.15.12:0/3199069511 shutdown_connections 2024-05-09T09:24:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.063+0000 7fa88c963640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa8700a2b90 0x7fa8700a5050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.063+0000 7fa88c963640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa888101d70 0x7fa88819e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.063+0000 7fa88c963640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8881013d0 0x7fa888196fc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.063+0000 7fa88c963640 1 --2- 172.21.15.12:0/3199069511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa888100a10 0x7fa888196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.063+0000 7fa88c963640 1 -- 172.21.15.12:0/3199069511 >> 172.21.15.12:0/3199069511 conn(0x7fa8880fb460 msgr2=0x7fa888104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.063+0000 7fa88c963640 1 -- 172.21.15.12:0/3199069511 shutdown_connections 2024-05-09T09:24:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.063+0000 7fa88c963640 1 -- 172.21.15.12:0/3199069511 wait complete. 2024-05-09T09:24:54.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:24:54.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.183+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/2740481655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c581023b0 msgr2=0x7f7c581047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.183+0000 7f7c5f6ee640 1 --2- 172.21.15.12:0/2740481655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c581023b0 0x7f7c581047a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009900 tx=0x7f7c5402a410 comp rx=0 tx=0).stop 2024-05-09T09:24:54.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.183+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/2740481655 shutdown_connections 2024-05-09T09:24:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.183+0000 7f7c5f6ee640 1 --2- 172.21.15.12:0/2740481655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58107610 0x7f7c58109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.183+0000 7f7c5f6ee640 1 --2- 172.21.15.12:0/2740481655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c58104ce0 0x7f7c581070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.183+0000 7f7c5f6ee640 1 --2- 172.21.15.12:0/2740481655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c581023b0 0x7f7c581047a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.183+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/2740481655 >> 172.21.15.12:0/2740481655 conn(0x7f7c580fbd20 msgr2=0x7f7c580fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.184+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/2740481655 shutdown_connections 2024-05-09T09:24:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.184+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/2740481655 wait complete. 2024-05-09T09:24:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.184+0000 7f7c5f6ee640 1 Processor -- start 2024-05-09T09:24:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.184+0000 7f7c5f6ee640 1 -- start start 2024-05-09T09:24:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.184+0000 7f7c5f6ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c581023b0 0x7f7c58197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.184+0000 7f7c5f6ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58104ce0 0x7f7c58198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.184+0000 7f7c5f6ee640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c58107610 0x7f7c5819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:54.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.184+0000 7f7c5f6ee640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5810d510 con 0x7f7c58107610 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.184+0000 7f7c5f6ee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5810d390 con 0x7f7c58104ce0 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.184+0000 7f7c5f6ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5810d690 con 0x7f7c581023b0 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c5dc8b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c58107610 0x7f7c5819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c5dc8b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c58107610 0x7f7c5819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50062/0 (socket says 172.21.15.12:50062) 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c5dc8b640 1 -- 172.21.15.12:0/3353882109 learned_addr learned my addr 172.21.15.12:0/3353882109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c5d48a640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c581023b0 0x7f7c58197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c5cc89640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58104ce0 0x7f7c58198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c5dc8b640 1 -- 172.21.15.12:0/3353882109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c581023b0 msgr2=0x7f7c58197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c5dc8b640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c581023b0 0x7f7c58197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c5dc8b640 1 -- 172.21.15.12:0/3353882109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58104ce0 msgr2=0x7f7c58198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c5dc8b640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58104ce0 0x7f7c58198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c5dc8b640 1 -- 172.21.15.12:0/3353882109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5819f850 con 0x7f7c58107610 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c5dc8b640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c58107610 0x7f7c5819f1b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000cad0 tx=0x7f7c5000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c4a7fc640 1 -- 172.21.15.12:0/3353882109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000ae20 con 0x7f7c58107610 2024-05-09T09:24:54.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/3353882109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5819fb40 con 0x7f7c58107610 2024-05-09T09:24:54.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c4a7fc640 1 -- 172.21.15.12:0/3353882109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7c50005020 con 0x7f7c58107610 2024-05-09T09:24:54.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c4a7fc640 1 -- 172.21.15.12:0/3353882109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000f690 con 0x7f7c58107610 2024-05-09T09:24:54.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.185+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/3353882109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c581a0040 con 0x7f7c58107610 2024-05-09T09:24:54.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.186+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/3353882109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c580fd810 con 0x7f7c58107610 2024-05-09T09:24:54.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.188+0000 7f7c4a7fc640 1 -- 172.21.15.12:0/3353882109 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7c500051d0 con 0x7f7c58107610 2024-05-09T09:24:54.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.188+0000 7f7c4a7fc640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7c440a2b90 0x7f7c440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:54.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.189+0000 7f7c5d48a640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7c440a2b90 0x7f7c440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:54.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.189+0000 7f7c4a7fc640 1 -- 172.21.15.12:0/3353882109 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7c500cf380 con 0x7f7c58107610 2024-05-09T09:24:54.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.189+0000 7f7c5d48a640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7c440a2b90 0x7f7c440a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7c54006010 tx=0x7f7c54031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:54.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.193+0000 7f7c4a7fc640 1 -- 172.21.15.12:0/3353882109 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7c50014030 con 0x7f7c58107610 2024-05-09T09:24:54.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.382+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/3353882109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7c580624d0 con 0x7f7c58107610 2024-05-09T09:24:54.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.382+0000 7f7c4a7fc640 1 -- 172.21.15.12:0/3353882109 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f7c5009b930 con 0x7f7c58107610 2024-05-09T09:24:54.382 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:24:54.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.384+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/3353882109 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7c440a2b90 msgr2=0x7f7c440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.384+0000 7f7c5f6ee640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7c440a2b90 0x7f7c440a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7c54006010 tx=0x7f7c54031510 comp rx=0 tx=0).stop 2024-05-09T09:24:54.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.384+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/3353882109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c58107610 msgr2=0x7f7c5819f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.384+0000 7f7c5f6ee640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c58107610 0x7f7c5819f1b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000cad0 tx=0x7f7c5000cf90 comp rx=0 tx=0).stop 2024-05-09T09:24:54.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.384+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/3353882109 shutdown_connections 2024-05-09T09:24:54.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.385+0000 7f7c5f6ee640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7c440a2b90 0x7f7c440a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.385+0000 7f7c5f6ee640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c58107610 0x7f7c5819f1b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.385+0000 7f7c5f6ee640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58104ce0 0x7f7c58198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.385+0000 7f7c5f6ee640 1 --2- 172.21.15.12:0/3353882109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c581023b0 0x7f7c58197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.385+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/3353882109 >> 172.21.15.12:0/3353882109 conn(0x7f7c580fbd20 msgr2=0x7f7c58102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:54.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.385+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/3353882109 shutdown_connections 2024-05-09T09:24:54.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.385+0000 7f7c5f6ee640 1 -- 172.21.15.12:0/3353882109 wait complete. 2024-05-09T09:24:54.394 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:54 smithi012 ceph-mon[30140]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:24:54.394 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:54 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/871005046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:24:54.395 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:54 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2745322674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:24:54.395 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:54 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3199069511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:24:54.396 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:24:54.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:54 smithi170 ceph-mon[33632]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:24:54.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:54 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/871005046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:24:54.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:54 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2745322674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:24:54.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:54 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3199069511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:24:54.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.501+0000 7f1731f5c640 1 -- 172.21.15.12:0/125993142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f172c1018a0 msgr2=0x7f172c101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.501+0000 7f1731f5c640 1 --2- 172.21.15.12:0/125993142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f172c1018a0 0x7f172c101cc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f171c009900 tx=0x7f171c02a410 comp rx=0 tx=0).stop 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 -- 172.21.15.12:0/125993142 shutdown_connections 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 --2- 172.21.15.12:0/125993142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c103450 0x7f172c109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 --2- 172.21.15.12:0/125993142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f172c102a90 0x7f172c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 --2- 172.21.15.12:0/125993142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f172c1018a0 0x7f172c101cc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 -- 172.21.15.12:0/125993142 >> 172.21.15.12:0/125993142 conn(0x7f172c0fce30 msgr2=0x7f172c0ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 -- 172.21.15.12:0/125993142 shutdown_connections 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 -- 172.21.15.12:0/125993142 wait complete. 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 Processor -- start 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 -- start start 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c1018a0 0x7f172c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f172c102a90 0x7f172c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f172c103450 0x7f172c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f172c10d500 con 0x7f172c103450 2024-05-09T09:24:54.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f172c10d380 con 0x7f172c1018a0 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f1731f5c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f172c10d680 con 0x7f172c102a90 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f172bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f172c103450 0x7f172c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f172bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f172c103450 0x7f172c19f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50074/0 (socket says 172.21.15.12:50074) 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.502+0000 7f172bfff640 1 -- 172.21.15.12:0/3731205221 learned_addr learned my addr 172.21.15.12:0/3731205221 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f1723fff640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f172c102a90 0x7f172c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f172b7fe640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c1018a0 0x7f172c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f172bfff640 1 -- 172.21.15.12:0/3731205221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f172c102a90 msgr2=0x7f172c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f172bfff640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f172c102a90 0x7f172c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f172bfff640 1 -- 172.21.15.12:0/3731205221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c1018a0 msgr2=0x7f172c197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f172bfff640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c1018a0 0x7f172c197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f172bfff640 1 -- 172.21.15.12:0/3731205221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c19f790 con 0x7f172c103450 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f172b7fe640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c1018a0 0x7f172c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f172bfff640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f172c103450 0x7f172c19f120 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f172400cce0 tx=0x7f172400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f17297fa640 1 -- 172.21.15.12:0/3731205221 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172400f450 con 0x7f172c103450 2024-05-09T09:24:54.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f17297fa640 1 -- 172.21.15.12:0/3731205221 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f172400fac0 con 0x7f172c103450 2024-05-09T09:24:54.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f1731f5c640 1 -- 172.21.15.12:0/3731205221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172c19fa80 con 0x7f172c103450 2024-05-09T09:24:54.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f17297fa640 1 -- 172.21.15.12:0/3731205221 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724018be0 con 0x7f172c103450 2024-05-09T09:24:54.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.503+0000 7f1731f5c640 1 -- 172.21.15.12:0/3731205221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172c19ff80 con 0x7f172c103450 2024-05-09T09:24:54.506 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:54 smithi016 ceph-mon[33335]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:24:54.506 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:54 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/871005046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:24:54.506 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:54 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2745322674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:24:54.507 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:54 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3199069511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:24:54.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.504+0000 7f1731f5c640 1 -- 172.21.15.12:0/3731205221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172c108bc0 con 0x7f172c103450 2024-05-09T09:24:54.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.505+0000 7f17297fa640 1 -- 172.21.15.12:0/3731205221 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f172400fc70 con 0x7f172c103450 2024-05-09T09:24:54.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.506+0000 7f17297fa640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17180a2b90 0x7f17180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:54.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.507+0000 7f172b7fe640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17180a2b90 0x7f17180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:54.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.507+0000 7f17297fa640 1 -- 172.21.15.12:0/3731205221 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f17240cfac0 con 0x7f172c103450 2024-05-09T09:24:54.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.507+0000 7f172b7fe640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17180a2b90 0x7f17180a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f171c000c00 tx=0x7f171c031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:54.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.511+0000 7f17297fa640 1 -- 172.21.15.12:0/3731205221 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1724014030 con 0x7f172c103450 2024-05-09T09:24:54.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.738+0000 7f1731f5c640 1 -- 172.21.15.12:0/3731205221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f172c02d1c0 con 0x7f172c103450 2024-05-09T09:24:54.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.739+0000 7f17297fa640 1 -- 172.21.15.12:0/3731205221 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11931 (secure 0 0 0) 0x7f172409c6c0 con 0x7f172c103450 2024-05-09T09:24:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:24:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:23:34.596143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[24523,24502,24515,24484,14628]},"id":1}]} 2024-05-09T09:24:54.739 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-05-09T09:24:54.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.741+0000 7f1731f5c640 1 -- 172.21.15.12:0/3731205221 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17180a2b90 msgr2=0x7f17180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.742+0000 7f1731f5c640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17180a2b90 0x7f17180a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f171c000c00 tx=0x7f171c031510 comp rx=0 tx=0).stop 2024-05-09T09:24:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.742+0000 7f1731f5c640 1 -- 172.21.15.12:0/3731205221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f172c103450 msgr2=0x7f172c19f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.742+0000 7f1731f5c640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f172c103450 0x7f172c19f120 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f172400cce0 tx=0x7f172400a620 comp rx=0 tx=0).stop 2024-05-09T09:24:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.742+0000 7f1731f5c640 1 -- 172.21.15.12:0/3731205221 shutdown_connections 2024-05-09T09:24:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.742+0000 7f1731f5c640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17180a2b90 0x7f17180a5050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.742+0000 7f1731f5c640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f172c103450 0x7f172c19f120 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.742+0000 7f1731f5c640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f172c102a90 0x7f172c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.742+0000 7f1731f5c640 1 --2- 172.21.15.12:0/3731205221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c1018a0 0x7f172c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.742+0000 7f1731f5c640 1 -- 172.21.15.12:0/3731205221 >> 172.21.15.12:0/3731205221 conn(0x7f172c0fce30 msgr2=0x7f172c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.742+0000 7f1731f5c640 1 -- 172.21.15.12:0/3731205221 shutdown_connections 2024-05-09T09:24:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.742+0000 7f1731f5c640 1 -- 172.21.15.12:0/3731205221 wait complete. 2024-05-09T09:24:54.753 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:24:54.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.858+0000 7fb7710da640 1 -- 172.21.15.12:0/3479687009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb76c100770 msgr2=0x7fb76c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.858+0000 7fb7710da640 1 --2- 172.21.15.12:0/3479687009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb76c100770 0x7fb76c100b90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009ca0 tx=0x7fb75c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.858+0000 7fb7710da640 1 -- 172.21.15.12:0/3479687009 shutdown_connections 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.858+0000 7fb7710da640 1 --2- 172.21.15.12:0/3479687009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb76c102320 0x7fb76c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.858+0000 7fb7710da640 1 --2- 172.21.15.12:0/3479687009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb76c101960 0x7fb76c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.858+0000 7fb7710da640 1 --2- 172.21.15.12:0/3479687009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb76c100770 0x7fb76c100b90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.858+0000 7fb7710da640 1 -- 172.21.15.12:0/3479687009 >> 172.21.15.12:0/3479687009 conn(0x7fb76c0fbce0 msgr2=0x7fb76c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.858+0000 7fb7710da640 1 -- 172.21.15.12:0/3479687009 shutdown_connections 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.858+0000 7fb7710da640 1 -- 172.21.15.12:0/3479687009 wait complete. 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.859+0000 7fb7710da640 1 Processor -- start 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.859+0000 7fb7710da640 1 -- start start 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.859+0000 7fb7710da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb76c100770 0x7fb76c104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.859+0000 7fb7710da640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb76c101960 0x7fb76c102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.859+0000 7fb7710da640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb76c102320 0x7fb76c103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.859+0000 7fb7710da640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c10c4b0 con 0x7fb76c102320 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.859+0000 7fb7710da640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c10c330 con 0x7fb76c100770 2024-05-09T09:24:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.859+0000 7fb7710da640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c10c630 con 0x7fb76c101960 2024-05-09T09:24:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb76b59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb76c102320 0x7fb76c103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb76b59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb76c102320 0x7fb76c103280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50080/0 (socket says 172.21.15.12:50080) 2024-05-09T09:24:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb76b59e640 1 -- 172.21.15.12:0/133371291 learned_addr learned my addr 172.21.15.12:0/133371291 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb76ad9d640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb76c100770 0x7fb76c104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb76a59c640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb76c101960 0x7fb76c102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb76b59e640 1 -- 172.21.15.12:0/133371291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb76c101960 msgr2=0x7fb76c102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb76b59e640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb76c101960 0x7fb76c102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb76b59e640 1 -- 172.21.15.12:0/133371291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb76c100770 msgr2=0x7fb76c104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb76b59e640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb76c100770 0x7fb76c104700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb76b59e640 1 -- 172.21.15.12:0/133371291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c103b10 con 0x7fb76c102320 2024-05-09T09:24:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb76b59e640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb76c102320 0x7fb76c103280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb764009fd0 tx=0x7fb76400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb753fff640 1 -- 172.21.15.12:0/133371291 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764011830 con 0x7fb76c102320 2024-05-09T09:24:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb753fff640 1 -- 172.21.15.12:0/133371291 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb764016070 con 0x7fb76c102320 2024-05-09T09:24:54.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb7710da640 1 -- 172.21.15.12:0/133371291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76c1a1e50 con 0x7fb76c102320 2024-05-09T09:24:54.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb753fff640 1 -- 172.21.15.12:0/133371291 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76400f2e0 con 0x7fb76c102320 2024-05-09T09:24:54.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.860+0000 7fb7710da640 1 -- 172.21.15.12:0/133371291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76c1a2330 con 0x7fb76c102320 2024-05-09T09:24:54.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.861+0000 7fb7710da640 1 -- 172.21.15.12:0/133371291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb76c1a2620 con 0x7fb76c102320 2024-05-09T09:24:54.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.862+0000 7fb753fff640 1 -- 172.21.15.12:0/133371291 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb7640119d0 con 0x7fb76c102320 2024-05-09T09:24:54.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.863+0000 7fb753fff640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb7540a2b90 0x7fb7540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:54.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.863+0000 7fb753fff640 1 -- 172.21.15.12:0/133371291 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb7640cead0 con 0x7fb76c102320 2024-05-09T09:24:54.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.863+0000 7fb76ad9d640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb7540a2b90 0x7fb7540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:54.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.864+0000 7fb76ad9d640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb7540a2b90 0x7fb7540a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb75c000f80 tx=0x7fb75c005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:54.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:54.868+0000 7fb753fff640 1 -- 172.21.15.12:0/133371291 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb76409b6d0 con 0x7fb76c102320 2024-05-09T09:24:55.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.056+0000 7fb7710da640 1 -- 172.21.15.12:0/133371291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb76c1a2980 con 0x7fb76c102320 2024-05-09T09:24:55.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.057+0000 7fb753fff640 1 -- 172.21.15.12:0/133371291 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24575 (secure 0 0 0) 0x7fb76409afd0 con 0x7fb76c102320 2024-05-09T09:24:55.057 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:24:55.057 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":91,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:24:05.727743+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:24:55.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.059+0000 7fb7710da640 1 -- 172.21.15.12:0/133371291 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb7540a2b90 msgr2=0x7fb7540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.059+0000 7fb7710da640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb7540a2b90 0x7fb7540a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb75c000f80 tx=0x7fb75c005f80 comp rx=0 tx=0).stop 2024-05-09T09:24:55.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.059+0000 7fb7710da640 1 -- 172.21.15.12:0/133371291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb76c102320 msgr2=0x7fb76c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.059+0000 7fb7710da640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb76c102320 0x7fb76c103280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb764009fd0 tx=0x7fb76400b8e0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.059+0000 7fb7710da640 1 -- 172.21.15.12:0/133371291 shutdown_connections 2024-05-09T09:24:55.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.059+0000 7fb7710da640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb7540a2b90 0x7fb7540a5050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.059+0000 7fb7710da640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb76c102320 0x7fb76c103280 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.059+0000 7fb7710da640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb76c101960 0x7fb76c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.059+0000 7fb7710da640 1 --2- 172.21.15.12:0/133371291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb76c100770 0x7fb76c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.059+0000 7fb7710da640 1 -- 172.21.15.12:0/133371291 >> 172.21.15.12:0/133371291 conn(0x7fb76c0fbce0 msgr2=0x7fb76c068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.059+0000 7fb7710da640 1 -- 172.21.15.12:0/133371291 shutdown_connections 2024-05-09T09:24:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.059+0000 7fb7710da640 1 -- 172.21.15.12:0/133371291 wait complete. 2024-05-09T09:24:55.071 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:23:08.167107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:23:09.065670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:23:12.773640+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:24:55.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:24:55.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.182+0000 7fa13f1d5640 1 -- 172.21.15.12:0/4125529835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa138102320 msgr2=0x7fa138108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.182+0000 7fa13f1d5640 1 --2- 172.21.15.12:0/4125529835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa138102320 0x7fa138108bb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa130009ca0 tx=0x7fa13002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.182+0000 7fa13f1d5640 1 -- 172.21.15.12:0/4125529835 shutdown_connections 2024-05-09T09:24:55.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.182+0000 7fa13f1d5640 1 --2- 172.21.15.12:0/4125529835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa138102320 0x7fa138108bb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.182+0000 7fa13f1d5640 1 --2- 172.21.15.12:0/4125529835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa138101960 0x7fa138101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.182+0000 7fa13f1d5640 1 --2- 172.21.15.12:0/4125529835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138100770 0x7fa138100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.182+0000 7fa13f1d5640 1 -- 172.21.15.12:0/4125529835 >> 172.21.15.12:0/4125529835 conn(0x7fa1380fbce0 msgr2=0x7fa1380fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.182+0000 7fa13f1d5640 1 -- 172.21.15.12:0/4125529835 shutdown_connections 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.182+0000 7fa13f1d5640 1 -- 172.21.15.12:0/4125529835 wait complete. 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.182+0000 7fa13f1d5640 1 Processor -- start 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.182+0000 7fa13f1d5640 1 -- start start 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa13f1d5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa138100770 0x7fa138196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa13f1d5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa138101960 0x7fa138196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa13f1d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138102320 0x7fa13819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa13f1d5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa13810c3d0 con 0x7fa138101960 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa13f1d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa13810c250 con 0x7fa138102320 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa13f1d5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa13810c550 con 0x7fa138100770 2024-05-09T09:24:55.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa137fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa138101960 0x7fa138196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa137fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa138101960 0x7fa138196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50094/0 (socket says 172.21.15.12:50094) 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa137fff640 1 -- 172.21.15.12:0/698064583 learned_addr learned my addr 172.21.15.12:0/698064583 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa13cf71640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa138100770 0x7fa138196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa13d772640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138102320 0x7fa13819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa137fff640 1 -- 172.21.15.12:0/698064583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa138100770 msgr2=0x7fa138196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa137fff640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa138100770 0x7fa138196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa137fff640 1 -- 172.21.15.12:0/698064583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138102320 msgr2=0x7fa13819e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa137fff640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138102320 0x7fa13819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa137fff640 1 -- 172.21.15.12:0/698064583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13819e690 con 0x7fa138101960 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa137fff640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa138101960 0x7fa138196fb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa12000e9f0 tx=0x7fa12000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa135ffb640 1 -- 172.21.15.12:0/698064583 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12000cd80 con 0x7fa138101960 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa135ffb640 1 -- 172.21.15.12:0/698064583 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa1200054f0 con 0x7fa138101960 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa135ffb640 1 -- 172.21.15.12:0/698064583 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120010690 con 0x7fa138101960 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.183+0000 7fa13f1d5640 1 -- 172.21.15.12:0/698064583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13819e980 con 0x7fa138101960 2024-05-09T09:24:55.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.184+0000 7fa13f1d5640 1 -- 172.21.15.12:0/698064583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13819eed0 con 0x7fa138101960 2024-05-09T09:24:55.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.185+0000 7fa13f1d5640 1 -- 172.21.15.12:0/698064583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa138107b10 con 0x7fa138101960 2024-05-09T09:24:55.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.186+0000 7fa135ffb640 1 -- 172.21.15.12:0/698064583 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa120004f20 con 0x7fa138101960 2024-05-09T09:24:55.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.187+0000 7fa135ffb640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa1240a2c10 0x7fa1240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:55.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.187+0000 7fa13cf71640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa1240a2c10 0x7fa1240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:55.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.187+0000 7fa135ffb640 1 -- 172.21.15.12:0/698064583 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa1200cf860 con 0x7fa138101960 2024-05-09T09:24:55.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.188+0000 7fa13cf71640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa1240a2c10 0x7fa1240a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa1280095f0 tx=0x7fa128005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:55.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.192+0000 7fa135ffb640 1 -- 172.21.15.12:0/698064583 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa12009c400 con 0x7fa138101960 2024-05-09T09:24:55.353 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:55 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3353882109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:24:55.353 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:55 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3731205221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:55.354 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:55 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/133371291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:24:55.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.380+0000 7fa13f1d5640 1 -- 172.21.15.12:0/698064583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa138062460 con 0x7fa138101960 2024-05-09T09:24:55.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.381+0000 7fa135ffb640 1 -- 172.21.15.12:0/698064583 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fa12009bd00 con 0x7fa138101960 2024-05-09T09:24:55.380 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:24:55.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.383+0000 7fa13f1d5640 1 -- 172.21.15.12:0/698064583 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa1240a2c10 msgr2=0x7fa1240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.383+0000 7fa13f1d5640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa1240a2c10 0x7fa1240a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa1280095f0 tx=0x7fa128005f90 comp rx=0 tx=0).stop 2024-05-09T09:24:55.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.383+0000 7fa13f1d5640 1 -- 172.21.15.12:0/698064583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa138101960 msgr2=0x7fa138196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.383+0000 7fa13f1d5640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa138101960 0x7fa138196fb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa12000e9f0 tx=0x7fa12000edf0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.383+0000 7fa13f1d5640 1 -- 172.21.15.12:0/698064583 shutdown_connections 2024-05-09T09:24:55.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.383+0000 7fa13f1d5640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa1240a2c10 0x7fa1240a50d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.383+0000 7fa13f1d5640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138102320 0x7fa13819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.383+0000 7fa13f1d5640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa138101960 0x7fa138196fb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.383+0000 7fa13f1d5640 1 --2- 172.21.15.12:0/698064583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa138100770 0x7fa138196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.383+0000 7fa13f1d5640 1 -- 172.21.15.12:0/698064583 >> 172.21.15.12:0/698064583 conn(0x7fa1380fbce0 msgr2=0x7fa138104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:55.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.383+0000 7fa13f1d5640 1 -- 172.21.15.12:0/698064583 shutdown_connections 2024-05-09T09:24:55.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.383+0000 7fa13f1d5640 1 -- 172.21.15.12:0/698064583 wait complete. 2024-05-09T09:24:55.394 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:24:55.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:55 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3353882109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:24:55.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:55 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3731205221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:55.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:55 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/133371291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:24:55.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:55 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3353882109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:24:55.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:55 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3731205221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:55.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:55 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/133371291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:24:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.505+0000 7f9fe31ed640 1 -- 172.21.15.12:0/4261767675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9fdc101970 msgr2=0x7f9fdc101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.505+0000 7f9fe31ed640 1 --2- 172.21.15.12:0/4261767675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9fdc101970 0x7f9fdc101df0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009900 tx=0x7f9fc802a410 comp rx=0 tx=0).stop 2024-05-09T09:24:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.505+0000 7f9fe31ed640 1 -- 172.21.15.12:0/4261767675 shutdown_connections 2024-05-09T09:24:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.505+0000 7f9fe31ed640 1 --2- 172.21.15.12:0/4261767675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fdc102330 0x7f9fdc108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.505+0000 7f9fe31ed640 1 --2- 172.21.15.12:0/4261767675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9fdc101970 0x7f9fdc101df0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.505+0000 7f9fe31ed640 1 --2- 172.21.15.12:0/4261767675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fdc100780 0x7f9fdc100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.505+0000 7f9fe31ed640 1 -- 172.21.15.12:0/4261767675 >> 172.21.15.12:0/4261767675 conn(0x7f9fdc0fbcf0 msgr2=0x7f9fdc0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.505+0000 7f9fe31ed640 1 -- 172.21.15.12:0/4261767675 shutdown_connections 2024-05-09T09:24:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.505+0000 7f9fe31ed640 1 -- 172.21.15.12:0/4261767675 wait complete. 2024-05-09T09:24:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.505+0000 7f9fe31ed640 1 Processor -- start 2024-05-09T09:24:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.505+0000 7f9fe31ed640 1 -- start start 2024-05-09T09:24:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe31ed640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9fdc100780 0x7f9fdc197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe31ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fdc101970 0x7f9fdc197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe31ed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fdc102330 0x7f9fdc19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe31ed640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc10c3e0 con 0x7f9fdc100780 2024-05-09T09:24:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe31ed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc10c260 con 0x7f9fdc101970 2024-05-09T09:24:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe31ed640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc10c560 con 0x7f9fdc102330 2024-05-09T09:24:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe0f89640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9fdc100780 0x7f9fdc197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe0f89640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9fdc100780 0x7f9fdc197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50112/0 (socket says 172.21.15.12:50112) 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe0f89640 1 -- 172.21.15.12:0/1979843965 learned_addr learned my addr 172.21.15.12:0/1979843965 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe178a640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fdc102330 0x7f9fdc19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fdbfff640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fdc101970 0x7f9fdc197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe0f89640 1 -- 172.21.15.12:0/1979843965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fdc102330 msgr2=0x7f9fdc19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe0f89640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fdc102330 0x7f9fdc19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe0f89640 1 -- 172.21.15.12:0/1979843965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fdc101970 msgr2=0x7f9fdc197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe0f89640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fdc101970 0x7f9fdc197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe0f89640 1 -- 172.21.15.12:0/1979843965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fdc19f6d0 con 0x7f9fdc100780 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe178a640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fdc102330 0x7f9fdc19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe0f89640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9fdc100780 0x7f9fdc197ab0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000c970 tx=0x7f9fd000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fd9ffb640 1 -- 172.21.15.12:0/1979843965 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0007c10 con 0x7f9fdc100780 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fd9ffb640 1 -- 172.21.15.12:0/1979843965 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9fd0007db0 con 0x7f9fdc100780 2024-05-09T09:24:55.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fe31ed640 1 -- 172.21.15.12:0/1979843965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fdc19f9c0 con 0x7f9fdc100780 2024-05-09T09:24:55.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.506+0000 7f9fd9ffb640 1 -- 172.21.15.12:0/1979843965 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd000f8e0 con 0x7f9fdc100780 2024-05-09T09:24:55.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.507+0000 7f9fe31ed640 1 -- 172.21.15.12:0/1979843965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fdc19ff10 con 0x7f9fdc100780 2024-05-09T09:24:55.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.508+0000 7f9fe31ed640 1 -- 172.21.15.12:0/1979843965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fdc1a01a0 con 0x7f9fdc100780 2024-05-09T09:24:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.509+0000 7f9fd9ffb640 1 -- 172.21.15.12:0/1979843965 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9fd000fa80 con 0x7f9fdc100780 2024-05-09T09:24:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.509+0000 7f9fd9ffb640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9fcc0a2b90 0x7f9fcc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.509+0000 7f9fdbfff640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9fcc0a2b90 0x7f9fcc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.509+0000 7f9fd9ffb640 1 -- 172.21.15.12:0/1979843965 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9fd00cf7c0 con 0x7f9fdc100780 2024-05-09T09:24:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.510+0000 7f9fdbfff640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9fcc0a2b90 0x7f9fcc0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8005c40 tx=0x7f9fc8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.514+0000 7f9fd9ffb640 1 -- 172.21.15.12:0/1979843965 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9fd009c450 con 0x7f9fdc100780 2024-05-09T09:24:55.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.706+0000 7f9fe31ed640 1 -- 172.21.15.12:0/1979843965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9fdc1a0370 con 0x7f9fdc100780 2024-05-09T09:24:55.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.706+0000 7f9fd9ffb640 1 -- 172.21.15.12:0/1979843965 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f9fd009bd50 con 0x7f9fdc100780 2024-05-09T09:24:55.706 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:24:55.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.708+0000 7f9fe31ed640 1 -- 172.21.15.12:0/1979843965 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9fcc0a2b90 msgr2=0x7f9fcc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.708+0000 7f9fe31ed640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9fcc0a2b90 0x7f9fcc0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8005c40 tx=0x7f9fc8000bc0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.708+0000 7f9fe31ed640 1 -- 172.21.15.12:0/1979843965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9fdc100780 msgr2=0x7f9fdc197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.708+0000 7f9fe31ed640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9fdc100780 0x7f9fdc197ab0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000c970 tx=0x7f9fd000ce30 comp rx=0 tx=0).stop 2024-05-09T09:24:55.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.708+0000 7f9fe31ed640 1 -- 172.21.15.12:0/1979843965 shutdown_connections 2024-05-09T09:24:55.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.708+0000 7f9fe31ed640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9fcc0a2b90 0x7f9fcc0a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.708+0000 7f9fe31ed640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fdc102330 0x7f9fdc19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.708+0000 7f9fe31ed640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fdc101970 0x7f9fdc197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.708+0000 7f9fe31ed640 1 --2- 172.21.15.12:0/1979843965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9fdc100780 0x7f9fdc197ab0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.708+0000 7f9fe31ed640 1 -- 172.21.15.12:0/1979843965 >> 172.21.15.12:0/1979843965 conn(0x7f9fdc0fbcf0 msgr2=0x7f9fdc069dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:55.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.709+0000 7f9fe31ed640 1 -- 172.21.15.12:0/1979843965 shutdown_connections 2024-05-09T09:24:55.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.709+0000 7f9fe31ed640 1 -- 172.21.15.12:0/1979843965 wait complete. 2024-05-09T09:24:55.720 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:24:55.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.832+0000 7f25576a5640 1 -- 172.21.15.12:0/3440031462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2550101960 msgr2=0x7f2550101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.832+0000 7f25576a5640 1 --2- 172.21.15.12:0/3440031462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2550101960 0x7f2550101de0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2544009900 tx=0x7f254402a410 comp rx=0 tx=0).stop 2024-05-09T09:24:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.832+0000 7f25576a5640 1 -- 172.21.15.12:0/3440031462 shutdown_connections 2024-05-09T09:24:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.832+0000 7f25576a5640 1 --2- 172.21.15.12:0/3440031462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2550102320 0x7f2550108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.832+0000 7f25576a5640 1 --2- 172.21.15.12:0/3440031462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2550101960 0x7f2550101de0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.832+0000 7f25576a5640 1 --2- 172.21.15.12:0/3440031462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2550100770 0x7f2550100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.832+0000 7f25576a5640 1 -- 172.21.15.12:0/3440031462 >> 172.21.15.12:0/3440031462 conn(0x7f25500fbce0 msgr2=0x7f25500fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.832+0000 7f25576a5640 1 -- 172.21.15.12:0/3440031462 shutdown_connections 2024-05-09T09:24:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.832+0000 7f25576a5640 1 -- 172.21.15.12:0/3440031462 wait complete. 2024-05-09T09:24:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.832+0000 7f25576a5640 1 Processor -- start 2024-05-09T09:24:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.832+0000 7f25576a5640 1 -- start start 2024-05-09T09:24:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f25576a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2550100770 0x7f255010b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f25576a5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2550101960 0x7f2550109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f25576a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2550102320 0x7f2550109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f25576a5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f255010d4e0 con 0x7f2550101960 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f25576a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f255010d360 con 0x7f2550100770 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f25576a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f255010d660 con 0x7f2550102320 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f2554c40640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2550101960 0x7f2550109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f2554c40640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2550101960 0x7f2550109730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50132/0 (socket says 172.21.15.12:50132) 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f2554c40640 1 -- 172.21.15.12:0/393289969 learned_addr learned my addr 172.21.15.12:0/393289969 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f2554c40640 1 -- 172.21.15.12:0/393289969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2550102320 msgr2=0x7f2550109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f2554c40640 1 --2- 172.21.15.12:0/393289969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2550102320 0x7f2550109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f2555441640 1 --2- 172.21.15.12:0/393289969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2550100770 0x7f255010b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f2554c40640 1 -- 172.21.15.12:0/393289969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2550100770 msgr2=0x7f255010b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f2554c40640 1 --2- 172.21.15.12:0/393289969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2550100770 0x7f255010b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f2554c40640 1 -- 172.21.15.12:0/393289969 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255010a500 con 0x7f2550101960 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f2554c40640 1 --2- 172.21.15.12:0/393289969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2550101960 0x7f2550109730 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2544009a20 tx=0x7f2544031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f254e7fc640 1 -- 172.21.15.12:0/393289969 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254403f030 con 0x7f2550101960 2024-05-09T09:24:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f254e7fc640 1 -- 172.21.15.12:0/393289969 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2544003ac0 con 0x7f2550101960 2024-05-09T09:24:55.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f25576a5640 1 -- 172.21.15.12:0/393289969 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25501a1c80 con 0x7f2550101960 2024-05-09T09:24:55.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f254e7fc640 1 -- 172.21.15.12:0/393289969 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544035070 con 0x7f2550101960 2024-05-09T09:24:55.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.833+0000 7f25576a5640 1 -- 172.21.15.12:0/393289969 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25501a21d0 con 0x7f2550101960 2024-05-09T09:24:55.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.834+0000 7f25576a5640 1 -- 172.21.15.12:0/393289969 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2550107b60 con 0x7f2550101960 2024-05-09T09:24:55.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.836+0000 7f254e7fc640 1 -- 172.21.15.12:0/393289969 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2544042030 con 0x7f2550101960 2024-05-09T09:24:55.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.837+0000 7f254e7fc640 1 --2- 172.21.15.12:0/393289969 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f25380a2c10 0x7f25380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:55.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.837+0000 7f2555441640 1 --2- 172.21.15.12:0/393289969 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f25380a2c10 0x7f25380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:55.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.837+0000 7f254e7fc640 1 -- 172.21.15.12:0/393289969 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f25440f0090 con 0x7f2550101960 2024-05-09T09:24:55.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.838+0000 7f2555441640 1 --2- 172.21.15.12:0/393289969 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f25380a2c10 0x7f25380a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f25501017c0 tx=0x7f2540008230 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:55.841+0000 7f254e7fc640 1 -- 172.21.15.12:0/393289969 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f25440bcc90 con 0x7f2550101960 2024-05-09T09:24:56.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.041+0000 7f25576a5640 1 -- 172.21.15.12:0/393289969 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f255002d350 con 0x7f2550101960 2024-05-09T09:24:56.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.042+0000 7f254e7fc640 1 -- 172.21.15.12:0/393289969 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f25440bc590 con 0x7f2550101960 2024-05-09T09:24:56.042 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T09:24:56.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.044+0000 7f25576a5640 1 -- 172.21.15.12:0/393289969 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f25380a2c10 msgr2=0x7f25380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.044+0000 7f25576a5640 1 --2- 172.21.15.12:0/393289969 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f25380a2c10 0x7f25380a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f25501017c0 tx=0x7f2540008230 comp rx=0 tx=0).stop 2024-05-09T09:24:56.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.044+0000 7f25576a5640 1 -- 172.21.15.12:0/393289969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2550101960 msgr2=0x7f2550109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.044+0000 7f25576a5640 1 --2- 172.21.15.12:0/393289969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2550101960 0x7f2550109730 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2544009a20 tx=0x7f2544031ac0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.044+0000 7f25576a5640 1 -- 172.21.15.12:0/393289969 shutdown_connections 2024-05-09T09:24:56.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.044+0000 7f25576a5640 1 --2- 172.21.15.12:0/393289969 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f25380a2c10 0x7f25380a50d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.044+0000 7f25576a5640 1 --2- 172.21.15.12:0/393289969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2550102320 0x7f2550109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.044+0000 7f25576a5640 1 --2- 172.21.15.12:0/393289969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2550101960 0x7f2550109730 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.044+0000 7f25576a5640 1 --2- 172.21.15.12:0/393289969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2550100770 0x7f255010b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.044+0000 7f25576a5640 1 -- 172.21.15.12:0/393289969 >> 172.21.15.12:0/393289969 conn(0x7f25500fbce0 msgr2=0x7f2550104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:56.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.044+0000 7f25576a5640 1 -- 172.21.15.12:0/393289969 shutdown_connections 2024-05-09T09:24:56.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.044+0000 7f25576a5640 1 -- 172.21.15.12:0/393289969 wait complete. 2024-05-09T09:24:56.058 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:24:56.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.169+0000 7f79ae989640 1 -- 172.21.15.12:0/3482392307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79a8104c20 msgr2=0x7f79a8107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.169+0000 7f79ae989640 1 --2- 172.21.15.12:0/3482392307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79a8104c20 0x7f79a8107010 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7990009900 tx=0x7f799002a410 comp rx=0 tx=0).stop 2024-05-09T09:24:56.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.169+0000 7f79ae989640 1 -- 172.21.15.12:0/3482392307 shutdown_connections 2024-05-09T09:24:56.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.169+0000 7f79ae989640 1 --2- 172.21.15.12:0/3482392307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79a8107550 0x7f79a81099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.169+0000 7f79ae989640 1 --2- 172.21.15.12:0/3482392307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79a8104c20 0x7f79a8107010 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.169+0000 7f79ae989640 1 --2- 172.21.15.12:0/3482392307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a81022f0 0x7f79a81046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.169+0000 7f79ae989640 1 -- 172.21.15.12:0/3482392307 >> 172.21.15.12:0/3482392307 conn(0x7f79a80fbce0 msgr2=0x7f79a80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:56.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.169+0000 7f79ae989640 1 -- 172.21.15.12:0/3482392307 shutdown_connections 2024-05-09T09:24:56.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.169+0000 7f79ae989640 1 -- 172.21.15.12:0/3482392307 wait complete. 2024-05-09T09:24:56.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.170+0000 7f79ae989640 1 Processor -- start 2024-05-09T09:24:56.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.170+0000 7f79ae989640 1 -- start start 2024-05-09T09:24:56.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.170+0000 7f79ae989640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79a81022f0 0x7f79a8197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.170+0000 7f79ae989640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79a8104c20 0x7f79a8198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.170+0000 7f79ae989640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a8107550 0x7f79a819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.170+0000 7f79ae989640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f79a810d4b0 con 0x7f79a8104c20 2024-05-09T09:24:56.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.170+0000 7f79ae989640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f79a810d330 con 0x7f79a8107550 2024-05-09T09:24:56.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.170+0000 7f79ae989640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f79a810d630 con 0x7f79a81022f0 2024-05-09T09:24:56.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.170+0000 7f799ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79a8104c20 0x7f79a8198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.170+0000 7f799ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79a8104c20 0x7f79a8198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50152/0 (socket says 172.21.15.12:50152) 2024-05-09T09:24:56.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.170+0000 7f799ffff640 1 -- 172.21.15.12:0/3353865990 learned_addr learned my addr 172.21.15.12:0/3353865990 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:56.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.170+0000 7f79acf26640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a8107550 0x7f79a819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.171+0000 7f79a7fff640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79a81022f0 0x7f79a8197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.171+0000 7f799ffff640 1 -- 172.21.15.12:0/3353865990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79a81022f0 msgr2=0x7f79a8197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.171+0000 7f799ffff640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79a81022f0 0x7f79a8197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.171+0000 7f799ffff640 1 -- 172.21.15.12:0/3353865990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a8107550 msgr2=0x7f79a819f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.171+0000 7f799ffff640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a8107550 0x7f79a819f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.171+0000 7f799ffff640 1 -- 172.21.15.12:0/3353865990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a819f7e0 con 0x7f79a8104c20 2024-05-09T09:24:56.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.171+0000 7f799ffff640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79a8104c20 0x7f79a8198100 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7990005ab0 tx=0x7f79900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:56.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.171+0000 7f79a5ffb640 1 -- 172.21.15.12:0/3353865990 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799003f030 con 0x7f79a8104c20 2024-05-09T09:24:56.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.171+0000 7f79a5ffb640 1 -- 172.21.15.12:0/3353865990 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7990003ac0 con 0x7f79a8104c20 2024-05-09T09:24:56.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.171+0000 7f79ae989640 1 -- 172.21.15.12:0/3353865990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a819fa70 con 0x7f79a8104c20 2024-05-09T09:24:56.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.171+0000 7f79a5ffb640 1 -- 172.21.15.12:0/3353865990 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990035070 con 0x7f79a8104c20 2024-05-09T09:24:56.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.171+0000 7f79ae989640 1 -- 172.21.15.12:0/3353865990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a819ff60 con 0x7f79a8104c20 2024-05-09T09:24:56.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.172+0000 7f79ae989640 1 -- 172.21.15.12:0/3353865990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a80fd7d0 con 0x7f79a8104c20 2024-05-09T09:24:56.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.173+0000 7f79a5ffb640 1 -- 172.21.15.12:0/3353865990 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7990042030 con 0x7f79a8104c20 2024-05-09T09:24:56.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.174+0000 7f79a5ffb640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f79940a2b90 0x7f79940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.174+0000 7f79a7fff640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f79940a2b90 0x7f79940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.174+0000 7f79a5ffb640 1 -- 172.21.15.12:0/3353865990 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f79900efa30 con 0x7f79a8104c20 2024-05-09T09:24:56.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.175+0000 7f79a7fff640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f79940a2b90 0x7f79940a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7998005fd0 tx=0x7f7998005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:56.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.178+0000 7f79a5ffb640 1 -- 172.21.15.12:0/3353865990 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f79900bc630 con 0x7f79a8104c20 2024-05-09T09:24:56.374 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:56 smithi012 ceph-mon[30140]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:24:56.374 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:56 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/698064583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:24:56.374 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:56 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1979843965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:24:56.374 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:56 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/393289969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:24:56.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.382+0000 7f79ae989640 1 -- 172.21.15.12:0/3353865990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f79a8062430 con 0x7f79a8104c20 2024-05-09T09:24:56.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.382+0000 7f79a5ffb640 1 -- 172.21.15.12:0/3353865990 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f79900bbf30 con 0x7f79a8104c20 2024-05-09T09:24:56.382 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:24:56.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.385+0000 7f79ae989640 1 -- 172.21.15.12:0/3353865990 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f79940a2b90 msgr2=0x7f79940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.385+0000 7f79ae989640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f79940a2b90 0x7f79940a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7998005fd0 tx=0x7f7998005850 comp rx=0 tx=0).stop 2024-05-09T09:24:56.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.385+0000 7f79ae989640 1 -- 172.21.15.12:0/3353865990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79a8104c20 msgr2=0x7f79a8198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.385+0000 7f79ae989640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79a8104c20 0x7f79a8198100 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7990005ab0 tx=0x7f79900042f0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.385+0000 7f79ae989640 1 -- 172.21.15.12:0/3353865990 shutdown_connections 2024-05-09T09:24:56.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.385+0000 7f79ae989640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f79940a2b90 0x7f79940a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.385+0000 7f79ae989640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a8107550 0x7f79a819f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.385+0000 7f79ae989640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79a8104c20 0x7f79a8198100 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.385+0000 7f79ae989640 1 --2- 172.21.15.12:0/3353865990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79a81022f0 0x7f79a8197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.385+0000 7f79ae989640 1 -- 172.21.15.12:0/3353865990 >> 172.21.15.12:0/3353865990 conn(0x7f79a80fbce0 msgr2=0x7f79a81041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:56.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.385+0000 7f79ae989640 1 -- 172.21.15.12:0/3353865990 shutdown_connections 2024-05-09T09:24:56.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.385+0000 7f79ae989640 1 -- 172.21.15.12:0/3353865990 wait complete. 2024-05-09T09:24:56.396 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:24:56.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:56 smithi170 ceph-mon[33632]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:24:56.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:56 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/698064583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:24:56.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:56 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1979843965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:24:56.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:56 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/393289969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:24:56.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.503+0000 7f5b93e8c640 1 -- 172.21.15.12:0/4067706524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b8c104ca0 msgr2=0x7f5b8c107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.503+0000 7f5b93e8c640 1 --2- 172.21.15.12:0/4067706524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b8c104ca0 0x7f5b8c107090 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009900 tx=0x7f5b7c02a410 comp rx=0 tx=0).stop 2024-05-09T09:24:56.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.503+0000 7f5b93e8c640 1 -- 172.21.15.12:0/4067706524 shutdown_connections 2024-05-09T09:24:56.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.503+0000 7f5b93e8c640 1 --2- 172.21.15.12:0/4067706524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b8c1075d0 0x7f5b8c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.503+0000 7f5b93e8c640 1 --2- 172.21.15.12:0/4067706524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b8c104ca0 0x7f5b8c107090 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.503+0000 7f5b93e8c640 1 --2- 172.21.15.12:0/4067706524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b8c102370 0x7f5b8c104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.503+0000 7f5b93e8c640 1 -- 172.21.15.12:0/4067706524 >> 172.21.15.12:0/4067706524 conn(0x7f5b8c0fbd20 msgr2=0x7f5b8c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:56.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.503+0000 7f5b93e8c640 1 -- 172.21.15.12:0/4067706524 shutdown_connections 2024-05-09T09:24:56.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.503+0000 7f5b93e8c640 1 -- 172.21.15.12:0/4067706524 wait complete. 2024-05-09T09:24:56.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b93e8c640 1 Processor -- start 2024-05-09T09:24:56.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b93e8c640 1 -- start start 2024-05-09T09:24:56.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b93e8c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b8c102370 0x7f5b8c101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b93e8c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b8c104ca0 0x7f5b8c100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b93e8c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b8c1075d0 0x7f5b8c100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b93e8c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c10d4d0 con 0x7f5b8c104ca0 2024-05-09T09:24:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b93e8c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c10d350 con 0x7f5b8c1075d0 2024-05-09T09:24:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b93e8c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c10d650 con 0x7f5b8c102370 2024-05-09T09:24:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b91427640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b8c104ca0 0x7f5b8c100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b91427640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b8c104ca0 0x7f5b8c100320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50170/0 (socket says 172.21.15.12:50170) 2024-05-09T09:24:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b91427640 1 -- 172.21.15.12:0/3716798 learned_addr learned my addr 172.21.15.12:0/3716798 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b91c28640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b8c102370 0x7f5b8c101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b92429640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b8c1075d0 0x7f5b8c100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b91427640 1 -- 172.21.15.12:0/3716798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b8c102370 msgr2=0x7f5b8c101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b91427640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b8c102370 0x7f5b8c101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b91427640 1 -- 172.21.15.12:0/3716798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b8c1075d0 msgr2=0x7f5b8c100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b91427640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b8c1075d0 0x7f5b8c100860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b91427640 1 -- 172.21.15.12:0/3716798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c1010f0 con 0x7f5b8c104ca0 2024-05-09T09:24:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b91c28640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b8c102370 0x7f5b8c101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.504+0000 7f5b92429640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b8c1075d0 0x7f5b8c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.505+0000 7f5b91427640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b8c104ca0 0x7f5b8c100320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c005ab0 tx=0x7f5b7c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.505+0000 7f5b82ffd640 1 -- 172.21.15.12:0/3716798 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c03f030 con 0x7f5b8c104ca0 2024-05-09T09:24:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.505+0000 7f5b93e8c640 1 -- 172.21.15.12:0/3716798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c1a1c90 con 0x7f5b8c104ca0 2024-05-09T09:24:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.505+0000 7f5b82ffd640 1 -- 172.21.15.12:0/3716798 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5b7c003ac0 con 0x7f5b8c104ca0 2024-05-09T09:24:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.505+0000 7f5b82ffd640 1 -- 172.21.15.12:0/3716798 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c035070 con 0x7f5b8c104ca0 2024-05-09T09:24:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.505+0000 7f5b93e8c640 1 -- 172.21.15.12:0/3716798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c1a2190 con 0x7f5b8c104ca0 2024-05-09T09:24:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.506+0000 7f5b93e8c640 1 -- 172.21.15.12:0/3716798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8c0fd810 con 0x7f5b8c104ca0 2024-05-09T09:24:56.509 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:56 smithi016 ceph-mon[33335]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:24:56.509 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:56 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/698064583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:24:56.509 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:56 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1979843965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:24:56.510 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:56 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/393289969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:24:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.507+0000 7f5b82ffd640 1 -- 172.21.15.12:0/3716798 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5b7c042030 con 0x7f5b8c104ca0 2024-05-09T09:24:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.508+0000 7f5b82ffd640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5b780a2b90 0x7f5b780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.508+0000 7f5b91c28640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5b780a2b90 0x7f5b780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.508+0000 7f5b82ffd640 1 -- 172.21.15.12:0/3716798 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5b7c0efa30 con 0x7f5b8c104ca0 2024-05-09T09:24:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.509+0000 7f5b91c28640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5b780a2b90 0x7f5b780a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009c30 tx=0x7f5b88005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.513+0000 7f5b82ffd640 1 -- 172.21.15.12:0/3716798 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5b7c0bc630 con 0x7f5b8c104ca0 2024-05-09T09:24:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.739+0000 7f5b93e8c640 1 -- 172.21.15.12:0/3716798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b8c062430 con 0x7f5b8c104ca0 2024-05-09T09:24:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.740+0000 7f5b82ffd640 1 -- 172.21.15.12:0/3716798 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11931 (secure 0 0 0) 0x7f5b7c0bbf30 con 0x7f5b8c104ca0 2024-05-09T09:24:56.739 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:24:56.740 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:23:34.596143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[24523,24502,24515,24484,14628]},"id":1}]} 2024-05-09T09:24:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-05-09T09:24:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.742+0000 7f5b93e8c640 1 -- 172.21.15.12:0/3716798 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5b780a2b90 msgr2=0x7f5b780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.742+0000 7f5b93e8c640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5b780a2b90 0x7f5b780a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009c30 tx=0x7f5b88005f90 comp rx=0 tx=0).stop 2024-05-09T09:24:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.742+0000 7f5b93e8c640 1 -- 172.21.15.12:0/3716798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b8c104ca0 msgr2=0x7f5b8c100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.742+0000 7f5b93e8c640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b8c104ca0 0x7f5b8c100320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c005ab0 tx=0x7f5b7c0042f0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.742+0000 7f5b93e8c640 1 -- 172.21.15.12:0/3716798 shutdown_connections 2024-05-09T09:24:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.742+0000 7f5b93e8c640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5b780a2b90 0x7f5b780a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.742+0000 7f5b93e8c640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b8c1075d0 0x7f5b8c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.742+0000 7f5b93e8c640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b8c104ca0 0x7f5b8c100320 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.742+0000 7f5b93e8c640 1 --2- 172.21.15.12:0/3716798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b8c102370 0x7f5b8c101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.742+0000 7f5b93e8c640 1 -- 172.21.15.12:0/3716798 >> 172.21.15.12:0/3716798 conn(0x7f5b8c0fbd20 msgr2=0x7f5b8c102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.742+0000 7f5b93e8c640 1 -- 172.21.15.12:0/3716798 shutdown_connections 2024-05-09T09:24:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.742+0000 7f5b93e8c640 1 -- 172.21.15.12:0/3716798 wait complete. 2024-05-09T09:24:56.753 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:24:56.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.859+0000 7f64bf5f4640 1 -- 172.21.15.12:0/3932188300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64b81035a0 msgr2=0x7f64b8109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.859+0000 7f64bf5f4640 1 --2- 172.21.15.12:0/3932188300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64b81035a0 0x7f64b8109e30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009cd0 tx=0x7f64b002a8e0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.859+0000 7f64bf5f4640 1 -- 172.21.15.12:0/3932188300 shutdown_connections 2024-05-09T09:24:56.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.859+0000 7f64bf5f4640 1 --2- 172.21.15.12:0/3932188300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64b81035a0 0x7f64b8109e30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.859+0000 7f64bf5f4640 1 --2- 172.21.15.12:0/3932188300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64b8102be0 0x7f64b8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.859+0000 7f64bf5f4640 1 --2- 172.21.15.12:0/3932188300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b80ffb70 0x7f64b80fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.859+0000 7f64bf5f4640 1 -- 172.21.15.12:0/3932188300 >> 172.21.15.12:0/3932188300 conn(0x7f64b80fb4a0 msgr2=0x7f64b80fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.859+0000 7f64bf5f4640 1 -- 172.21.15.12:0/3932188300 shutdown_connections 2024-05-09T09:24:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.859+0000 7f64bf5f4640 1 -- 172.21.15.12:0/3932188300 wait complete. 2024-05-09T09:24:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bf5f4640 1 Processor -- start 2024-05-09T09:24:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bf5f4640 1 -- start start 2024-05-09T09:24:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bf5f4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64b80ffb70 0x7f64b8102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bf5f4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64b8102be0 0x7f64b8100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bf5f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b81035a0 0x7f64b81010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bf5f4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f64b810d500 con 0x7f64b8102be0 2024-05-09T09:24:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bf5f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64b810d380 con 0x7f64b81035a0 2024-05-09T09:24:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bf5f4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f64b810d680 con 0x7f64b80ffb70 2024-05-09T09:24:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bcb8f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64b8102be0 0x7f64b8100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bcb8f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64b8102be0 0x7f64b8100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50188/0 (socket says 172.21.15.12:50188) 2024-05-09T09:24:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bcb8f640 1 -- 172.21.15.12:0/954309830 learned_addr learned my addr 172.21.15.12:0/954309830 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bdb91640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b81035a0 0x7f64b81010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bd390640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64b80ffb70 0x7f64b8102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bcb8f640 1 -- 172.21.15.12:0/954309830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64b80ffb70 msgr2=0x7f64b8102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bcb8f640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64b80ffb70 0x7f64b8102550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bcb8f640 1 -- 172.21.15.12:0/954309830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b81035a0 msgr2=0x7f64b81010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bcb8f640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b81035a0 0x7f64b81010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bcb8f640 1 -- 172.21.15.12:0/954309830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b8101960 con 0x7f64b8102be0 2024-05-09T09:24:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bd390640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64b80ffb70 0x7f64b8102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.860+0000 7f64bcb8f640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64b8102be0 0x7f64b8100b90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f64a800c920 tx=0x7f64a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.861+0000 7f64ae7fc640 1 -- 172.21.15.12:0/954309830 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8007c70 con 0x7f64b8102be0 2024-05-09T09:24:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.861+0000 7f64bf5f4640 1 -- 172.21.15.12:0/954309830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b81a1c70 con 0x7f64b8102be0 2024-05-09T09:24:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.861+0000 7f64ae7fc640 1 -- 172.21.15.12:0/954309830 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f64a8007e10 con 0x7f64b8102be0 2024-05-09T09:24:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.861+0000 7f64ae7fc640 1 -- 172.21.15.12:0/954309830 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8018800 con 0x7f64b8102be0 2024-05-09T09:24:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.861+0000 7f64bf5f4640 1 -- 172.21.15.12:0/954309830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b81a21c0 con 0x7f64b8102be0 2024-05-09T09:24:56.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.862+0000 7f64bf5f4640 1 -- 172.21.15.12:0/954309830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b8108c40 con 0x7f64b8102be0 2024-05-09T09:24:56.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.863+0000 7f64ae7fc640 1 -- 172.21.15.12:0/954309830 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f64a8018a20 con 0x7f64b8102be0 2024-05-09T09:24:56.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.864+0000 7f64ae7fc640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f64a40a2b90 0x7f64a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.864+0000 7f64bd390640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f64a40a2b90 0x7f64a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.864+0000 7f64ae7fc640 1 -- 172.21.15.12:0/954309830 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f64a80cf5b0 con 0x7f64b8102be0 2024-05-09T09:24:56.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.864+0000 7f64bd390640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f64a40a2b90 0x7f64a40a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f64b8069c10 tx=0x7f64b400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:56.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:56.868+0000 7f64ae7fc640 1 -- 172.21.15.12:0/954309830 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f64a8014030 con 0x7f64b8102be0 2024-05-09T09:24:57.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.055+0000 7f64bf5f4640 1 -- 172.21.15.12:0/954309830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64b8062430 con 0x7f64b8102be0 2024-05-09T09:24:57.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.057+0000 7f64ae7fc640 1 -- 172.21.15.12:0/954309830 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24575 (secure 0 0 0) 0x7f64a800de20 con 0x7f64b8102be0 2024-05-09T09:24:57.057 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:24:57.057 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":91,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:24:05.727743+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:24:57.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.059+0000 7f64bf5f4640 1 -- 172.21.15.12:0/954309830 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f64a40a2b90 msgr2=0x7f64a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.059+0000 7f64bf5f4640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f64a40a2b90 0x7f64a40a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f64b8069c10 tx=0x7f64b400a560 comp rx=0 tx=0).stop 2024-05-09T09:24:57.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.059+0000 7f64bf5f4640 1 -- 172.21.15.12:0/954309830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64b8102be0 msgr2=0x7f64b8100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.059+0000 7f64bf5f4640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64b8102be0 0x7f64b8100b90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f64a800c920 tx=0x7f64a800cde0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.059+0000 7f64bf5f4640 1 -- 172.21.15.12:0/954309830 shutdown_connections 2024-05-09T09:24:57.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.059+0000 7f64bf5f4640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f64a40a2b90 0x7f64a40a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.059+0000 7f64bf5f4640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b81035a0 0x7f64b81010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.059+0000 7f64bf5f4640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64b8102be0 0x7f64b8100b90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.059+0000 7f64bf5f4640 1 --2- 172.21.15.12:0/954309830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64b80ffb70 0x7f64b8102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.059+0000 7f64bf5f4640 1 -- 172.21.15.12:0/954309830 >> 172.21.15.12:0/954309830 conn(0x7f64b80fb4a0 msgr2=0x7f64b8105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.059+0000 7f64bf5f4640 1 -- 172.21.15.12:0/954309830 shutdown_connections 2024-05-09T09:24:57.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.059+0000 7f64bf5f4640 1 -- 172.21.15.12:0/954309830 wait complete. 2024-05-09T09:24:57.070 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:24:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.176+0000 7ff1d1f90640 1 -- 172.21.15.12:0/3189508630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1cc102310 msgr2=0x7ff1cc104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.176+0000 7ff1d1f90640 1 --2- 172.21.15.12:0/3189508630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1cc102310 0x7ff1cc104700 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009900 tx=0x7ff1bc02a410 comp rx=0 tx=0).stop 2024-05-09T09:24:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.176+0000 7ff1d1f90640 1 -- 172.21.15.12:0/3189508630 shutdown_connections 2024-05-09T09:24:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.176+0000 7ff1d1f90640 1 --2- 172.21.15.12:0/3189508630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1cc107570 0x7ff1cc1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.176+0000 7ff1d1f90640 1 --2- 172.21.15.12:0/3189508630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc104c40 0x7ff1cc107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.176+0000 7ff1d1f90640 1 --2- 172.21.15.12:0/3189508630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1cc102310 0x7ff1cc104700 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.176+0000 7ff1d1f90640 1 -- 172.21.15.12:0/3189508630 >> 172.21.15.12:0/3189508630 conn(0x7ff1cc0fbd20 msgr2=0x7ff1cc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.176+0000 7ff1d1f90640 1 -- 172.21.15.12:0/3189508630 shutdown_connections 2024-05-09T09:24:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.176+0000 7ff1d1f90640 1 -- 172.21.15.12:0/3189508630 wait complete. 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.177+0000 7ff1d1f90640 1 Processor -- start 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.177+0000 7ff1d1f90640 1 -- start start 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.177+0000 7ff1d1f90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1cc102310 0x7ff1cc197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.177+0000 7ff1d1f90640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1cc104c40 0x7ff1cc198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.177+0000 7ff1d1f90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc107570 0x7ff1cc19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.177+0000 7ff1d1f90640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc10d510 con 0x7ff1cc102310 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.177+0000 7ff1d1f90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc10d390 con 0x7ff1cc107570 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.177+0000 7ff1d1f90640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc10d690 con 0x7ff1cc104c40 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.177+0000 7ff1cb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1cc102310 0x7ff1cc197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.177+0000 7ff1cb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1cc102310 0x7ff1cc197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50216/0 (socket says 172.21.15.12:50216) 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.177+0000 7ff1cb7fe640 1 -- 172.21.15.12:0/4099698508 learned_addr learned my addr 172.21.15.12:0/4099698508 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.177+0000 7ff1caffd640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1cc104c40 0x7ff1cc198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.177+0000 7ff1cbfff640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc107570 0x7ff1cc19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.178+0000 7ff1cb7fe640 1 -- 172.21.15.12:0/4099698508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1cc104c40 msgr2=0x7ff1cc198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.178+0000 7ff1cb7fe640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1cc104c40 0x7ff1cc198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.178+0000 7ff1cb7fe640 1 -- 172.21.15.12:0/4099698508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc107570 msgr2=0x7ff1cc19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.178+0000 7ff1cb7fe640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc107570 0x7ff1cc19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.178+0000 7ff1cb7fe640 1 -- 172.21.15.12:0/4099698508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc19f800 con 0x7ff1cc102310 2024-05-09T09:24:57.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.178+0000 7ff1cb7fe640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1cc102310 0x7ff1cc197be0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009a20 tx=0x7ff1bc031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.178+0000 7ff1c8ff9640 1 -- 172.21.15.12:0/4099698508 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc03f030 con 0x7ff1cc102310 2024-05-09T09:24:57.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.178+0000 7ff1c8ff9640 1 -- 172.21.15.12:0/4099698508 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff1bc003ac0 con 0x7ff1cc102310 2024-05-09T09:24:57.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.178+0000 7ff1c8ff9640 1 -- 172.21.15.12:0/4099698508 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc035070 con 0x7ff1cc102310 2024-05-09T09:24:57.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.178+0000 7ff1d1f90640 1 -- 172.21.15.12:0/4099698508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc19fa90 con 0x7ff1cc102310 2024-05-09T09:24:57.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.178+0000 7ff1d1f90640 1 -- 172.21.15.12:0/4099698508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc19ffd0 con 0x7ff1cc102310 2024-05-09T09:24:57.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.179+0000 7ff1d1f90640 1 -- 172.21.15.12:0/4099698508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1cc0fd810 con 0x7ff1cc102310 2024-05-09T09:24:57.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.180+0000 7ff1c8ff9640 1 -- 172.21.15.12:0/4099698508 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff1bc042030 con 0x7ff1cc102310 2024-05-09T09:24:57.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.181+0000 7ff1c8ff9640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff1b40a2c10 0x7ff1b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.181+0000 7ff1c8ff9640 1 -- 172.21.15.12:0/4099698508 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff1bc0effa0 con 0x7ff1cc102310 2024-05-09T09:24:57.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.181+0000 7ff1caffd640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff1b40a2c10 0x7ff1b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.181+0000 7ff1caffd640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff1b40a2c10 0x7ff1b40a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc1990f0 tx=0x7ff1c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.184+0000 7ff1c8ff9640 1 -- 172.21.15.12:0/4099698508 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff1bc0bc550 con 0x7ff1cc102310 2024-05-09T09:24:57.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.409+0000 7ff1d1f90640 1 -- 172.21.15.12:0/4099698508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1cc02d1c0 con 0x7ff1cc102310 2024-05-09T09:24:57.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.409+0000 7ff1c8ff9640 1 -- 172.21.15.12:0/4099698508 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11931 (secure 0 0 0) 0x7ff1bc0c0850 con 0x7ff1cc102310 2024-05-09T09:24:57.409 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:24:57.409 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:23:34.596143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[24523,24502,24515,24484,14628]},"id":1}]} 2024-05-09T09:24:57.409 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-05-09T09:24:57.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.412+0000 7ff1d1f90640 1 -- 172.21.15.12:0/4099698508 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff1b40a2c10 msgr2=0x7ff1b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.412+0000 7ff1d1f90640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff1b40a2c10 0x7ff1b40a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc1990f0 tx=0x7ff1c000b410 comp rx=0 tx=0).stop 2024-05-09T09:24:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.412+0000 7ff1d1f90640 1 -- 172.21.15.12:0/4099698508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1cc102310 msgr2=0x7ff1cc197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.412+0000 7ff1d1f90640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1cc102310 0x7ff1cc197be0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009a20 tx=0x7ff1bc031ac0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.412+0000 7ff1d1f90640 1 -- 172.21.15.12:0/4099698508 shutdown_connections 2024-05-09T09:24:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.412+0000 7ff1d1f90640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff1b40a2c10 0x7ff1b40a50d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.412+0000 7ff1d1f90640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc107570 0x7ff1cc19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.412+0000 7ff1d1f90640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1cc104c40 0x7ff1cc198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.412+0000 7ff1d1f90640 1 --2- 172.21.15.12:0/4099698508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1cc102310 0x7ff1cc197be0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.412+0000 7ff1d1f90640 1 -- 172.21.15.12:0/4099698508 >> 172.21.15.12:0/4099698508 conn(0x7ff1cc0fbd20 msgr2=0x7ff1cc102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.412+0000 7ff1d1f90640 1 -- 172.21.15.12:0/4099698508 shutdown_connections 2024-05-09T09:24:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.412+0000 7ff1d1f90640 1 -- 172.21.15.12:0/4099698508 wait complete. 2024-05-09T09:24:57.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:57 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3353865990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:24:57.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:57 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3716798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:57.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:57 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/954309830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:24:57.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T09:23:09.656296+0000', 'modified': '2024-05-09T09:23:34.596143+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24523, 'mds_1': 24515, 'mds_2': 24502, 'mds_3': 24484, 'mds_4': 14628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.12:6839/901613691', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6838', 'nonce': 901613691}, {'type': 'v1', 'addr': '172.21.15.12:6839', 'nonce': 901613691}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.12:6841/1434721532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6840', 'nonce': 1434721532}, {'type': 'v1', 'addr': '172.21.15.12:6841', 'nonce': 1434721532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6833/3659922754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6832', 'nonce': 3659922754}, {'type': 'v1', 'addr': '172.21.15.16:6833', 'nonce': 3659922754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6833/4010027838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 4010027838}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 4010027838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6835/1098165328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 1098165328}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 1098165328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6837/3413591353', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6836', 'nonce': 3413591353}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 3413591353}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.170:6835/277363225', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 277363225}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 277363225}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.170:6837/2973369023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 2973369023}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 2973369023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.16:6839/3932062083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 3932062083}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 3932062083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.170:6839/3731005466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 3731005466}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 3731005466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24523, 'qdb_cluster': [24523, 24502, 24515, 24484, 14628]} 2024-05-09T09:24:57.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T09:24:57.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-09T09:24:57.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:57 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3353865990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:24:57.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:57 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3716798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:57.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:57 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/954309830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:24:57.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:57 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3353865990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:24:57.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:57 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3716798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:57.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:57 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/954309830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:24:57.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.531+0000 7fdbebe64640 1 -- 172.21.15.12:0/1518977753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbe4102340 msgr2=0x7fdbe4108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.531+0000 7fdbebe64640 1 --2- 172.21.15.12:0/1518977753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbe4102340 0x7fdbe4108bd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009ca0 tx=0x7fdbdc02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.531+0000 7fdbebe64640 1 -- 172.21.15.12:0/1518977753 shutdown_connections 2024-05-09T09:24:57.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.531+0000 7fdbebe64640 1 --2- 172.21.15.12:0/1518977753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbe4102340 0x7fdbe4108bd0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.531+0000 7fdbebe64640 1 --2- 172.21.15.12:0/1518977753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbe4101980 0x7fdbe4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.531+0000 7fdbebe64640 1 --2- 172.21.15.12:0/1518977753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe4100790 0x7fdbe4100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.531+0000 7fdbebe64640 1 -- 172.21.15.12:0/1518977753 >> 172.21.15.12:0/1518977753 conn(0x7fdbe40fbd20 msgr2=0x7fdbe40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.531+0000 7fdbebe64640 1 -- 172.21.15.12:0/1518977753 shutdown_connections 2024-05-09T09:24:57.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.531+0000 7fdbebe64640 1 -- 172.21.15.12:0/1518977753 wait complete. 2024-05-09T09:24:57.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbebe64640 1 Processor -- start 2024-05-09T09:24:57.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbebe64640 1 -- start start 2024-05-09T09:24:57.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbebe64640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbe4100790 0x7fdbe4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbebe64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe4101980 0x7fdbe4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbebe64640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbe4102340 0x7fdbe419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbebe64640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe410d500 con 0x7fdbe4102340 2024-05-09T09:24:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbebe64640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe410d380 con 0x7fdbe4101980 2024-05-09T09:24:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbebe64640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe410d680 con 0x7fdbe4100790 2024-05-09T09:24:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbea401640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbe4102340 0x7fdbe419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbea401640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbe4102340 0x7fdbe419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50230/0 (socket says 172.21.15.12:50230) 2024-05-09T09:24:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbea401640 1 -- 172.21.15.12:0/4046241649 learned_addr learned my addr 172.21.15.12:0/4046241649 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe4101980 0x7fdbe4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbe9c00640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbe4100790 0x7fdbe4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbea401640 1 -- 172.21.15.12:0/4046241649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbe4100790 msgr2=0x7fdbe4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbea401640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbe4100790 0x7fdbe4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbea401640 1 -- 172.21.15.12:0/4046241649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe4101980 msgr2=0x7fdbe4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbea401640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe4101980 0x7fdbe4197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbea401640 1 -- 172.21.15.12:0/4046241649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe419f6d0 con 0x7fdbe4102340 2024-05-09T09:24:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.532+0000 7fdbe9c00640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbe4100790 0x7fdbe4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.533+0000 7fdbea401640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbe4102340 0x7fdbe419f060 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009430 tx=0x7fdbdc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.533+0000 7fdbdaffd640 1 -- 172.21.15.12:0/4046241649 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc003bf0 con 0x7fdbe4102340 2024-05-09T09:24:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.533+0000 7fdbebe64640 1 -- 172.21.15.12:0/4046241649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe419f960 con 0x7fdbe4102340 2024-05-09T09:24:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.533+0000 7fdbdaffd640 1 -- 172.21.15.12:0/4046241649 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdbdc003d90 con 0x7fdbe4102340 2024-05-09T09:24:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.533+0000 7fdbdaffd640 1 -- 172.21.15.12:0/4046241649 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc02fcc0 con 0x7fdbe4102340 2024-05-09T09:24:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.533+0000 7fdbebe64640 1 -- 172.21.15.12:0/4046241649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe419ff00 con 0x7fdbe4102340 2024-05-09T09:24:57.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.535+0000 7fdbdaffd640 1 -- 172.21.15.12:0/4046241649 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdbdc004040 con 0x7fdbe4102340 2024-05-09T09:24:57.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.535+0000 7fdbeeb71740 1 Processor -- start 2024-05-09T09:24:57.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbeeb71740 1 -- start start 2024-05-09T09:24:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbeeb71740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 0x55f3a589dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbeeb71740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 0x55f3a58a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbeeb71740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a58a3420 0x55f3a58a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbeeb71740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f3a58014d0 con 0x55f3a589d8c0 2024-05-09T09:24:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbeeb71740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f3a57a93e0 con 0x55f3a589e2b0 2024-05-09T09:24:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbeeb71740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f3a57fd490 con 0x55f3a58a3420 2024-05-09T09:24:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbe93ff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 0x55f3a589dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbe93ff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 0x55f3a589dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50242/0 (socket says 172.21.15.12:50242) 2024-05-09T09:24:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbe93ff640 1 -- 172.21.15.12:0/3667160534 learned_addr learned my addr 172.21.15.12:0/3667160534 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbdaffd640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdbd00a2b90 0x7fdbd00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbe9c00640 1 --2- 172.21.15.12:0/3667160534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 0x55f3a58a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbea401640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdbd00a2b90 0x7fdbd00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbe93ff640 1 -- 172.21.15.12:0/3667160534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a58a3420 msgr2=0x55f3a58a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/3667160534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a58a3420 0x55f3a58a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbe93ff640 1 -- 172.21.15.12:0/3667160534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 msgr2=0x55f3a58a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/3667160534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 0x55f3a58a2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbe93ff640 1 -- 172.21.15.12:0/3667160534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a55152d0 con 0x55f3a589d8c0 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbe9c00640 1 --2- 172.21.15.12:0/3667160534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 0x55f3a58a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.536+0000 7fdbdaffd640 1 -- 172.21.15.12:0/4046241649 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdbdc0f0b70 con 0x7fdbe4102340 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.537+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/3667160534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 0x55f3a589dce0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009ce0 tx=0x7fdbd40137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2831d6a59f3f68de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.537+0000 7fdbcffff640 1 -- 172.21.15.12:0/3667160534 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4013070 con 0x55f3a589d8c0 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.537+0000 7fdbea401640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdbd00a2b90 0x7fdbd00a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0ee680 tx=0x7fdbdc0dd040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.537+0000 7fdbcffff640 1 -- 172.21.15.12:0/3667160534 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdbd4017030 con 0x55f3a589d8c0 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.537+0000 7fdbcffff640 1 -- 172.21.15.12:0/3667160534 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40116a0 con 0x55f3a589d8c0 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.537+0000 7fdbeeb71740 1 -- 172.21.15.12:0/3667160534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 msgr2=0x55f3a589dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.537+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/3667160534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 0x55f3a589dce0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009ce0 tx=0x7fdbd40137d0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 -- 172.21.15.12:0/3667160534 shutdown_connections 2024-05-09T09:24:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/3667160534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a58a3420 0x55f3a58a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/3667160534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 0x55f3a58a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/3667160534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 0x55f3a589dce0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 -- 172.21.15.12:0/3667160534 >> 172.21.15.12:0/3667160534 conn(0x55f3a5898f30 msgr2=0x55f3a5899330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 -- 172.21.15.12:0/3667160534 shutdown_connections 2024-05-09T09:24:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 -- 172.21.15.12:0/3667160534 wait complete. 2024-05-09T09:24:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 -- start start 2024-05-09T09:24:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 Processor -- start 2024-05-09T09:24:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a589d8c0 0x55f3a58a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589e2b0 0x55f3a58a8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a58a3420 0x55f3a58af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f3a58a63d0 con 0x55f3a589e2b0 2024-05-09T09:24:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f3a57bc8e0 con 0x55f3a58a3420 2024-05-09T09:24:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.538+0000 7fdbeeb71740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f3a58a6550 con 0x55f3a589d8c0 2024-05-09T09:24:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbeeb71740 10 client.0 _ll_register_callbacks cb 0x55f3a57899c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbe9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589e2b0 0x55f3a58a8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbe9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589e2b0 0x55f3a58a8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50256/0 (socket says 172.21.15.12:50256) 2024-05-09T09:24:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbe9c00640 1 -- 172.21.15.12:0/644992785 learned_addr learned my addr 172.21.15.12:0/644992785 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/644992785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a589d8c0 0x55f3a58a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/644992785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a58a3420 0x55f3a58af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbe9c00640 1 -- 172.21.15.12:0/644992785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a589d8c0 msgr2=0x55f3a58a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbe9c00640 1 --2- 172.21.15.12:0/644992785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a589d8c0 0x55f3a58a7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbe9c00640 1 -- 172.21.15.12:0/644992785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a58a3420 msgr2=0x55f3a58af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbe9c00640 1 --2- 172.21.15.12:0/644992785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a58a3420 0x55f3a58af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbe9c00640 1 -- 172.21.15.12:0/644992785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a55df6b0 con 0x55f3a589e2b0 2024-05-09T09:24:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbe9c00640 1 --2- 172.21.15.12:0/644992785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589e2b0 0x55f3a58a8280 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000e720 tx=0x7fdbe000ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe69e7353c2bc69 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:24:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbcd7fa640 1 -- 172.21.15.12:0/644992785 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe001c060 con 0x55f3a589e2b0 2024-05-09T09:24:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbeeb71740 1 -- 172.21.15.12:0/644992785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd4017030 con 0x55f3a589e2b0 2024-05-09T09:24:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbcd7fa640 1 -- 172.21.15.12:0/644992785 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdbe0009a80 con 0x55f3a589e2b0 2024-05-09T09:24:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbcd7fa640 1 -- 172.21.15.12:0/644992785 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdbe0023050 con 0x55f3a589e2b0 2024-05-09T09:24:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbeeb71740 10 client.15219 fetch_fsmap learned FSMap version 28 2024-05-09T09:24:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.539+0000 7fdbeeb71740 1 -- 172.21.15.12:0/644992785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f3a58b22e0 con 0x55f3a589e2b0 2024-05-09T09:24:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.540+0000 7fdbcd7fa640 1 -- 172.21.15.12:0/644992785 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdbd4017030 con 0x55f3a589e2b0 2024-05-09T09:24:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.540+0000 7fdbcd7fa640 1 -- 172.21.15.12:0/644992785 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdbe0015340 con 0x55f3a589e2b0 2024-05-09T09:24:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.540+0000 7fdbeeb71740 10 client.15219 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:24:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.540+0000 7fdbeeb71740 10 client.15219 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T09:24:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.540+0000 7fdbeeb71740 20 client.15219 populate_metadata read hostname 'smithi012' 2024-05-09T09:24:57.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.540+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/644992785 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55f3a58b4920 0x55f3a58b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.540+0000 7fdbeeb71740 4 client.15219 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:57.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.540+0000 7fdbeeb71740 1 -- 172.21.15.12:0/644992785 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3a580ec00 con 0x55f3a58b4920 2024-05-09T09:24:57.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.540+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/644992785 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55f3a58b4920 0x55f3a58b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.540+0000 7fdbcd7fa640 10 client.15219 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:24:57.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.540+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/644992785 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55f3a58b4920 0x55f3a58b5d90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a9420c5439694716 server_cookie=10916c663481da21 in_seq=0 out_seq=0 2024-05-09T09:24:57.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbcd7fa640 1 -- 172.21.15.12:0/644992785 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3a580ec00 con 0x55f3a58b4920 2024-05-09T09:24:57.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbcd7fa640 10 client.15219 handle_command_reply: tid=0 2024-05-09T09:24:57.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbeeb71740 1 client.15219 shutdown 2024-05-09T09:24:57.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbeeb71740 1 -- 172.21.15.12:0/644992785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589e2b0 msgr2=0x55f3a58a8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/644992785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589e2b0 0x55f3a58a8280 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000e720 tx=0x7fdbe000ebe0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbeeb71740 1 -- 172.21.15.12:0/644992785 shutdown_connections 2024-05-09T09:24:57.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/644992785 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55f3a58b4920 0x55f3a58b5d90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/644992785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a58a3420 0x55f3a58af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/644992785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589e2b0 0x55f3a58a8280 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/644992785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a589d8c0 0x55f3a58a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbcd7fa640 0 client.15219 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:24:57.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbeeb71740 1 -- 172.21.15.12:0/644992785 >> 172.21.15.12:0/644992785 conn(0x55f3a5898f30 msgr2=0x55f3a5898000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbeeb71740 1 -- 172.21.15.12:0/644992785 shutdown_connections 2024-05-09T09:24:57.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbeeb71740 1 -- 172.21.15.12:0/644992785 wait complete. 2024-05-09T09:24:57.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.541+0000 7fdbeeb71740 20 client.15219 trim_cache size 0 max 16384 2024-05-09T09:24:57.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbeeb71740 1 Processor -- start 2024-05-09T09:24:57.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbeeb71740 1 -- start start 2024-05-09T09:24:57.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbeeb71740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 0x55f3a58a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbeeb71740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 0x55f3a58af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbeeb71740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a58a3420 0x55f3a58a5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbeeb71740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f3a58b40d0 con 0x55f3a589d8c0 2024-05-09T09:24:57.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbeeb71740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f3a58b47a0 con 0x55f3a589e2b0 2024-05-09T09:24:57.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbeeb71740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f3a58b4e70 con 0x55f3a58a3420 2024-05-09T09:24:57.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbe93ff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 0x55f3a58a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbe93ff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 0x55f3a58a2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50264/0 (socket says 172.21.15.12:50264) 2024-05-09T09:24:57.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbe93ff640 1 -- 172.21.15.12:0/3331595463 learned_addr learned my addr 172.21.15.12:0/3331595463 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbe93ff640 1 -- 172.21.15.12:0/3331595463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a58a3420 msgr2=0x55f3a58a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/3331595463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a58a3420 0x55f3a58a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbe93ff640 1 -- 172.21.15.12:0/3331595463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 msgr2=0x55f3a58af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/3331595463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 0x55f3a58af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbe93ff640 1 -- 172.21.15.12:0/3331595463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a58a6710 con 0x55f3a589d8c0 2024-05-09T09:24:57.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.560+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/3331595463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 0x55f3a58a2ce0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009af0 tx=0x7fdbd4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9694fc0fc4567c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.561+0000 7fdbcf7fe640 1 -- 172.21.15.12:0/3331595463 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4013190 con 0x55f3a589d8c0 2024-05-09T09:24:57.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.561+0000 7fdbcf7fe640 1 -- 172.21.15.12:0/3331595463 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdbd4017050 con 0x55f3a589d8c0 2024-05-09T09:24:57.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.561+0000 7fdbcf7fe640 1 -- 172.21.15.12:0/3331595463 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40035f0 con 0x55f3a589d8c0 2024-05-09T09:24:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.561+0000 7fdbeeb71740 1 -- 172.21.15.12:0/3331595463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 msgr2=0x55f3a58a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.561+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/3331595463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 0x55f3a58a2ce0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009af0 tx=0x7fdbd4009310 comp rx=0 tx=0).stop 2024-05-09T09:24:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.561+0000 7fdbeeb71740 1 -- 172.21.15.12:0/3331595463 shutdown_connections 2024-05-09T09:24:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.561+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/3331595463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a58a3420 0x55f3a58a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.561+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/3331595463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 0x55f3a58af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.561+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/3331595463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a589d8c0 0x55f3a58a2ce0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.561+0000 7fdbeeb71740 1 -- 172.21.15.12:0/3331595463 >> 172.21.15.12:0/3331595463 conn(0x55f3a5898f30 msgr2=0x55f3a5899310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.561+0000 7fdbeeb71740 1 -- 172.21.15.12:0/3331595463 shutdown_connections 2024-05-09T09:24:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.561+0000 7fdbeeb71740 1 -- 172.21.15.12:0/3331595463 wait complete. 2024-05-09T09:24:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbeeb71740 1 -- start start 2024-05-09T09:24:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbeeb71740 1 Processor -- start 2024-05-09T09:24:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbeeb71740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a589d8c0 0x55f3a58aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbeeb71740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 0x55f3a58aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbeeb71740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a58a3420 0x55f3a58ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbeeb71740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f3a58b5c10 con 0x55f3a58a3420 2024-05-09T09:24:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbeeb71740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f3a58b5540 con 0x55f3a589e2b0 2024-05-09T09:24:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbeeb71740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f3a58b62e0 con 0x55f3a589d8c0 2024-05-09T09:24:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbeeb71740 10 client.0 _ll_register_callbacks cb 0x7fdbe000c3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbe93ff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a58a3420 0x55f3a58ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbe93ff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a58a3420 0x55f3a58ada20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50274/0 (socket says 172.21.15.12:50274) 2024-05-09T09:24:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbe93ff640 1 -- 172.21.15.12:0/2097683363 learned_addr learned my addr 172.21.15.12:0/2097683363 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbe9c00640 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 0x55f3a58aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a589d8c0 0x55f3a58aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbe93ff640 1 -- 172.21.15.12:0/2097683363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a589d8c0 msgr2=0x55f3a58aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a589d8c0 0x55f3a58aa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbe93ff640 1 -- 172.21.15.12:0/2097683363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 msgr2=0x55f3a58aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 0x55f3a58aaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbe93ff640 1 -- 172.21.15.12:0/2097683363 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a58ae010 con 0x55f3a58a3420 2024-05-09T09:24:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.562+0000 7fdbe9c00640 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 0x55f3a58aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbe93ff640 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a58a3420 0x55f3a58ada20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4013d50 tx=0x7fdbd4009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d8308865573e70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:24:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbcd7fa640 1 -- 172.21.15.12:0/2097683363 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4034040 con 0x55f3a58a3420 2024-05-09T09:24:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbcd7fa640 1 -- 172.21.15.12:0/2097683363 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdbd400f810 con 0x55f3a58a3420 2024-05-09T09:24:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbeeb71740 1 -- 172.21.15.12:0/2097683363 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd4017050 con 0x55f3a58a3420 2024-05-09T09:24:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbcd7fa640 1 -- 172.21.15.12:0/2097683363 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdbd4005040 con 0x55f3a58a3420 2024-05-09T09:24:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbeeb71740 10 client.15225 fetch_fsmap learned FSMap version 28 2024-05-09T09:24:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbeeb71740 1 -- 172.21.15.12:0/2097683363 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f3a58af310 con 0x55f3a58a3420 2024-05-09T09:24:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbcd7fa640 1 -- 172.21.15.12:0/2097683363 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdbd4017050 con 0x55f3a58a3420 2024-05-09T09:24:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbcd7fa640 1 -- 172.21.15.12:0/2097683363 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdbd402b020 con 0x55f3a58a3420 2024-05-09T09:24:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbeeb71740 10 client.15225 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:24:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbeeb71740 10 client.15225 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T09:24:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbeeb71740 20 client.15225 populate_metadata read hostname 'smithi012' 2024-05-09T09:24:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55f3a591e890 0x55f3a591d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbeeb71740 4 client.15225 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:24:57.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.563+0000 7fdbeeb71740 1 -- 172.21.15.12:0/2097683363 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "status"}) -- 0x55f3a5808f50 con 0x55f3a591e890 2024-05-09T09:24:57.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbe9c00640 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55f3a591e890 0x55f3a591d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbcd7fa640 10 client.15225 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:24:57.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbe9c00640 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55f3a591e890 0x55f3a591d5a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5fcb0f499986b240 server_cookie=79e4187474747473 in_seq=0 out_seq=0 2024-05-09T09:24:57.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbcd7fa640 1 -- 172.21.15.12:0/2097683363 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f3a5808f50 con 0x55f3a591e890 2024-05-09T09:24:57.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbcd7fa640 10 client.15225 handle_command_reply: tid=0 2024-05-09T09:24:57.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbeeb71740 1 client.15225 shutdown 2024-05-09T09:24:57.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbeeb71740 1 -- 172.21.15.12:0/2097683363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a58a3420 msgr2=0x55f3a58ada20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a58a3420 0x55f3a58ada20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4013d50 tx=0x7fdbd4009590 comp rx=0 tx=0).stop 2024-05-09T09:24:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbeeb71740 1 -- 172.21.15.12:0/2097683363 shutdown_connections 2024-05-09T09:24:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55f3a591e890 0x55f3a591d5a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3a58a3420 0x55f3a58ada20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbcd7fa640 0 client.15225 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:24:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3a589e2b0 0x55f3a58aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbeeb71740 1 --2- 172.21.15.12:0/2097683363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f3a589d8c0 0x55f3a58aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbeeb71740 1 -- 172.21.15.12:0/2097683363 >> 172.21.15.12:0/2097683363 conn(0x55f3a5898f30 msgr2=0x55f3a589eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbeeb71740 1 -- 172.21.15.12:0/2097683363 shutdown_connections 2024-05-09T09:24:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.564+0000 7fdbeeb71740 1 -- 172.21.15.12:0/2097683363 wait complete. 2024-05-09T09:24:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.565+0000 7fdbeeb71740 20 client.15225 trim_cache size 0 max 16384 2024-05-09T09:24:57.573 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:24:57.573 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:24:57.573 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 4, 2024-05-09T09:24:57.573 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 14628, 2024-05-09T09:24:57.573 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-09T09:24:57.573 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-09T09:24:57.573 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-09T09:24:57.574 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 83.973068002000005, 2024-05-09T09:24:57.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-05-09T09:24:57.574 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 91, 2024-05-09T09:24:57.574 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T09:24:57.574 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 116.477069051 2024-05-09T09:24:57.574 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:24:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.565+0000 7fdbebe64640 1 -- 172.21.15.12:0/4046241649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdbd00a2b90 msgr2=0x7fdbd00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.565+0000 7fdbebe64640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdbd00a2b90 0x7fdbd00a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0ee680 tx=0x7fdbdc0dd040 comp rx=0 tx=0).stop 2024-05-09T09:24:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.565+0000 7fdbebe64640 1 -- 172.21.15.12:0/4046241649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbe4102340 msgr2=0x7fdbe419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.565+0000 7fdbebe64640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbe4102340 0x7fdbe419f060 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009430 tx=0x7fdbdc02f380 comp rx=0 tx=0).stop 2024-05-09T09:24:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.565+0000 7fdbebe64640 1 -- 172.21.15.12:0/4046241649 shutdown_connections 2024-05-09T09:24:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.565+0000 7fdbebe64640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdbd00a2b90 0x7fdbd00a5050 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.565+0000 7fdbebe64640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbe4102340 0x7fdbe419f060 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.565+0000 7fdbebe64640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe4101980 0x7fdbe4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.565+0000 7fdbebe64640 1 --2- 172.21.15.12:0/4046241649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbe4100790 0x7fdbe4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.565+0000 7fdbebe64640 1 -- 172.21.15.12:0/4046241649 >> 172.21.15.12:0/4046241649 conn(0x7fdbe40fbd20 msgr2=0x7fdbe4104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.565+0000 7fdbebe64640 1 -- 172.21.15.12:0/4046241649 shutdown_connections 2024-05-09T09:24:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.565+0000 7fdbebe64640 1 -- 172.21.15.12:0/4046241649 wait complete. 2024-05-09T09:24:57.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-05-09T09:24:57.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.684+0000 7f2ec28a5640 1 -- 172.21.15.12:0/3861663632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ebc075810 msgr2=0x7f2ebc073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.684+0000 7f2ec28a5640 1 --2- 172.21.15.12:0/3861663632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ebc075810 0x7f2ebc073c60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009900 tx=0x7f2eac02a480 comp rx=0 tx=0).stop 2024-05-09T09:24:57.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.684+0000 7f2ec28a5640 1 -- 172.21.15.12:0/3861663632 shutdown_connections 2024-05-09T09:24:57.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.684+0000 7f2ec28a5640 1 --2- 172.21.15.12:0/3861663632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ebc109720 0x7f2ebc10bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.684+0000 7f2ec28a5640 1 --2- 172.21.15.12:0/3861663632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ebc074230 0x7f2ebc074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.684+0000 7f2ec28a5640 1 --2- 172.21.15.12:0/3861663632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ebc075810 0x7f2ebc073c60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.684+0000 7f2ec28a5640 1 -- 172.21.15.12:0/3861663632 >> 172.21.15.12:0/3861663632 conn(0x7f2ebc0fce00 msgr2=0x7f2ebc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.684+0000 7f2ec28a5640 1 -- 172.21.15.12:0/3861663632 shutdown_connections 2024-05-09T09:24:57.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.684+0000 7f2ec28a5640 1 -- 172.21.15.12:0/3861663632 wait complete. 2024-05-09T09:24:57.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.684+0000 7f2ec28a5640 1 Processor -- start 2024-05-09T09:24:57.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.684+0000 7f2ec28a5640 1 -- start start 2024-05-09T09:24:57.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ec28a5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ebc074230 0x7f2ebc199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ec28a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ebc075810 0x7f2ebc19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ec28a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ebc109720 0x7f2ebc1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ec28a5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc10f6f0 con 0x7f2ebc074230 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ec28a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc10f570 con 0x7f2ebc075810 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ec28a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc10f870 con 0x7f2ebc109720 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ebbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ebc074230 0x7f2ebc199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ebbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ebc074230 0x7f2ebc199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50280/0 (socket says 172.21.15.12:50280) 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ebbfff640 1 -- 172.21.15.12:0/2550279153 learned_addr learned my addr 172.21.15.12:0/2550279153 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ebb7fe640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ebc075810 0x7f2ebc19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ec0e42640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ebc109720 0x7f2ebc1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ebbfff640 1 -- 172.21.15.12:0/2550279153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ebc109720 msgr2=0x7f2ebc1a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ebbfff640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ebc109720 0x7f2ebc1a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ebbfff640 1 -- 172.21.15.12:0/2550279153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ebc075810 msgr2=0x7f2ebc19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ebbfff640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ebc075810 0x7f2ebc19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ebbfff640 1 -- 172.21.15.12:0/2550279153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc1a19c0 con 0x7f2ebc074230 2024-05-09T09:24:57.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ebbfff640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ebc074230 0x7f2ebc199db0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009230 tx=0x7f2eac031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2eb97fa640 1 -- 172.21.15.12:0/2550279153 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac03f030 con 0x7f2ebc074230 2024-05-09T09:24:57.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2ec28a5640 1 -- 172.21.15.12:0/2550279153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ebc1a1c50 con 0x7f2ebc074230 2024-05-09T09:24:57.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2eb97fa640 1 -- 172.21.15.12:0/2550279153 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2eac003ac0 con 0x7f2ebc074230 2024-05-09T09:24:57.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.685+0000 7f2eb97fa640 1 -- 172.21.15.12:0/2550279153 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac035070 con 0x7f2ebc074230 2024-05-09T09:24:57.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.686+0000 7f2ec28a5640 1 -- 172.21.15.12:0/2550279153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ebc1a2190 con 0x7f2ebc074230 2024-05-09T09:24:57.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.688+0000 7f2eb97fa640 1 -- 172.21.15.12:0/2550279153 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2eac042030 con 0x7f2ebc074230 2024-05-09T09:24:57.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.688+0000 7f2ec55b5740 1 Processor -- start 2024-05-09T09:24:57.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.688+0000 7f2ec55b5740 1 -- start start 2024-05-09T09:24:57.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.688+0000 7f2ec55b5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 0x55685bf6ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.688+0000 7f2ec55b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf6fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.688+0000 7f2eb97fa640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ea40a2b90 0x7f2ea40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.688+0000 7f2ec55b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf70420 0x55685bf728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.688+0000 7f2ec55b5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55685bece4d0 con 0x55685bf6a8c0 2024-05-09T09:24:57.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.688+0000 7f2ec55b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55685be763e0 con 0x55685bf6b2b0 2024-05-09T09:24:57.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.688+0000 7f2ec55b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55685beca490 con 0x55685bf70420 2024-05-09T09:24:57.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.688+0000 7f2ec0e42640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 0x55685bf6ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ec0e42640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 0x55685bf6ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50284/0 (socket says 172.21.15.12:50284) 2024-05-09T09:24:57.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ec0e42640 1 -- 172.21.15.12:0/2961573540 learned_addr learned my addr 172.21.15.12:0/2961573540 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ebb7fe640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ea40a2b90 0x7f2ea40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2eb97fa640 1 -- 172.21.15.12:0/2550279153 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2eac0efa40 con 0x7f2ebc074230 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ebbfff640 1 --2- 172.21.15.12:0/2961573540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf6fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ec0e42640 1 --2- 172.21.15.12:0/2961573540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf70420 0x55685bf728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ec0e42640 1 -- 172.21.15.12:0/2961573540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf70420 msgr2=0x55685bf728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ec0e42640 1 --2- 172.21.15.12:0/2961573540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf70420 0x55685bf728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ec0e42640 1 -- 172.21.15.12:0/2961573540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 msgr2=0x55685bf6fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ec0e42640 1 --2- 172.21.15.12:0/2961573540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf6fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ec0e42640 1 -- 172.21.15.12:0/2961573540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55685bbe22d0 con 0x55685bf6a8c0 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ebb7fe640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ea40a2b90 0x7f2ea40a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0005fd0 tx=0x7f2eb000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ec0e42640 1 --2- 172.21.15.12:0/2961573540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 0x55685bf6ace0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4012610 tx=0x7f2eb4009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8206befb0d76ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ea27fc640 1 -- 172.21.15.12:0/2961573540 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4004b20 con 0x55685bf6a8c0 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ea27fc640 1 -- 172.21.15.12:0/2961573540 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2eb4003e40 con 0x55685bf6a8c0 2024-05-09T09:24:57.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.689+0000 7f2ea27fc640 1 -- 172.21.15.12:0/2961573540 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40047b0 con 0x55685bf6a8c0 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.690+0000 7f2ec55b5740 1 -- 172.21.15.12:0/2961573540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 msgr2=0x55685bf6ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.690+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/2961573540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 0x55685bf6ace0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4012610 tx=0x7f2eb4009b90 comp rx=0 tx=0).stop 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.690+0000 7f2ec55b5740 1 -- 172.21.15.12:0/2961573540 shutdown_connections 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.690+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/2961573540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf70420 0x55685bf728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.690+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/2961573540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf6fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.690+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/2961573540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 0x55685bf6ace0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.690+0000 7f2ec55b5740 1 -- 172.21.15.12:0/2961573540 >> 172.21.15.12:0/2961573540 conn(0x55685bf65f30 msgr2=0x55685bf66330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.690+0000 7f2ec55b5740 1 -- 172.21.15.12:0/2961573540 shutdown_connections 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.690+0000 7f2ec55b5740 1 -- 172.21.15.12:0/2961573540 wait complete. 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.690+0000 7f2ec55b5740 1 -- start start 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ec55b5740 1 Processor -- start 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ec55b5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 0x55685bf74cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ec55b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf75200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ec55b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf70420 0x55685bf7c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ec55b5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55685bf733d0 con 0x55685bf6a8c0 2024-05-09T09:24:57.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ec55b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55685be898e0 con 0x55685bf6b2b0 2024-05-09T09:24:57.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ec55b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55685bf73550 con 0x55685bf70420 2024-05-09T09:24:57.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ec55b5740 10 client.0 _ll_register_callbacks cb 0x55685be569c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:57.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ebbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 0x55685bf74cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ebbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 0x55685bf74cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50288/0 (socket says 172.21.15.12:50288) 2024-05-09T09:24:57.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ebbfff640 1 -- 172.21.15.12:0/3354166085 learned_addr learned my addr 172.21.15.12:0/3354166085 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ebb7fe640 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf75200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ec0e42640 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf70420 0x55685bf7c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ebbfff640 1 -- 172.21.15.12:0/3354166085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf70420 msgr2=0x55685bf7c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ebbfff640 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf70420 0x55685bf7c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ebbfff640 1 -- 172.21.15.12:0/3354166085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 msgr2=0x55685bf75200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ebbfff640 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf75200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ebbfff640 1 -- 172.21.15.12:0/3354166085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55685bcac6b0 con 0x55685bf6a8c0 2024-05-09T09:24:57.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ebb7fe640 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf75200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:57.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ec0e42640 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf70420 0x55685bf7c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:57.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2ebbfff640 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 0x55685bf74cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0ed1e0 tx=0x7f2eac009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d4e312b5be1c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.691+0000 7f2e9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:24:57.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/3354166085 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac049040 con 0x55685bf6a8c0 2024-05-09T09:24:57.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/3354166085 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2eac0b8520 con 0x55685bf6a8c0 2024-05-09T09:24:57.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2ec55b5740 1 -- 172.21.15.12:0/3354166085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55685bedbc00 con 0x55685bf6a8c0 2024-05-09T09:24:57.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/3354166085 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2eac0bbf40 con 0x55685bf6a8c0 2024-05-09T09:24:57.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2ec55b5740 10 client.15237 fetch_fsmap learned FSMap version 28 2024-05-09T09:24:57.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2ec55b5740 1 -- 172.21.15.12:0/3354166085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55685bf7f210 con 0x55685bf6a8c0 2024-05-09T09:24:57.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/3354166085 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55685bedbc00 con 0x55685bf6a8c0 2024-05-09T09:24:57.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/3354166085 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2eac0d7e10 con 0x55685bf6a8c0 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2ec55b5740 10 client.15237 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2ec55b5740 10 client.15237 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2ec55b5740 20 client.15237 populate_metadata read hostname 'smithi012' 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55685bf81870 0x55685bf82ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2ec55b5740 4 client.15237 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2ec55b5740 1 -- 172.21.15.12:0/3354166085 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55685bed5f50 con 0x55685bf81870 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.692+0000 7f2ebb7fe640 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55685bf81870 0x55685bf82ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.693+0000 7f2ebb7fe640 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55685bf81870 0x55685bf82ce0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=73db13229821788 server_cookie=acdfe34d0a0ad9c6 in_seq=0 out_seq=0 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.693+0000 7f2e9b7fe640 10 client.15237 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/3354166085 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55685bed5f50 con 0x55685bf81870 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2e9b7fe640 10 client.15237 handle_command_reply: tid=0 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2ec55b5740 1 client.15237 shutdown 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2ec55b5740 1 -- 172.21.15.12:0/3354166085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 msgr2=0x55685bf74cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 0x55685bf74cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0ed1e0 tx=0x7f2eac009680 comp rx=0 tx=0).stop 2024-05-09T09:24:57.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2ec55b5740 1 -- 172.21.15.12:0/3354166085 shutdown_connections 2024-05-09T09:24:57.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55685bf81870 0x55685bf82ce0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf70420 0x55685bf7c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf75200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/3354166085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf6a8c0 0x55685bf74cc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2e9b7fe640 0 client.15237 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:24:57.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2ec55b5740 1 -- 172.21.15.12:0/3354166085 >> 172.21.15.12:0/3354166085 conn(0x55685bf65f30 msgr2=0x55685bf65000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2ec55b5740 1 -- 172.21.15.12:0/3354166085 shutdown_connections 2024-05-09T09:24:57.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2ec55b5740 1 -- 172.21.15.12:0/3354166085 wait complete. 2024-05-09T09:24:57.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.694+0000 7f2ec55b5740 20 client.15237 trim_cache size 0 max 16384 2024-05-09T09:24:57.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.712+0000 7f2ec55b5740 1 Processor -- start 2024-05-09T09:24:57.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.712+0000 7f2ec55b5740 1 -- start start 2024-05-09T09:24:57.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec55b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf6a8c0 0x55685bf6bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec55b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf7c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec55b5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 0x55685bf748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec55b5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55685bf81020 con 0x55685bf70420 2024-05-09T09:24:57.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec55b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55685bf816f0 con 0x55685bf6b2b0 2024-05-09T09:24:57.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec55b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55685bf81dc0 con 0x55685bf6a8c0 2024-05-09T09:24:57.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec0e42640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 0x55685bf748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec0e42640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 0x55685bf748d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50302/0 (socket says 172.21.15.12:50302) 2024-05-09T09:24:57.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec0e42640 1 -- 172.21.15.12:0/2078772207 learned_addr learned my addr 172.21.15.12:0/2078772207 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ebb7fe640 1 --2- 172.21.15.12:0/2078772207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf7c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ebbfff640 1 --2- 172.21.15.12:0/2078772207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf6a8c0 0x55685bf6bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec0e42640 1 -- 172.21.15.12:0/2078772207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf6a8c0 msgr2=0x55685bf6bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec0e42640 1 --2- 172.21.15.12:0/2078772207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf6a8c0 0x55685bf6bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec0e42640 1 -- 172.21.15.12:0/2078772207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 msgr2=0x55685bf7c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec0e42640 1 --2- 172.21.15.12:0/2078772207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf7c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec0e42640 1 -- 172.21.15.12:0/2078772207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55685bf6b690 con 0x55685bf70420 2024-05-09T09:24:57.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ec0e42640 1 --2- 172.21.15.12:0/2078772207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 0x55685bf748d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4019000 tx=0x7f2eb4005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2cd5bea257f9b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.713+0000 7f2ea1ffb640 1 -- 172.21.15.12:0/2078772207 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4012130 con 0x55685bf70420 2024-05-09T09:24:57.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.714+0000 7f2ea1ffb640 1 -- 172.21.15.12:0/2078772207 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2eb4005070 con 0x55685bf70420 2024-05-09T09:24:57.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.714+0000 7f2ea1ffb640 1 -- 172.21.15.12:0/2078772207 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb400e770 con 0x55685bf70420 2024-05-09T09:24:57.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.714+0000 7f2ec55b5740 1 -- 172.21.15.12:0/2078772207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 msgr2=0x55685bf748d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.714+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/2078772207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 0x55685bf748d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4019000 tx=0x7f2eb4005c70 comp rx=0 tx=0).stop 2024-05-09T09:24:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.714+0000 7f2ec55b5740 1 -- 172.21.15.12:0/2078772207 shutdown_connections 2024-05-09T09:24:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.714+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/2078772207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 0x55685bf748d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.714+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/2078772207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf7c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.714+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/2078772207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf6a8c0 0x55685bf6bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.714+0000 7f2ec55b5740 1 -- 172.21.15.12:0/2078772207 >> 172.21.15.12:0/2078772207 conn(0x55685bf65f30 msgr2=0x55685bf66310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.714+0000 7f2ec55b5740 1 -- 172.21.15.12:0/2078772207 shutdown_connections 2024-05-09T09:24:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.714+0000 7f2ec55b5740 1 -- 172.21.15.12:0/2078772207 wait complete. 2024-05-09T09:24:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.714+0000 7f2ec55b5740 1 -- start start 2024-05-09T09:24:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec55b5740 1 Processor -- start 2024-05-09T09:24:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec55b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf6a8c0 0x55685bf780d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec55b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf78610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec55b5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 0x55685bf7b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec55b5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55685bf82b60 con 0x55685bf70420 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec55b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55685bf82490 con 0x55685bf6b2b0 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec55b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55685bf83230 con 0x55685bf6a8c0 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec55b5740 10 client.0 _ll_register_callbacks cb 0x7f2eac03b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec0e42640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 0x55685bf7b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ebb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf78610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec0e42640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 0x55685bf7b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50308/0 (socket says 172.21.15.12:50308) 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec0e42640 1 -- 172.21.15.12:0/743765930 learned_addr learned my addr 172.21.15.12:0/743765930 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ebbfff640 1 --2- 172.21.15.12:0/743765930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf6a8c0 0x55685bf780d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec0e42640 1 -- 172.21.15.12:0/743765930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf6a8c0 msgr2=0x55685bf780d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec0e42640 1 --2- 172.21.15.12:0/743765930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf6a8c0 0x55685bf780d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec0e42640 1 -- 172.21.15.12:0/743765930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 msgr2=0x55685bf78610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec0e42640 1 --2- 172.21.15.12:0/743765930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf78610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.715+0000 7f2ec0e42640 1 -- 172.21.15.12:0/743765930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55685bf7bbd0 con 0x55685bf70420 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2ec0e42640 1 --2- 172.21.15.12:0/743765930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 0x55685bf7b5e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4012610 tx=0x7f2eb4003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d2d719fd06e1c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2e9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/743765930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb401f040 con 0x55685bf70420 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/743765930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2eb40055b0 con 0x55685bf70420 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2ec55b5740 1 -- 172.21.15.12:0/743765930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb4005070 con 0x55685bf70420 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/743765930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2eb40057b0 con 0x55685bf70420 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2ec55b5740 10 client.15243 fetch_fsmap learned FSMap version 28 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2ec55b5740 1 -- 172.21.15.12:0/743765930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55685bfe7ee0 con 0x55685bf70420 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/743765930 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2eb4005070 con 0x55685bf70420 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/743765930 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2eb4010690 con 0x55685bf70420 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2ec55b5740 10 client.15243 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:24:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2ec55b5740 10 client.15243 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T09:24:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2ec55b5740 20 client.15243 populate_metadata read hostname 'smithi012' 2024-05-09T09:24:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/743765930 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55685bfec4b0 0x55685bfe8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2ec55b5740 4 client.15243 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:24:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.716+0000 7f2ec55b5740 1 -- 172.21.15.12:0/743765930 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "status"}) -- 0x55685be53b80 con 0x55685bfec4b0 2024-05-09T09:24:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2ebbfff640 1 --2- 172.21.15.12:0/743765930 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55685bfec4b0 0x55685bfe8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2ebbfff640 1 --2- 172.21.15.12:0/743765930 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55685bfec4b0 0x55685bfe8fc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=91c0a760b7592471 server_cookie=980917ab19bfbfec in_seq=0 out_seq=0 2024-05-09T09:24:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2e9b7fe640 10 client.15243 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:24:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/743765930 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55685be53b80 con 0x55685bfec4b0 2024-05-09T09:24:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2e9b7fe640 10 client.15243 handle_command_reply: tid=0 2024-05-09T09:24:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2ec55b5740 1 client.15243 shutdown 2024-05-09T09:24:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2ec55b5740 1 -- 172.21.15.12:0/743765930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 msgr2=0x55685bf7b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/743765930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 0x55685bf7b5e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4012610 tx=0x7f2eb4003db0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2ec55b5740 1 -- 172.21.15.12:0/743765930 shutdown_connections 2024-05-09T09:24:57.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/743765930 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55685bfec4b0 0x55685bfe8fc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/743765930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55685bf70420 0x55685bf7b5e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/743765930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685bf6b2b0 0x55685bf78610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2e9b7fe640 0 client.15243 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:24:57.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2ec55b5740 1 --2- 172.21.15.12:0/743765930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685bf6a8c0 0x55685bf780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2ec55b5740 1 -- 172.21.15.12:0/743765930 >> 172.21.15.12:0/743765930 conn(0x55685bf65f30 msgr2=0x55685bf6bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.717+0000 7f2ec55b5740 1 -- 172.21.15.12:0/743765930 shutdown_connections 2024-05-09T09:24:57.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec55b5740 1 -- 172.21.15.12:0/743765930 wait complete. 2024-05-09T09:24:57.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec55b5740 20 client.15243 trim_cache size 0 max 16384 2024-05-09T09:24:57.725 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:24:57.725 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:24:57.725 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 3, 2024-05-09T09:24:57.725 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24484, 2024-05-09T09:24:57.725 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-09T09:24:57.725 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-09T09:24:57.725 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-09T09:24:57.725 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 85.049536708000005, 2024-05-09T09:24:57.725 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-05-09T09:24:57.725 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 91, 2024-05-09T09:24:57.725 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T09:24:57.726 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 131.09085997700001 2024-05-09T09:24:57.726 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:24:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec28a5640 1 -- 172.21.15.12:0/2550279153 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ea40a2b90 msgr2=0x7f2ea40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec28a5640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ea40a2b90 0x7f2ea40a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0005fd0 tx=0x7f2eb000b390 comp rx=0 tx=0).stop 2024-05-09T09:24:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec28a5640 1 -- 172.21.15.12:0/2550279153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ebc074230 msgr2=0x7f2ebc199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec28a5640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ebc074230 0x7f2ebc199db0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009230 tx=0x7f2eac031ac0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec28a5640 1 -- 172.21.15.12:0/2550279153 shutdown_connections 2024-05-09T09:24:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec28a5640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ea40a2b90 0x7f2ea40a5050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec28a5640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ebc109720 0x7f2ebc1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec28a5640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ebc075810 0x7f2ebc19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec28a5640 1 --2- 172.21.15.12:0/2550279153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ebc074230 0x7f2ebc199db0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec28a5640 1 -- 172.21.15.12:0/2550279153 >> 172.21.15.12:0/2550279153 conn(0x7f2ebc0fce00 msgr2=0x7f2ebc1056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec28a5640 1 -- 172.21.15.12:0/2550279153 shutdown_connections 2024-05-09T09:24:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.718+0000 7f2ec28a5640 1 -- 172.21.15.12:0/2550279153 wait complete. 2024-05-09T09:24:57.729 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-09T09:24:57.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 -- 172.21.15.12:0/2015314809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99b8101960 msgr2=0x7f99b8101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 --2- 172.21.15.12:0/2015314809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99b8101960 0x7f99b8101de0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009900 tx=0x7f99a802a410 comp rx=0 tx=0).stop 2024-05-09T09:24:57.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 -- 172.21.15.12:0/2015314809 shutdown_connections 2024-05-09T09:24:57.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 --2- 172.21.15.12:0/2015314809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b8102320 0x7f99b8108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 --2- 172.21.15.12:0/2015314809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99b8101960 0x7f99b8101de0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 --2- 172.21.15.12:0/2015314809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b8100770 0x7f99b8100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 -- 172.21.15.12:0/2015314809 >> 172.21.15.12:0/2015314809 conn(0x7f99b80fbce0 msgr2=0x7f99b80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 -- 172.21.15.12:0/2015314809 shutdown_connections 2024-05-09T09:24:57.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 -- 172.21.15.12:0/2015314809 wait complete. 2024-05-09T09:24:57.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 Processor -- start 2024-05-09T09:24:57.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 -- start start 2024-05-09T09:24:57.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b8100770 0x7f99b81969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b8101960 0x7f99b8196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99b8102320 0x7f99b819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f99b810c3d0 con 0x7f99b8102320 2024-05-09T09:24:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f99b810c250 con 0x7f99b8100770 2024-05-09T09:24:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.835+0000 7f99c02d2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f99b810c550 con 0x7f99b8101960 2024-05-09T09:24:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99be86f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99b8102320 0x7f99b819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99be86f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99b8102320 0x7f99b819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50332/0 (socket says 172.21.15.12:50332) 2024-05-09T09:24:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99be86f640 1 -- 172.21.15.12:0/2721309119 learned_addr learned my addr 172.21.15.12:0/2721309119 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99be06e640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b8100770 0x7f99b81969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99bd86d640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b8101960 0x7f99b8196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99be86f640 1 -- 172.21.15.12:0/2721309119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b8101960 msgr2=0x7f99b8196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99be86f640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b8101960 0x7f99b8196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99be86f640 1 -- 172.21.15.12:0/2721309119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b8100770 msgr2=0x7f99b81969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99be86f640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b8100770 0x7f99b81969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99be86f640 1 -- 172.21.15.12:0/2721309119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b819e5c0 con 0x7f99b8102320 2024-05-09T09:24:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99be86f640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99b8102320 0x7f99b819df50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f99b000cce0 tx=0x7f99b000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99af7fe640 1 -- 172.21.15.12:0/2721309119 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b000f450 con 0x7f99b8102320 2024-05-09T09:24:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99c02d2640 1 -- 172.21.15.12:0/2721309119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b819e8b0 con 0x7f99b8102320 2024-05-09T09:24:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99af7fe640 1 -- 172.21.15.12:0/2721309119 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f99b000fac0 con 0x7f99b8102320 2024-05-09T09:24:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99af7fe640 1 -- 172.21.15.12:0/2721309119 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0018be0 con 0x7f99b8102320 2024-05-09T09:24:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.836+0000 7f99c02d2640 1 -- 172.21.15.12:0/2721309119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b819ee00 con 0x7f99b8102320 2024-05-09T09:24:57.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.839+0000 7f99af7fe640 1 -- 172.21.15.12:0/2721309119 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f99b000fc70 con 0x7f99b8102320 2024-05-09T09:24:57.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.839+0000 7f99c2fdd740 1 Processor -- start 2024-05-09T09:24:57.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.839+0000 7f99c2fdd740 1 -- start start 2024-05-09T09:24:57.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.839+0000 7f99c2fdd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 0x55f2248f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.839+0000 7f99c2fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f54c0 0x55f2248f9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.839+0000 7f99c2fdd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248fa630 0x55f2248fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99c2fdd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f2248564d0 con 0x55f2248f4ad0 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99c2fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f2247fe3e0 con 0x55f2248f54c0 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99c2fdd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f224852490 con 0x55f2248fa630 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99bd86d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 0x55f2248f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99bd86d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 0x55f2248f4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50340/0 (socket says 172.21.15.12:50340) 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99bd86d640 1 -- 172.21.15.12:0/15552564 learned_addr learned my addr 172.21.15.12:0/15552564 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99af7fe640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99a40a2b90 0x7f99a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99be06e640 1 --2- 172.21.15.12:0/15552564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f54c0 0x55f2248f9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99bd86d640 1 --2- 172.21.15.12:0/15552564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248fa630 0x55f2248fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99bd86d640 1 -- 172.21.15.12:0/15552564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248fa630 msgr2=0x55f2248fcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99bd86d640 1 --2- 172.21.15.12:0/15552564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248fa630 0x55f2248fcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99bd86d640 1 -- 172.21.15.12:0/15552564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f54c0 msgr2=0x55f2248f9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99bd86d640 1 --2- 172.21.15.12:0/15552564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f54c0 0x55f2248f9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99bd86d640 1 -- 172.21.15.12:0/15552564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f22456a2d0 con 0x55f2248f4ad0 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99be06e640 1 --2- 172.21.15.12:0/15552564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f54c0 0x55f2248f9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99be86f640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99a40a2b90 0x7f99a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99af7fe640 1 -- 172.21.15.12:0/2721309119 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f99b00cfaa0 con 0x7f99b8102320 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99bd86d640 1 --2- 172.21.15.12:0/15552564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 0x55f2248f4ef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f99a8005ab0 tx=0x7f99a8005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61dbac29329a6b79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.840+0000 7f99acbfe640 1 -- 172.21.15.12:0/15552564 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8043020 con 0x55f2248f4ad0 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.841+0000 7f99acbfe640 1 -- 172.21.15.12:0/15552564 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f99a8005020 con 0x55f2248f4ad0 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.841+0000 7f99acbfe640 1 -- 172.21.15.12:0/15552564 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8035050 con 0x55f2248f4ad0 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.841+0000 7f99be86f640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99a40a2b90 0x7f99a40a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f99b00cd290 tx=0x7f99b00516b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.841+0000 7f99c2fdd740 1 -- 172.21.15.12:0/15552564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 msgr2=0x55f2248f4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.841+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/15552564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 0x55f2248f4ef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f99a8005ab0 tx=0x7f99a8005a40 comp rx=0 tx=0).stop 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.841+0000 7f99c2fdd740 1 -- 172.21.15.12:0/15552564 shutdown_connections 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.841+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/15552564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248fa630 0x55f2248fcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.841+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/15552564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f54c0 0x55f2248f9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.841+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/15552564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 0x55f2248f4ef0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.841+0000 7f99c2fdd740 1 -- 172.21.15.12:0/15552564 >> 172.21.15.12:0/15552564 conn(0x55f2248f0140 msgr2=0x55f2248f0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.841+0000 7f99c2fdd740 1 -- 172.21.15.12:0/15552564 shutdown_connections 2024-05-09T09:24:57.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.841+0000 7f99c2fdd740 1 -- 172.21.15.12:0/15552564 wait complete. 2024-05-09T09:24:57.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99c2fdd740 1 -- start start 2024-05-09T09:24:57.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99c2fdd740 1 Processor -- start 2024-05-09T09:24:57.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99c2fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f4ad0 0x55f2248fef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99c2fdd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f54c0 0x55f2248ff470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99c2fdd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248fa630 0x55f2249064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99c2fdd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f2248fd580 con 0x55f2248f54c0 2024-05-09T09:24:57.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99c2fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f2248118e0 con 0x55f2248f4ad0 2024-05-09T09:24:57.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99c2fdd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f2248fd700 con 0x55f2248fa630 2024-05-09T09:24:57.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99c2fdd740 10 client.0 _ll_register_callbacks cb 0x55f2247de9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:57.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99be06e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f54c0 0x55f2248ff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99be06e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f54c0 0x55f2248ff470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50342/0 (socket says 172.21.15.12:50342) 2024-05-09T09:24:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99be06e640 1 -- 172.21.15.12:0/2811721150 learned_addr learned my addr 172.21.15.12:0/2811721150 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99bd86d640 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f4ad0 0x55f2248fef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99bd86d640 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248fa630 0x55f2249064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99be06e640 1 -- 172.21.15.12:0/2811721150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248fa630 msgr2=0x55f2249064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99be06e640 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248fa630 0x55f2249064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99be06e640 1 -- 172.21.15.12:0/2811721150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f4ad0 msgr2=0x55f2248fef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99be06e640 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f4ad0 0x55f2248fef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99be06e640 1 -- 172.21.15.12:0/2811721150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2246346b0 con 0x55f2248f54c0 2024-05-09T09:24:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.842+0000 7f99bd86d640 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f4ad0 0x55f2248fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99be06e640 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f54c0 0x55f2248ff470 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f99b4012780 tx=0x7f99b4012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab9905c4e84844f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99a1ffb640 1 -- 172.21.15.12:0/2811721150 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b4010d60 con 0x55f2248f54c0 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99a1ffb640 1 -- 172.21.15.12:0/2811721150 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f99b4005020 con 0x55f2248f54c0 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2811721150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99a8005020 con 0x55f2248f54c0 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99a1ffb640 1 -- 172.21.15.12:0/2811721150 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f99b4025050 con 0x55f2248f54c0 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99c2fdd740 10 client.15255 fetch_fsmap learned FSMap version 28 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2811721150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f224909560 con 0x55f2248f54c0 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99a1ffb640 1 -- 172.21.15.12:0/2811721150 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f99a8005020 con 0x55f2248f54c0 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99a1ffb640 1 -- 172.21.15.12:0/2811721150 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f99b401a430 con 0x55f2248f54c0 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99c2fdd740 10 client.15255 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99c2fdd740 10 client.15255 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99c2fdd740 20 client.15255 populate_metadata read hostname 'smithi012' 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55f22490bc50 0x55f22490d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99c2fdd740 4 client.15255 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:57.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.843+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2811721150 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f224863c00 con 0x55f22490bc50 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.844+0000 7f99bd86d640 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55f22490bc50 0x55f22490d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.844+0000 7f99bd86d640 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55f22490bc50 0x55f22490d0c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77ebd83259d0195a server_cookie=f9a98931ec3a526 in_seq=0 out_seq=0 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.844+0000 7f99a1ffb640 10 client.15255 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99a1ffb640 1 -- 172.21.15.12:0/2811721150 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f224863c00 con 0x55f22490bc50 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99a1ffb640 10 client.15255 handle_command_reply: tid=0 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99c2fdd740 1 client.15255 shutdown 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2811721150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f54c0 msgr2=0x55f2248ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f54c0 0x55f2248ff470 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f99b4012780 tx=0x7f99b4012c40 comp rx=0 tx=0).stop 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2811721150 shutdown_connections 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55f22490bc50 0x55f22490d0c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248fa630 0x55f2249064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99a1ffb640 0 client.15255 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f54c0 0x55f2248ff470 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/2811721150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f4ad0 0x55f2248fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2811721150 >> 172.21.15.12:0/2811721150 conn(0x55f2248f0140 msgr2=0x55f2248ed000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2811721150 shutdown_connections 2024-05-09T09:24:57.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2811721150 wait complete. 2024-05-09T09:24:57.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.845+0000 7f99c2fdd740 20 client.15255 trim_cache size 0 max 16384 2024-05-09T09:24:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99c2fdd740 1 Processor -- start 2024-05-09T09:24:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99c2fdd740 1 -- start start 2024-05-09T09:24:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99c2fdd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 0x55f2248f0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99c2fdd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248f54c0 0x55f224906470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99c2fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248fa630 0x55f2248feb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99c2fdd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f22490b400 con 0x55f2248f4ad0 2024-05-09T09:24:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99c2fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f22490bad0 con 0x55f2248fa630 2024-05-09T09:24:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99c2fdd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f22490c1a0 con 0x55f2248f54c0 2024-05-09T09:24:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99bd86d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 0x55f2248f0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99bd86d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 0x55f2248f0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50344/0 (socket says 172.21.15.12:50344) 2024-05-09T09:24:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99bd86d640 1 -- 172.21.15.12:0/1959119544 learned_addr learned my addr 172.21.15.12:0/1959119544 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99bd86d640 1 -- 172.21.15.12:0/1959119544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248f54c0 msgr2=0x55f224906470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99bd86d640 1 --2- 172.21.15.12:0/1959119544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248f54c0 0x55f224906470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99bd86d640 1 -- 172.21.15.12:0/1959119544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248fa630 msgr2=0x55f2248feb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99bd86d640 1 --2- 172.21.15.12:0/1959119544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248fa630 0x55f2248feb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99bd86d640 1 -- 172.21.15.12:0/1959119544 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2248f58a0 con 0x55f2248f4ad0 2024-05-09T09:24:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99bd86d640 1 --2- 172.21.15.12:0/1959119544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 0x55f2248f0520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009900 tx=0x7f99a802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165bc926ee7772dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.864+0000 7f99a3fff640 1 -- 172.21.15.12:0/1959119544 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8043040 con 0x55f2248f4ad0 2024-05-09T09:24:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.865+0000 7f99a3fff640 1 -- 172.21.15.12:0/1959119544 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f99a8050260 con 0x55f2248f4ad0 2024-05-09T09:24:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.865+0000 7f99a3fff640 1 -- 172.21.15.12:0/1959119544 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8035040 con 0x55f2248f4ad0 2024-05-09T09:24:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.865+0000 7f99c2fdd740 1 -- 172.21.15.12:0/1959119544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 msgr2=0x55f2248f0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.865+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/1959119544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 0x55f2248f0520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009900 tx=0x7f99a802a410 comp rx=0 tx=0).stop 2024-05-09T09:24:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.865+0000 7f99c2fdd740 1 -- 172.21.15.12:0/1959119544 shutdown_connections 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.865+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/1959119544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248fa630 0x55f2248feb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.865+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/1959119544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248f54c0 0x55f224906470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.865+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/1959119544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248f4ad0 0x55f2248f0520 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.865+0000 7f99c2fdd740 1 -- 172.21.15.12:0/1959119544 >> 172.21.15.12:0/1959119544 conn(0x55f2248f0140 msgr2=0x55f2248fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.865+0000 7f99c2fdd740 1 -- 172.21.15.12:0/1959119544 shutdown_connections 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.865+0000 7f99c2fdd740 1 -- 172.21.15.12:0/1959119544 wait complete. 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.865+0000 7f99c2fdd740 1 -- start start 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.865+0000 7f99c2fdd740 1 Processor -- start 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99c2fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f4ad0 0x55f224902030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99c2fdd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248f54c0 0x55f224902570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99c2fdd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248fa630 0x55f224905540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99c2fdd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f22490cf40 con 0x55f2248fa630 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99c2fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f22490c870 con 0x55f2248f4ad0 2024-05-09T09:24:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99c2fdd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f22490d610 con 0x55f2248f54c0 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99c2fdd740 10 client.0 _ll_register_callbacks cb 0x7f99b400a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99bd86d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248fa630 0x55f224905540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99bd86d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248fa630 0x55f224905540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50356/0 (socket says 172.21.15.12:50356) 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99bd86d640 1 -- 172.21.15.12:0/2138658798 learned_addr learned my addr 172.21.15.12:0/2138658798 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99be06e640 1 --2- 172.21.15.12:0/2138658798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248f54c0 0x55f224902570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99bd86d640 1 -- 172.21.15.12:0/2138658798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248f54c0 msgr2=0x55f224902570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99bd86d640 1 --2- 172.21.15.12:0/2138658798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248f54c0 0x55f224902570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99bd86d640 1 -- 172.21.15.12:0/2138658798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f4ad0 msgr2=0x55f224902030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99bd86d640 1 --2- 172.21.15.12:0/2138658798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f4ad0 0x55f224902030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99bd86d640 1 -- 172.21.15.12:0/2138658798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f224905b30 con 0x55f2248fa630 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99bd86d640 1 --2- 172.21.15.12:0/2138658798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248fa630 0x55f224905540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f99a8006010 tx=0x7f99a8031c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada725c73aa87fd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99a1ffb640 1 -- 172.21.15.12:0/2138658798 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8035070 con 0x55f2248fa630 2024-05-09T09:24:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99a1ffb640 1 -- 172.21.15.12:0/2138658798 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f99a80049a0 con 0x55f2248fa630 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.866+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2138658798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99a8050260 con 0x55f2248fa630 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.867+0000 7f99a1ffb640 1 -- 172.21.15.12:0/2138658798 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f99a8004ba0 con 0x55f2248fa630 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.867+0000 7f99c2fdd740 10 client.15261 fetch_fsmap learned FSMap version 28 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.867+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2138658798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f224972030 con 0x55f2248fa630 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.867+0000 7f99a1ffb640 1 -- 172.21.15.12:0/2138658798 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f99a8050260 con 0x55f2248fa630 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.867+0000 7f99a1ffb640 1 -- 172.21.15.12:0/2138658798 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f99a805b030 con 0x55f2248fa630 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.867+0000 7f99c2fdd740 10 client.15261 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.867+0000 7f99c2fdd740 10 client.15261 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.867+0000 7f99c2fdd740 20 client.15261 populate_metadata read hostname 'smithi012' 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.867+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/2138658798 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55f224976520 0x55f224975230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.867+0000 7f99c2fdd740 4 client.15261 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.867+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2138658798 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "status"}) -- 0x55f22485df50 con 0x55f224976520 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.867+0000 7f99be06e640 1 --2- 172.21.15.12:0/2138658798 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55f224976520 0x55f224975230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99be06e640 1 --2- 172.21.15.12:0/2138658798 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55f224976520 0x55f224975230 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d87cafc5e2008fc9 server_cookie=9a1e9bfdb6ae7b0e in_seq=0 out_seq=0 2024-05-09T09:24:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99a1ffb640 10 client.15261 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99a1ffb640 1 -- 172.21.15.12:0/2138658798 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f22485df50 con 0x55f224976520 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99a1ffb640 10 client.15261 handle_command_reply: tid=0 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99c2fdd740 1 client.15261 shutdown 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2138658798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248fa630 msgr2=0x55f224905540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/2138658798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248fa630 0x55f224905540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f99a8006010 tx=0x7f99a8031c80 comp rx=0 tx=0).stop 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2138658798 shutdown_connections 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/2138658798 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55f224976520 0x55f224975230 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/2138658798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2248fa630 0x55f224905540 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/2138658798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2248f54c0 0x55f224902570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99a1ffb640 0 client.15261 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99c2fdd740 1 --2- 172.21.15.12:0/2138658798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2248f4ad0 0x55f224902030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2138658798 >> 172.21.15.12:0/2138658798 conn(0x55f2248f0140 msgr2=0x55f2248f0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2138658798 shutdown_connections 2024-05-09T09:24:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99c2fdd740 1 -- 172.21.15.12:0/2138658798 wait complete. 2024-05-09T09:24:57.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.868+0000 7f99c2fdd740 20 client.15261 trim_cache size 0 max 16384 2024-05-09T09:24:57.875 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:24:57.875 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:24:57.875 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-05-09T09:24:57.875 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24502, 2024-05-09T09:24:57.875 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-09T09:24:57.875 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-09T09:24:57.875 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-09T09:24:57.875 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 86.213036715000001, 2024-05-09T09:24:57.875 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-05-09T09:24:57.875 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 91, 2024-05-09T09:24:57.876 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T09:24:57.876 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 128.40274839400001 2024-05-09T09:24:57.876 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:24:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.869+0000 7f99c02d2640 1 -- 172.21.15.12:0/2721309119 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99a40a2b90 msgr2=0x7f99a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.869+0000 7f99c02d2640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99a40a2b90 0x7f99a40a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f99b00cd290 tx=0x7f99b00516b0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.869+0000 7f99c02d2640 1 -- 172.21.15.12:0/2721309119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99b8102320 msgr2=0x7f99b819df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.869+0000 7f99c02d2640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99b8102320 0x7f99b819df50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f99b000cce0 tx=0x7f99b000a620 comp rx=0 tx=0).stop 2024-05-09T09:24:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.869+0000 7f99c02d2640 1 -- 172.21.15.12:0/2721309119 shutdown_connections 2024-05-09T09:24:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.869+0000 7f99c02d2640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99a40a2b90 0x7f99a40a5050 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.869+0000 7f99c02d2640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99b8102320 0x7f99b819df50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.869+0000 7f99c02d2640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b8101960 0x7f99b8196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.869+0000 7f99c02d2640 1 --2- 172.21.15.12:0/2721309119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b8100770 0x7f99b81969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.869+0000 7f99c02d2640 1 -- 172.21.15.12:0/2721309119 >> 172.21.15.12:0/2721309119 conn(0x7f99b80fbce0 msgr2=0x7f99b8104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.869+0000 7f99c02d2640 1 -- 172.21.15.12:0/2721309119 shutdown_connections 2024-05-09T09:24:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.869+0000 7f99c02d2640 1 -- 172.21.15.12:0/2721309119 wait complete. 2024-05-09T09:24:57.879 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-09T09:24:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.986+0000 7fa4ac8fc640 1 -- 172.21.15.12:0/1983103251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4a8104c40 msgr2=0x7fa4a8107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.986+0000 7fa4ac8fc640 1 --2- 172.21.15.12:0/1983103251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4a8104c40 0x7fa4a8107030 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa490009900 tx=0x7fa49002a410 comp rx=0 tx=0).stop 2024-05-09T09:24:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.986+0000 7fa4ac8fc640 1 -- 172.21.15.12:0/1983103251 shutdown_connections 2024-05-09T09:24:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.986+0000 7fa4ac8fc640 1 --2- 172.21.15.12:0/1983103251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a8107570 0x7fa4a81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.986+0000 7fa4ac8fc640 1 --2- 172.21.15.12:0/1983103251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4a8104c40 0x7fa4a8107030 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.986+0000 7fa4ac8fc640 1 --2- 172.21.15.12:0/1983103251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4a8102310 0x7fa4a8104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.986+0000 7fa4ac8fc640 1 -- 172.21.15.12:0/1983103251 >> 172.21.15.12:0/1983103251 conn(0x7fa4a80fbd00 msgr2=0x7fa4a80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.986+0000 7fa4ac8fc640 1 -- 172.21.15.12:0/1983103251 shutdown_connections 2024-05-09T09:24:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.986+0000 7fa4ac8fc640 1 -- 172.21.15.12:0/1983103251 wait complete. 2024-05-09T09:24:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.986+0000 7fa4ac8fc640 1 Processor -- start 2024-05-09T09:24:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4ac8fc640 1 -- start start 2024-05-09T09:24:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4ac8fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4a8102310 0x7fa4a8197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4ac8fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4a8104c40 0x7fa4a8198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4ac8fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a8107570 0x7fa4a819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4ac8fc640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a810d4f0 con 0x7fa4a8102310 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4ac8fc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a810d370 con 0x7fa4a8107570 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4ac8fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a810d670 con 0x7fa4a8104c40 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4a659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4a8102310 0x7fa4a8197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4a659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4a8102310 0x7fa4a8197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50378/0 (socket says 172.21.15.12:50378) 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4a659c640 1 -- 172.21.15.12:0/1301835315 learned_addr learned my addr 172.21.15.12:0/1301835315 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4a6d9d640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a8107570 0x7fa4a819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4a5d9b640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4a8104c40 0x7fa4a8198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4a659c640 1 -- 172.21.15.12:0/1301835315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4a8104c40 msgr2=0x7fa4a8198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4a659c640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4a8104c40 0x7fa4a8198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4a659c640 1 -- 172.21.15.12:0/1301835315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a8107570 msgr2=0x7fa4a819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4a659c640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a8107570 0x7fa4a819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4a659c640 1 -- 172.21.15.12:0/1301835315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a819f810 con 0x7fa4a8102310 2024-05-09T09:24:57.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.987+0000 7fa4a659c640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4a8102310 0x7fa4a8197bf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa49800d770 tx=0x7fa49800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.988+0000 7fa49f7fe640 1 -- 172.21.15.12:0/1301835315 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498017070 con 0x7fa4a8102310 2024-05-09T09:24:57.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.988+0000 7fa4ac8fc640 1 -- 172.21.15.12:0/1301835315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a819fb00 con 0x7fa4a8102310 2024-05-09T09:24:57.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.988+0000 7fa49f7fe640 1 -- 172.21.15.12:0/1301835315 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa4980054f0 con 0x7fa4a8102310 2024-05-09T09:24:57.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.988+0000 7fa49f7fe640 1 -- 172.21.15.12:0/1301835315 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498004d60 con 0x7fa4a8102310 2024-05-09T09:24:57.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.988+0000 7fa4ac8fc640 1 -- 172.21.15.12:0/1301835315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a81a0020 con 0x7fa4a8102310 2024-05-09T09:24:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.990+0000 7fa49f7fe640 1 -- 172.21.15.12:0/1301835315 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa498004f00 con 0x7fa4a8102310 2024-05-09T09:24:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.990+0000 7fa4af609740 1 Processor -- start 2024-05-09T09:24:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.990+0000 7fa4af609740 1 -- start start 2024-05-09T09:24:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.990+0000 7fa4af609740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384ecd0 0x563ec384f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.990+0000 7fa4af609740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384f6c0 0x563ec38541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.990+0000 7fa4af609740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec3854830 0x563ec3856cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4af609740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563ec37ae4d0 con 0x563ec384f6c0 2024-05-09T09:24:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4af609740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563ec37563e0 con 0x563ec3854830 2024-05-09T09:24:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4af609740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563ec37aa490 con 0x563ec384ecd0 2024-05-09T09:24:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4a659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384f6c0 0x563ec38541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4a659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384f6c0 0x563ec38541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50388/0 (socket says 172.21.15.12:50388) 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4a659c640 1 -- 172.21.15.12:0/2107943697 learned_addr learned my addr 172.21.15.12:0/2107943697 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4a6d9d640 1 --2- 172.21.15.12:0/2107943697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384ecd0 0x563ec384f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4a5d9b640 1 --2- 172.21.15.12:0/2107943697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec3854830 0x563ec3856cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa49f7fe640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa4940a2b90 0x7fa4940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4a659c640 1 -- 172.21.15.12:0/2107943697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384ecd0 msgr2=0x563ec384f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4a659c640 1 --2- 172.21.15.12:0/2107943697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384ecd0 0x563ec384f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4a659c640 1 -- 172.21.15.12:0/2107943697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec3854830 msgr2=0x563ec3856cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4a659c640 1 --2- 172.21.15.12:0/2107943697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec3854830 0x563ec3856cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4a659c640 1 -- 172.21.15.12:0/2107943697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec34c22d0 con 0x563ec384f6c0 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4a5d9b640 1 --2- 172.21.15.12:0/2107943697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec3854830 0x563ec3856cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4a6d9d640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa4940a2b90 0x7fa4940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa49f7fe640 1 -- 172.21.15.12:0/1301835315 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa4980ceda0 con 0x7fa4a8102310 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa4a659c640 1 --2- 172.21.15.12:0/2107943697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384f6c0 0x563ec38541c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa4980ba9c0 tx=0x7fa4980d2f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a60e3e4f4b7a12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa49cff9640 1 -- 172.21.15.12:0/2107943697 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498020070 con 0x563ec384f6c0 2024-05-09T09:24:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa49cff9640 1 -- 172.21.15.12:0/2107943697 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa4980056a0 con 0x563ec384f6c0 2024-05-09T09:24:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.991+0000 7fa49cff9640 1 -- 172.21.15.12:0/2107943697 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49809a7a0 con 0x563ec384f6c0 2024-05-09T09:24:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.992+0000 7fa4af609740 1 -- 172.21.15.12:0/2107943697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384f6c0 msgr2=0x563ec38541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.992+0000 7fa4a6d9d640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa4940a2b90 0x7fa4940a5050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x563ec384ff20 tx=0x7fa4a0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.992+0000 7fa4af609740 1 --2- 172.21.15.12:0/2107943697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384f6c0 0x563ec38541c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa4980ba9c0 tx=0x7fa4980d2f30 comp rx=0 tx=0).stop 2024-05-09T09:24:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.992+0000 7fa4af609740 1 -- 172.21.15.12:0/2107943697 shutdown_connections 2024-05-09T09:24:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.992+0000 7fa4af609740 1 --2- 172.21.15.12:0/2107943697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec3854830 0x563ec3856cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.992+0000 7fa4af609740 1 --2- 172.21.15.12:0/2107943697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384f6c0 0x563ec38541c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.992+0000 7fa4af609740 1 --2- 172.21.15.12:0/2107943697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384ecd0 0x563ec384f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.992+0000 7fa4af609740 1 -- 172.21.15.12:0/2107943697 >> 172.21.15.12:0/2107943697 conn(0x563ec384a340 msgr2=0x563ec384a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.992+0000 7fa4af609740 1 -- 172.21.15.12:0/2107943697 shutdown_connections 2024-05-09T09:24:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.992+0000 7fa4af609740 1 -- 172.21.15.12:0/2107943697 wait complete. 2024-05-09T09:24:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4af609740 1 -- start start 2024-05-09T09:24:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4af609740 1 Processor -- start 2024-05-09T09:24:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4af609740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 0x563ec38590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4af609740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384f6c0 0x563ec3859600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4af609740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec3854830 0x563ec3860640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4af609740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563ec3857780 con 0x563ec384ecd0 2024-05-09T09:24:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4af609740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563ec37698e0 con 0x563ec3854830 2024-05-09T09:24:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4af609740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563ec3857900 con 0x563ec384f6c0 2024-05-09T09:24:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4af609740 10 client.0 _ll_register_callbacks cb 0x563ec37369c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4a659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 0x563ec38590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4a659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 0x563ec38590c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50404/0 (socket says 172.21.15.12:50404) 2024-05-09T09:24:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4a659c640 1 -- 172.21.15.12:0/1781260059 learned_addr learned my addr 172.21.15.12:0/1781260059 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4a5d9b640 1 --2- 172.21.15.12:0/1781260059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384f6c0 0x563ec3859600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4a659c640 1 -- 172.21.15.12:0/1781260059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384f6c0 msgr2=0x563ec3859600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4a659c640 1 --2- 172.21.15.12:0/1781260059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384f6c0 0x563ec3859600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4a659c640 1 -- 172.21.15.12:0/1781260059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec3854830 msgr2=0x563ec3860640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4a659c640 1 --2- 172.21.15.12:0/1781260059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec3854830 0x563ec3860640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4a659c640 1 -- 172.21.15.12:0/1781260059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec358c6b0 con 0x563ec384ecd0 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa4a659c640 1 --2- 172.21.15.12:0/1781260059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 0x563ec38590c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa4980cc590 tx=0x7fa4980d2410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18921419da83785f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.993+0000 7fa48dffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa48dffb640 1 -- 172.21.15.12:0/1781260059 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498020040 con 0x563ec384ecd0 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa48dffb640 1 -- 172.21.15.12:0/1781260059 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa49809a9e0 con 0x563ec384ecd0 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa4af609740 1 -- 172.21.15.12:0/1781260059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4980056a0 con 0x563ec384ecd0 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa48dffb640 1 -- 172.21.15.12:0/1781260059 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4980046c0 con 0x563ec384ecd0 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa4af609740 10 client.15273 fetch_fsmap learned FSMap version 28 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa4af609740 1 -- 172.21.15.12:0/1781260059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563ec3863590 con 0x563ec384ecd0 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa48dffb640 1 -- 172.21.15.12:0/1781260059 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa4980056a0 con 0x563ec384ecd0 2024-05-09T09:24:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa48dffb640 1 -- 172.21.15.12:0/1781260059 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4980d1040 con 0x563ec384ecd0 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa4af609740 10 client.15273 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa4af609740 10 client.15273 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa4af609740 20 client.15273 populate_metadata read hostname 'smithi012' 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa4af609740 1 --2- 172.21.15.12:0/1781260059 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563ec3865810 0x563ec3866c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa4af609740 4 client.15273 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa4af609740 1 -- 172.21.15.12:0/1781260059 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ec37bbc00 con 0x563ec3865810 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.994+0000 7fa4a5d9b640 1 --2- 172.21.15.12:0/1781260059 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563ec3865810 0x563ec3866c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.995+0000 7fa48dffb640 10 client.15273 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.995+0000 7fa4a5d9b640 1 --2- 172.21.15.12:0/1781260059 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563ec3865810 0x563ec3866c80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab871c393882109c server_cookie=dd3701349787ebb9 in_seq=0 out_seq=0 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa48dffb640 1 -- 172.21.15.12:0/1781260059 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ec37bbc00 con 0x563ec3865810 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa48dffb640 10 client.15273 handle_command_reply: tid=0 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa4af609740 1 client.15273 shutdown 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa4af609740 1 -- 172.21.15.12:0/1781260059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 msgr2=0x563ec38590c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa4af609740 1 --2- 172.21.15.12:0/1781260059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 0x563ec38590c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa4980cc590 tx=0x7fa4980d2410 comp rx=0 tx=0).stop 2024-05-09T09:24:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa4af609740 1 -- 172.21.15.12:0/1781260059 shutdown_connections 2024-05-09T09:24:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa4af609740 1 --2- 172.21.15.12:0/1781260059 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563ec3865810 0x563ec3866c80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa4af609740 1 --2- 172.21.15.12:0/1781260059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec3854830 0x563ec3860640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa48dffb640 0 client.15273 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:24:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa4af609740 1 --2- 172.21.15.12:0/1781260059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384f6c0 0x563ec3859600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa4af609740 1 --2- 172.21.15.12:0/1781260059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 0x563ec38590c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa4af609740 1 -- 172.21.15.12:0/1781260059 >> 172.21.15.12:0/1781260059 conn(0x563ec384a340 msgr2=0x563ec3845000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa4af609740 1 -- 172.21.15.12:0/1781260059 shutdown_connections 2024-05-09T09:24:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa4af609740 1 -- 172.21.15.12:0/1781260059 wait complete. 2024-05-09T09:24:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:57.996+0000 7fa4af609740 20 client.15273 trim_cache size 0 max 16384 2024-05-09T09:24:58.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4af609740 1 Processor -- start 2024-05-09T09:24:58.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4af609740 1 -- start start 2024-05-09T09:24:58.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4af609740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec384ecd0 0x563ec3859560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4af609740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384f6c0 0x563ec3860600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4af609740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec3854830 0x563ec384a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4af609740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563ec3864fc0 con 0x563ec3854830 2024-05-09T09:24:58.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4af609740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563ec3865690 con 0x563ec384ecd0 2024-05-09T09:24:58.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4af609740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563ec3865d60 con 0x563ec384f6c0 2024-05-09T09:24:58.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4a6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec3854830 0x563ec384a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4a6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec3854830 0x563ec384a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50410/0 (socket says 172.21.15.12:50410) 2024-05-09T09:24:58.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4a6d9d640 1 -- 172.21.15.12:0/606939559 learned_addr learned my addr 172.21.15.12:0/606939559 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:58.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4a5d9b640 1 --2- 172.21.15.12:0/606939559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384f6c0 0x563ec3860600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4a659c640 1 --2- 172.21.15.12:0/606939559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec384ecd0 0x563ec3859560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4a6d9d640 1 -- 172.21.15.12:0/606939559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384f6c0 msgr2=0x563ec3860600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4a6d9d640 1 --2- 172.21.15.12:0/606939559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384f6c0 0x563ec3860600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4a6d9d640 1 -- 172.21.15.12:0/606939559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec384ecd0 msgr2=0x563ec3859560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4a6d9d640 1 --2- 172.21.15.12:0/606939559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec384ecd0 0x563ec3859560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.015+0000 7fa4a6d9d640 1 -- 172.21.15.12:0/606939559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec3854540 con 0x563ec3854830 2024-05-09T09:24:58.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.016+0000 7fa4a6d9d640 1 --2- 172.21.15.12:0/606939559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec3854830 0x563ec384a720 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0005f40 tx=0x7fa4a0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655fefac6561fbf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.016+0000 7fa48ffff640 1 -- 172.21.15.12:0/606939559 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a001c460 con 0x563ec3854830 2024-05-09T09:24:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.016+0000 7fa48ffff640 1 -- 172.21.15.12:0/606939559 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa4a001cad0 con 0x563ec3854830 2024-05-09T09:24:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.016+0000 7fa48ffff640 1 -- 172.21.15.12:0/606939559 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a001ec50 con 0x563ec3854830 2024-05-09T09:24:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.016+0000 7fa4af609740 1 -- 172.21.15.12:0/606939559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec3854830 msgr2=0x563ec384a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.016+0000 7fa4af609740 1 --2- 172.21.15.12:0/606939559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec3854830 0x563ec384a720 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0005f40 tx=0x7fa4a0014040 comp rx=0 tx=0).stop 2024-05-09T09:24:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.016+0000 7fa4af609740 1 -- 172.21.15.12:0/606939559 shutdown_connections 2024-05-09T09:24:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.016+0000 7fa4af609740 1 --2- 172.21.15.12:0/606939559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec3854830 0x563ec384a720 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.016+0000 7fa4af609740 1 --2- 172.21.15.12:0/606939559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec384f6c0 0x563ec3860600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.016+0000 7fa4af609740 1 --2- 172.21.15.12:0/606939559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec384ecd0 0x563ec3859560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.016+0000 7fa4af609740 1 -- 172.21.15.12:0/606939559 >> 172.21.15.12:0/606939559 conn(0x563ec384a340 msgr2=0x563ec384ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.016+0000 7fa4af609740 1 -- 172.21.15.12:0/606939559 shutdown_connections 2024-05-09T09:24:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.016+0000 7fa4af609740 1 -- 172.21.15.12:0/606939559 wait complete. 2024-05-09T09:24:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4af609740 1 -- start start 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4af609740 1 Processor -- start 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4af609740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 0x563ec385bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4af609740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec384f6c0 0x563ec385c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4af609740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec3854830 0x563ec385f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4af609740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563ec3866b00 con 0x563ec384ecd0 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4af609740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563ec3866430 con 0x563ec384f6c0 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4af609740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563ec38671d0 con 0x563ec3854830 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4af609740 10 client.0 _ll_register_callbacks cb 0x7fa4980d2dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4a659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 0x563ec385bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4a659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 0x563ec385bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50412/0 (socket says 172.21.15.12:50412) 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4a659c640 1 -- 172.21.15.12:0/3692486113 learned_addr learned my addr 172.21.15.12:0/3692486113 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4a5d9b640 1 --2- 172.21.15.12:0/3692486113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec384f6c0 0x563ec385c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4a6d9d640 1 --2- 172.21.15.12:0/3692486113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec3854830 0x563ec385f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4a659c640 1 -- 172.21.15.12:0/3692486113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec3854830 msgr2=0x563ec385f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4a659c640 1 --2- 172.21.15.12:0/3692486113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec3854830 0x563ec385f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4a659c640 1 -- 172.21.15.12:0/3692486113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec384f6c0 msgr2=0x563ec385c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4a659c640 1 --2- 172.21.15.12:0/3692486113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec384f6c0 0x563ec385c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.017+0000 7fa4a659c640 1 -- 172.21.15.12:0/3692486113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec385f8e0 con 0x563ec384ecd0 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa4a659c640 1 --2- 172.21.15.12:0/3692486113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 0x563ec385bd90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa4980977b0 tx=0x7fa498097660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37077d8e3d6af270 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa48dffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa48dffb640 1 -- 172.21.15.12:0/3692486113 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498020070 con 0x563ec384ecd0 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa48dffb640 1 -- 172.21.15.12:0/3692486113 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa4980051f0 con 0x563ec384ecd0 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa4af609740 1 -- 172.21.15.12:0/3692486113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a001cad0 con 0x563ec384ecd0 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa48dffb640 1 -- 172.21.15.12:0/3692486113 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa49809acb0 con 0x563ec384ecd0 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa4af609740 10 client.15279 fetch_fsmap learned FSMap version 28 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa4af609740 1 -- 172.21.15.12:0/3692486113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563ec38cb880 con 0x563ec384ecd0 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa48dffb640 1 -- 172.21.15.12:0/3692486113 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa4a001cad0 con 0x563ec384ecd0 2024-05-09T09:24:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa48dffb640 1 -- 172.21.15.12:0/3692486113 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4980d1070 con 0x563ec384ecd0 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa4af609740 10 client.15279 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa4af609740 10 client.15279 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa4af609740 20 client.15279 populate_metadata read hostname 'smithi012' 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa4af609740 1 --2- 172.21.15.12:0/3692486113 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563ec38d01f0 0x563ec38cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa4af609740 4 client.15279 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.018+0000 7fa4af609740 1 -- 172.21.15.12:0/3692486113 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "status"}) -- 0x563ec37b5f50 con 0x563ec38d01f0 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4a5d9b640 1 --2- 172.21.15.12:0/3692486113 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563ec38d01f0 0x563ec38cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4a5d9b640 1 --2- 172.21.15.12:0/3692486113 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563ec38d01f0 0x563ec38cef00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2f73fc0411ab3e9 server_cookie=1c43a4f6c8f8b4c in_seq=0 out_seq=0 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa48dffb640 10 client.15279 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa48dffb640 1 -- 172.21.15.12:0/3692486113 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563ec37b5f50 con 0x563ec38d01f0 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa48dffb640 10 client.15279 handle_command_reply: tid=0 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4af609740 1 client.15279 shutdown 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4af609740 1 -- 172.21.15.12:0/3692486113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 msgr2=0x563ec385bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4af609740 1 --2- 172.21.15.12:0/3692486113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 0x563ec385bd90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa4980977b0 tx=0x7fa498097660 comp rx=0 tx=0).stop 2024-05-09T09:24:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4af609740 1 -- 172.21.15.12:0/3692486113 shutdown_connections 2024-05-09T09:24:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4af609740 1 --2- 172.21.15.12:0/3692486113 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563ec38d01f0 0x563ec38cef00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4af609740 1 --2- 172.21.15.12:0/3692486113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec3854830 0x563ec385f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4af609740 1 --2- 172.21.15.12:0/3692486113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec384f6c0 0x563ec385c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa48dffb640 0 client.15279 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:24:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4af609740 1 --2- 172.21.15.12:0/3692486113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ec384ecd0 0x563ec385bd90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4af609740 1 -- 172.21.15.12:0/3692486113 >> 172.21.15.12:0/3692486113 conn(0x563ec384a340 msgr2=0x563ec384a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4af609740 1 -- 172.21.15.12:0/3692486113 shutdown_connections 2024-05-09T09:24:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4af609740 1 -- 172.21.15.12:0/3692486113 wait complete. 2024-05-09T09:24:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.019+0000 7fa4af609740 20 client.15279 trim_cache size 0 max 16384 2024-05-09T09:24:58.027 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:24:58.027 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:24:58.027 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-05-09T09:24:58.027 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24515, 2024-05-09T09:24:58.027 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-09T09:24:58.027 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-09T09:24:58.027 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-09T09:24:58.027 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 87.377190976999998, 2024-05-09T09:24:58.027 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-05-09T09:24:58.027 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 91, 2024-05-09T09:24:58.027 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T09:24:58.027 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 113.916660612 2024-05-09T09:24:58.027 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:24:58.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.020+0000 7fa4ac8fc640 1 -- 172.21.15.12:0/1301835315 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa4940a2b90 msgr2=0x7fa4940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.020+0000 7fa4ac8fc640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa4940a2b90 0x7fa4940a5050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x563ec384ff20 tx=0x7fa4a0004a40 comp rx=0 tx=0).stop 2024-05-09T09:24:58.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.020+0000 7fa4ac8fc640 1 -- 172.21.15.12:0/1301835315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4a8102310 msgr2=0x7fa4a8197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.020+0000 7fa4ac8fc640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4a8102310 0x7fa4a8197bf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa49800d770 tx=0x7fa49800dc30 comp rx=0 tx=0).stop 2024-05-09T09:24:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.020+0000 7fa4ac8fc640 1 -- 172.21.15.12:0/1301835315 shutdown_connections 2024-05-09T09:24:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.020+0000 7fa4ac8fc640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa4940a2b90 0x7fa4940a5050 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.020+0000 7fa4ac8fc640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a8107570 0x7fa4a819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.020+0000 7fa4ac8fc640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4a8104c40 0x7fa4a8198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.020+0000 7fa4ac8fc640 1 --2- 172.21.15.12:0/1301835315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4a8102310 0x7fa4a8197bf0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.020+0000 7fa4ac8fc640 1 -- 172.21.15.12:0/1301835315 >> 172.21.15.12:0/1301835315 conn(0x7fa4a80fbd00 msgr2=0x7fa4a8102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.020+0000 7fa4ac8fc640 1 -- 172.21.15.12:0/1301835315 shutdown_connections 2024-05-09T09:24:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.020+0000 7fa4ac8fc640 1 -- 172.21.15.12:0/1301835315 wait complete. 2024-05-09T09:24:58.031 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-09T09:24:58.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.139+0000 7fd6cf742640 1 -- 172.21.15.12:0/2623252031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c8101960 msgr2=0x7fd6c8101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.139+0000 7fd6cf742640 1 --2- 172.21.15.12:0/2623252031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c8101960 0x7fd6c8101de0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009900 tx=0x7fd6b802a410 comp rx=0 tx=0).stop 2024-05-09T09:24:58.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.139+0000 7fd6cf742640 1 -- 172.21.15.12:0/2623252031 shutdown_connections 2024-05-09T09:24:58.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.139+0000 7fd6cf742640 1 --2- 172.21.15.12:0/2623252031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6c8102320 0x7fd6c8108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.139+0000 7fd6cf742640 1 --2- 172.21.15.12:0/2623252031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c8101960 0x7fd6c8101de0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.139+0000 7fd6cf742640 1 --2- 172.21.15.12:0/2623252031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6c8100770 0x7fd6c8100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.139+0000 7fd6cf742640 1 -- 172.21.15.12:0/2623252031 >> 172.21.15.12:0/2623252031 conn(0x7fd6c80fbce0 msgr2=0x7fd6c80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:58.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.140+0000 7fd6cf742640 1 -- 172.21.15.12:0/2623252031 shutdown_connections 2024-05-09T09:24:58.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.140+0000 7fd6cf742640 1 -- 172.21.15.12:0/2623252031 wait complete. 2024-05-09T09:24:58.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.140+0000 7fd6cf742640 1 Processor -- start 2024-05-09T09:24:58.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.140+0000 7fd6cf742640 1 -- start start 2024-05-09T09:24:58.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.140+0000 7fd6cf742640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c8100770 0x7fd6c8197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.140+0000 7fd6cf742640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6c8101960 0x7fd6c8198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.140+0000 7fd6cf742640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6c8102320 0x7fd6c819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.140+0000 7fd6cf742640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c810c420 con 0x7fd6c8100770 2024-05-09T09:24:58.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.140+0000 7fd6cf742640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c810c2a0 con 0x7fd6c8102320 2024-05-09T09:24:58.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.140+0000 7fd6cf742640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c810c5a0 con 0x7fd6c8101960 2024-05-09T09:24:58.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.140+0000 7fd6cd4de640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c8100770 0x7fd6c8197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6cd4de640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c8100770 0x7fd6c8197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50432/0 (socket says 172.21.15.12:50432) 2024-05-09T09:24:58.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6cd4de640 1 -- 172.21.15.12:0/3899216823 learned_addr learned my addr 172.21.15.12:0/3899216823 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:58.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6cdcdf640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6c8102320 0x7fd6c819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6cccdd640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6c8101960 0x7fd6c8198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6cd4de640 1 -- 172.21.15.12:0/3899216823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6c8101960 msgr2=0x7fd6c8198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6cd4de640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6c8101960 0x7fd6c8198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6cd4de640 1 -- 172.21.15.12:0/3899216823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6c8102320 msgr2=0x7fd6c819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6cd4de640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6c8102320 0x7fd6c819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6cd4de640 1 -- 172.21.15.12:0/3899216823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c819f810 con 0x7fd6c8100770 2024-05-09T09:24:58.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6cd4de640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c8100770 0x7fd6c8197bf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400e9c0 tx=0x7fd6c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:58.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6be7fc640 1 -- 172.21.15.12:0/3899216823 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c400cd80 con 0x7fd6c8100770 2024-05-09T09:24:58.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6be7fc640 1 -- 172.21.15.12:0/3899216823 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd6c40035f0 con 0x7fd6c8100770 2024-05-09T09:24:58.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6cf742640 1 -- 172.21.15.12:0/3899216823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c819fb00 con 0x7fd6c8100770 2024-05-09T09:24:58.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6be7fc640 1 -- 172.21.15.12:0/3899216823 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4010690 con 0x7fd6c8100770 2024-05-09T09:24:58.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.141+0000 7fd6cf742640 1 -- 172.21.15.12:0/3899216823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c819ffd0 con 0x7fd6c8100770 2024-05-09T09:24:58.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.143+0000 7fd6be7fc640 1 -- 172.21.15.12:0/3899216823 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd6c40037a0 con 0x7fd6c8100770 2024-05-09T09:24:58.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.144+0000 7fd6d244e740 1 Processor -- start 2024-05-09T09:24:58.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.144+0000 7fd6d244e740 1 -- start start 2024-05-09T09:24:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.144+0000 7fd6d244e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.144+0000 7fd6d244e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c256c0 0x55e156c2a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.144+0000 7fd6d244e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c2a830 0x55e156c2ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.144+0000 7fd6d244e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e156b844d0 con 0x55e156c24cd0 2024-05-09T09:24:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.144+0000 7fd6d244e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e156b2c3e0 con 0x55e156c2a830 2024-05-09T09:24:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.144+0000 7fd6d244e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e156b80490 con 0x55e156c256c0 2024-05-09T09:24:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.144+0000 7fd6cdcdf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.144+0000 7fd6cdcdf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c250f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50434/0 (socket says 172.21.15.12:50434) 2024-05-09T09:24:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.144+0000 7fd6cdcdf640 1 -- 172.21.15.12:0/921725821 learned_addr learned my addr 172.21.15.12:0/921725821 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.144+0000 7fd6cccdd640 1 --2- 172.21.15.12:0/921725821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c2a830 0x55e156c2ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.144+0000 7fd6cd4de640 1 --2- 172.21.15.12:0/921725821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c256c0 0x55e156c2a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.145+0000 7fd6be7fc640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd6b40a2b90 0x7fd6b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.145+0000 7fd6cdcdf640 1 -- 172.21.15.12:0/921725821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c256c0 msgr2=0x55e156c2a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.145+0000 7fd6cdcdf640 1 --2- 172.21.15.12:0/921725821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c256c0 0x55e156c2a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.145+0000 7fd6cdcdf640 1 -- 172.21.15.12:0/921725821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c2a830 msgr2=0x55e156c2ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.145+0000 7fd6cdcdf640 1 --2- 172.21.15.12:0/921725821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c2a830 0x55e156c2ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.145+0000 7fd6cdcdf640 1 -- 172.21.15.12:0/921725821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1568982d0 con 0x55e156c24cd0 2024-05-09T09:24:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.145+0000 7fd6cccdd640 1 --2- 172.21.15.12:0/921725821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c2a830 0x55e156c2ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.145+0000 7fd6be7fc640 1 -- 172.21.15.12:0/3899216823 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd6c40cf7c0 con 0x7fd6c8100770 2024-05-09T09:24:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.145+0000 7fd6cdcdf640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd6b40a2b90 0x7fd6b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.145+0000 7fd6cdcdf640 1 --2- 172.21.15.12:0/921725821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c250f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0011590 tx=0x7fd6c0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2956ed0eee6fe10a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.145+0000 7fd6b37fe640 1 -- 172.21.15.12:0/921725821 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0017050 con 0x55e156c24cd0 2024-05-09T09:24:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.145+0000 7fd6b37fe640 1 -- 172.21.15.12:0/921725821 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd6c000fd10 con 0x55e156c24cd0 2024-05-09T09:24:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.145+0000 7fd6b37fe640 1 -- 172.21.15.12:0/921725821 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c001bb00 con 0x55e156c24cd0 2024-05-09T09:24:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.146+0000 7fd6d244e740 1 -- 172.21.15.12:0/921725821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 msgr2=0x55e156c250f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.146+0000 7fd6d244e740 1 --2- 172.21.15.12:0/921725821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c250f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0011590 tx=0x7fd6c0011a50 comp rx=0 tx=0).stop 2024-05-09T09:24:58.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.146+0000 7fd6cdcdf640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd6b40a2b90 0x7fd6b40a5050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0022000 tx=0x7fd6c0023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:58.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.146+0000 7fd6d244e740 1 -- 172.21.15.12:0/921725821 shutdown_connections 2024-05-09T09:24:58.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.146+0000 7fd6d244e740 1 --2- 172.21.15.12:0/921725821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c2a830 0x55e156c2ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.146+0000 7fd6d244e740 1 --2- 172.21.15.12:0/921725821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c256c0 0x55e156c2a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.146+0000 7fd6d244e740 1 --2- 172.21.15.12:0/921725821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c250f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.146+0000 7fd6d244e740 1 -- 172.21.15.12:0/921725821 >> 172.21.15.12:0/921725821 conn(0x55e156c1bf30 msgr2=0x55e156c1c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:58.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.146+0000 7fd6d244e740 1 -- 172.21.15.12:0/921725821 shutdown_connections 2024-05-09T09:24:58.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.146+0000 7fd6d244e740 1 -- 172.21.15.12:0/921725821 wait complete. 2024-05-09T09:24:58.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.146+0000 7fd6d244e740 1 -- start start 2024-05-09T09:24:58.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.146+0000 7fd6d244e740 1 Processor -- start 2024-05-09T09:24:58.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6d244e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c24cd0 0x55e156c2f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6d244e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c256c0 0x55e156c2f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6d244e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c2a830 0x55e156c36680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6d244e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e156c2d7e0 con 0x55e156c2a830 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6d244e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e156b3f8e0 con 0x55e156c24cd0 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6d244e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e156c2d960 con 0x55e156c256c0 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6d244e740 10 client.0 _ll_register_callbacks cb 0x55e156b0c9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6cdcdf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c2a830 0x55e156c36680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6cdcdf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c2a830 0x55e156c36680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50438/0 (socket says 172.21.15.12:50438) 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6cdcdf640 1 -- 172.21.15.12:0/850689801 learned_addr learned my addr 172.21.15.12:0/850689801 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6cd4de640 1 --2- 172.21.15.12:0/850689801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c24cd0 0x55e156c2f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6cccdd640 1 --2- 172.21.15.12:0/850689801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c256c0 0x55e156c2f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6cdcdf640 1 -- 172.21.15.12:0/850689801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c256c0 msgr2=0x55e156c2f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6cdcdf640 1 --2- 172.21.15.12:0/850689801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c256c0 0x55e156c2f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6cdcdf640 1 -- 172.21.15.12:0/850689801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c24cd0 msgr2=0x55e156c2f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6cdcdf640 1 --2- 172.21.15.12:0/850689801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c24cd0 0x55e156c2f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.147+0000 7fd6cdcdf640 1 -- 172.21.15.12:0/850689801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1569626b0 con 0x55e156c2a830 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6cdcdf640 1 --2- 172.21.15.12:0/850689801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c2a830 0x55e156c36680 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000f850 tx=0x7fd6c000f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69360f1413594bf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6b0ff9640 1 -- 172.21.15.12:0/850689801 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c00049e0 con 0x55e156c2a830 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6b0ff9640 1 -- 172.21.15.12:0/850689801 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd6c0004b80 con 0x55e156c2a830 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6d244e740 1 -- 172.21.15.12:0/850689801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6c000fd10 con 0x55e156c2a830 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6b0ff9640 1 -- 172.21.15.12:0/850689801 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6c0017020 con 0x55e156c2a830 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6d244e740 10 client.15291 fetch_fsmap learned FSMap version 28 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6d244e740 1 -- 172.21.15.12:0/850689801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e156c396d0 con 0x55e156c2a830 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6b0ff9640 1 -- 172.21.15.12:0/850689801 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd6c000fd10 con 0x55e156c2a830 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6b0ff9640 1 -- 172.21.15.12:0/850689801 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd6c001a070 con 0x55e156c2a830 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6d244e740 10 client.15291 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6d244e740 10 client.15291 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6d244e740 20 client.15291 populate_metadata read hostname 'smithi012' 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6d244e740 1 --2- 172.21.15.12:0/850689801 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e156c3bd30 0x55e156c3d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6d244e740 4 client.15291 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.148+0000 7fd6d244e740 1 -- 172.21.15.12:0/850689801 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e156b91c00 con 0x55e156c3bd30 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.149+0000 7fd6cd4de640 1 --2- 172.21.15.12:0/850689801 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e156c3bd30 0x55e156c3d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.149+0000 7fd6cd4de640 1 --2- 172.21.15.12:0/850689801 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e156c3bd30 0x55e156c3d1a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ba859de8503f9 server_cookie=f8a203f246de0cc5 in_seq=0 out_seq=0 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.149+0000 7fd6b0ff9640 10 client.15291 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6b0ff9640 1 -- 172.21.15.12:0/850689801 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e156b91c00 con 0x55e156c3bd30 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6b0ff9640 10 client.15291 handle_command_reply: tid=0 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6d244e740 1 client.15291 shutdown 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6d244e740 1 -- 172.21.15.12:0/850689801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c2a830 msgr2=0x55e156c36680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6d244e740 1 --2- 172.21.15.12:0/850689801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c2a830 0x55e156c36680 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000f850 tx=0x7fd6c000f880 comp rx=0 tx=0).stop 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6d244e740 1 -- 172.21.15.12:0/850689801 shutdown_connections 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6d244e740 1 --2- 172.21.15.12:0/850689801 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e156c3bd30 0x55e156c3d1a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6d244e740 1 --2- 172.21.15.12:0/850689801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c2a830 0x55e156c36680 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6d244e740 1 --2- 172.21.15.12:0/850689801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c256c0 0x55e156c2f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6d244e740 1 --2- 172.21.15.12:0/850689801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c24cd0 0x55e156c2f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6d244e740 1 -- 172.21.15.12:0/850689801 >> 172.21.15.12:0/850689801 conn(0x55e156c1bf30 msgr2=0x55e156c1b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6b0ff9640 0 client.15291 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:24:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6d244e740 1 -- 172.21.15.12:0/850689801 shutdown_connections 2024-05-09T09:24:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6d244e740 1 -- 172.21.15.12:0/850689801 wait complete. 2024-05-09T09:24:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.150+0000 7fd6d244e740 20 client.15291 trim_cache size 0 max 16384 2024-05-09T09:24:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6d244e740 1 Processor -- start 2024-05-09T09:24:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6d244e740 1 -- start start 2024-05-09T09:24:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6d244e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c2cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6d244e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c256c0 0x55e156c365a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6d244e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c2a830 0x55e156c16d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6d244e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e156c3b4e0 con 0x55e156c24cd0 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6d244e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e156c3bbb0 con 0x55e156c256c0 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6d244e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e156c3c280 con 0x55e156c2a830 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6cd4de640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c2cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6cd4de640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c2cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50442/0 (socket says 172.21.15.12:50442) 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6cd4de640 1 -- 172.21.15.12:0/1275947147 learned_addr learned my addr 172.21.15.12:0/1275947147 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6cdcdf640 1 --2- 172.21.15.12:0/1275947147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c2a830 0x55e156c16d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6cccdd640 1 --2- 172.21.15.12:0/1275947147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c256c0 0x55e156c365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6cd4de640 1 -- 172.21.15.12:0/1275947147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c2a830 msgr2=0x55e156c16d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6cd4de640 1 --2- 172.21.15.12:0/1275947147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c2a830 0x55e156c16d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6cd4de640 1 -- 172.21.15.12:0/1275947147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c256c0 msgr2=0x55e156c365a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6cd4de640 1 --2- 172.21.15.12:0/1275947147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c256c0 0x55e156c365a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6cd4de640 1 -- 172.21.15.12:0/1275947147 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e156c1ba70 con 0x55e156c24cd0 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6cdcdf640 1 --2- 172.21.15.12:0/1275947147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c2a830 0x55e156c16d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.169+0000 7fd6cd4de640 1 --2- 172.21.15.12:0/1275947147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c2cc20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4018000 tx=0x7fd6c406f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2900c301be02d04a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.170+0000 7fd6b2ffd640 1 -- 172.21.15.12:0/1275947147 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c406a020 con 0x55e156c24cd0 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.170+0000 7fd6b2ffd640 1 -- 172.21.15.12:0/1275947147 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd6c4003bc0 con 0x55e156c24cd0 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.170+0000 7fd6b2ffd640 1 -- 172.21.15.12:0/1275947147 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c40d5040 con 0x55e156c24cd0 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.170+0000 7fd6d244e740 1 -- 172.21.15.12:0/1275947147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 msgr2=0x55e156c2cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.170+0000 7fd6d244e740 1 --2- 172.21.15.12:0/1275947147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c2cc20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4018000 tx=0x7fd6c406f420 comp rx=0 tx=0).stop 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.170+0000 7fd6d244e740 1 -- 172.21.15.12:0/1275947147 shutdown_connections 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.170+0000 7fd6d244e740 1 --2- 172.21.15.12:0/1275947147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c2a830 0x55e156c16d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.170+0000 7fd6d244e740 1 --2- 172.21.15.12:0/1275947147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c256c0 0x55e156c365a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.170+0000 7fd6d244e740 1 --2- 172.21.15.12:0/1275947147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c2cc20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.170+0000 7fd6d244e740 1 -- 172.21.15.12:0/1275947147 >> 172.21.15.12:0/1275947147 conn(0x55e156c1bf30 msgr2=0x55e156c1c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.170+0000 7fd6d244e740 1 -- 172.21.15.12:0/1275947147 shutdown_connections 2024-05-09T09:24:58.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.170+0000 7fd6d244e740 1 -- 172.21.15.12:0/1275947147 wait complete. 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6d244e740 1 -- start start 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6d244e740 1 Processor -- start 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6d244e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c31f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6d244e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c256c0 0x55e156c32480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6d244e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c2a830 0x55e156c35450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6d244e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e156c3d020 con 0x55e156c24cd0 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6d244e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e156c3c950 con 0x55e156c256c0 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6d244e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e156c3d6f0 con 0x55e156c2a830 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6d244e740 10 client.0 _ll_register_callbacks cb 0x7fd6c00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6cd4de640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c31f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6cd4de640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c31f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50456/0 (socket says 172.21.15.12:50456) 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6cd4de640 1 -- 172.21.15.12:0/3391169792 learned_addr learned my addr 172.21.15.12:0/3391169792 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6cccdd640 1 --2- 172.21.15.12:0/3391169792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c256c0 0x55e156c32480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6cd4de640 1 -- 172.21.15.12:0/3391169792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c2a830 msgr2=0x55e156c35450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6cd4de640 1 --2- 172.21.15.12:0/3391169792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c2a830 0x55e156c35450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6cd4de640 1 -- 172.21.15.12:0/3391169792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c256c0 msgr2=0x55e156c32480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:58.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6cd4de640 1 --2- 172.21.15.12:0/3391169792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c256c0 0x55e156c32480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.171+0000 7fd6cd4de640 1 -- 172.21.15.12:0/3391169792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e156c30230 con 0x55e156c24cd0 2024-05-09T09:24:58.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6cd4de640 1 --2- 172.21.15.12:0/3391169792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c31f40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4098690 tx=0x7fd6c4097fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99aef48c73b8ff4c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:58.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:24:58.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6b0ff9640 1 -- 172.21.15.12:0/3391169792 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c406a070 con 0x55e156c24cd0 2024-05-09T09:24:58.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6b0ff9640 1 -- 172.21.15.12:0/3391169792 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd6c4098330 con 0x55e156c24cd0 2024-05-09T09:24:58.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6d244e740 1 -- 172.21.15.12:0/3391169792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e156b8bf50 con 0x55e156c24cd0 2024-05-09T09:24:58.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6b0ff9640 1 -- 172.21.15.12:0/3391169792 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6c40d5040 con 0x55e156c24cd0 2024-05-09T09:24:58.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6d244e740 10 client.15297 fetch_fsmap learned FSMap version 28 2024-05-09T09:24:58.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6d244e740 1 -- 172.21.15.12:0/3391169792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e156ca1ce0 con 0x55e156c24cd0 2024-05-09T09:24:58.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6b0ff9640 1 -- 172.21.15.12:0/3391169792 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55e156b8bf50 con 0x55e156c24cd0 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6b0ff9640 1 -- 172.21.15.12:0/3391169792 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd6c4097a10 con 0x55e156c24cd0 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6d244e740 10 client.15297 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6d244e740 10 client.15297 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6d244e740 20 client.15297 populate_metadata read hostname 'smithi012' 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6d244e740 1 --2- 172.21.15.12:0/3391169792 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e156ca6270 0x55e156ca4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6d244e740 4 client.15297 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.172+0000 7fd6d244e740 1 -- 172.21.15.12:0/3391169792 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "status"}) -- 0x55e156b09b80 con 0x55e156ca6270 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6cccdd640 1 --2- 172.21.15.12:0/3391169792 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e156ca6270 0x55e156ca4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6cccdd640 1 --2- 172.21.15.12:0/3391169792 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e156ca6270 0x55e156ca4f80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6885944c9181b712 server_cookie=7a0af082e03e544c in_seq=0 out_seq=0 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6b0ff9640 10 client.15297 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6b0ff9640 1 -- 172.21.15.12:0/3391169792 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e156b09b80 con 0x55e156ca6270 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6b0ff9640 10 client.15297 handle_command_reply: tid=0 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6d244e740 1 client.15297 shutdown 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6d244e740 1 -- 172.21.15.12:0/3391169792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 msgr2=0x55e156c31f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6d244e740 1 --2- 172.21.15.12:0/3391169792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c31f40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4098690 tx=0x7fd6c4097fd0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6d244e740 1 -- 172.21.15.12:0/3391169792 shutdown_connections 2024-05-09T09:24:58.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6d244e740 1 --2- 172.21.15.12:0/3391169792 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e156ca6270 0x55e156ca4f80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6d244e740 1 --2- 172.21.15.12:0/3391169792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e156c2a830 0x55e156c35450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6d244e740 1 --2- 172.21.15.12:0/3391169792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e156c256c0 0x55e156c32480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6d244e740 1 --2- 172.21.15.12:0/3391169792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c24cd0 0x55e156c31f40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6b0ff9640 0 client.15297 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:24:58.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.173+0000 7fd6d244e740 1 -- 172.21.15.12:0/3391169792 >> 172.21.15.12:0/3391169792 conn(0x55e156c1bf30 msgr2=0x55e156c26ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:58.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6d244e740 1 -- 172.21.15.12:0/3391169792 shutdown_connections 2024-05-09T09:24:58.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6d244e740 1 -- 172.21.15.12:0/3391169792 wait complete. 2024-05-09T09:24:58.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6d244e740 20 client.15297 trim_cache size 0 max 16384 2024-05-09T09:24:58.180 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:24:58.181 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:24:58.181 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-05-09T09:24:58.181 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24523, 2024-05-09T09:24:58.181 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-09T09:24:58.181 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-09T09:24:58.181 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-09T09:24:58.181 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 107.864568283, 2024-05-09T09:24:58.181 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-05-09T09:24:58.181 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 91, 2024-05-09T09:24:58.181 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T09:24:58.181 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 122.861668956 2024-05-09T09:24:58.181 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:24:58.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6cf742640 1 -- 172.21.15.12:0/3899216823 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd6b40a2b90 msgr2=0x7fd6b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6cf742640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd6b40a2b90 0x7fd6b40a5050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0022000 tx=0x7fd6c0023000 comp rx=0 tx=0).stop 2024-05-09T09:24:58.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6cf742640 1 -- 172.21.15.12:0/3899216823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c8100770 msgr2=0x7fd6c8197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:58.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6cf742640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c8100770 0x7fd6c8197bf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400e9c0 tx=0x7fd6c400ee80 comp rx=0 tx=0).stop 2024-05-09T09:24:58.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6cf742640 1 -- 172.21.15.12:0/3899216823 shutdown_connections 2024-05-09T09:24:58.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6cf742640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd6b40a2b90 0x7fd6b40a5050 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6cf742640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6c8102320 0x7fd6c819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6cf742640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6c8101960 0x7fd6c8198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6cf742640 1 --2- 172.21.15.12:0/3899216823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c8100770 0x7fd6c8197bf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:58.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6cf742640 1 -- 172.21.15.12:0/3899216823 >> 172.21.15.12:0/3899216823 conn(0x7fd6c80fbce0 msgr2=0x7fd6c80fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:58.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6cf742640 1 -- 172.21.15.12:0/3899216823 shutdown_connections 2024-05-09T09:24:58.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:24:58.174+0000 7fd6cf742640 1 -- 172.21.15.12:0/3899216823 wait complete. 2024-05-09T09:24:58.185 INFO:tasks.cephadm:Setup complete, yielding 2024-05-09T09:24:58.185 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T09:24:58.195 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-09T09:24:58.196 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-09T09:24:58.447 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:24:58.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:58 smithi012 ceph-mon[30140]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:24:58.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:24:58 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4099698508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:58.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:58 smithi170 ceph-mon[33632]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:24:58.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:24:58 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/4099698508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:58.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:58 smithi016 ceph-mon[33335]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:24:58.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:24:58 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/4099698508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:00.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:00 smithi012 ceph-mon[30140]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T09:25:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.340+0000 7f08b4edc700 1 -- 172.21.15.12:0/3757939753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b0102380 msgr2=0x7f08b01027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.340+0000 7f08b4edc700 1 --2- 172.21.15.12:0/3757939753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b0102380 0x7f08b01027a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f08a00099c0 tx=0x7f08a0009cd0 comp rx=0 tx=0).stop 2024-05-09T09:25:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.341+0000 7f08b4edc700 1 -- 172.21.15.12:0/3757939753 shutdown_connections 2024-05-09T09:25:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.341+0000 7f08b4edc700 1 --2- 172.21.15.12:0/3757939753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08b0103f20 0x7f08b010a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.341+0000 7f08b4edc700 1 --2- 172.21.15.12:0/3757939753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b0102380 0x7f08b01027a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.341+0000 7f08b4edc700 1 --2- 172.21.15.12:0/3757939753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08b0103580 0x7f08b01039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.341+0000 7f08b4edc700 1 -- 172.21.15.12:0/3757939753 >> 172.21.15.12:0/3757939753 conn(0x7f08b00fd920 msgr2=0x7f08b00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.341+0000 7f08b4edc700 1 -- 172.21.15.12:0/3757939753 shutdown_connections 2024-05-09T09:25:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.341+0000 7f08b4edc700 1 -- 172.21.15.12:0/3757939753 wait complete. 2024-05-09T09:25:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.341+0000 7f08b4edc700 1 Processor -- start 2024-05-09T09:25:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08b4edc700 1 -- start start 2024-05-09T09:25:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08b4edc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08b0102380 0x7f08b0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08b4edc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08b0103580 0x7f08b0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08b4edc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b0103f20 0x7f08b019fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08b4edc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f08b010dfc0 con 0x7f08b0103f20 2024-05-09T09:25:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08b4edc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08b010de40 con 0x7f08b0103580 2024-05-09T09:25:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08b4edc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f08b010e140 con 0x7f08b0102380 2024-05-09T09:25:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08aed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b0103f20 0x7f08b019fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08aed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b0103f20 0x7f08b019fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50484/0 (socket says 172.21.15.12:50484) 2024-05-09T09:25:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08aed9d700 1 -- 172.21.15.12:0/3887473698 learned_addr learned my addr 172.21.15.12:0/3887473698 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08ae59c700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08b0102380 0x7f08b0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08a5d9b700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08b0103580 0x7f08b0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08aed9d700 1 -- 172.21.15.12:0/3887473698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08b0102380 msgr2=0x7f08b0198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08aed9d700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08b0102380 0x7f08b0198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08aed9d700 1 -- 172.21.15.12:0/3887473698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08b0103580 msgr2=0x7f08b0198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08aed9d700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08b0103580 0x7f08b0198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08aed9d700 1 -- 172.21.15.12:0/3887473698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b01a01c0 con 0x7f08b0103f20 2024-05-09T09:25:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.342+0000 7f08ae59c700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08b0102380 0x7f08b0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.343+0000 7f08aed9d700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b0103f20 0x7f08b019fb20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f08a8007da0 tx=0x7f08a800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.343+0000 7f08a7fff700 1 -- 172.21.15.12:0/3887473698 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8016020 con 0x7f08b0103f20 2024-05-09T09:25:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.343+0000 7f08a7fff700 1 -- 172.21.15.12:0/3887473698 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f08a800ad00 con 0x7f08b0103f20 2024-05-09T09:25:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.343+0000 7f08b4edc700 1 -- 172.21.15.12:0/3887473698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b01a04b0 con 0x7f08b0103f20 2024-05-09T09:25:00.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.343+0000 7f08a7fff700 1 -- 172.21.15.12:0/3887473698 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a80115e0 con 0x7f08b0103f20 2024-05-09T09:25:00.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.343+0000 7f08b4edc700 1 -- 172.21.15.12:0/3887473698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b01a09f0 con 0x7f08b0103f20 2024-05-09T09:25:00.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.344+0000 7f08b4edc700 1 -- 172.21.15.12:0/3887473698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b0069010 con 0x7f08b0103f20 2024-05-09T09:25:00.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.345+0000 7f08a7fff700 1 -- 172.21.15.12:0/3887473698 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f08a8019020 con 0x7f08b0103f20 2024-05-09T09:25:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.346+0000 7f08a7fff700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f089c0a2ca0 0x7f089c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.346+0000 7f08ae59c700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f089c0a2ca0 0x7f089c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.346+0000 7f08a7fff700 1 -- 172.21.15.12:0/3887473698 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f08a80cf740 con 0x7f08b0103f20 2024-05-09T09:25:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.346+0000 7f08ae59c700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f089c0a2ca0 0x7f089c0a5160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f08a000b580 tx=0x7f08a0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:00.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.349+0000 7f08a7fff700 1 -- 172.21.15.12:0/3887473698 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f08a8020020 con 0x7f08b0103f20 2024-05-09T09:25:00.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:00 smithi170 ceph-mon[33632]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T09:25:00.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:00 smithi016 ceph-mon[33335]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T09:25:00.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.576+0000 7f08b4edc700 1 -- 172.21.15.12:0/3887473698 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f08b0062760 con 0x7f089c0a2ca0 2024-05-09T09:25:00.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.577+0000 7f08a7fff700 1 -- 172.21.15.12:0/3887473698 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f08b0062760 con 0x7f089c0a2ca0 2024-05-09T09:25:00.577 INFO:teuthology.orchestra.run.smithi012.stdout:Backend: cephadm 2024-05-09T09:25:00.577 INFO:teuthology.orchestra.run.smithi012.stdout:Available: Yes 2024-05-09T09:25:00.577 INFO:teuthology.orchestra.run.smithi012.stdout:Paused: No 2024-05-09T09:25:00.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.580+0000 7f08b4edc700 1 -- 172.21.15.12:0/3887473698 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f089c0a2ca0 msgr2=0x7f089c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:00.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.580+0000 7f08b4edc700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f089c0a2ca0 0x7f089c0a5160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f08a000b580 tx=0x7f08a0005fb0 comp rx=0 tx=0).stop 2024-05-09T09:25:00.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.580+0000 7f08b4edc700 1 -- 172.21.15.12:0/3887473698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b0103f20 msgr2=0x7f08b019fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:00.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.580+0000 7f08b4edc700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b0103f20 0x7f08b019fb20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f08a8007da0 tx=0x7f08a800a340 comp rx=0 tx=0).stop 2024-05-09T09:25:00.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.580+0000 7f08b4edc700 1 -- 172.21.15.12:0/3887473698 shutdown_connections 2024-05-09T09:25:00.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.580+0000 7f08b4edc700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f089c0a2ca0 0x7f089c0a5160 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:00.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.580+0000 7f08b4edc700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b0103f20 0x7f08b019fb20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:00.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.580+0000 7f08b4edc700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08b0102380 0x7f08b0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:00.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.580+0000 7f08b4edc700 1 --2- 172.21.15.12:0/3887473698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08b0103580 0x7f08b0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:00.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.580+0000 7f08b4edc700 1 -- 172.21.15.12:0/3887473698 >> 172.21.15.12:0/3887473698 conn(0x7f08b00fd920 msgr2=0x7f08b0106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:00.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.580+0000 7f08b4edc700 1 -- 172.21.15.12:0/3887473698 shutdown_connections 2024-05-09T09:25:00.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:00.580+0000 7f08b4edc700 1 -- 172.21.15.12:0/3887473698 wait complete. 2024-05-09T09:25:01.421 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-09T09:25:01.681 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:25:02.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:02 smithi012 ceph-mon[30140]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:02.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:02 smithi012 ceph-mon[30140]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-09T09:25:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:02 smithi170 ceph-mon[33632]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:02 smithi170 ceph-mon[33632]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-09T09:25:02.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:02 smithi016 ceph-mon[33335]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:02.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:02 smithi016 ceph-mon[33335]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-09T09:25:03.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.491+0000 7fbfe8935700 1 -- 172.21.15.12:0/1704673940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe4103f40 msgr2=0x7fbfe410a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:03.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.491+0000 7fbfe8935700 1 --2- 172.21.15.12:0/1704673940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe4103f40 0x7fbfe410a7f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00b0d0 tx=0x7fbfdc00b3e0 comp rx=0 tx=0).stop 2024-05-09T09:25:03.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.491+0000 7fbfe8935700 1 -- 172.21.15.12:0/1704673940 shutdown_connections 2024-05-09T09:25:03.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.491+0000 7fbfe8935700 1 --2- 172.21.15.12:0/1704673940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe4103f40 0x7fbfe410a7f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:03.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.491+0000 7fbfe8935700 1 --2- 172.21.15.12:0/1704673940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe41023a0 0x7fbfe41027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:03.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.491+0000 7fbfe8935700 1 --2- 172.21.15.12:0/1704673940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfe41035a0 0x7fbfe4103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:03.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.491+0000 7fbfe8935700 1 -- 172.21.15.12:0/1704673940 >> 172.21.15.12:0/1704673940 conn(0x7fbfe40fd940 msgr2=0x7fbfe40ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:03.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.491+0000 7fbfe8935700 1 -- 172.21.15.12:0/1704673940 shutdown_connections 2024-05-09T09:25:03.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.491+0000 7fbfe8935700 1 -- 172.21.15.12:0/1704673940 wait complete. 2024-05-09T09:25:03.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe8935700 1 Processor -- start 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe8935700 1 -- start start 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe8935700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfe41023a0 0x7fbfe41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe8935700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe41035a0 0x7fbfe4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe8935700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe4103f40 0x7fbfe419fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe8935700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe410dfd0 con 0x7fbfe4103f40 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe8935700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe410de50 con 0x7fbfe41023a0 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe8935700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe410e150 con 0x7fbfe41035a0 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe4103f40 0x7fbfe419fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe4103f40 0x7fbfe419fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48268/0 (socket says 172.21.15.12:48268) 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe2d9d700 1 -- 172.21.15.12:0/2394709534 learned_addr learned my addr 172.21.15.12:0/2394709534 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe1d9b700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe41035a0 0x7fbfe4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe259c700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfe41023a0 0x7fbfe41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe2d9d700 1 -- 172.21.15.12:0/2394709534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe41035a0 msgr2=0x7fbfe4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:03.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe2d9d700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe41035a0 0x7fbfe4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe2d9d700 1 -- 172.21.15.12:0/2394709534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfe41023a0 msgr2=0x7fbfe41984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe2d9d700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfe41023a0 0x7fbfe41984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.492+0000 7fbfe2d9d700 1 -- 172.21.15.12:0/2394709534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe41a0130 con 0x7fbfe4103f40 2024-05-09T09:25:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.493+0000 7fbfe2d9d700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe4103f40 0x7fbfe419fa90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc007a20 tx=0x7fbfdc007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.493+0000 7fbfdb7fe700 1 -- 172.21.15.12:0/2394709534 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc021070 con 0x7fbfe4103f40 2024-05-09T09:25:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.493+0000 7fbfe8935700 1 -- 172.21.15.12:0/2394709534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe41a03c0 con 0x7fbfe4103f40 2024-05-09T09:25:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.493+0000 7fbfdb7fe700 1 -- 172.21.15.12:0/2394709534 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbfdc01db20 con 0x7fbfe4103f40 2024-05-09T09:25:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.493+0000 7fbfdb7fe700 1 -- 172.21.15.12:0/2394709534 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc01bb00 con 0x7fbfe4103f40 2024-05-09T09:25:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.493+0000 7fbfe8935700 1 -- 172.21.15.12:0/2394709534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe41a0870 con 0x7fbfe4103f40 2024-05-09T09:25:03.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.494+0000 7fbfe8935700 1 -- 172.21.15.12:0/2394709534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe4069010 con 0x7fbfe4103f40 2024-05-09T09:25:03.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.495+0000 7fbfdb7fe700 1 -- 172.21.15.12:0/2394709534 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbfdc024070 con 0x7fbfe4103f40 2024-05-09T09:25:03.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.496+0000 7fbfdb7fe700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbfd00a2c20 0x7fbfd00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:03.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.496+0000 7fbfe259c700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbfd00a2c20 0x7fbfd00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:03.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.496+0000 7fbfdb7fe700 1 -- 172.21.15.12:0/2394709534 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbfdc0d0ec0 con 0x7fbfe4103f40 2024-05-09T09:25:03.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.497+0000 7fbfe259c700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbfd00a2c20 0x7fbfd00a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4103400 tx=0x7fbfd400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:03.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.499+0000 7fbfdb7fe700 1 -- 172.21.15.12:0/2394709534 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbfdc09db50 con 0x7fbfe4103f40 2024-05-09T09:25:03.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.714+0000 7fbfe8935700 1 -- 172.21.15.12:0/2394709534 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbfe4062760 con 0x7fbfd00a2c20 2024-05-09T09:25:03.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.728+0000 7fbfdb7fe700 1 -- 172.21.15.12:0/2394709534 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fbfe4062760 con 0x7fbfd00a2c20 2024-05-09T09:25:03.727 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T09:25:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:mds.a smithi012 running (2m) 20s ago 2m 19.4M - 19.0.0-2515-g96805dcf d7acac7d5657 512799093bbe 2024-05-09T09:25:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:mds.b smithi016 running (2m) 52s ago 119s 16.7M - 19.0.0-2515-g96805dcf d7acac7d5657 19bdab9e4e0c 2024-05-09T09:25:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:mds.c smithi170 running (2m) 52s ago 2m 21.2M - 19.0.0-2515-g96805dcf d7acac7d5657 552f76904383 2024-05-09T09:25:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:mds.d smithi012 running (2m) 20s ago 2m 13.9M - 19.0.0-2515-g96805dcf d7acac7d5657 b27c3a53d845 2024-05-09T09:25:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:mds.e smithi016 running (2m) 52s ago 2m 16.2M - 19.0.0-2515-g96805dcf d7acac7d5657 c2904ab19e3e 2024-05-09T09:25:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:mds.f smithi170 running (2m) 52s ago 2m 17.2M - 19.0.0-2515-g96805dcf d7acac7d5657 d55293678f89 2024-05-09T09:25:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:mds.g smithi012 running (2m) 20s ago 2m 17.9M - 19.0.0-2515-g96805dcf d7acac7d5657 087bd705b6e3 2024-05-09T09:25:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:mds.h smithi016 running (2m) 52s ago 2m 18.7M - 19.0.0-2515-g96805dcf d7acac7d5657 881bf909381e 2024-05-09T09:25:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:mds.i smithi170 running (2m) 52s ago 2m 26.2M - 19.0.0-2515-g96805dcf d7acac7d5657 0aa652887d7e 2024-05-09T09:25:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:mds.j smithi012 running (2m) 20s ago 2m 15.4M - 19.0.0-2515-g96805dcf d7acac7d5657 b9e335ae8c04 2024-05-09T09:25:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:mds.k smithi016 running (2m) 52s ago 2m 16.2M - 19.0.0-2515-g96805dcf d7acac7d5657 e88be52650b2 2024-05-09T09:25:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:mds.l smithi170 running (2m) 52s ago 2m 15.9M - 19.0.0-2515-g96805dcf d7acac7d5657 415b7f396a93 2024-05-09T09:25:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.x smithi012 *:9283,8765 running (9m) 20s ago 9m 516M - 19.0.0-2515-g96805dcf d7acac7d5657 b9494772fc03 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.y smithi016 *:8443,8765 running (7m) 52s ago 7m 453M - 19.0.0-2515-g96805dcf d7acac7d5657 b92a20322afa 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.z smithi170 *:8443,8765 running (7m) 52s ago 7m 453M - 19.0.0-2515-g96805dcf d7acac7d5657 8a125eab5f62 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:mon.a smithi012 running (9m) 20s ago 10m 51.2M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 1e0b9433ce3e 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:mon.b smithi016 running (8m) 52s ago 8m 43.1M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 f6666148f064 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:mon.c smithi170 running (8m) 52s ago 8m 41.0M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 84836b4b5320 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (7m) 20s ago 7m 46.6M 4273M 19.0.0-2515-g96805dcf d7acac7d5657 c82ef4d02395 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi016 running (7m) 52s ago 7m 53.5M 4273M 19.0.0-2515-g96805dcf d7acac7d5657 839098fb8fed 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi170 running (6m) 52s ago 6m 55.7M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 2f7d263810f7 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi012 running (6m) 20s ago 6m 44.0M 4273M 19.0.0-2515-g96805dcf d7acac7d5657 13d309c55ae5 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi016 running (5m) 52s ago 5m 53.7M 4273M 19.0.0-2515-g96805dcf d7acac7d5657 f3341ccd5d2a 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi170 running (5m) 52s ago 5m 45.8M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 ec65f60c30c7 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 smithi012 running (5m) 20s ago 5m 49.7M 4273M 19.0.0-2515-g96805dcf d7acac7d5657 275ea822210c 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 smithi016 running (4m) 52s ago 4m 44.2M 4273M 19.0.0-2515-g96805dcf d7acac7d5657 3020c7f30ffb 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 smithi170 running (4m) 52s ago 4m 51.9M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 e6fade36a900 2024-05-09T09:25:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 smithi012 running (3m) 20s ago 3m 41.7M 4273M 19.0.0-2515-g96805dcf d7acac7d5657 6bde3e41e342 2024-05-09T09:25:03.730 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 smithi016 running (3m) 52s ago 3m 47.1M 4273M 19.0.0-2515-g96805dcf d7acac7d5657 2ada98632da7 2024-05-09T09:25:03.730 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 smithi170 running (2m) 52s ago 2m 51.0M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 de818b5f651e 2024-05-09T09:25:03.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.730+0000 7fbfe8935700 1 -- 172.21.15.12:0/2394709534 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbfd00a2c20 msgr2=0x7fbfd00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:03.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.730+0000 7fbfe8935700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbfd00a2c20 0x7fbfd00a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4103400 tx=0x7fbfd400b410 comp rx=0 tx=0).stop 2024-05-09T09:25:03.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.730+0000 7fbfe8935700 1 -- 172.21.15.12:0/2394709534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe4103f40 msgr2=0x7fbfe419fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:03.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.730+0000 7fbfe8935700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe4103f40 0x7fbfe419fa90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc007a20 tx=0x7fbfdc007a50 comp rx=0 tx=0).stop 2024-05-09T09:25:03.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.730+0000 7fbfe8935700 1 -- 172.21.15.12:0/2394709534 shutdown_connections 2024-05-09T09:25:03.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.730+0000 7fbfe8935700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbfd00a2c20 0x7fbfd00a50e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:03.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.730+0000 7fbfe8935700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe4103f40 0x7fbfe419fa90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:03.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.730+0000 7fbfe8935700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfe41023a0 0x7fbfe41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:03.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.730+0000 7fbfe8935700 1 --2- 172.21.15.12:0/2394709534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe41035a0 0x7fbfe4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:03.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.730+0000 7fbfe8935700 1 -- 172.21.15.12:0/2394709534 >> 172.21.15.12:0/2394709534 conn(0x7fbfe40fd940 msgr2=0x7fbfe4106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:03.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.730+0000 7fbfe8935700 1 -- 172.21.15.12:0/2394709534 shutdown_connections 2024-05-09T09:25:03.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:03.730+0000 7fbfe8935700 1 -- 172.21.15.12:0/2394709534 wait complete. 2024-05-09T09:25:04.470 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:04 smithi012 ceph-mon[30140]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:25:04.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:04 smithi170 ceph-mon[33632]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:25:04.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-09T09:25:04.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:04 smithi016 ceph-mon[33335]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:25:04.765 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:25:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:05 smithi012 ceph-mon[30140]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-09T09:25:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-09T09:25:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-09T09:25:05.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:05 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]: dispatch 2024-05-09T09:25:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:05 smithi170 ceph-mon[33632]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-09T09:25:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-09T09:25:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-09T09:25:05.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:05 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]: dispatch 2024-05-09T09:25:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:05 smithi016 ceph-mon[33335]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-09T09:25:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-09T09:25:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-09T09:25:05.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:05 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]: dispatch 2024-05-09T09:25:06.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:06 smithi012 ceph-mon[30140]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:25:06.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-09T09:25:06.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-09T09:25:06.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-09T09:25:06.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:06 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]': finished 2024-05-09T09:25:06.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:06 smithi012 ceph-mon[30140]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T09:25:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:06 smithi170 ceph-mon[33632]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:25:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-09T09:25:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-09T09:25:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-09T09:25:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:06 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]': finished 2024-05-09T09:25:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:06 smithi170 ceph-mon[33632]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T09:25:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:06 smithi016 ceph-mon[33335]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:25:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-09T09:25:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-09T09:25:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-09T09:25:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:06 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [1, 9]}]': finished 2024-05-09T09:25:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:06 smithi016 ceph-mon[33335]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T09:25:06.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.526+0000 7fe406f83700 1 -- 172.21.15.12:0/4023754801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe400102e60 msgr2=0x7fe400105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.526+0000 7fe406f83700 1 --2- 172.21.15.12:0/4023754801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe400102e60 0x7fe400105250 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000b0a0 tx=0x7fe3f000b3b0 comp rx=0 tx=0).stop 2024-05-09T09:25:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.526+0000 7fe406f83700 1 -- 172.21.15.12:0/4023754801 shutdown_connections 2024-05-09T09:25:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.526+0000 7fe406f83700 1 --2- 172.21.15.12:0/4023754801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4001080c0 0x7fe40010a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.526+0000 7fe406f83700 1 --2- 172.21.15.12:0/4023754801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe400102e60 0x7fe400105250 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.526+0000 7fe406f83700 1 --2- 172.21.15.12:0/4023754801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe400105790 0x7fe400107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.526+0000 7fe406f83700 1 -- 172.21.15.12:0/4023754801 >> 172.21.15.12:0/4023754801 conn(0x7fe4000fc810 msgr2=0x7fe4000fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.526+0000 7fe406f83700 1 -- 172.21.15.12:0/4023754801 shutdown_connections 2024-05-09T09:25:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.526+0000 7fe406f83700 1 -- 172.21.15.12:0/4023754801 wait complete. 2024-05-09T09:25:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.526+0000 7fe406f83700 1 Processor -- start 2024-05-09T09:25:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe406f83700 1 -- start start 2024-05-09T09:25:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe406f83700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe400102e60 0x7fe4001027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe406f83700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe400105790 0x7fe400100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe406f83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4001080c0 0x7fe400101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe406f83700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe40010df80 con 0x7fe4001080c0 2024-05-09T09:25:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe406f83700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe40010de00 con 0x7fe400102e60 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe406f83700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe40010e100 con 0x7fe400105790 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe405520700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4001080c0 0x7fe400101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe405520700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4001080c0 0x7fe400101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48296/0 (socket says 172.21.15.12:48296) 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe405520700 1 -- 172.21.15.12:0/665257954 learned_addr learned my addr 172.21.15.12:0/665257954 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe3fffff700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe400105790 0x7fe400100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe404d1f700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe400102e60 0x7fe4001027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe405520700 1 -- 172.21.15.12:0/665257954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe400105790 msgr2=0x7fe400100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe405520700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe400105790 0x7fe400100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe405520700 1 -- 172.21.15.12:0/665257954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe400102e60 msgr2=0x7fe4001027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe405520700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe400102e60 0x7fe4001027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe405520700 1 -- 172.21.15.12:0/665257954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe400101c00 con 0x7fe4001080c0 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe3fffff700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe400105790 0x7fe400100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe404d1f700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe400102e60 0x7fe4001027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.527+0000 7fe405520700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4001080c0 0x7fe400101350 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c970 tx=0x7fe3f800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:06.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.528+0000 7fe3fdffb700 1 -- 172.21.15.12:0/665257954 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8007b00 con 0x7fe4001080c0 2024-05-09T09:25:06.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.528+0000 7fe3fdffb700 1 -- 172.21.15.12:0/665257954 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe3f800ce50 con 0x7fe4001080c0 2024-05-09T09:25:06.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.528+0000 7fe406f83700 1 -- 172.21.15.12:0/665257954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4001a2760 con 0x7fe4001080c0 2024-05-09T09:25:06.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.528+0000 7fe3fdffb700 1 -- 172.21.15.12:0/665257954 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8005640 con 0x7fe4001080c0 2024-05-09T09:25:06.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.528+0000 7fe406f83700 1 -- 172.21.15.12:0/665257954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4001a2c20 con 0x7fe4001080c0 2024-05-09T09:25:06.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.529+0000 7fe406f83700 1 -- 172.21.15.12:0/665257954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe400069010 con 0x7fe4001080c0 2024-05-09T09:25:06.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.530+0000 7fe3fdffb700 1 -- 172.21.15.12:0/665257954 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe3f8004750 con 0x7fe4001080c0 2024-05-09T09:25:06.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.531+0000 7fe3fdffb700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe3ec0a2c20 0x7fe3ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:06.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.531+0000 7fe404d1f700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe3ec0a2c20 0x7fe3ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:06.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.531+0000 7fe3fdffb700 1 -- 172.21.15.12:0/665257954 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fe3f80cefe0 con 0x7fe4001080c0 2024-05-09T09:25:06.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.531+0000 7fe404d1f700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe3ec0a2c20 0x7fe3ec0a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000b070 tx=0x7fe3f000b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:06.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.535+0000 7fe3fdffb700 1 -- 172.21.15.12:0/665257954 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe3f8014030 con 0x7fe4001080c0 2024-05-09T09:25:06.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.749+0000 7fe406f83700 1 -- 172.21.15.12:0/665257954 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fe400062760 con 0x7fe3ec0a2c20 2024-05-09T09:25:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.755+0000 7fe3fdffb700 1 -- 172.21.15.12:0/665257954 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fe400062760 con 0x7fe3ec0a2c20 2024-05-09T09:25:06.754 INFO:teuthology.orchestra.run.smithi012.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-09T09:25:06.755 INFO:teuthology.orchestra.run.smithi012.stdout:mds.all 12/12 55s ago 2m smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-05-09T09:25:06.755 INFO:teuthology.orchestra.run.smithi012.stdout:mgr 3/3 55s ago 7m smithi012=x;smithi016=y;smithi170=z;count:3 2024-05-09T09:25:06.755 INFO:teuthology.orchestra.run.smithi012.stdout:mon 3/3 55s ago 8m smithi012:172.21.15.12=a;smithi016:172.21.15.16=b;smithi170:172.21.15.170=c;count:3 2024-05-09T09:25:06.755 INFO:teuthology.orchestra.run.smithi012.stdout:osd 12 55s ago - 2024-05-09T09:25:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.757+0000 7fe406f83700 1 -- 172.21.15.12:0/665257954 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe3ec0a2c20 msgr2=0x7fe3ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.757+0000 7fe406f83700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe3ec0a2c20 0x7fe3ec0a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000b070 tx=0x7fe3f000b7b0 comp rx=0 tx=0).stop 2024-05-09T09:25:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.757+0000 7fe406f83700 1 -- 172.21.15.12:0/665257954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4001080c0 msgr2=0x7fe400101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.757+0000 7fe406f83700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4001080c0 0x7fe400101350 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c970 tx=0x7fe3f800cd30 comp rx=0 tx=0).stop 2024-05-09T09:25:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.757+0000 7fe406f83700 1 -- 172.21.15.12:0/665257954 shutdown_connections 2024-05-09T09:25:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.757+0000 7fe406f83700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe3ec0a2c20 0x7fe3ec0a50e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.757+0000 7fe406f83700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4001080c0 0x7fe400101350 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.757+0000 7fe406f83700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe400102e60 0x7fe4001027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.757+0000 7fe406f83700 1 --2- 172.21.15.12:0/665257954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe400105790 0x7fe400100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.757+0000 7fe406f83700 1 -- 172.21.15.12:0/665257954 >> 172.21.15.12:0/665257954 conn(0x7fe4000fc810 msgr2=0x7fe4001037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.757+0000 7fe406f83700 1 -- 172.21.15.12:0/665257954 shutdown_connections 2024-05-09T09:25:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:06.757+0000 7fe406f83700 1 -- 172.21.15.12:0/665257954 wait complete. 2024-05-09T09:25:07.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:07 smithi012 ceph-mon[30140]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T09:25:07.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:07 smithi170 ceph-mon[33632]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T09:25:07.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:07 smithi016 ceph-mon[33335]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T09:25:07.519 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-09T09:25:07.772 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:25:08.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:08 smithi012 ceph-mon[30140]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-09T09:25:08.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:08 smithi012 ceph-mon[30140]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:08.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:08 smithi170 ceph-mon[33632]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-09T09:25:08.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:08 smithi170 ceph-mon[33632]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:08.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:08 smithi016 ceph-mon[33335]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-09T09:25:08.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:08 smithi016 ceph-mon[33335]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:09.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.490+0000 7f74bb115700 1 -- 172.21.15.12:0/1146386944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4100ac0 msgr2=0x7f74b4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.490+0000 7f74bb115700 1 --2- 172.21.15.12:0/1146386944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4100ac0 0x7f74b4100ee0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f74a800aad0 tx=0x7f74a800ade0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.490+0000 7f74bb115700 1 -- 172.21.15.12:0/1146386944 shutdown_connections 2024-05-09T09:25:09.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.490+0000 7f74bb115700 1 --2- 172.21.15.12:0/1146386944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74b4102660 0x7f74b4108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.490+0000 7f74bb115700 1 --2- 172.21.15.12:0/1146386944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4100ac0 0x7f74b4100ee0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.490+0000 7f74bb115700 1 --2- 172.21.15.12:0/1146386944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b4101cc0 0x7f74b4102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.490+0000 7f74bb115700 1 -- 172.21.15.12:0/1146386944 >> 172.21.15.12:0/1146386944 conn(0x7f74b40fc060 msgr2=0x7f74b40fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:09.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.491+0000 7f74bb115700 1 -- 172.21.15.12:0/1146386944 shutdown_connections 2024-05-09T09:25:09.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.491+0000 7f74bb115700 1 -- 172.21.15.12:0/1146386944 wait complete. 2024-05-09T09:25:09.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.491+0000 7f74bb115700 1 Processor -- start 2024-05-09T09:25:09.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.491+0000 7f74bb115700 1 -- start start 2024-05-09T09:25:09.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.491+0000 7f74bb115700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b4100ac0 0x7f74b419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.491+0000 7f74bb115700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4101cc0 0x7f74b419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.491+0000 7f74bb115700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74b4102660 0x7f74b41a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.491+0000 7f74bb115700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f74b410bf30 con 0x7f74b4101cc0 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.491+0000 7f74bb115700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74b410bdb0 con 0x7f74b4100ac0 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.491+0000 7f74bb115700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74b410c0b0 con 0x7f74b4102660 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4101cc0 0x7f74b419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b8eb1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b4100ac0 0x7f74b419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4101cc0 0x7f74b419ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48316/0 (socket says 172.21.15.12:48316) 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b3fff700 1 -- 172.21.15.12:0/3653981695 learned_addr learned my addr 172.21.15.12:0/3653981695 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b96b2700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74b4102660 0x7f74b41a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b3fff700 1 -- 172.21.15.12:0/3653981695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74b4102660 msgr2=0x7f74b41a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b3fff700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74b4102660 0x7f74b41a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b3fff700 1 -- 172.21.15.12:0/3653981695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b4100ac0 msgr2=0x7f74b419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b3fff700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b4100ac0 0x7f74b419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b3fff700 1 -- 172.21.15.12:0/3653981695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b41a25b0 con 0x7f74b4101cc0 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b8eb1700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b4100ac0 0x7f74b419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:09.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b3fff700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4101cc0 0x7f74b419ae90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f74a000b560 tx=0x7f74a000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b1ffb700 1 -- 172.21.15.12:0/3653981695 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0015020 con 0x7f74b4101cc0 2024-05-09T09:25:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b1ffb700 1 -- 172.21.15.12:0/3653981695 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f74a0011d00 con 0x7f74b4101cc0 2024-05-09T09:25:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74bb115700 1 -- 172.21.15.12:0/3653981695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b41a28a0 con 0x7f74b4101cc0 2024-05-09T09:25:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.492+0000 7f74b1ffb700 1 -- 172.21.15.12:0/3653981695 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0010dd0 con 0x7f74b4101cc0 2024-05-09T09:25:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.493+0000 7f74bb115700 1 -- 172.21.15.12:0/3653981695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b41a2e60 con 0x7f74b4101cc0 2024-05-09T09:25:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.494+0000 7f74bb115700 1 -- 172.21.15.12:0/3653981695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b4069010 con 0x7f74b4101cc0 2024-05-09T09:25:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.495+0000 7f74b1ffb700 1 -- 172.21.15.12:0/3653981695 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f74a0018020 con 0x7f74b4101cc0 2024-05-09T09:25:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.495+0000 7f74b1ffb700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f74a40a2c20 0x7f74a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.495+0000 7f74b8eb1700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f74a40a2c20 0x7f74a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.495+0000 7f74b1ffb700 1 -- 172.21.15.12:0/3653981695 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f74a00ceae0 con 0x7f74b4101cc0 2024-05-09T09:25:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.496+0000 7f74b8eb1700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f74a40a2c20 0x7f74a40a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f74a800c690 tx=0x7f74a8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.500+0000 7f74b1ffb700 1 -- 172.21.15.12:0/3653981695 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f74a001f020 con 0x7f74b4101cc0 2024-05-09T09:25:09.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.712+0000 7f74bb115700 1 -- 172.21.15.12:0/3653981695 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f74b4062760 con 0x7f74a40a2c20 2024-05-09T09:25:09.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.714+0000 7f74b1ffb700 1 -- 172.21.15.12:0/3653981695 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f74b4062760 con 0x7f74a40a2c20 2024-05-09T09:25:09.713 INFO:teuthology.orchestra.run.smithi012.stdout:HOST ADDR LABELS STATUS 2024-05-09T09:25:09.714 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 172.21.15.12 2024-05-09T09:25:09.714 INFO:teuthology.orchestra.run.smithi012.stdout:smithi016 172.21.15.16 2024-05-09T09:25:09.714 INFO:teuthology.orchestra.run.smithi012.stdout:smithi170 172.21.15.170 2024-05-09T09:25:09.714 INFO:teuthology.orchestra.run.smithi012.stdout:3 hosts in cluster 2024-05-09T09:25:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.716+0000 7f74bb115700 1 -- 172.21.15.12:0/3653981695 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f74a40a2c20 msgr2=0x7f74a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.716+0000 7f74bb115700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f74a40a2c20 0x7f74a40a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f74a800c690 tx=0x7f74a8005fb0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.716+0000 7f74bb115700 1 -- 172.21.15.12:0/3653981695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4101cc0 msgr2=0x7f74b419ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.716+0000 7f74bb115700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4101cc0 0x7f74b419ae90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f74a000b560 tx=0x7f74a000b870 comp rx=0 tx=0).stop 2024-05-09T09:25:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.716+0000 7f74bb115700 1 -- 172.21.15.12:0/3653981695 shutdown_connections 2024-05-09T09:25:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.716+0000 7f74bb115700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f74a40a2c20 0x7f74a40a50e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.716+0000 7f74bb115700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74b4102660 0x7f74b41a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.716+0000 7f74bb115700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b4100ac0 0x7f74b419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.716+0000 7f74bb115700 1 --2- 172.21.15.12:0/3653981695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4101cc0 0x7f74b419ae90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.716+0000 7f74bb115700 1 -- 172.21.15.12:0/3653981695 >> 172.21.15.12:0/3653981695 conn(0x7f74b40fc060 msgr2=0x7f74b4104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:09.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.716+0000 7f74bb115700 1 -- 172.21.15.12:0/3653981695 shutdown_connections 2024-05-09T09:25:09.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:09.716+0000 7f74bb115700 1 -- 172.21.15.12:0/3653981695 wait complete. 2024-05-09T09:25:10.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:10 smithi012 ceph-mon[30140]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-09T09:25:10.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:10 smithi170 ceph-mon[33632]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-09T09:25:10.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:10 smithi016 ceph-mon[33335]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-09T09:25:10.540 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-09T09:25:10.807 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:25:11.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:11 smithi012 ceph-mon[30140]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:11.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:11 smithi170 ceph-mon[33632]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:11.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:11 smithi016 ceph-mon[33335]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:12.457 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:12 smithi012 ceph-mon[30140]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-09T09:25:12.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:12 smithi170 ceph-mon[33632]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-09T09:25:12.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:12 smithi016 ceph-mon[33335]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-09T09:25:12.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.567+0000 7fa2ea828700 1 -- 172.21.15.12:0/1006552298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4101f20 msgr2=0x7fa2e41097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:12.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.567+0000 7fa2ea828700 1 --2- 172.21.15.12:0/1006552298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4101f20 0x7fa2e41097e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00b0d0 tx=0x7fa2dc00b3e0 comp rx=0 tx=0).stop 2024-05-09T09:25:12.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.567+0000 7fa2ea828700 1 -- 172.21.15.12:0/1006552298 shutdown_connections 2024-05-09T09:25:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.567+0000 7fa2ea828700 1 --2- 172.21.15.12:0/1006552298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4101f20 0x7fa2e41097e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.567+0000 7fa2ea828700 1 --2- 172.21.15.12:0/1006552298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4102840 0x7fa2e4102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.567+0000 7fa2ea828700 1 --2- 172.21.15.12:0/1006552298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2e4101580 0x7fa2e41019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.567+0000 7fa2ea828700 1 -- 172.21.15.12:0/1006552298 >> 172.21.15.12:0/1006552298 conn(0x7fa2e40fbfd0 msgr2=0x7fa2e40fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.567+0000 7fa2ea828700 1 -- 172.21.15.12:0/1006552298 shutdown_connections 2024-05-09T09:25:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.567+0000 7fa2ea828700 1 -- 172.21.15.12:0/1006552298 wait complete. 2024-05-09T09:25:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.568+0000 7fa2ea828700 1 Processor -- start 2024-05-09T09:25:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.568+0000 7fa2ea828700 1 -- start start 2024-05-09T09:25:12.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.568+0000 7fa2ea828700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4101580 0x7fa2e4197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:12.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.568+0000 7fa2ea828700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2e4101f20 0x7fa2e4197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:12.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.568+0000 7fa2ea828700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4102840 0x7fa2e419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:12.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.568+0000 7fa2ea828700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e410ced0 con 0x7fa2e4102840 2024-05-09T09:25:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.568+0000 7fa2ea828700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e410cd50 con 0x7fa2e4101580 2024-05-09T09:25:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.568+0000 7fa2ea828700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e410d050 con 0x7fa2e4101f20 2024-05-09T09:25:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.568+0000 7fa2e8dc5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4102840 0x7fa2e419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.568+0000 7fa2e8dc5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4102840 0x7fa2e419e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52744/0 (socket says 172.21.15.12:52744) 2024-05-09T09:25:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.568+0000 7fa2e8dc5700 1 -- 172.21.15.12:0/4086251578 learned_addr learned my addr 172.21.15.12:0/4086251578 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.568+0000 7fa2e3fff700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4101580 0x7fa2e4197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.569+0000 7fa2e37fe700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2e4101f20 0x7fa2e4197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.569+0000 7fa2e8dc5700 1 -- 172.21.15.12:0/4086251578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2e4101f20 msgr2=0x7fa2e4197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.569+0000 7fa2e8dc5700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2e4101f20 0x7fa2e4197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.569+0000 7fa2e8dc5700 1 -- 172.21.15.12:0/4086251578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4101580 msgr2=0x7fa2e4197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.569+0000 7fa2e8dc5700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4101580 0x7fa2e4197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.569+0000 7fa2e8dc5700 1 -- 172.21.15.12:0/4086251578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e419f090 con 0x7fa2e4102840 2024-05-09T09:25:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.569+0000 7fa2e3fff700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4101580 0x7fa2e4197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.569+0000 7fa2e8dc5700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4102840 0x7fa2e419e9f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00bd60 tx=0x7fa2dc00be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.569+0000 7fa2e17fa700 1 -- 172.21.15.12:0/4086251578 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc021020 con 0x7fa2e4102840 2024-05-09T09:25:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.569+0000 7fa2e17fa700 1 -- 172.21.15.12:0/4086251578 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa2dc01b900 con 0x7fa2e4102840 2024-05-09T09:25:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.569+0000 7fa2ea828700 1 -- 172.21.15.12:0/4086251578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e419f320 con 0x7fa2e4102840 2024-05-09T09:25:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.569+0000 7fa2e17fa700 1 -- 172.21.15.12:0/4086251578 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc012a70 con 0x7fa2e4102840 2024-05-09T09:25:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.569+0000 7fa2ea828700 1 -- 172.21.15.12:0/4086251578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e419f7d0 con 0x7fa2e4102840 2024-05-09T09:25:12.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.570+0000 7fa2ea828700 1 -- 172.21.15.12:0/4086251578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e4069010 con 0x7fa2e4102840 2024-05-09T09:25:12.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.571+0000 7fa2e17fa700 1 -- 172.21.15.12:0/4086251578 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa2dc024020 con 0x7fa2e4102840 2024-05-09T09:25:12.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.572+0000 7fa2e17fa700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa2cc0a2ca0 0x7fa2cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:12.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.572+0000 7fa2e3fff700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa2cc0a2ca0 0x7fa2cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:12.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.572+0000 7fa2e17fa700 1 -- 172.21.15.12:0/4086251578 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fa2dc00f020 con 0x7fa2e4102840 2024-05-09T09:25:12.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.573+0000 7fa2e3fff700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa2cc0a2ca0 0x7fa2cc0a5160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa2e406a810 tx=0x7fa2d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:12.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.577+0000 7fa2e17fa700 1 -- 172.21.15.12:0/4086251578 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa2dc019320 con 0x7fa2e4102840 2024-05-09T09:25:12.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.791+0000 7fa2ea828700 1 -- 172.21.15.12:0/4086251578 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa2e4062760 con 0x7fa2cc0a2ca0 2024-05-09T09:25:12.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.792+0000 7fa2e17fa700 1 -- 172.21.15.12:0/4086251578 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa2e4062760 con 0x7fa2cc0a2ca0 2024-05-09T09:25:12.792 INFO:teuthology.orchestra.run.smithi012.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-09T09:25:12.792 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005J400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-09T09:25:12.792 INFO:teuthology.orchestra.run.smithi012.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-05-09T09:25:12.792 INFO:teuthology.orchestra.run.smithi012.stdout:smithi170 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002R480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-05-09T09:25:12.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.794+0000 7fa2ea828700 1 -- 172.21.15.12:0/4086251578 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa2cc0a2ca0 msgr2=0x7fa2cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:12.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.794+0000 7fa2ea828700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa2cc0a2ca0 0x7fa2cc0a5160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa2e406a810 tx=0x7fa2d4005f90 comp rx=0 tx=0).stop 2024-05-09T09:25:12.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.794+0000 7fa2ea828700 1 -- 172.21.15.12:0/4086251578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4102840 msgr2=0x7fa2e419e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.794+0000 7fa2ea828700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4102840 0x7fa2e419e9f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00bd60 tx=0x7fa2dc00be40 comp rx=0 tx=0).stop 2024-05-09T09:25:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.795+0000 7fa2ea828700 1 -- 172.21.15.12:0/4086251578 shutdown_connections 2024-05-09T09:25:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.795+0000 7fa2ea828700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa2cc0a2ca0 0x7fa2cc0a5160 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.795+0000 7fa2ea828700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4102840 0x7fa2e419e9f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.795+0000 7fa2ea828700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4101580 0x7fa2e4197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.795+0000 7fa2ea828700 1 --2- 172.21.15.12:0/4086251578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2e4101f20 0x7fa2e4197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.795+0000 7fa2ea828700 1 -- 172.21.15.12:0/4086251578 >> 172.21.15.12:0/4086251578 conn(0x7fa2e40fbfd0 msgr2=0x7fa2e41057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.795+0000 7fa2ea828700 1 -- 172.21.15.12:0/4086251578 shutdown_connections 2024-05-09T09:25:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:12.795+0000 7fa2ea828700 1 -- 172.21.15.12:0/4086251578 wait complete. 2024-05-09T09:25:13.454 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T09:25:13.464 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-09T09:25:13.464 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-09T09:25:13.722 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:25:14.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:14 smithi012 ceph-mon[30140]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-05-09T09:25:14.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:14 smithi012 ceph-mon[30140]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:14.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:14 smithi170 ceph-mon[33632]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-05-09T09:25:14.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:14 smithi170 ceph-mon[33632]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:14.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:14 smithi016 ceph-mon[33335]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-05-09T09:25:14.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:14 smithi016 ceph-mon[33335]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:25:15.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.604+0000 7f6734c87700 1 -- 172.21.15.12:0/3337226798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730100630 msgr2=0x7f6730100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.604+0000 7f6734c87700 1 --2- 172.21.15.12:0/3337226798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730100630 0x7f6730100a50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6720009990 tx=0x7f6720009ca0 comp rx=0 tx=0).stop 2024-05-09T09:25:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.604+0000 7f6734c87700 1 -- 172.21.15.12:0/3337226798 shutdown_connections 2024-05-09T09:25:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.604+0000 7f6734c87700 1 --2- 172.21.15.12:0/3337226798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6730103fb0 0x7f673010a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.604+0000 7f6734c87700 1 --2- 172.21.15.12:0/3337226798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730100630 0x7f6730100a50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.604+0000 7f6734c87700 1 --2- 172.21.15.12:0/3337226798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6730103610 0x7f6730103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.604+0000 7f6734c87700 1 -- 172.21.15.12:0/3337226798 >> 172.21.15.12:0/3337226798 conn(0x7f67300fbfb0 msgr2=0x7f67300fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.604+0000 7f6734c87700 1 -- 172.21.15.12:0/3337226798 shutdown_connections 2024-05-09T09:25:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.604+0000 7f6734c87700 1 -- 172.21.15.12:0/3337226798 wait complete. 2024-05-09T09:25:15.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.605+0000 7f6734c87700 1 Processor -- start 2024-05-09T09:25:15.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.605+0000 7f6734c87700 1 -- start start 2024-05-09T09:25:15.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.605+0000 7f6734c87700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730100630 0x7f67301984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:15.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.605+0000 7f6734c87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6730103610 0x7f6730198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:15.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.605+0000 7f6734c87700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6730103fb0 0x7f673019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:15.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.605+0000 7f6734c87700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f673010df50 con 0x7f6730100630 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.605+0000 7f6734c87700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f673010ddd0 con 0x7f6730103fb0 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.605+0000 7f6734c87700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f673010e0d0 con 0x7f6730103610 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.606+0000 7f672e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730100630 0x7f67301984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.606+0000 7f672e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730100630 0x7f67301984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52768/0 (socket says 172.21.15.12:52768) 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.606+0000 7f672e59c700 1 -- 172.21.15.12:0/4023150000 learned_addr learned my addr 172.21.15.12:0/4023150000 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.606+0000 7f6727fff700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6730103610 0x7f6730198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.606+0000 7f672ed9d700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6730103fb0 0x7f673019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.606+0000 7f672e59c700 1 -- 172.21.15.12:0/4023150000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6730103610 msgr2=0x7f6730198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.606+0000 7f672e59c700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6730103610 0x7f6730198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.606+0000 7f672e59c700 1 -- 172.21.15.12:0/4023150000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6730103fb0 msgr2=0x7f673019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.606+0000 7f672e59c700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6730103fb0 0x7f673019fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.606+0000 7f672e59c700 1 -- 172.21.15.12:0/4023150000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67301a0150 con 0x7f6730100630 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.606+0000 7f672e59c700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730100630 0x7f67301984f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f67200098e0 tx=0x7f67200057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.606+0000 7f67277fe700 1 -- 172.21.15.12:0/4023150000 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720027070 con 0x7f6730100630 2024-05-09T09:25:15.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.607+0000 7f67277fe700 1 -- 172.21.15.12:0/4023150000 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6720004140 con 0x7f6730100630 2024-05-09T09:25:15.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.607+0000 7f67277fe700 1 -- 172.21.15.12:0/4023150000 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672000f460 con 0x7f6730100630 2024-05-09T09:25:15.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.607+0000 7f6734c87700 1 -- 172.21.15.12:0/4023150000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67301a03e0 con 0x7f6730100630 2024-05-09T09:25:15.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.607+0000 7f6734c87700 1 -- 172.21.15.12:0/4023150000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67301a07e0 con 0x7f6730100630 2024-05-09T09:25:15.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.608+0000 7f6734c87700 1 -- 172.21.15.12:0/4023150000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6730069010 con 0x7f6730100630 2024-05-09T09:25:15.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.609+0000 7f67277fe700 1 -- 172.21.15.12:0/4023150000 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f672002a070 con 0x7f6730100630 2024-05-09T09:25:15.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.610+0000 7f67277fe700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f671c0a2c20 0x7f671c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:15.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.610+0000 7f6727fff700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f671c0a2c20 0x7f671c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:15.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.610+0000 7f67277fe700 1 -- 172.21.15.12:0/4023150000 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f67200d04a0 con 0x7f6730100630 2024-05-09T09:25:15.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.610+0000 7f6727fff700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f671c0a2c20 0x7f671c0a50e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f671800ac50 tx=0x7f671800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:15.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.613+0000 7f67277fe700 1 -- 172.21.15.12:0/4023150000 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6720013070 con 0x7f6730100630 2024-05-09T09:25:15.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.863+0000 7f6734c87700 1 -- 172.21.15.12:0/4023150000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6730062f30 con 0x7f6730100630 2024-05-09T09:25:15.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.864+0000 7f67277fe700 1 -- 172.21.15.12:0/4023150000 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3091 (secure 0 0 0) 0x7f672009d0c0 con 0x7f6730100630 2024-05-09T09:25:15.864 INFO:teuthology.orchestra.run.smithi012.stdout:e28 2024-05-09T09:25:15.864 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T09:25:15.864 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T09:25:15.864 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-05-09T09:25:15.864 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:15.865 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-05-09T09:25:15.865 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-05-09T09:25:15.865 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 28 2024-05-09T09:25:15.865 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T09:25:15.865 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-05-09T09:23:09.656296+0000 2024-05-09T09:25:15.865 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-05-09T09:23:34.596143+0000 2024-05-09T09:25:15.865 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-05-09T09:25:15.865 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-05-09T09:25:15.865 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 300 2024-05-09T09:25:15.865 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-05-09T09:25:15.865 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-05-09T09:25:15.865 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-05-09T09:25:15.865 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-05-09T09:25:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-05-09T09:25:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-05-09T09:25:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T09:25:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 5 2024-05-09T09:25:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:in 0,1,2,3,4 2024-05-09T09:25:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=24523,1=24515,2=24502,3=24484,4=14628} 2024-05-09T09:25:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-05-09T09:25:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-05-09T09:25:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-05-09T09:25:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3,4] 2024-05-09T09:25:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-05-09T09:25:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-05-09T09:25:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 0 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 24523 members: 24523,24502,24515,24484,14628 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.i{0:24523} state up:active seq 5 addr [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.h{0:24482} state up:standby-replay seq 1 addr [v2:172.21.15.16:6836/3413591353,v1:172.21.15.16:6837/3413591353] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b{1:24515} state up:active seq 8 addr [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.e{1:24449} state up:standby-replay seq 1 addr [v2:172.21.15.16:6834/1098165328,v1:172.21.15.16:6835/1098165328] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.c{2:24502} state up:active seq 12 addr [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.l{2:24436} state up:standby-replay seq 1 addr [v2:172.21.15.170:6832/4010027838,v1:172.21.15.170:6833/4010027838] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.f{3:24484} state up:active seq 13 addr [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.k{3:24410} state up:standby-replay seq 1 addr [v2:172.21.15.16:6832/3659922754,v1:172.21.15.16:6833/3659922754] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a{4:14628} state up:active seq 10 addr [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.g{4:14616} state up:standby-replay seq 1 addr [v2:172.21.15.12:6838/901613691,v1:172.21.15.12:6839/901613691] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:25:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:15.868 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:15.868 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-05-09T09:25:15.868 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:15.868 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.j{-1:14535} state up:standby seq 1 addr [v2:172.21.15.12:6834/2151845805,v1:172.21.15.12:6835/2151845805] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:25:15.868 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.d{-1:14547} state up:standby seq 1 addr [v2:172.21.15.12:6836/3490188582,v1:172.21.15.12:6837/3490188582] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:25:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.866+0000 7f6734c87700 1 -- 172.21.15.12:0/4023150000 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f671c0a2c20 msgr2=0x7f671c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.866+0000 7f6734c87700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f671c0a2c20 0x7f671c0a50e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f671800ac50 tx=0x7f671800a380 comp rx=0 tx=0).stop 2024-05-09T09:25:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.866+0000 7f6734c87700 1 -- 172.21.15.12:0/4023150000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730100630 msgr2=0x7f67301984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.866+0000 7f6734c87700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730100630 0x7f67301984f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f67200098e0 tx=0x7f67200057c0 comp rx=0 tx=0).stop 2024-05-09T09:25:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.866+0000 7f6734c87700 1 -- 172.21.15.12:0/4023150000 shutdown_connections 2024-05-09T09:25:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.866+0000 7f6734c87700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f671c0a2c20 0x7f671c0a50e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.866+0000 7f6734c87700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6730103fb0 0x7f673019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.866+0000 7f6734c87700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730100630 0x7f67301984f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.866+0000 7f6734c87700 1 --2- 172.21.15.12:0/4023150000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6730103610 0x7f6730198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.866+0000 7f6734c87700 1 -- 172.21.15.12:0/4023150000 >> 172.21.15.12:0/4023150000 conn(0x7f67300fbfb0 msgr2=0x7f6730106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.866+0000 7f6734c87700 1 -- 172.21.15.12:0/4023150000 shutdown_connections 2024-05-09T09:25:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:15.866+0000 7f6734c87700 1 -- 172.21.15.12:0/4023150000 wait complete. 2024-05-09T09:25:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-05-09T09:25:16.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:16 smithi012 ceph-mon[30140]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-09T09:25:16.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:16 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4023150000' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T09:25:16.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:16 smithi170 ceph-mon[33632]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-09T09:25:16.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:16 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/4023150000' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T09:25:16.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:16 smithi016 ceph-mon[33335]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-09T09:25:16.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:16 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/4023150000' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T09:25:16.522 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-09T09:25:16.781 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:25:18.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:18 smithi012 ceph-mon[30140]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T09:25:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:18 smithi170 ceph-mon[33632]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T09:25:18.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:18 smithi016 ceph-mon[33335]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T09:25:18.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.523+0000 7fddfd18f700 1 -- 172.21.15.12:0/4067545636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf8102380 msgr2=0x7fddf81027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:18.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.523+0000 7fddfd18f700 1 --2- 172.21.15.12:0/4067545636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf8102380 0x7fddf81027a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdde80099c0 tx=0x7fdde8009cd0 comp rx=0 tx=0).stop 2024-05-09T09:25:18.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.523+0000 7fddfd18f700 1 -- 172.21.15.12:0/4067545636 shutdown_connections 2024-05-09T09:25:18.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.523+0000 7fddfd18f700 1 --2- 172.21.15.12:0/4067545636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf8103f20 0x7fddf810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:18.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.523+0000 7fddfd18f700 1 --2- 172.21.15.12:0/4067545636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf8102380 0x7fddf81027a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:18.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.523+0000 7fddfd18f700 1 --2- 172.21.15.12:0/4067545636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddf8103580 0x7fddf81039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:18.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.523+0000 7fddfd18f700 1 -- 172.21.15.12:0/4067545636 >> 172.21.15.12:0/4067545636 conn(0x7fddf80fd920 msgr2=0x7fddf80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:18.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.523+0000 7fddfd18f700 1 -- 172.21.15.12:0/4067545636 shutdown_connections 2024-05-09T09:25:18.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.523+0000 7fddfd18f700 1 -- 172.21.15.12:0/4067545636 wait complete. 2024-05-09T09:25:18.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.524+0000 7fddfd18f700 1 Processor -- start 2024-05-09T09:25:18.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.524+0000 7fddfd18f700 1 -- start start 2024-05-09T09:25:18.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddfd18f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddf8102380 0x7fddf80760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:18.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddfd18f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf8103580 0x7fddf8074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:18.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddfd18f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf8103f20 0x7fddf8074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:18.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddfd18f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fddf810df80 con 0x7fddf8103580 2024-05-09T09:25:18.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddfd18f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddf810de00 con 0x7fddf8103f20 2024-05-09T09:25:18.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddfd18f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fddf810e100 con 0x7fddf8102380 2024-05-09T09:25:18.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddf659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf8103580 0x7fddf8074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:18.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddf659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf8103580 0x7fddf8074710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52784/0 (socket says 172.21.15.12:52784) 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddf659c700 1 -- 172.21.15.12:0/3839569270 learned_addr learned my addr 172.21.15.12:0/3839569270 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddf6d9d700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddf8102380 0x7fddf80760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddf759e700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf8103f20 0x7fddf8074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddf659c700 1 -- 172.21.15.12:0/3839569270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddf8102380 msgr2=0x7fddf80760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddf659c700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddf8102380 0x7fddf80760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddf659c700 1 -- 172.21.15.12:0/3839569270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf8103f20 msgr2=0x7fddf8074c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddf659c700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf8103f20 0x7fddf8074c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddf659c700 1 -- 172.21.15.12:0/3839569270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf8075500 con 0x7fddf8103580 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.525+0000 7fddf759e700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf8103f20 0x7fddf8074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.526+0000 7fddf659c700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf8103580 0x7fddf8074710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fddec00d980 tx=0x7fddec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.526+0000 7fdddffff700 1 -- 172.21.15.12:0/3839569270 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec017070 con 0x7fddf8103580 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.526+0000 7fdddffff700 1 -- 172.21.15.12:0/3839569270 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fddec005020 con 0x7fddf8103580 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.526+0000 7fddfd18f700 1 -- 172.21.15.12:0/3839569270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf81a27b0 con 0x7fddf8103580 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.526+0000 7fdddffff700 1 -- 172.21.15.12:0/3839569270 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec009bd0 con 0x7fddf8103580 2024-05-09T09:25:18.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.526+0000 7fddfd18f700 1 -- 172.21.15.12:0/3839569270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf81a2cf0 con 0x7fddf8103580 2024-05-09T09:25:18.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.527+0000 7fddfd18f700 1 -- 172.21.15.12:0/3839569270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf8069010 con 0x7fddf8103580 2024-05-09T09:25:18.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.528+0000 7fdddffff700 1 -- 172.21.15.12:0/3839569270 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fddec0051d0 con 0x7fddf8103580 2024-05-09T09:25:18.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.529+0000 7fdddffff700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdde00a2c20 0x7fdde00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:18.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.529+0000 7fddf6d9d700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdde00a2c20 0x7fdde00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:18.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.529+0000 7fdddffff700 1 -- 172.21.15.12:0/3839569270 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fddec0cf6d0 con 0x7fddf8103580 2024-05-09T09:25:18.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.529+0000 7fddf6d9d700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdde00a2c20 0x7fdde00a50e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdde8003900 tx=0x7fdde8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:18.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.532+0000 7fdddffff700 1 -- 172.21.15.12:0/3839569270 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fddec020020 con 0x7fddf8103580 2024-05-09T09:25:18.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.738+0000 7fddfd18f700 1 -- 172.21.15.12:0/3839569270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fddf802db80 con 0x7fddf8103580 2024-05-09T09:25:18.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.739+0000 7fdddffff700 1 -- 172.21.15.12:0/3839569270 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5697 (secure 0 0 0) 0x7fddec09c260 con 0x7fddf8103580 2024-05-09T09:25:18.739 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 93 2024-05-09T09:25:18.739 INFO:teuthology.orchestra.run.smithi012.stdout:fsid 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:25:18.739 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-05-09T09:15:02.891661+0000 2024-05-09T09:25:18.739 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-05-09T09:25:06.152764+0000 2024-05-09T09:25:18.739 INFO:teuthology.orchestra.run.smithi012.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-09T09:25:18.739 INFO:teuthology.orchestra.run.smithi012.stdout:crush_version 27 2024-05-09T09:25:18.739 INFO:teuthology.orchestra.run.smithi012.stdout:full_ratio 0.95 2024-05-09T09:25:18.739 INFO:teuthology.orchestra.run.smithi012.stdout:backfillfull_ratio 0.9 2024-05-09T09:25:18.739 INFO:teuthology.orchestra.run.smithi012.stdout:nearfull_ratio 0.85 2024-05-09T09:25:18.740 INFO:teuthology.orchestra.run.smithi012.stdout:require_min_compat_client luminous 2024-05-09T09:25:18.740 INFO:teuthology.orchestra.run.smithi012.stdout:min_compat_client luminous 2024-05-09T09:25:18.740 INFO:teuthology.orchestra.run.smithi012.stdout:require_osd_release squid 2024-05-09T09:25:18.740 INFO:teuthology.orchestra.run.smithi012.stdout:stretch_mode_enabled false 2024-05-09T09:25:18.740 INFO:teuthology.orchestra.run.smithi012.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-09T09:25:18.740 INFO:teuthology.orchestra.run.smithi012.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-09T09:25:18.740 INFO:teuthology.orchestra.run.smithi012.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-09T09:25:18.740 INFO:teuthology.orchestra.run.smithi012.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-09T09:25:18.740 INFO:teuthology.orchestra.run.smithi012.stdout:max_osd 12 2024-05-09T09:25:18.740 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6802/2958883281,v1:172.21.15.12:6803/2958883281] [v2:172.21.15.12:6804/2958883281,v1:172.21.15.12:6805/2958883281] exists,up 3abc7d81-df98-4a60-8798-58a765426230 2024-05-09T09:25:18.740 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 up in weight 1 up_from 13 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6800/2346791948,v1:172.21.15.16:6801/2346791948] [v2:172.21.15.16:6802/2346791948,v1:172.21.15.16:6803/2346791948] exists,up a4a1fa5a-7444-4090-a12d-c626ed8abdf6 2024-05-09T09:25:18.740 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 up in weight 1 up_from 18 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6800/4106675859,v1:172.21.15.170:6801/4106675859] [v2:172.21.15.170:6802/4106675859,v1:172.21.15.170:6803/4106675859] exists,up cc35c021-d74c-4b11-8e03-c54002bf99e7 2024-05-09T09:25:18.740 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 up in weight 1 up_from 25 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6810/2152501762,v1:172.21.15.12:6811/2152501762] [v2:172.21.15.12:6812/2152501762,v1:172.21.15.12:6813/2152501762] exists,up d122daa0-3729-41b6-b07d-3c2db6f9d514 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 up in weight 1 up_from 33 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6808/587973120,v1:172.21.15.16:6809/587973120] [v2:172.21.15.16:6810/587973120,v1:172.21.15.16:6811/587973120] exists,up d05507ed-2594-4791-93e2-0bf32b70942a 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6808/2002026273,v1:172.21.15.170:6809/2002026273] [v2:172.21.15.170:6810/2002026273,v1:172.21.15.170:6811/2002026273] exists,up 40f06cd7-6010-43cb-9dab-71ea07e02745 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6818/3412258494,v1:172.21.15.12:6819/3412258494] [v2:172.21.15.12:6820/3412258494,v1:172.21.15.12:6821/3412258494] exists,up 580dac7a-d03d-4830-9996-84ed2c9e9bf5 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6816/1916732888,v1:172.21.15.16:6817/1916732888] [v2:172.21.15.16:6818/1916732888,v1:172.21.15.16:6819/1916732888] exists,up 0c7f5786-3123-417c-b8eb-9c44e5284858 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 up in weight 1 up_from 59 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6816/4075084847,v1:172.21.15.170:6817/4075084847] [v2:172.21.15.170:6818/4075084847,v1:172.21.15.170:6819/4075084847] exists,up d78f2036-c1f3-4efc-a4bc-d173ed8cd846 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 up in weight 1 up_from 67 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6826/1889064834,v1:172.21.15.12:6827/1889064834] [v2:172.21.15.12:6828/1889064834,v1:172.21.15.12:6829/1889064834] exists,up be6068d1-eaf5-4694-a49f-e7baa8613fe3 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6824/1985793423,v1:172.21.15.16:6825/1985793423] [v2:172.21.15.16:6826/1985793423,v1:172.21.15.16:6827/1985793423] exists,up caa08b1c-2f8b-4853-accc-f84809ba98f1 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6824/3375809659,v1:172.21.15.170:6825/3375809659] [v2:172.21.15.170:6826/3375809659,v1:172.21.15.170:6827/3375809659] exists,up 2d7bab10-d9ad-4917-9a11-a4b4bd363e70 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.4 [2,11] 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.6 [8,11] 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.e [4,8] 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.14 [4,7] 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.1a [5,6] 2024-05-09T09:25:18.741 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.22 [3,9] 2024-05-09T09:25:18.742 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.25 [5,9] 2024-05-09T09:25:18.742 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.27 [4,6] 2024-05-09T09:25:18.742 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.2a [1,9] 2024-05-09T09:25:18.742 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.31 [4,8] 2024-05-09T09:25:18.742 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.5 [9,10] 2024-05-09T09:25:18.742 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.b [9,8] 2024-05-09T09:25:18.742 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.e [5,8] 2024-05-09T09:25:18.742 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.1b [9,8] 2024-05-09T09:25:18.742 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/1454579683 expires 2024-05-10T09:16:04.466876+0000 2024-05-09T09:25:18.742 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/2168557623 expires 2024-05-10T09:16:04.466876+0000 2024-05-09T09:25:18.742 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/2168557623 expires 2024-05-10T09:16:04.466876+0000 2024-05-09T09:25:18.743 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/545109421 expires 2024-05-10T09:15:36.182637+0000 2024-05-09T09:25:18.743 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/1153421462 expires 2024-05-10T09:16:04.466876+0000 2024-05-09T09:25:18.743 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/1731765200 expires 2024-05-10T09:15:36.182637+0000 2024-05-09T09:25:18.743 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/2508497448 expires 2024-05-10T09:15:36.182637+0000 2024-05-09T09:25:18.743 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/1876427478 expires 2024-05-10T09:15:36.182637+0000 2024-05-09T09:25:18.743 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/3479121957 expires 2024-05-10T09:16:04.466876+0000 2024-05-09T09:25:18.743 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/1876427478 expires 2024-05-10T09:15:36.182637+0000 2024-05-09T09:25:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.741+0000 7fddfd18f700 1 -- 172.21.15.12:0/3839569270 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdde00a2c20 msgr2=0x7fdde00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.741+0000 7fddfd18f700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdde00a2c20 0x7fdde00a50e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdde8003900 tx=0x7fdde8005fb0 comp rx=0 tx=0).stop 2024-05-09T09:25:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.741+0000 7fddfd18f700 1 -- 172.21.15.12:0/3839569270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf8103580 msgr2=0x7fddf8074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.741+0000 7fddfd18f700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf8103580 0x7fddf8074710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fddec00d980 tx=0x7fddec00dc90 comp rx=0 tx=0).stop 2024-05-09T09:25:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.742+0000 7fddfd18f700 1 -- 172.21.15.12:0/3839569270 shutdown_connections 2024-05-09T09:25:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.742+0000 7fddfd18f700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdde00a2c20 0x7fdde00a50e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:18.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.742+0000 7fddfd18f700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf8103f20 0x7fddf8074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:18.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.742+0000 7fddfd18f700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddf8102380 0x7fddf80760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:18.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.742+0000 7fddfd18f700 1 --2- 172.21.15.12:0/3839569270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf8103580 0x7fddf8074710 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:18.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.742+0000 7fddfd18f700 1 -- 172.21.15.12:0/3839569270 >> 172.21.15.12:0/3839569270 conn(0x7fddf80fd920 msgr2=0x7fddf8106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:18.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.742+0000 7fddfd18f700 1 -- 172.21.15.12:0/3839569270 shutdown_connections 2024-05-09T09:25:18.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:18.742+0000 7fddfd18f700 1 -- 172.21.15.12:0/3839569270 wait complete. 2024-05-09T09:25:19.375 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:19 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3839569270' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T09:25:19.409 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-09T09:25:19.420 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:25:19.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:19 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3839569270' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T09:25:19.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:19 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3839569270' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T09:25:19.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.536+0000 7f6341ce1640 1 -- 172.21.15.12:0/450569446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c100770 msgr2=0x7f633c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:19.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.536+0000 7f6341ce1640 1 --2- 172.21.15.12:0/450569446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c100770 0x7f633c100b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f632c009900 tx=0x7f632c02a410 comp rx=0 tx=0).stop 2024-05-09T09:25:19.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.536+0000 7f6341ce1640 1 -- 172.21.15.12:0/450569446 shutdown_connections 2024-05-09T09:25:19.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.536+0000 7f6341ce1640 1 --2- 172.21.15.12:0/450569446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f633c102320 0x7f633c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.536+0000 7f6341ce1640 1 --2- 172.21.15.12:0/450569446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f633c101960 0x7f633c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.536+0000 7f6341ce1640 1 --2- 172.21.15.12:0/450569446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c100770 0x7f633c100b90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.536+0000 7f6341ce1640 1 -- 172.21.15.12:0/450569446 >> 172.21.15.12:0/450569446 conn(0x7f633c0fbce0 msgr2=0x7f633c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:19.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.536+0000 7f6341ce1640 1 -- 172.21.15.12:0/450569446 shutdown_connections 2024-05-09T09:25:19.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.536+0000 7f6341ce1640 1 -- 172.21.15.12:0/450569446 wait complete. 2024-05-09T09:25:19.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.537+0000 7f6341ce1640 1 Processor -- start 2024-05-09T09:25:19.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.537+0000 7f6341ce1640 1 -- start start 2024-05-09T09:25:19.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.537+0000 7f6341ce1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f633c100770 0x7f633c104820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:19.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.537+0000 7f6341ce1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f633c101960 0x7f633c102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.537+0000 7f6341ce1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c102320 0x7f633c1033a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.537+0000 7f6341ce1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f633c10c430 con 0x7f633c102320 2024-05-09T09:25:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.537+0000 7f6341ce1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f633c10c2b0 con 0x7f633c101960 2024-05-09T09:25:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.537+0000 7f6341ce1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f633c10c5b0 con 0x7f633c100770 2024-05-09T09:25:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.537+0000 7f633bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c102320 0x7f633c1033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.537+0000 7f633bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c102320 0x7f633c1033a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52812/0 (socket says 172.21.15.12:52812) 2024-05-09T09:25:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.537+0000 7f633bfff640 1 -- 172.21.15.12:0/1370182649 learned_addr learned my addr 172.21.15.12:0/1370182649 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.537+0000 7f6332ffd640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f633c101960 0x7f633c102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.537+0000 7f633b7fe640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f633c100770 0x7f633c104820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.538+0000 7f633bfff640 1 -- 172.21.15.12:0/1370182649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f633c100770 msgr2=0x7f633c104820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.538+0000 7f633bfff640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f633c100770 0x7f633c104820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.538+0000 7f633bfff640 1 -- 172.21.15.12:0/1370182649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f633c101960 msgr2=0x7f633c102e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:19.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.538+0000 7f633bfff640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f633c101960 0x7f633c102e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.538+0000 7f633bfff640 1 -- 172.21.15.12:0/1370182649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633c103c30 con 0x7f633c102320 2024-05-09T09:25:19.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.538+0000 7f633bfff640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c102320 0x7f633c1033a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f633400b970 tx=0x7f633400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:19.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.538+0000 7f63397fa640 1 -- 172.21.15.12:0/1370182649 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334019070 con 0x7f633c102320 2024-05-09T09:25:19.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.538+0000 7f6341ce1640 1 -- 172.21.15.12:0/1370182649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633c1a1d60 con 0x7f633c102320 2024-05-09T09:25:19.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.538+0000 7f63397fa640 1 -- 172.21.15.12:0/1370182649 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f633400cb40 con 0x7f633c102320 2024-05-09T09:25:19.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.538+0000 7f63397fa640 1 -- 172.21.15.12:0/1370182649 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334007610 con 0x7f633c102320 2024-05-09T09:25:19.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.538+0000 7f6341ce1640 1 -- 172.21.15.12:0/1370182649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633c1a2270 con 0x7f633c102320 2024-05-09T09:25:19.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.539+0000 7f6341ce1640 1 -- 172.21.15.12:0/1370182649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f633c1a2560 con 0x7f633c102320 2024-05-09T09:25:19.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.540+0000 7f63397fa640 1 -- 172.21.15.12:0/1370182649 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f633401c070 con 0x7f633c102320 2024-05-09T09:25:19.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.541+0000 7f63397fa640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f63280a2b90 0x7f63280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:19.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.541+0000 7f63397fa640 1 -- 172.21.15.12:0/1370182649 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f63340cea70 con 0x7f633c102320 2024-05-09T09:25:19.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.541+0000 7f633b7fe640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f63280a2b90 0x7f63280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:19.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.542+0000 7f633b7fe640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f63280a2b90 0x7f63280a5050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f632c000c00 tx=0x7f632c031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:19.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.546+0000 7f63397fa640 1 -- 172.21.15.12:0/1370182649 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f633409af50 con 0x7f633c102320 2024-05-09T09:25:19.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.735+0000 7f6341ce1640 1 -- 172.21.15.12:0/1370182649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f633c1a28f0 con 0x7f633c102320 2024-05-09T09:25:19.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.736+0000 7f63397fa640 1 -- 172.21.15.12:0/1370182649 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24762 (secure 0 0 0) 0x7f633409f250 con 0x7f633c102320 2024-05-09T09:25:19.736 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:19.736 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":93,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:25:06.152764+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:25:19.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.738+0000 7f6341ce1640 1 -- 172.21.15.12:0/1370182649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f63280a2b90 msgr2=0x7f63280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:19.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.738+0000 7f6341ce1640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f63280a2b90 0x7f63280a5050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f632c000c00 tx=0x7f632c031510 comp rx=0 tx=0).stop 2024-05-09T09:25:19.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.738+0000 7f6341ce1640 1 -- 172.21.15.12:0/1370182649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c102320 msgr2=0x7f633c1033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:19.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.738+0000 7f6341ce1640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c102320 0x7f633c1033a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f633400b970 tx=0x7f633400be30 comp rx=0 tx=0).stop 2024-05-09T09:25:19.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.738+0000 7f6341ce1640 1 -- 172.21.15.12:0/1370182649 shutdown_connections 2024-05-09T09:25:19.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.738+0000 7f6341ce1640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f63280a2b90 0x7f63280a5050 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.738+0000 7f6341ce1640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c102320 0x7f633c1033a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.738+0000 7f6341ce1640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f633c101960 0x7f633c102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.738+0000 7f6341ce1640 1 --2- 172.21.15.12:0/1370182649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f633c100770 0x7f633c104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.738+0000 7f6341ce1640 1 -- 172.21.15.12:0/1370182649 >> 172.21.15.12:0/1370182649 conn(0x7f633c0fbce0 msgr2=0x7f633c0689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:19.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.738+0000 7f6341ce1640 1 -- 172.21.15.12:0/1370182649 shutdown_connections 2024-05-09T09:25:19.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.738+0000 7f6341ce1640 1 -- 172.21.15.12:0/1370182649 wait complete. 2024-05-09T09:25:19.750 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:23:08.167107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:23:09.065670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:23:12.773640+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:25:19.750 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:25:19.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.860+0000 7f8712b02640 1 -- 172.21.15.12:0/1222429530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c104ce0 msgr2=0x7f870c1070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.860+0000 7f8712b02640 1 --2- 172.21.15.12:0/1222429530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c104ce0 0x7f870c1070d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009900 tx=0x7f86f402a410 comp rx=0 tx=0).stop 2024-05-09T09:25:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.860+0000 7f8712b02640 1 -- 172.21.15.12:0/1222429530 shutdown_connections 2024-05-09T09:25:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.860+0000 7f8712b02640 1 --2- 172.21.15.12:0/1222429530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f870c107610 0x7f870c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.860+0000 7f8712b02640 1 --2- 172.21.15.12:0/1222429530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c104ce0 0x7f870c1070d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.860+0000 7f8712b02640 1 --2- 172.21.15.12:0/1222429530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c1023b0 0x7f870c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.860+0000 7f8712b02640 1 -- 172.21.15.12:0/1222429530 >> 172.21.15.12:0/1222429530 conn(0x7f870c0fbce0 msgr2=0x7f870c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.860+0000 7f8712b02640 1 -- 172.21.15.12:0/1222429530 shutdown_connections 2024-05-09T09:25:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.860+0000 7f8712b02640 1 -- 172.21.15.12:0/1222429530 wait complete. 2024-05-09T09:25:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.860+0000 7f8712b02640 1 Processor -- start 2024-05-09T09:25:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.860+0000 7f8712b02640 1 -- start start 2024-05-09T09:25:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f8712b02640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c1023b0 0x7f870c101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f8712b02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f870c104ce0 0x7f870c100360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f8712b02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c107610 0x7f870c1008a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f8712b02640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f870c10d4c0 con 0x7f870c1023b0 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f8712b02640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f870c10d340 con 0x7f870c107610 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f8712b02640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f870c10d640 con 0x7f870c104ce0 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f871089e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c1023b0 0x7f870c101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f871089e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c1023b0 0x7f870c101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52822/0 (socket says 172.21.15.12:52822) 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f871089e640 1 -- 172.21.15.12:0/784180274 learned_addr learned my addr 172.21.15.12:0/784180274 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f870bfff640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f870c104ce0 0x7f870c100360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f871109f640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c107610 0x7f870c1008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f871089e640 1 -- 172.21.15.12:0/784180274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f870c104ce0 msgr2=0x7f870c100360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f871089e640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f870c104ce0 0x7f870c100360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f871089e640 1 -- 172.21.15.12:0/784180274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c107610 msgr2=0x7f870c1008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f871089e640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c107610 0x7f870c1008a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f871089e640 1 -- 172.21.15.12:0/784180274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c101130 con 0x7f870c1023b0 2024-05-09T09:25:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f871109f640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c107610 0x7f870c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f871089e640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c1023b0 0x7f870c101d20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00c920 tx=0x7f86fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f8709ffb640 1 -- 172.21.15.12:0/784180274 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc007c70 con 0x7f870c1023b0 2024-05-09T09:25:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f8709ffb640 1 -- 172.21.15.12:0/784180274 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f86fc007e10 con 0x7f870c1023b0 2024-05-09T09:25:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f8709ffb640 1 -- 172.21.15.12:0/784180274 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc018850 con 0x7f870c1023b0 2024-05-09T09:25:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.861+0000 7f8712b02640 1 -- 172.21.15.12:0/784180274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870c1a1c90 con 0x7f870c1023b0 2024-05-09T09:25:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.862+0000 7f8712b02640 1 -- 172.21.15.12:0/784180274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870c1a21b0 con 0x7f870c1023b0 2024-05-09T09:25:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.863+0000 7f8712b02640 1 -- 172.21.15.12:0/784180274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f870c068650 con 0x7f870c1023b0 2024-05-09T09:25:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.864+0000 7f8709ffb640 1 -- 172.21.15.12:0/784180274 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f86fc018a30 con 0x7f870c1023b0 2024-05-09T09:25:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.864+0000 7f8709ffb640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86f80a2b90 0x7f86f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.865+0000 7f8709ffb640 1 -- 172.21.15.12:0/784180274 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f86fc0cf630 con 0x7f870c1023b0 2024-05-09T09:25:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.865+0000 7f870bfff640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86f80a2b90 0x7f86f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.865+0000 7f870bfff640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86f80a2b90 0x7f86f80a5050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f86f4006010 tx=0x7f86f4031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:19.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:19.869+0000 7f8709ffb640 1 -- 172.21.15.12:0/784180274 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f86fc014030 con 0x7f870c1023b0 2024-05-09T09:25:20.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.056+0000 7f8712b02640 1 -- 172.21.15.12:0/784180274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f870c062460 con 0x7f870c1023b0 2024-05-09T09:25:20.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.056+0000 7f8709ffb640 1 -- 172.21.15.12:0/784180274 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f86fc09c160 con 0x7f870c1023b0 2024-05-09T09:25:20.056 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:25:20.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.058+0000 7f8712b02640 1 -- 172.21.15.12:0/784180274 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86f80a2b90 msgr2=0x7f86f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.058+0000 7f8712b02640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86f80a2b90 0x7f86f80a5050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f86f4006010 tx=0x7f86f4031510 comp rx=0 tx=0).stop 2024-05-09T09:25:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.058+0000 7f8712b02640 1 -- 172.21.15.12:0/784180274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c1023b0 msgr2=0x7f870c101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.058+0000 7f8712b02640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c1023b0 0x7f870c101d20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00c920 tx=0x7f86fc00cde0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.058+0000 7f8712b02640 1 -- 172.21.15.12:0/784180274 shutdown_connections 2024-05-09T09:25:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.058+0000 7f8712b02640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86f80a2b90 0x7f86f80a5050 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.058+0000 7f8712b02640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c107610 0x7f870c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.058+0000 7f8712b02640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f870c104ce0 0x7f870c100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.058+0000 7f8712b02640 1 --2- 172.21.15.12:0/784180274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c1023b0 0x7f870c101d20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.058+0000 7f8712b02640 1 -- 172.21.15.12:0/784180274 >> 172.21.15.12:0/784180274 conn(0x7f870c0fbce0 msgr2=0x7f870c102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.058+0000 7f8712b02640 1 -- 172.21.15.12:0/784180274 shutdown_connections 2024-05-09T09:25:20.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.058+0000 7f8712b02640 1 -- 172.21.15.12:0/784180274 wait complete. 2024-05-09T09:25:20.069 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:25:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.179+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3292122762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a4102310 msgr2=0x7f20a4104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.179+0000 7f20a8bb7640 1 --2- 172.21.15.12:0/3292122762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a4102310 0x7f20a4104700 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2094009900 tx=0x7f209402a410 comp rx=0 tx=0).stop 2024-05-09T09:25:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.179+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3292122762 shutdown_connections 2024-05-09T09:25:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.179+0000 7f20a8bb7640 1 --2- 172.21.15.12:0/3292122762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a4107570 0x7f20a41099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.179+0000 7f20a8bb7640 1 --2- 172.21.15.12:0/3292122762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20a4104c40 0x7f20a4107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.179+0000 7f20a8bb7640 1 --2- 172.21.15.12:0/3292122762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a4102310 0x7f20a4104700 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.179+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3292122762 >> 172.21.15.12:0/3292122762 conn(0x7f20a40fbd20 msgr2=0x7f20a40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.179+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3292122762 shutdown_connections 2024-05-09T09:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.179+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3292122762 wait complete. 2024-05-09T09:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.180+0000 7f20a8bb7640 1 Processor -- start 2024-05-09T09:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.180+0000 7f20a8bb7640 1 -- start start 2024-05-09T09:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.180+0000 7f20a8bb7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a4102310 0x7f20a4197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.180+0000 7f20a8bb7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20a4104c40 0x7f20a4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.180+0000 7f20a8bb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a4107570 0x7f20a419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.180+0000 7f20a8bb7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f20a410d4f0 con 0x7f20a4102310 2024-05-09T09:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.180+0000 7f20a8bb7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20a410d370 con 0x7f20a4107570 2024-05-09T09:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.180+0000 7f20a8bb7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f20a410d670 con 0x7f20a4104c40 2024-05-09T09:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.180+0000 7f20a259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a4102310 0x7f20a4197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.180+0000 7f20a259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a4102310 0x7f20a4197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52850/0 (socket says 172.21.15.12:52850) 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.180+0000 7f20a259c640 1 -- 172.21.15.12:0/3812646872 learned_addr learned my addr 172.21.15.12:0/3812646872 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20a2d9d640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a4107570 0x7f20a419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20a1d9b640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20a4104c40 0x7f20a4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20a259c640 1 -- 172.21.15.12:0/3812646872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20a4104c40 msgr2=0x7f20a4198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20a259c640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20a4104c40 0x7f20a4198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20a259c640 1 -- 172.21.15.12:0/3812646872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a4107570 msgr2=0x7f20a419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20a259c640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a4107570 0x7f20a419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20a259c640 1 -- 172.21.15.12:0/3812646872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a419f7e0 con 0x7f20a4102310 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20a1d9b640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20a4104c40 0x7f20a4198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20a259c640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a4102310 0x7f20a4197bc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2094009230 tx=0x7f2094031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20937fe640 1 -- 172.21.15.12:0/3812646872 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209403f030 con 0x7f20a4102310 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3812646872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a419fa70 con 0x7f20a4102310 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20937fe640 1 -- 172.21.15.12:0/3812646872 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2094003ac0 con 0x7f20a4102310 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20937fe640 1 -- 172.21.15.12:0/3812646872 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094035070 con 0x7f20a4102310 2024-05-09T09:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.181+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3812646872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a419ffb0 con 0x7f20a4102310 2024-05-09T09:25:20.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.182+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3812646872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a40fd810 con 0x7f20a4102310 2024-05-09T09:25:20.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.183+0000 7f20937fe640 1 -- 172.21.15.12:0/3812646872 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2094042030 con 0x7f20a4102310 2024-05-09T09:25:20.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.184+0000 7f20937fe640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f208c0a2b90 0x7f208c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:20.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.184+0000 7f20a1d9b640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f208c0a2b90 0x7f208c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.184+0000 7f20937fe640 1 -- 172.21.15.12:0/3812646872 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f20940efaa0 con 0x7f20a4102310 2024-05-09T09:25:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.185+0000 7f20a1d9b640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f208c0a2b90 0x7f208c0a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f20a41990d0 tx=0x7f2098005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.188+0000 7f20937fe640 1 -- 172.21.15.12:0/3812646872 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f20940bc5d0 con 0x7f20a4102310 2024-05-09T09:25:20.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.372+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3812646872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f20a4062430 con 0x7f20a4102310 2024-05-09T09:25:20.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.372+0000 7f20937fe640 1 -- 172.21.15.12:0/3812646872 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f20940bbed0 con 0x7f20a4102310 2024-05-09T09:25:20.372 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:25:20.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.374+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3812646872 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f208c0a2b90 msgr2=0x7f208c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.374+0000 7f20a8bb7640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f208c0a2b90 0x7f208c0a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f20a41990d0 tx=0x7f2098005d30 comp rx=0 tx=0).stop 2024-05-09T09:25:20.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.374+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3812646872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a4102310 msgr2=0x7f20a4197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.374+0000 7f20a8bb7640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a4102310 0x7f20a4197bc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2094009230 tx=0x7f2094031ac0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.374+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3812646872 shutdown_connections 2024-05-09T09:25:20.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.374+0000 7f20a8bb7640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f208c0a2b90 0x7f208c0a5050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.374+0000 7f20a8bb7640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a4107570 0x7f20a419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.374+0000 7f20a8bb7640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20a4104c40 0x7f20a4198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.374+0000 7f20a8bb7640 1 --2- 172.21.15.12:0/3812646872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a4102310 0x7f20a4197bc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.374+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3812646872 >> 172.21.15.12:0/3812646872 conn(0x7f20a40fbd20 msgr2=0x7f20a4102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:20.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.374+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3812646872 shutdown_connections 2024-05-09T09:25:20.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.374+0000 7f20a8bb7640 1 -- 172.21.15.12:0/3812646872 wait complete. 2024-05-09T09:25:20.384 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:20 smithi012 ceph-mon[30140]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T09:25:20.384 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:20 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1370182649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:20.384 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:20 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/784180274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:20.385 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:25:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:20 smithi170 ceph-mon[33632]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T09:25:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:20 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1370182649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:20 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/784180274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:20.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.493+0000 7fb423088640 1 -- 172.21.15.12:0/1163602083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb41c101960 msgr2=0x7fb41c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.493+0000 7fb423088640 1 --2- 172.21.15.12:0/1163602083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb41c101960 0x7fb41c101de0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb404009900 tx=0x7fb40402a410 comp rx=0 tx=0).stop 2024-05-09T09:25:20.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 -- 172.21.15.12:0/1163602083 shutdown_connections 2024-05-09T09:25:20.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 --2- 172.21.15.12:0/1163602083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb41c102320 0x7fb41c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 --2- 172.21.15.12:0/1163602083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb41c101960 0x7fb41c101de0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 --2- 172.21.15.12:0/1163602083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb41c100770 0x7fb41c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 -- 172.21.15.12:0/1163602083 >> 172.21.15.12:0/1163602083 conn(0x7fb41c0fbce0 msgr2=0x7fb41c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 -- 172.21.15.12:0/1163602083 shutdown_connections 2024-05-09T09:25:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 -- 172.21.15.12:0/1163602083 wait complete. 2024-05-09T09:25:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 Processor -- start 2024-05-09T09:25:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 -- start start 2024-05-09T09:25:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb41c100770 0x7fb41c069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb41c101960 0x7fb41c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb41c102320 0x7fb41c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c10c430 con 0x7fb41c102320 2024-05-09T09:25:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c10c2b0 con 0x7fb41c100770 2024-05-09T09:25:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb423088640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c10c5b0 con 0x7fb41c101960 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.494+0000 7fb421625640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb41c102320 0x7fb41c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb421625640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb41c102320 0x7fb41c19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52870/0 (socket says 172.21.15.12:52870) 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb421625640 1 -- 172.21.15.12:0/2836431092 learned_addr learned my addr 172.21.15.12:0/2836431092 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb420e24640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb41c100770 0x7fb41c069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb41bfff640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb41c101960 0x7fb41c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb421625640 1 -- 172.21.15.12:0/2836431092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb41c101960 msgr2=0x7fb41c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb421625640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb41c101960 0x7fb41c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb421625640 1 -- 172.21.15.12:0/2836431092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb41c100770 msgr2=0x7fb41c069550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb421625640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb41c100770 0x7fb41c069550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb421625640 1 -- 172.21.15.12:0/2836431092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c19f860 con 0x7fb41c102320 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb41bfff640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb41c101960 0x7fb41c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb421625640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb41c102320 0x7fb41c19efa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00b970 tx=0x7fb40c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb419ffb640 1 -- 172.21.15.12:0/2836431092 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c019070 con 0x7fb41c102320 2024-05-09T09:25:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb419ffb640 1 -- 172.21.15.12:0/2836431092 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb40c00cb40 con 0x7fb41c102320 2024-05-09T09:25:20.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb423088640 1 -- 172.21.15.12:0/2836431092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41c19fb50 con 0x7fb41c102320 2024-05-09T09:25:20.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb419ffb640 1 -- 172.21.15.12:0/2836431092 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c007650 con 0x7fb41c102320 2024-05-09T09:25:20.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.495+0000 7fb423088640 1 -- 172.21.15.12:0/2836431092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41c1a0120 con 0x7fb41c102320 2024-05-09T09:25:20.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.496+0000 7fb423088640 1 -- 172.21.15.12:0/2836431092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41c107b60 con 0x7fb41c102320 2024-05-09T09:25:20.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.497+0000 7fb419ffb640 1 -- 172.21.15.12:0/2836431092 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb40c01c070 con 0x7fb41c102320 2024-05-09T09:25:20.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.498+0000 7fb419ffb640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb4080a2b90 0x7fb4080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:20.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.498+0000 7fb419ffb640 1 -- 172.21.15.12:0/2836431092 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fb40c0ceaa0 con 0x7fb41c102320 2024-05-09T09:25:20.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.498+0000 7fb420e24640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb4080a2b90 0x7fb4080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:20.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.499+0000 7fb420e24640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb4080a2b90 0x7fb4080a5050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb414005db0 tx=0x7fb414005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:20.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.503+0000 7fb419ffb640 1 -- 172.21.15.12:0/2836431092 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb40c09b630 con 0x7fb41c102320 2024-05-09T09:25:20.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:20 smithi016 ceph-mon[33335]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T09:25:20.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:20 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1370182649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:20.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:20 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/784180274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:20.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.686+0000 7fb423088640 1 -- 172.21.15.12:0/2836431092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb41c062430 con 0x7fb41c102320 2024-05-09T09:25:20.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.687+0000 7fb419ffb640 1 -- 172.21.15.12:0/2836431092 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fb40c09af30 con 0x7fb41c102320 2024-05-09T09:25:20.686 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T09:25:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.689+0000 7fb423088640 1 -- 172.21.15.12:0/2836431092 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb4080a2b90 msgr2=0x7fb4080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.689+0000 7fb423088640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb4080a2b90 0x7fb4080a5050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb414005db0 tx=0x7fb414005d40 comp rx=0 tx=0).stop 2024-05-09T09:25:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.689+0000 7fb423088640 1 -- 172.21.15.12:0/2836431092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb41c102320 msgr2=0x7fb41c19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.689+0000 7fb423088640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb41c102320 0x7fb41c19efa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00b970 tx=0x7fb40c00be30 comp rx=0 tx=0).stop 2024-05-09T09:25:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.689+0000 7fb423088640 1 -- 172.21.15.12:0/2836431092 shutdown_connections 2024-05-09T09:25:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.689+0000 7fb423088640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb4080a2b90 0x7fb4080a5050 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.689+0000 7fb423088640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb41c102320 0x7fb41c19efa0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.689+0000 7fb423088640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb41c101960 0x7fb41c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.689+0000 7fb423088640 1 --2- 172.21.15.12:0/2836431092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb41c100770 0x7fb41c069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.689+0000 7fb423088640 1 -- 172.21.15.12:0/2836431092 >> 172.21.15.12:0/2836431092 conn(0x7fb41c0fbce0 msgr2=0x7fb41c104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:20.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.689+0000 7fb423088640 1 -- 172.21.15.12:0/2836431092 shutdown_connections 2024-05-09T09:25:20.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.689+0000 7fb423088640 1 -- 172.21.15.12:0/2836431092 wait complete. 2024-05-09T09:25:20.700 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:25:20.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.809+0000 7fcdfd79d640 1 -- 172.21.15.12:0/3201111239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf8102a70 msgr2=0x7fcdf8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.809+0000 7fcdfd79d640 1 --2- 172.21.15.12:0/3201111239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf8102a70 0x7fcdf8102ef0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009900 tx=0x7fcde002a410 comp rx=0 tx=0).stop 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.810+0000 7fcdfd79d640 1 -- 172.21.15.12:0/3201111239 shutdown_connections 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.810+0000 7fcdfd79d640 1 --2- 172.21.15.12:0/3201111239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf8103430 0x7fcdf8109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.810+0000 7fcdfd79d640 1 --2- 172.21.15.12:0/3201111239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf8102a70 0x7fcdf8102ef0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.810+0000 7fcdfd79d640 1 --2- 172.21.15.12:0/3201111239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf8101880 0x7fcdf8101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.810+0000 7fcdfd79d640 1 -- 172.21.15.12:0/3201111239 >> 172.21.15.12:0/3201111239 conn(0x7fcdf80fce10 msgr2=0x7fcdf80ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.810+0000 7fcdfd79d640 1 -- 172.21.15.12:0/3201111239 shutdown_connections 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.810+0000 7fcdfd79d640 1 -- 172.21.15.12:0/3201111239 wait complete. 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.810+0000 7fcdfd79d640 1 Processor -- start 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.810+0000 7fcdfd79d640 1 -- start start 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdfd79d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf8101880 0x7fcdf8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdfd79d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf8102a70 0x7fcdf8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdfd79d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf8103430 0x7fcdf819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdfd79d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf810d4e0 con 0x7fcdf8103430 2024-05-09T09:25:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdfd79d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf810d360 con 0x7fcdf8101880 2024-05-09T09:25:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdfd79d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf810d660 con 0x7fcdf8102a70 2024-05-09T09:25:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf8103430 0x7fcdf819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf8103430 0x7fcdf819f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52890/0 (socket says 172.21.15.12:52890) 2024-05-09T09:25:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdf77fe640 1 -- 172.21.15.12:0/1846666392 learned_addr learned my addr 172.21.15.12:0/1846666392 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdf67fc640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf8102a70 0x7fcdf8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdf6ffd640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf8101880 0x7fcdf8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdf77fe640 1 -- 172.21.15.12:0/1846666392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf8102a70 msgr2=0x7fcdf8197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdf77fe640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf8102a70 0x7fcdf8197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdf77fe640 1 -- 172.21.15.12:0/1846666392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf8101880 msgr2=0x7fcdf8197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdf77fe640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf8101880 0x7fcdf8197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdf77fe640 1 -- 172.21.15.12:0/1846666392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf819f6d0 con 0x7fcdf8103430 2024-05-09T09:25:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdf6ffd640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf8101880 0x7fcdf8197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdf77fe640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf8103430 0x7fcdf819f060 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcde800eaa0 tx=0x7fcde800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:20.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdeffff640 1 -- 172.21.15.12:0/1846666392 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde800cdb0 con 0x7fcdf8103430 2024-05-09T09:25:20.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdfd79d640 1 -- 172.21.15.12:0/1846666392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf819f9c0 con 0x7fcdf8103430 2024-05-09T09:25:20.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdeffff640 1 -- 172.21.15.12:0/1846666392 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcde8005020 con 0x7fcdf8103430 2024-05-09T09:25:20.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.811+0000 7fcdeffff640 1 -- 172.21.15.12:0/1846666392 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80116b0 con 0x7fcdf8103430 2024-05-09T09:25:20.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.812+0000 7fcdfd79d640 1 -- 172.21.15.12:0/1846666392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf819fe90 con 0x7fcdf8103430 2024-05-09T09:25:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.813+0000 7fcdfd79d640 1 -- 172.21.15.12:0/1846666392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf8108b40 con 0x7fcdf8103430 2024-05-09T09:25:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.814+0000 7fcdeffff640 1 -- 172.21.15.12:0/1846666392 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcde80051d0 con 0x7fcdf8103430 2024-05-09T09:25:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.815+0000 7fcdeffff640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcde40a2b90 0x7fcde40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.815+0000 7fcdf6ffd640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcde40a2b90 0x7fcde40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.815+0000 7fcdeffff640 1 -- 172.21.15.12:0/1846666392 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fcde80cfa70 con 0x7fcdf8103430 2024-05-09T09:25:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.815+0000 7fcdf6ffd640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcde40a2b90 0x7fcde40a5050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcdf81028d0 tx=0x7fcdf000a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:20.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:20.818+0000 7fcdeffff640 1 -- 172.21.15.12:0/1846666392 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcde8016030 con 0x7fcdf8103430 2024-05-09T09:25:21.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.004+0000 7fcdfd79d640 1 -- 172.21.15.12:0/1846666392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fcdf802d1c0 con 0x7fcdf8103430 2024-05-09T09:25:21.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.004+0000 7fcdeffff640 1 -- 172.21.15.12:0/1846666392 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fcde809c600 con 0x7fcdf8103430 2024-05-09T09:25:21.004 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:25:21.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.007+0000 7fcdfd79d640 1 -- 172.21.15.12:0/1846666392 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcde40a2b90 msgr2=0x7fcde40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.007+0000 7fcdfd79d640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcde40a2b90 0x7fcde40a5050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcdf81028d0 tx=0x7fcdf000a5e0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.007+0000 7fcdfd79d640 1 -- 172.21.15.12:0/1846666392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf8103430 msgr2=0x7fcdf819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.007+0000 7fcdfd79d640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf8103430 0x7fcdf819f060 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcde800eaa0 tx=0x7fcde800ef60 comp rx=0 tx=0).stop 2024-05-09T09:25:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.007+0000 7fcdfd79d640 1 -- 172.21.15.12:0/1846666392 shutdown_connections 2024-05-09T09:25:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.007+0000 7fcdfd79d640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcde40a2b90 0x7fcde40a5050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.007+0000 7fcdfd79d640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf8103430 0x7fcdf819f060 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.007+0000 7fcdfd79d640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf8102a70 0x7fcdf8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.007+0000 7fcdfd79d640 1 --2- 172.21.15.12:0/1846666392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf8101880 0x7fcdf8197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.007+0000 7fcdfd79d640 1 -- 172.21.15.12:0/1846666392 >> 172.21.15.12:0/1846666392 conn(0x7fcdf80fce10 msgr2=0x7fcdf8105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.007+0000 7fcdfd79d640 1 -- 172.21.15.12:0/1846666392 shutdown_connections 2024-05-09T09:25:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.007+0000 7fcdfd79d640 1 -- 172.21.15.12:0/1846666392 wait complete. 2024-05-09T09:25:21.018 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:25:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.123+0000 7efffdee1640 1 -- 172.21.15.12:0/4147898504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8101980 msgr2=0x7efff8101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.123+0000 7efffdee1640 1 --2- 172.21.15.12:0/4147898504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8101980 0x7efff8101e00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7effec009900 tx=0x7effec02a410 comp rx=0 tx=0).stop 2024-05-09T09:25:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.123+0000 7efffdee1640 1 -- 172.21.15.12:0/4147898504 shutdown_connections 2024-05-09T09:25:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.123+0000 7efffdee1640 1 --2- 172.21.15.12:0/4147898504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff8102340 0x7efff8108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.123+0000 7efffdee1640 1 --2- 172.21.15.12:0/4147898504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8101980 0x7efff8101e00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.123+0000 7efffdee1640 1 --2- 172.21.15.12:0/4147898504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efff8100790 0x7efff8100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.123+0000 7efffdee1640 1 -- 172.21.15.12:0/4147898504 >> 172.21.15.12:0/4147898504 conn(0x7efff80fbd20 msgr2=0x7efff80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.123+0000 7efffdee1640 1 -- 172.21.15.12:0/4147898504 shutdown_connections 2024-05-09T09:25:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.123+0000 7efffdee1640 1 -- 172.21.15.12:0/4147898504 wait complete. 2024-05-09T09:25:21.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.123+0000 7efffdee1640 1 Processor -- start 2024-05-09T09:25:21.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efffdee1640 1 -- start start 2024-05-09T09:25:21.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efffdee1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8100790 0x7efff8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efffdee1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff8101980 0x7efff81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efffdee1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efff8102340 0x7efff819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efffdee1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efff810c440 con 0x7efff8100790 2024-05-09T09:25:21.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efffdee1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efff810c2c0 con 0x7efff8101980 2024-05-09T09:25:21.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efffdee1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efff810c5c0 con 0x7efff8102340 2024-05-09T09:25:21.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efff77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8100790 0x7efff8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efff77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8100790 0x7efff8197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52900/0 (socket says 172.21.15.12:52900) 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efff77fe640 1 -- 172.21.15.12:0/2605190769 learned_addr learned my addr 172.21.15.12:0/2605190769 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efff6ffd640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff8101980 0x7efff81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efff7fff640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efff8102340 0x7efff819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efff77fe640 1 -- 172.21.15.12:0/2605190769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efff8102340 msgr2=0x7efff819f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efff77fe640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efff8102340 0x7efff819f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efff77fe640 1 -- 172.21.15.12:0/2605190769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff8101980 msgr2=0x7efff81980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efff77fe640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff8101980 0x7efff81980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efff77fe640 1 -- 172.21.15.12:0/2605190769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff819f750 con 0x7efff8100790 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.124+0000 7efff77fe640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8100790 0x7efff8197b80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7effe800c970 tx=0x7effe800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.125+0000 7efff4ff9640 1 -- 172.21.15.12:0/2605190769 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8004220 con 0x7efff8100790 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.125+0000 7efffdee1640 1 -- 172.21.15.12:0/2605190769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff819fa40 con 0x7efff8100790 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.125+0000 7efff4ff9640 1 -- 172.21.15.12:0/2605190769 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7effe80043c0 con 0x7efff8100790 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.125+0000 7efff4ff9640 1 -- 172.21.15.12:0/2605190769 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8004a00 con 0x7efff8100790 2024-05-09T09:25:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.125+0000 7efffdee1640 1 -- 172.21.15.12:0/2605190769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff819ff10 con 0x7efff8100790 2024-05-09T09:25:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.126+0000 7efffdee1640 1 -- 172.21.15.12:0/2605190769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff81a01a0 con 0x7efff8100790 2024-05-09T09:25:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.127+0000 7efff4ff9640 1 -- 172.21.15.12:0/2605190769 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7effe8004ba0 con 0x7efff8100790 2024-05-09T09:25:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.127+0000 7efff4ff9640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7effe00a2b90 0x7effe00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.128+0000 7efff4ff9640 1 -- 172.21.15.12:0/2605190769 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7effe80cf1d0 con 0x7efff8100790 2024-05-09T09:25:21.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.128+0000 7efff6ffd640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7effe00a2b90 0x7effe00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.128+0000 7efff6ffd640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7effe00a2b90 0x7effe00a5050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7effec000c00 tx=0x7effec031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.132+0000 7efff4ff9640 1 -- 172.21.15.12:0/2605190769 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7effe8014030 con 0x7efff8100790 2024-05-09T09:25:21.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:21 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3812646872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:21.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:21 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2836431092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:21.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:21 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1846666392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.359+0000 7efffdee1640 1 -- 172.21.15.12:0/2605190769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efff81a0910 con 0x7efff8100790 2024-05-09T09:25:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.359+0000 7efff4ff9640 1 -- 172.21.15.12:0/2605190769 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11931 (secure 0 0 0) 0x7effe809bd60 con 0x7efff8100790 2024-05-09T09:25:21.359 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:21.359 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:23:34.596143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[24523,24502,24515,24484,14628]},"id":1}]} 2024-05-09T09:25:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-05-09T09:25:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.361+0000 7efffdee1640 1 -- 172.21.15.12:0/2605190769 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7effe00a2b90 msgr2=0x7effe00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.361+0000 7efffdee1640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7effe00a2b90 0x7effe00a5050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7effec000c00 tx=0x7effec031510 comp rx=0 tx=0).stop 2024-05-09T09:25:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.361+0000 7efffdee1640 1 -- 172.21.15.12:0/2605190769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8100790 msgr2=0x7efff8197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.361+0000 7efffdee1640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8100790 0x7efff8197b80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7effe800c970 tx=0x7effe800ce30 comp rx=0 tx=0).stop 2024-05-09T09:25:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.361+0000 7efffdee1640 1 -- 172.21.15.12:0/2605190769 shutdown_connections 2024-05-09T09:25:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.362+0000 7efffdee1640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7effe00a2b90 0x7effe00a5050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.362+0000 7efffdee1640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efff8102340 0x7efff819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.362+0000 7efffdee1640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff8101980 0x7efff81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.362+0000 7efffdee1640 1 --2- 172.21.15.12:0/2605190769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8100790 0x7efff8197b80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.362+0000 7efffdee1640 1 -- 172.21.15.12:0/2605190769 >> 172.21.15.12:0/2605190769 conn(0x7efff80fbd20 msgr2=0x7efff80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.362+0000 7efffdee1640 1 -- 172.21.15.12:0/2605190769 shutdown_connections 2024-05-09T09:25:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.362+0000 7efffdee1640 1 -- 172.21.15.12:0/2605190769 wait complete. 2024-05-09T09:25:21.373 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:25:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.478+0000 7fdd8129a640 1 -- 172.21.15.12:0/3646563348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd7c104c40 msgr2=0x7fdd7c107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.478+0000 7fdd8129a640 1 --2- 172.21.15.12:0/3646563348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd7c104c40 0x7fdd7c107030 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009900 tx=0x7fdd7002a410 comp rx=0 tx=0).stop 2024-05-09T09:25:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.479+0000 7fdd8129a640 1 -- 172.21.15.12:0/3646563348 shutdown_connections 2024-05-09T09:25:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.479+0000 7fdd8129a640 1 --2- 172.21.15.12:0/3646563348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd7c107570 0x7fdd7c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.479+0000 7fdd8129a640 1 --2- 172.21.15.12:0/3646563348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd7c104c40 0x7fdd7c107030 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.479+0000 7fdd8129a640 1 --2- 172.21.15.12:0/3646563348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd7c102310 0x7fdd7c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.479+0000 7fdd8129a640 1 -- 172.21.15.12:0/3646563348 >> 172.21.15.12:0/3646563348 conn(0x7fdd7c0fbd20 msgr2=0x7fdd7c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.479+0000 7fdd8129a640 1 -- 172.21.15.12:0/3646563348 shutdown_connections 2024-05-09T09:25:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.479+0000 7fdd8129a640 1 -- 172.21.15.12:0/3646563348 wait complete. 2024-05-09T09:25:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.479+0000 7fdd8129a640 1 Processor -- start 2024-05-09T09:25:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.479+0000 7fdd8129a640 1 -- start start 2024-05-09T09:25:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd8129a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd7c102310 0x7fdd7c197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd8129a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd7c104c40 0x7fdd7c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd8129a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd7c107570 0x7fdd7c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd8129a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c10d4d0 con 0x7fdd7c107570 2024-05-09T09:25:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd8129a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c10d350 con 0x7fdd7c104c40 2024-05-09T09:25:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd8129a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c10d650 con 0x7fdd7c102310 2024-05-09T09:25:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd7b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd7c107570 0x7fdd7c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd7b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd7c107570 0x7fdd7c19f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52914/0 (socket says 172.21.15.12:52914) 2024-05-09T09:25:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd7b7fe640 1 -- 172.21.15.12:0/3012651573 learned_addr learned my addr 172.21.15.12:0/3012651573 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd7affd640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd7c102310 0x7fdd7c197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd7a7fc640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd7c104c40 0x7fdd7c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd7b7fe640 1 -- 172.21.15.12:0/3012651573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd7c102310 msgr2=0x7fdd7c197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd7b7fe640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd7c102310 0x7fdd7c197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd7b7fe640 1 -- 172.21.15.12:0/3012651573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd7c104c40 msgr2=0x7fdd7c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd7b7fe640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd7c104c40 0x7fdd7c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd7b7fe640 1 -- 172.21.15.12:0/3012651573 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c19f7f0 con 0x7fdd7c107570 2024-05-09T09:25:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd7b7fe640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd7c107570 0x7fdd7c19f180 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400eaa0 tx=0x7fdd7400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd63fff640 1 -- 172.21.15.12:0/3012651573 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7400cdb0 con 0x7fdd7c107570 2024-05-09T09:25:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd8129a640 1 -- 172.21.15.12:0/3012651573 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd7c19fae0 con 0x7fdd7c107570 2024-05-09T09:25:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd63fff640 1 -- 172.21.15.12:0/3012651573 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdd74005020 con 0x7fdd7c107570 2024-05-09T09:25:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.480+0000 7fdd63fff640 1 -- 172.21.15.12:0/3012651573 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd740116b0 con 0x7fdd7c107570 2024-05-09T09:25:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.481+0000 7fdd8129a640 1 -- 172.21.15.12:0/3012651573 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd7c1a0000 con 0x7fdd7c107570 2024-05-09T09:25:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.481+0000 7fdd8129a640 1 -- 172.21.15.12:0/3012651573 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd7c0fd810 con 0x7fdd7c107570 2024-05-09T09:25:21.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.483+0000 7fdd63fff640 1 -- 172.21.15.12:0/3012651573 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdd740051d0 con 0x7fdd7c107570 2024-05-09T09:25:21.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.484+0000 7fdd63fff640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdd640a2b90 0x7fdd640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.484+0000 7fdd7affd640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdd640a2b90 0x7fdd640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.484+0000 7fdd63fff640 1 -- 172.21.15.12:0/3012651573 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fdd740cf640 con 0x7fdd7c107570 2024-05-09T09:25:21.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.484+0000 7fdd7affd640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdd640a2b90 0x7fdd640a5050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c005d00 tx=0x7fdd6c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.488+0000 7fdd63fff640 1 -- 172.21.15.12:0/3012651573 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdd74016030 con 0x7fdd7c107570 2024-05-09T09:25:21.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:21 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3812646872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:21.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:21 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2836431092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:21.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:21 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1846666392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:21.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:21 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3812646872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:21.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:21 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2836431092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:21.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:21 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1846666392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:21.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.677+0000 7fdd8129a640 1 -- 172.21.15.12:0/3012651573 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd7c062430 con 0x7fdd7c107570 2024-05-09T09:25:21.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.679+0000 7fdd63fff640 1 -- 172.21.15.12:0/3012651573 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24762 (secure 0 0 0) 0x7fdd7409c1d0 con 0x7fdd7c107570 2024-05-09T09:25:21.679 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:21.679 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":93,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:25:06.152764+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:25:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.681+0000 7fdd8129a640 1 -- 172.21.15.12:0/3012651573 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdd640a2b90 msgr2=0x7fdd640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.681+0000 7fdd8129a640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdd640a2b90 0x7fdd640a5050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c005d00 tx=0x7fdd6c00b410 comp rx=0 tx=0).stop 2024-05-09T09:25:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.681+0000 7fdd8129a640 1 -- 172.21.15.12:0/3012651573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd7c107570 msgr2=0x7fdd7c19f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.681+0000 7fdd8129a640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd7c107570 0x7fdd7c19f180 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400eaa0 tx=0x7fdd7400ef60 comp rx=0 tx=0).stop 2024-05-09T09:25:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.681+0000 7fdd8129a640 1 -- 172.21.15.12:0/3012651573 shutdown_connections 2024-05-09T09:25:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.681+0000 7fdd8129a640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdd640a2b90 0x7fdd640a5050 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.681+0000 7fdd8129a640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd7c107570 0x7fdd7c19f180 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.681+0000 7fdd8129a640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd7c104c40 0x7fdd7c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.681+0000 7fdd8129a640 1 --2- 172.21.15.12:0/3012651573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd7c102310 0x7fdd7c197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.681+0000 7fdd8129a640 1 -- 172.21.15.12:0/3012651573 >> 172.21.15.12:0/3012651573 conn(0x7fdd7c0fbd20 msgr2=0x7fdd7c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.681+0000 7fdd8129a640 1 -- 172.21.15.12:0/3012651573 shutdown_connections 2024-05-09T09:25:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.681+0000 7fdd8129a640 1 -- 172.21.15.12:0/3012651573 wait complete. 2024-05-09T09:25:21.693 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:23:08.167107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:23:09.065670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:23:12.773640+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:25:21.693 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:25:21.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.802+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1902729794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88100770 msgr2=0x7f0d88100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.802+0000 7f0d8ecee640 1 --2- 172.21.15.12:0/1902729794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88100770 0x7f0d88100b90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009900 tx=0x7f0d8002a410 comp rx=0 tx=0).stop 2024-05-09T09:25:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1902729794 shutdown_connections 2024-05-09T09:25:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 --2- 172.21.15.12:0/1902729794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d88102320 0x7f0d88108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 --2- 172.21.15.12:0/1902729794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d88101960 0x7f0d88101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 --2- 172.21.15.12:0/1902729794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88100770 0x7f0d88100b90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1902729794 >> 172.21.15.12:0/1902729794 conn(0x7f0d880fbce0 msgr2=0x7f0d880fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1902729794 shutdown_connections 2024-05-09T09:25:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1902729794 wait complete. 2024-05-09T09:25:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 Processor -- start 2024-05-09T09:25:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 -- start start 2024-05-09T09:25:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d88100770 0x7f0d88197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d88101960 0x7f0d88198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88102320 0x7f0d8819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8810c3d0 con 0x7f0d88102320 2024-05-09T09:25:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8810c250 con 0x7f0d88101960 2024-05-09T09:25:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.803+0000 7f0d8ecee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8810c550 con 0x7f0d88100770 2024-05-09T09:25:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d8d28b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88102320 0x7f0d8819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d8d28b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88102320 0x7f0d8819f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39134/0 (socket says 172.21.15.12:39134) 2024-05-09T09:25:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d8d28b640 1 -- 172.21.15.12:0/1050991342 learned_addr learned my addr 172.21.15.12:0/1050991342 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d87fff640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d88101960 0x7f0d88198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d8ca8a640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d88100770 0x7f0d88197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d8d28b640 1 -- 172.21.15.12:0/1050991342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d88100770 msgr2=0x7f0d88197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d8d28b640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d88100770 0x7f0d88197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d8d28b640 1 -- 172.21.15.12:0/1050991342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d88101960 msgr2=0x7f0d88198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d8d28b640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d88101960 0x7f0d88198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d8d28b640 1 -- 172.21.15.12:0/1050991342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8819f810 con 0x7f0d88102320 2024-05-09T09:25:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d8d28b640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88102320 0x7f0d8819f1a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d750 tx=0x7f0d7000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d85ffb640 1 -- 172.21.15.12:0/1050991342 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70017070 con 0x7f0d88102320 2024-05-09T09:25:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d85ffb640 1 -- 172.21.15.12:0/1050991342 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0d700054f0 con 0x7f0d88102320 2024-05-09T09:25:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1050991342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8819fb00 con 0x7f0d88102320 2024-05-09T09:25:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d85ffb640 1 -- 172.21.15.12:0/1050991342 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70004cf0 con 0x7f0d88102320 2024-05-09T09:25:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.804+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1050991342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d881a00a0 con 0x7f0d88102320 2024-05-09T09:25:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.806+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1050991342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d881a0270 con 0x7f0d88102320 2024-05-09T09:25:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.807+0000 7f0d85ffb640 1 -- 172.21.15.12:0/1050991342 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0d70004e90 con 0x7f0d88102320 2024-05-09T09:25:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.808+0000 7f0d85ffb640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0d740a2c10 0x7f0d740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.808+0000 7f0d8ca8a640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0d740a2c10 0x7f0d740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.808+0000 7f0d85ffb640 1 -- 172.21.15.12:0/1050991342 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f0d700cec80 con 0x7f0d88102320 2024-05-09T09:25:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.808+0000 7f0d8ca8a640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0d740a2c10 0x7f0d740a50d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0d80006010 tx=0x7f0d80031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:21.812+0000 7f0d85ffb640 1 -- 172.21.15.12:0/1050991342 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0d7009b1c0 con 0x7f0d88102320 2024-05-09T09:25:22.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.000+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1050991342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d881a0440 con 0x7f0d88102320 2024-05-09T09:25:22.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.001+0000 7f0d85ffb640 1 -- 172.21.15.12:0/1050991342 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f0d7009f4c0 con 0x7f0d88102320 2024-05-09T09:25:22.001 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:25:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.003+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1050991342 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0d740a2c10 msgr2=0x7f0d740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.003+0000 7f0d8ecee640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0d740a2c10 0x7f0d740a50d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0d80006010 tx=0x7f0d80031510 comp rx=0 tx=0).stop 2024-05-09T09:25:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.003+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1050991342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88102320 msgr2=0x7f0d8819f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.003+0000 7f0d8ecee640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88102320 0x7f0d8819f1a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d750 tx=0x7f0d7000dc10 comp rx=0 tx=0).stop 2024-05-09T09:25:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.003+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1050991342 shutdown_connections 2024-05-09T09:25:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.003+0000 7f0d8ecee640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0d740a2c10 0x7f0d740a50d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.003+0000 7f0d8ecee640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88102320 0x7f0d8819f1a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.003+0000 7f0d8ecee640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d88101960 0x7f0d88198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.003+0000 7f0d8ecee640 1 --2- 172.21.15.12:0/1050991342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d88100770 0x7f0d88197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.003+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1050991342 >> 172.21.15.12:0/1050991342 conn(0x7f0d880fbce0 msgr2=0x7f0d880689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.003+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1050991342 shutdown_connections 2024-05-09T09:25:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.003+0000 7f0d8ecee640 1 -- 172.21.15.12:0/1050991342 wait complete. 2024-05-09T09:25:22.014 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:25:22.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.122+0000 7f8b31202640 1 -- 172.21.15.12:0/19641814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b2c102220 msgr2=0x7f8b2c108ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.122+0000 7f8b31202640 1 --2- 172.21.15.12:0/19641814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b2c102220 0x7f8b2c108ab0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009900 tx=0x7f8b1c02a410 comp rx=0 tx=0).stop 2024-05-09T09:25:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.122+0000 7f8b31202640 1 -- 172.21.15.12:0/19641814 shutdown_connections 2024-05-09T09:25:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.122+0000 7f8b31202640 1 --2- 172.21.15.12:0/19641814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b2c102220 0x7f8b2c108ab0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.122+0000 7f8b31202640 1 --2- 172.21.15.12:0/19641814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b2c101860 0x7f8b2c101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.122+0000 7f8b31202640 1 --2- 172.21.15.12:0/19641814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b2c100670 0x7f8b2c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.122+0000 7f8b31202640 1 -- 172.21.15.12:0/19641814 >> 172.21.15.12:0/19641814 conn(0x7f8b2c0fbc00 msgr2=0x7f8b2c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.122+0000 7f8b31202640 1 -- 172.21.15.12:0/19641814 shutdown_connections 2024-05-09T09:25:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.122+0000 7f8b31202640 1 -- 172.21.15.12:0/19641814 wait complete. 2024-05-09T09:25:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.123+0000 7f8b31202640 1 Processor -- start 2024-05-09T09:25:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.123+0000 7f8b31202640 1 -- start start 2024-05-09T09:25:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.123+0000 7f8b31202640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b2c100670 0x7f8b2c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.123+0000 7f8b31202640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b2c101860 0x7f8b2c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.123+0000 7f8b31202640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b2c102220 0x7f8b2c1a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.123+0000 7f8b31202640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c10c2c0 con 0x7f8b2c101860 2024-05-09T09:25:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.123+0000 7f8b31202640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c10c140 con 0x7f8b2c100670 2024-05-09T09:25:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.123+0000 7f8b31202640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c10c440 con 0x7f8b2c102220 2024-05-09T09:25:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b2a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b2c101860 0x7f8b2c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b2a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b2c101860 0x7f8b2c19a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39150/0 (socket says 172.21.15.12:39150) 2024-05-09T09:25:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b2a59c640 1 -- 172.21.15.12:0/2948700632 learned_addr learned my addr 172.21.15.12:0/2948700632 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b2b59e640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b2c102220 0x7f8b2c1a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b2a59c640 1 -- 172.21.15.12:0/2948700632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b2c102220 msgr2=0x7f8b2c1a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b2a59c640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b2c102220 0x7f8b2c1a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b2ad9d640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b2c100670 0x7f8b2c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b2a59c640 1 -- 172.21.15.12:0/2948700632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b2c100670 msgr2=0x7f8b2c199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b2a59c640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b2c100670 0x7f8b2c199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b2a59c640 1 -- 172.21.15.12:0/2948700632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c1a17c0 con 0x7f8b2c101860 2024-05-09T09:25:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b2b59e640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b2c102220 0x7f8b2c1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b2ad9d640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b2c100670 0x7f8b2c199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b2a59c640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b2c101860 0x7f8b2c19a0e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400d950 tx=0x7f8b1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b23fff640 1 -- 172.21.15.12:0/2948700632 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14017030 con 0x7f8b2c101860 2024-05-09T09:25:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b31202640 1 -- 172.21.15.12:0/2948700632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c1a1ab0 con 0x7f8b2c101860 2024-05-09T09:25:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b23fff640 1 -- 172.21.15.12:0/2948700632 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8b14004020 con 0x7f8b2c101860 2024-05-09T09:25:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b23fff640 1 -- 172.21.15.12:0/2948700632 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b140047a0 con 0x7f8b2c101860 2024-05-09T09:25:22.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.124+0000 7f8b31202640 1 -- 172.21.15.12:0/2948700632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c1a2000 con 0x7f8b2c101860 2024-05-09T09:25:22.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.125+0000 7f8b31202640 1 -- 172.21.15.12:0/2948700632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c1a2230 con 0x7f8b2c101860 2024-05-09T09:25:22.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.127+0000 7f8b23fff640 1 -- 172.21.15.12:0/2948700632 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8b140041d0 con 0x7f8b2c101860 2024-05-09T09:25:22.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.127+0000 7f8b23fff640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8b180a2b90 0x7f8b180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.127+0000 7f8b2ad9d640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8b180a2b90 0x7f8b180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.127+0000 7f8b23fff640 1 -- 172.21.15.12:0/2948700632 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f8b140cf550 con 0x7f8b2c101860 2024-05-09T09:25:22.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.128+0000 7f8b2ad9d640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8b180a2b90 0x7f8b180a5050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c1016c0 tx=0x7f8b24008370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.131+0000 7f8b23fff640 1 -- 172.21.15.12:0/2948700632 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8b1409c0e0 con 0x7f8b2c101860 2024-05-09T09:25:22.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.319+0000 7f8b31202640 1 -- 172.21.15.12:0/2948700632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8b2c1a2400 con 0x7f8b2c101860 2024-05-09T09:25:22.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.319+0000 7f8b23fff640 1 -- 172.21.15.12:0/2948700632 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f8b1409b9e0 con 0x7f8b2c101860 2024-05-09T09:25:22.319 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:25:22.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.321+0000 7f8b31202640 1 -- 172.21.15.12:0/2948700632 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8b180a2b90 msgr2=0x7f8b180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.321+0000 7f8b31202640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8b180a2b90 0x7f8b180a5050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c1016c0 tx=0x7f8b24008370 comp rx=0 tx=0).stop 2024-05-09T09:25:22.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.321+0000 7f8b31202640 1 -- 172.21.15.12:0/2948700632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b2c101860 msgr2=0x7f8b2c19a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.321+0000 7f8b31202640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b2c101860 0x7f8b2c19a0e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400d950 tx=0x7f8b1400de10 comp rx=0 tx=0).stop 2024-05-09T09:25:22.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.321+0000 7f8b31202640 1 -- 172.21.15.12:0/2948700632 shutdown_connections 2024-05-09T09:25:22.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.321+0000 7f8b31202640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8b180a2b90 0x7f8b180a5050 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.321+0000 7f8b31202640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b2c102220 0x7f8b2c1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.321+0000 7f8b31202640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b2c101860 0x7f8b2c19a0e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.321+0000 7f8b31202640 1 --2- 172.21.15.12:0/2948700632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b2c100670 0x7f8b2c199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.321+0000 7f8b31202640 1 -- 172.21.15.12:0/2948700632 >> 172.21.15.12:0/2948700632 conn(0x7f8b2c0fbc00 msgr2=0x7f8b2c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.321+0000 7f8b31202640 1 -- 172.21.15.12:0/2948700632 shutdown_connections 2024-05-09T09:25:22.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.321+0000 7f8b31202640 1 -- 172.21.15.12:0/2948700632 wait complete. 2024-05-09T09:25:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:22 smithi012 ceph-mon[30140]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-09T09:25:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:22 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2605190769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:22 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3012651573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:22 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1050991342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:22.332 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:25:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.442+0000 7f0e2789a640 1 -- 172.21.15.12:0/2065040877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e20102a70 msgr2=0x7f0e20104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.442+0000 7f0e2789a640 1 --2- 172.21.15.12:0/2065040877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e20102a70 0x7f0e20104ee0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009900 tx=0x7f0e1002a3f0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.442+0000 7f0e2789a640 1 -- 172.21.15.12:0/2065040877 shutdown_connections 2024-05-09T09:25:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.442+0000 7f0e2789a640 1 --2- 172.21.15.12:0/2065040877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e20105420 0x7f0e201058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.442+0000 7f0e2789a640 1 --2- 172.21.15.12:0/2065040877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e20102a70 0x7f0e20104ee0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.442+0000 7f0e2789a640 1 --2- 172.21.15.12:0/2065040877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e20100100 0x7f0e20102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.442+0000 7f0e2789a640 1 -- 172.21.15.12:0/2065040877 >> 172.21.15.12:0/2065040877 conn(0x7f0e200fbd20 msgr2=0x7f0e200fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.442+0000 7f0e2789a640 1 -- 172.21.15.12:0/2065040877 shutdown_connections 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.442+0000 7f0e2789a640 1 -- 172.21.15.12:0/2065040877 wait complete. 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e2789a640 1 Processor -- start 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e2789a640 1 -- start start 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e2789a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e20100100 0x7f0e2019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e2789a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e20102a70 0x7f0e2019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e2789a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e20105420 0x7f0e201a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e2789a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20113b30 con 0x7f0e20100100 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e2789a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e201139b0 con 0x7f0e20105420 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e2789a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20113cb0 con 0x7f0e20102a70 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e25636640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e20100100 0x7f0e2019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e25636640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e20100100 0x7f0e2019e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39168/0 (socket says 172.21.15.12:39168) 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e25636640 1 -- 172.21.15.12:0/3030209136 learned_addr learned my addr 172.21.15.12:0/3030209136 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e24e35640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e20102a70 0x7f0e2019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e25636640 1 -- 172.21.15.12:0/3030209136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e20102a70 msgr2=0x7f0e2019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e25636640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e20102a70 0x7f0e2019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e25636640 1 -- 172.21.15.12:0/3030209136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e20105420 msgr2=0x7f0e201a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e25e37640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e20105420 0x7f0e201a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e25636640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e20105420 0x7f0e201a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e25636640 1 -- 172.21.15.12:0/3030209136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e201a5d40 con 0x7f0e20100100 2024-05-09T09:25:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.443+0000 7f0e24e35640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e20102a70 0x7f0e2019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.444+0000 7f0e25636640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e20100100 0x7f0e2019e120 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00c890 tx=0x7f0e1c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.444+0000 7f0e167fc640 1 -- 172.21.15.12:0/3030209136 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c004220 con 0x7f0e20100100 2024-05-09T09:25:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.444+0000 7f0e167fc640 1 -- 172.21.15.12:0/3030209136 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0e1c0043c0 con 0x7f0e20100100 2024-05-09T09:25:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.444+0000 7f0e2789a640 1 -- 172.21.15.12:0/3030209136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e201a5fd0 con 0x7f0e20100100 2024-05-09T09:25:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.444+0000 7f0e167fc640 1 -- 172.21.15.12:0/3030209136 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c004a00 con 0x7f0e20100100 2024-05-09T09:25:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.444+0000 7f0e2789a640 1 -- 172.21.15.12:0/3030209136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e201a6490 con 0x7f0e20100100 2024-05-09T09:25:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.445+0000 7f0e2789a640 1 -- 172.21.15.12:0/3030209136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e200fd810 con 0x7f0e20100100 2024-05-09T09:25:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.447+0000 7f0e167fc640 1 -- 172.21.15.12:0/3030209136 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0e1c004ba0 con 0x7f0e20100100 2024-05-09T09:25:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.447+0000 7f0e167fc640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0e0c0a2b90 0x7f0e0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.448+0000 7f0e24e35640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0e0c0a2b90 0x7f0e0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.448+0000 7f0e167fc640 1 -- 172.21.15.12:0/3030209136 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f0e1c0cf630 con 0x7f0e20100100 2024-05-09T09:25:22.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.448+0000 7f0e24e35640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0e0c0a2b90 0x7f0e0c0a5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0e10000c00 tx=0x7f0e10031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.451+0000 7f0e167fc640 1 -- 172.21.15.12:0/3030209136 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0e1c014030 con 0x7f0e20100100 2024-05-09T09:25:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:22 smithi170 ceph-mon[33632]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-09T09:25:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:22 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2605190769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:22 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3012651573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:22 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1050991342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:22 smithi016 ceph-mon[33335]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-09T09:25:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:22 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2605190769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:22 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3012651573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:22 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1050991342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.636+0000 7f0e2789a640 1 -- 172.21.15.12:0/3030209136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0e2002d470 con 0x7f0e20100100 2024-05-09T09:25:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.637+0000 7f0e167fc640 1 -- 172.21.15.12:0/3030209136 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f0e1c09c1c0 con 0x7f0e20100100 2024-05-09T09:25:22.636 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T09:25:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.639+0000 7f0e2789a640 1 -- 172.21.15.12:0/3030209136 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0e0c0a2b90 msgr2=0x7f0e0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.639+0000 7f0e2789a640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0e0c0a2b90 0x7f0e0c0a5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0e10000c00 tx=0x7f0e10031510 comp rx=0 tx=0).stop 2024-05-09T09:25:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.639+0000 7f0e2789a640 1 -- 172.21.15.12:0/3030209136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e20100100 msgr2=0x7f0e2019e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.639+0000 7f0e2789a640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e20100100 0x7f0e2019e120 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00c890 tx=0x7f0e1c00cd50 comp rx=0 tx=0).stop 2024-05-09T09:25:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.639+0000 7f0e2789a640 1 -- 172.21.15.12:0/3030209136 shutdown_connections 2024-05-09T09:25:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.639+0000 7f0e2789a640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0e0c0a2b90 0x7f0e0c0a5050 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.639+0000 7f0e2789a640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e20105420 0x7f0e201a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.639+0000 7f0e2789a640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e20102a70 0x7f0e2019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.639+0000 7f0e2789a640 1 --2- 172.21.15.12:0/3030209136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e20100100 0x7f0e2019e120 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.639+0000 7f0e2789a640 1 -- 172.21.15.12:0/3030209136 >> 172.21.15.12:0/3030209136 conn(0x7f0e200fbd20 msgr2=0x7f0e20100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.639+0000 7f0e2789a640 1 -- 172.21.15.12:0/3030209136 shutdown_connections 2024-05-09T09:25:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.639+0000 7f0e2789a640 1 -- 172.21.15.12:0/3030209136 wait complete. 2024-05-09T09:25:22.650 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:25:22.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.759+0000 7fbb6238b640 1 -- 172.21.15.12:0/2867344088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5c100790 msgr2=0x7fbb5c100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.759+0000 7fbb6238b640 1 --2- 172.21.15.12:0/2867344088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5c100790 0x7fbb5c100bb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009900 tx=0x7fbb4c02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.759+0000 7fbb6238b640 1 -- 172.21.15.12:0/2867344088 shutdown_connections 2024-05-09T09:25:22.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.759+0000 7fbb6238b640 1 --2- 172.21.15.12:0/2867344088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c102340 0x7fbb5c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.759+0000 7fbb6238b640 1 --2- 172.21.15.12:0/2867344088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb5c101980 0x7fbb5c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.759+0000 7fbb6238b640 1 --2- 172.21.15.12:0/2867344088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5c100790 0x7fbb5c100bb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.759+0000 7fbb6238b640 1 -- 172.21.15.12:0/2867344088 >> 172.21.15.12:0/2867344088 conn(0x7fbb5c0fbd20 msgr2=0x7fbb5c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.759+0000 7fbb6238b640 1 -- 172.21.15.12:0/2867344088 shutdown_connections 2024-05-09T09:25:22.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.759+0000 7fbb6238b640 1 -- 172.21.15.12:0/2867344088 wait complete. 2024-05-09T09:25:22.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb6238b640 1 Processor -- start 2024-05-09T09:25:22.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb6238b640 1 -- start start 2024-05-09T09:25:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb6238b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb5c100790 0x7fbb5c06a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb6238b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5c101980 0x7fbb5c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb6238b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c102340 0x7fbb5c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb6238b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c10c3f0 con 0x7fbb5c101980 2024-05-09T09:25:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb6238b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c10c270 con 0x7fbb5c102340 2024-05-09T09:25:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb6238b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c10c570 con 0x7fbb5c100790 2024-05-09T09:25:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb5b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5c101980 0x7fbb5c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb5b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5c101980 0x7fbb5c19c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39194/0 (socket says 172.21.15.12:39194) 2024-05-09T09:25:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb5b7fe640 1 -- 172.21.15.12:0/312684707 learned_addr learned my addr 172.21.15.12:0/312684707 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb5bfff640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb5c100790 0x7fbb5c06a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb60928640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c102340 0x7fbb5c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb5b7fe640 1 -- 172.21.15.12:0/312684707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb5c100790 msgr2=0x7fbb5c06a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb5b7fe640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb5c100790 0x7fbb5c06a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb5b7fe640 1 -- 172.21.15.12:0/312684707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c102340 msgr2=0x7fbb5c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb5b7fe640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c102340 0x7fbb5c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.760+0000 7fbb5b7fe640 1 -- 172.21.15.12:0/312684707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c19f7a0 con 0x7fbb5c101980 2024-05-09T09:25:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.761+0000 7fbb5b7fe640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5c101980 0x7fbb5c19c8a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000c970 tx=0x7fbb5000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.761+0000 7fbb597fa640 1 -- 172.21.15.12:0/312684707 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50007bb0 con 0x7fbb5c101980 2024-05-09T09:25:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.761+0000 7fbb597fa640 1 -- 172.21.15.12:0/312684707 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbb50007d50 con 0x7fbb5c101980 2024-05-09T09:25:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.761+0000 7fbb6238b640 1 -- 172.21.15.12:0/312684707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5c19fa90 con 0x7fbb5c101980 2024-05-09T09:25:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.761+0000 7fbb597fa640 1 -- 172.21.15.12:0/312684707 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5000f450 con 0x7fbb5c101980 2024-05-09T09:25:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.761+0000 7fbb6238b640 1 -- 172.21.15.12:0/312684707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5c19ffe0 con 0x7fbb5c101980 2024-05-09T09:25:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.762+0000 7fbb6238b640 1 -- 172.21.15.12:0/312684707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb5c107b80 con 0x7fbb5c101980 2024-05-09T09:25:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.763+0000 7fbb597fa640 1 -- 172.21.15.12:0/312684707 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbb50004fb0 con 0x7fbb5c101980 2024-05-09T09:25:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.764+0000 7fbb597fa640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbb440a2c10 0x7fbb440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.764+0000 7fbb5bfff640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbb440a2c10 0x7fbb440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.764+0000 7fbb597fa640 1 -- 172.21.15.12:0/312684707 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbb500cf1d0 con 0x7fbb5c101980 2024-05-09T09:25:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.765+0000 7fbb5bfff640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbb440a2c10 0x7fbb440a50d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c000c00 tx=0x7fbb4c031ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.769+0000 7fbb597fa640 1 -- 172.21.15.12:0/312684707 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbb50014030 con 0x7fbb5c101980 2024-05-09T09:25:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.955+0000 7fbb6238b640 1 -- 172.21.15.12:0/312684707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbb5c062430 con 0x7fbb5c101980 2024-05-09T09:25:22.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.955+0000 7fbb597fa640 1 -- 172.21.15.12:0/312684707 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fbb5009bd60 con 0x7fbb5c101980 2024-05-09T09:25:22.955 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:25:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.958+0000 7fbb6238b640 1 -- 172.21.15.12:0/312684707 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbb440a2c10 msgr2=0x7fbb440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.958+0000 7fbb6238b640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbb440a2c10 0x7fbb440a50d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c000c00 tx=0x7fbb4c031ca0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.958+0000 7fbb6238b640 1 -- 172.21.15.12:0/312684707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5c101980 msgr2=0x7fbb5c19c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.958+0000 7fbb6238b640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5c101980 0x7fbb5c19c8a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000c970 tx=0x7fbb5000ce30 comp rx=0 tx=0).stop 2024-05-09T09:25:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.958+0000 7fbb6238b640 1 -- 172.21.15.12:0/312684707 shutdown_connections 2024-05-09T09:25:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.958+0000 7fbb6238b640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbb440a2c10 0x7fbb440a50d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.958+0000 7fbb6238b640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c102340 0x7fbb5c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.958+0000 7fbb6238b640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5c101980 0x7fbb5c19c8a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.958+0000 7fbb6238b640 1 --2- 172.21.15.12:0/312684707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb5c100790 0x7fbb5c06a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.958+0000 7fbb6238b640 1 -- 172.21.15.12:0/312684707 >> 172.21.15.12:0/312684707 conn(0x7fbb5c0fbd20 msgr2=0x7fbb5c104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.958+0000 7fbb6238b640 1 -- 172.21.15.12:0/312684707 shutdown_connections 2024-05-09T09:25:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:22.958+0000 7fbb6238b640 1 -- 172.21.15.12:0/312684707 wait complete. 2024-05-09T09:25:22.969 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:25:23.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.074+0000 7f72ffedf640 1 -- 172.21.15.12:0/2366101847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f810de70 msgr2=0x7f72f8110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.074+0000 7f72ffedf640 1 --2- 172.21.15.12:0/2366101847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f810de70 0x7f72f8110260 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009cd0 tx=0x7f72f002a8e0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.074+0000 7f72ffedf640 1 -- 172.21.15.12:0/2366101847 shutdown_connections 2024-05-09T09:25:23.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.074+0000 7f72ffedf640 1 --2- 172.21.15.12:0/2366101847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f810de70 0x7f72f8110260 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.074+0000 7f72ffedf640 1 --2- 172.21.15.12:0/2366101847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72f8074290 0x7f72f8074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.074+0000 7f72ffedf640 1 --2- 172.21.15.12:0/2366101847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f8075870 0x7f72f8073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.074+0000 7f72ffedf640 1 -- 172.21.15.12:0/2366101847 >> 172.21.15.12:0/2366101847 conn(0x7f72f80fce30 msgr2=0x7f72f80ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.074+0000 7f72ffedf640 1 -- 172.21.15.12:0/2366101847 shutdown_connections 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.074+0000 7f72ffedf640 1 -- 172.21.15.12:0/2366101847 wait complete. 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.075+0000 7f72ffedf640 1 Processor -- start 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.075+0000 7f72ffedf640 1 -- start start 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.075+0000 7f72ffedf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72f8074290 0x7f72f819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.075+0000 7f72ffedf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f8075870 0x7f72f819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.075+0000 7f72ffedf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f810de70 0x7f72f81a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.075+0000 7f72ffedf640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f72f8113b60 con 0x7f72f8075870 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.075+0000 7f72ffedf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81139e0 con 0x7f72f810de70 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.075+0000 7f72ffedf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72f8113ce0 con 0x7f72f8074290 2024-05-09T09:25:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.075+0000 7f72fd47a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f8075870 0x7f72f819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.075+0000 7f72fd47a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f8075870 0x7f72f819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39212/0 (socket says 172.21.15.12:39212) 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.075+0000 7f72fd47a640 1 -- 172.21.15.12:0/2489923454 learned_addr learned my addr 172.21.15.12:0/2489923454 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.075+0000 7f72fe47c640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f810de70 0x7f72f81a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.075+0000 7f72fdc7b640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72f8074290 0x7f72f819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.076+0000 7f72fd47a640 1 -- 172.21.15.12:0/2489923454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72f8074290 msgr2=0x7f72f819e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.076+0000 7f72fd47a640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72f8074290 0x7f72f819e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.076+0000 7f72fd47a640 1 -- 172.21.15.12:0/2489923454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f810de70 msgr2=0x7f72f81a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.076+0000 7f72fd47a640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f810de70 0x7f72f81a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.076+0000 7f72fd47a640 1 -- 172.21.15.12:0/2489923454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f81a5dd0 con 0x7f72f8075870 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.076+0000 7f72fdc7b640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72f8074290 0x7f72f819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.076+0000 7f72fd47a640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f8075870 0x7f72f819e6f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f72e800b4a0 tx=0x7f72e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.076+0000 7f72eeffd640 1 -- 172.21.15.12:0/2489923454 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8014070 con 0x7f72f8075870 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.076+0000 7f72ffedf640 1 -- 172.21.15.12:0/2489923454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f81a60c0 con 0x7f72f8075870 2024-05-09T09:25:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.076+0000 7f72eeffd640 1 -- 172.21.15.12:0/2489923454 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f72e8010ae0 con 0x7f72f8075870 2024-05-09T09:25:23.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.076+0000 7f72ffedf640 1 -- 172.21.15.12:0/2489923454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f81a6690 con 0x7f72f8075870 2024-05-09T09:25:23.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.076+0000 7f72eeffd640 1 -- 172.21.15.12:0/2489923454 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e801e060 con 0x7f72f8075870 2024-05-09T09:25:23.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.077+0000 7f72ffedf640 1 -- 172.21.15.12:0/2489923454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72f8104040 con 0x7f72f8075870 2024-05-09T09:25:23.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.078+0000 7f72eeffd640 1 -- 172.21.15.12:0/2489923454 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f72e8017070 con 0x7f72f8075870 2024-05-09T09:25:23.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.079+0000 7f72eeffd640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f72e40a2b90 0x7f72e40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.079+0000 7f72eeffd640 1 -- 172.21.15.12:0/2489923454 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f72e80cefe0 con 0x7f72f8075870 2024-05-09T09:25:23.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.079+0000 7f72fdc7b640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f72e40a2b90 0x7f72e40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.079+0000 7f72fdc7b640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f72e40a2b90 0x7f72e40a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f72f40058c0 tx=0x7f72f4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.082+0000 7f72eeffd640 1 -- 172.21.15.12:0/2489923454 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f72e809bb70 con 0x7f72f8075870 2024-05-09T09:25:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:23 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2948700632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:23 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3030209136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:23 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/312684707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:23.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.306+0000 7f72ffedf640 1 -- 172.21.15.12:0/2489923454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72f802d410 con 0x7f72f8075870 2024-05-09T09:25:23.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.307+0000 7f72eeffd640 1 -- 172.21.15.12:0/2489923454 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11931 (secure 0 0 0) 0x7f72e809b470 con 0x7f72f8075870 2024-05-09T09:25:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:23.307 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:23:34.596143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[24523,24502,24515,24484,14628]},"id":1}]} 2024-05-09T09:25:23.308 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-05-09T09:25:23.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.309+0000 7f72ffedf640 1 -- 172.21.15.12:0/2489923454 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f72e40a2b90 msgr2=0x7f72e40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.309+0000 7f72ffedf640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f72e40a2b90 0x7f72e40a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f72f40058c0 tx=0x7f72f4005850 comp rx=0 tx=0).stop 2024-05-09T09:25:23.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.309+0000 7f72ffedf640 1 -- 172.21.15.12:0/2489923454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f8075870 msgr2=0x7f72f819e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.309+0000 7f72ffedf640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f8075870 0x7f72f819e6f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f72e800b4a0 tx=0x7f72e800b960 comp rx=0 tx=0).stop 2024-05-09T09:25:23.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.309+0000 7f72ffedf640 1 -- 172.21.15.12:0/2489923454 shutdown_connections 2024-05-09T09:25:23.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.309+0000 7f72ffedf640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f72e40a2b90 0x7f72e40a5050 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.309+0000 7f72ffedf640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f810de70 0x7f72f81a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.309+0000 7f72ffedf640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f8075870 0x7f72f819e6f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.309+0000 7f72ffedf640 1 --2- 172.21.15.12:0/2489923454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72f8074290 0x7f72f819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.309+0000 7f72ffedf640 1 -- 172.21.15.12:0/2489923454 >> 172.21.15.12:0/2489923454 conn(0x7f72f80fce30 msgr2=0x7f72f8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.309+0000 7f72ffedf640 1 -- 172.21.15.12:0/2489923454 shutdown_connections 2024-05-09T09:25:23.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.309+0000 7f72ffedf640 1 -- 172.21.15.12:0/2489923454 wait complete. 2024-05-09T09:25:23.321 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:25:23.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.427+0000 7f86e1c3f640 1 -- 172.21.15.12:0/3235077056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86dc103440 msgr2=0x7f86dc109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.427+0000 7f86e1c3f640 1 --2- 172.21.15.12:0/3235077056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86dc103440 0x7f86dc109cd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009900 tx=0x7f86d402a410 comp rx=0 tx=0).stop 2024-05-09T09:25:23.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.427+0000 7f86e1c3f640 1 -- 172.21.15.12:0/3235077056 shutdown_connections 2024-05-09T09:25:23.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.427+0000 7f86e1c3f640 1 --2- 172.21.15.12:0/3235077056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86dc103440 0x7f86dc109cd0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.427+0000 7f86e1c3f640 1 --2- 172.21.15.12:0/3235077056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86dc102a80 0x7f86dc102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.427+0000 7f86e1c3f640 1 --2- 172.21.15.12:0/3235077056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc101890 0x7f86dc101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.427+0000 7f86e1c3f640 1 -- 172.21.15.12:0/3235077056 >> 172.21.15.12:0/3235077056 conn(0x7f86dc0fcde0 msgr2=0x7f86dc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.427+0000 7f86e1c3f640 1 -- 172.21.15.12:0/3235077056 shutdown_connections 2024-05-09T09:25:23.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.427+0000 7f86e1c3f640 1 -- 172.21.15.12:0/3235077056 wait complete. 2024-05-09T09:25:23.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86e1c3f640 1 Processor -- start 2024-05-09T09:25:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86e1c3f640 1 -- start start 2024-05-09T09:25:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86e1c3f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86dc101890 0x7f86dc197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86e1c3f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86dc102a80 0x7f86dc198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86e1c3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc103440 0x7f86dc19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86e1c3f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc10d4e0 con 0x7f86dc101890 2024-05-09T09:25:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86e1c3f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc10d360 con 0x7f86dc103440 2024-05-09T09:25:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86e1c3f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc10d660 con 0x7f86dc102a80 2024-05-09T09:25:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86db7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86dc101890 0x7f86dc197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86db7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86dc101890 0x7f86dc197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39218/0 (socket says 172.21.15.12:39218) 2024-05-09T09:25:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86db7fe640 1 -- 172.21.15.12:0/1435778221 learned_addr learned my addr 172.21.15.12:0/1435778221 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86daffd640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86dc102a80 0x7f86dc198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86dbfff640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc103440 0x7f86dc19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86db7fe640 1 -- 172.21.15.12:0/1435778221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86dc102a80 msgr2=0x7f86dc198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86db7fe640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86dc102a80 0x7f86dc198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86db7fe640 1 -- 172.21.15.12:0/1435778221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc103440 msgr2=0x7f86dc19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86db7fe640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc103440 0x7f86dc19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86db7fe640 1 -- 172.21.15.12:0/1435778221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc19f740 con 0x7f86dc101890 2024-05-09T09:25:23.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86daffd640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86dc102a80 0x7f86dc198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:23.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.428+0000 7f86dbfff640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc103440 0x7f86dc19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:23.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.429+0000 7f86db7fe640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86dc101890 0x7f86dc197b20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00ea80 tx=0x7f86cc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.429+0000 7f86d8ff9640 1 -- 172.21.15.12:0/1435778221 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc00ce40 con 0x7f86dc101890 2024-05-09T09:25:23.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.429+0000 7f86e1c3f640 1 -- 172.21.15.12:0/1435778221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86dc19fa30 con 0x7f86dc101890 2024-05-09T09:25:23.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.429+0000 7f86d8ff9640 1 -- 172.21.15.12:0/1435778221 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f86cc011940 con 0x7f86dc101890 2024-05-09T09:25:23.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.429+0000 7f86d8ff9640 1 -- 172.21.15.12:0/1435778221 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc01ab30 con 0x7f86dc101890 2024-05-09T09:25:23.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.429+0000 7f86e1c3f640 1 -- 172.21.15.12:0/1435778221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86dc19ffd0 con 0x7f86dc101890 2024-05-09T09:25:23.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.430+0000 7f86e1c3f640 1 -- 172.21.15.12:0/1435778221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86dc068650 con 0x7f86dc101890 2024-05-09T09:25:23.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.431+0000 7f86d8ff9640 1 -- 172.21.15.12:0/1435778221 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f86cc01acd0 con 0x7f86dc101890 2024-05-09T09:25:23.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.432+0000 7f86d8ff9640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86c40a2b90 0x7f86c40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.432+0000 7f86daffd640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86c40a2b90 0x7f86c40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.432+0000 7f86d8ff9640 1 -- 172.21.15.12:0/1435778221 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f86cc0cfab0 con 0x7f86dc101890 2024-05-09T09:25:23.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.432+0000 7f86daffd640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86c40a2b90 0x7f86c40a5050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f86dc199030 tx=0x7f86d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.436+0000 7f86d8ff9640 1 -- 172.21.15.12:0/1435778221 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f86cc016030 con 0x7f86dc101890 2024-05-09T09:25:23.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:23 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2948700632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:23.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:23 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3030209136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:23.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:23 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/312684707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:23.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:23 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2948700632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:23.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:23 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3030209136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:23.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:23 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/312684707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:23.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.624+0000 7f86e1c3f640 1 -- 172.21.15.12:0/1435778221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86dc062430 con 0x7f86dc101890 2024-05-09T09:25:23.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.626+0000 7f86d8ff9640 1 -- 172.21.15.12:0/1435778221 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24762 (secure 0 0 0) 0x7f86cc09c640 con 0x7f86dc101890 2024-05-09T09:25:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:23.626 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":93,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:25:06.152764+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:25:23.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.628+0000 7f86e1c3f640 1 -- 172.21.15.12:0/1435778221 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86c40a2b90 msgr2=0x7f86c40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.628+0000 7f86e1c3f640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86c40a2b90 0x7f86c40a5050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f86dc199030 tx=0x7f86d0005f90 comp rx=0 tx=0).stop 2024-05-09T09:25:23.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.628+0000 7f86e1c3f640 1 -- 172.21.15.12:0/1435778221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86dc101890 msgr2=0x7f86dc197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.628+0000 7f86e1c3f640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86dc101890 0x7f86dc197b20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00ea80 tx=0x7f86cc00ef40 comp rx=0 tx=0).stop 2024-05-09T09:25:23.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.628+0000 7f86e1c3f640 1 -- 172.21.15.12:0/1435778221 shutdown_connections 2024-05-09T09:25:23.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.628+0000 7f86e1c3f640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86c40a2b90 0x7f86c40a5050 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.628+0000 7f86e1c3f640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc103440 0x7f86dc19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.628+0000 7f86e1c3f640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86dc102a80 0x7f86dc198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.628+0000 7f86e1c3f640 1 --2- 172.21.15.12:0/1435778221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86dc101890 0x7f86dc197b20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.628+0000 7f86e1c3f640 1 -- 172.21.15.12:0/1435778221 >> 172.21.15.12:0/1435778221 conn(0x7f86dc0fcde0 msgr2=0x7f86dc105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.628+0000 7f86e1c3f640 1 -- 172.21.15.12:0/1435778221 shutdown_connections 2024-05-09T09:25:23.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.628+0000 7f86e1c3f640 1 -- 172.21.15.12:0/1435778221 wait complete. 2024-05-09T09:25:23.640 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T09:23:09.656296+0000', 'modified': '2024-05-09T09:23:34.596143+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24523, 'mds_1': 24515, 'mds_2': 24502, 'mds_3': 24484, 'mds_4': 14628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.12:6839/901613691', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6838', 'nonce': 901613691}, {'type': 'v1', 'addr': '172.21.15.12:6839', 'nonce': 901613691}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.12:6841/1434721532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6840', 'nonce': 1434721532}, {'type': 'v1', 'addr': '172.21.15.12:6841', 'nonce': 1434721532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6833/3659922754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6832', 'nonce': 3659922754}, {'type': 'v1', 'addr': '172.21.15.16:6833', 'nonce': 3659922754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6833/4010027838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 4010027838}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 4010027838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6835/1098165328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 1098165328}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 1098165328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6837/3413591353', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6836', 'nonce': 3413591353}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 3413591353}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.170:6835/277363225', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 277363225}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 277363225}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.170:6837/2973369023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 2973369023}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 2973369023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.16:6839/3932062083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 3932062083}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 3932062083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.170:6839/3731005466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 3731005466}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 3731005466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24523, 'qdb_cluster': [24523, 24502, 24515, 24484, 14628]} 2024-05-09T09:25:23.640 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T09:25:23.640 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-09T09:25:23.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.747+0000 7f0f7c495640 1 -- 172.21.15.12:0/2666878332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f74101960 msgr2=0x7f0f74101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.747+0000 7f0f7c495640 1 --2- 172.21.15.12:0/2666878332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f74101960 0x7f0f74101de0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009900 tx=0x7f0f6402a3f0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.747+0000 7f0f7c495640 1 -- 172.21.15.12:0/2666878332 shutdown_connections 2024-05-09T09:25:23.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.747+0000 7f0f7c495640 1 --2- 172.21.15.12:0/2666878332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f74102320 0x7f0f74108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.747+0000 7f0f7c495640 1 --2- 172.21.15.12:0/2666878332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f74101960 0x7f0f74101de0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.747+0000 7f0f7c495640 1 --2- 172.21.15.12:0/2666878332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f74100770 0x7f0f74100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.747+0000 7f0f7c495640 1 -- 172.21.15.12:0/2666878332 >> 172.21.15.12:0/2666878332 conn(0x7f0f740fbce0 msgr2=0x7f0f740fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.747+0000 7f0f7c495640 1 -- 172.21.15.12:0/2666878332 shutdown_connections 2024-05-09T09:25:23.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.747+0000 7f0f7c495640 1 -- 172.21.15.12:0/2666878332 wait complete. 2024-05-09T09:25:23.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.747+0000 7f0f7c495640 1 Processor -- start 2024-05-09T09:25:23.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.747+0000 7f0f7c495640 1 -- start start 2024-05-09T09:25:23.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.747+0000 7f0f7c495640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f74100770 0x7f0f74197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7c495640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f74101960 0x7f0f741980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7c495640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f74102320 0x7f0f7419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7c495640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7410c3d0 con 0x7f0f74102320 2024-05-09T09:25:23.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7c495640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7410c250 con 0x7f0f74101960 2024-05-09T09:25:23.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7c495640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7410c550 con 0x7f0f74100770 2024-05-09T09:25:23.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7aa32640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f74102320 0x7f0f7419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7aa32640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f74102320 0x7f0f7419f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39240/0 (socket says 172.21.15.12:39240) 2024-05-09T09:25:23.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7aa32640 1 -- 172.21.15.12:0/2183079350 learned_addr learned my addr 172.21.15.12:0/2183079350 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:23.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7a231640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f74100770 0x7f0f74197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f79a30640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f74101960 0x7f0f741980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7aa32640 1 -- 172.21.15.12:0/2183079350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f74100770 msgr2=0x7f0f74197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7aa32640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f74100770 0x7f0f74197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7aa32640 1 -- 172.21.15.12:0/2183079350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f74101960 msgr2=0x7f0f741980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7aa32640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f74101960 0x7f0f741980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7aa32640 1 -- 172.21.15.12:0/2183079350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7419f790 con 0x7f0f74102320 2024-05-09T09:25:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f79a30640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f74101960 0x7f0f741980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7aa32640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f74102320 0x7f0f7419f120 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00cce0 tx=0x7f0f6c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f6b7fe640 1 -- 172.21.15.12:0/2183079350 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c00f450 con 0x7f0f74102320 2024-05-09T09:25:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f6b7fe640 1 -- 172.21.15.12:0/2183079350 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0f6c00fac0 con 0x7f0f74102320 2024-05-09T09:25:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7c495640 1 -- 172.21.15.12:0/2183079350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7419fa80 con 0x7f0f74102320 2024-05-09T09:25:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f6b7fe640 1 -- 172.21.15.12:0/2183079350 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c018be0 con 0x7f0f74102320 2024-05-09T09:25:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.748+0000 7f0f7c495640 1 -- 172.21.15.12:0/2183079350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f7419ff80 con 0x7f0f74102320 2024-05-09T09:25:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.751+0000 7f0f6b7fe640 1 -- 172.21.15.12:0/2183079350 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0f6c00fc70 con 0x7f0f74102320 2024-05-09T09:25:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.751+0000 7f0f7f19b740 1 Processor -- start 2024-05-09T09:25:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.751+0000 7f0f7f19b740 1 -- start start 2024-05-09T09:25:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f7f19b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad0fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f7f19b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad100db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f7f19b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 0x55a9ad1038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f7f19b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a9ad05f4d0 con 0x55a9ad0fb8c0 2024-05-09T09:25:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f7f19b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a9ad0073e0 con 0x55a9ad101420 2024-05-09T09:25:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f7f19b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a9ad05b490 con 0x55a9ad0fc2b0 2024-05-09T09:25:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f79a30640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad0fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f79a30640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad0fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39250/0 (socket says 172.21.15.12:39250) 2024-05-09T09:25:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f79a30640 1 -- 172.21.15.12:0/3004232936 learned_addr learned my addr 172.21.15.12:0/3004232936 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f7a231640 1 --2- 172.21.15.12:0/3004232936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad100db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f6b7fe640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0f600a2b90 0x7f0f600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f79a30640 1 --2- 172.21.15.12:0/3004232936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 0x55a9ad1038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f7aa32640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0f600a2b90 0x7f0f600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f79a30640 1 -- 172.21.15.12:0/3004232936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 msgr2=0x55a9ad100db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f79a30640 1 --2- 172.21.15.12:0/3004232936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad100db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f79a30640 1 -- 172.21.15.12:0/3004232936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 msgr2=0x55a9ad1038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f79a30640 1 --2- 172.21.15.12:0/3004232936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 0x55a9ad1038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f79a30640 1 -- 172.21.15.12:0/3004232936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9acd732d0 con 0x55a9ad0fb8c0 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f6b7fe640 1 -- 172.21.15.12:0/2183079350 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f0f6c0cfba0 con 0x7f0f74102320 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f79a30640 1 --2- 172.21.15.12:0/3004232936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad0fbce0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0f64005ab0 tx=0x7f0f64009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e04d751c795a48f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.752+0000 7f0f68bfe640 1 -- 172.21.15.12:0/3004232936 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64044020 con 0x55a9ad0fb8c0 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.753+0000 7f0f68bfe640 1 -- 172.21.15.12:0/3004232936 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0f64005020 con 0x55a9ad0fb8c0 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.753+0000 7f0f68bfe640 1 -- 172.21.15.12:0/3004232936 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64035070 con 0x55a9ad0fb8c0 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.753+0000 7f0f7f19b740 1 -- 172.21.15.12:0/3004232936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 msgr2=0x55a9ad0fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.753+0000 7f0f7aa32640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0f600a2b90 0x7f0f600a5050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c098490 tx=0x7f0f6c066c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.753+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/3004232936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad0fbce0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0f64005ab0 tx=0x7f0f64009560 comp rx=0 tx=0).stop 2024-05-09T09:25:23.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.753+0000 7f0f7f19b740 1 -- 172.21.15.12:0/3004232936 shutdown_connections 2024-05-09T09:25:23.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.753+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/3004232936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 0x55a9ad1038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.753+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/3004232936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.753+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/3004232936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad0fbce0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.753+0000 7f0f7f19b740 1 -- 172.21.15.12:0/3004232936 >> 172.21.15.12:0/3004232936 conn(0x55a9ad0f6f30 msgr2=0x55a9ad0f7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.753+0000 7f0f7f19b740 1 -- 172.21.15.12:0/3004232936 shutdown_connections 2024-05-09T09:25:23.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.753+0000 7f0f7f19b740 1 -- 172.21.15.12:0/3004232936 wait complete. 2024-05-09T09:25:23.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7f19b740 1 -- start start 2024-05-09T09:25:23.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7f19b740 1 Processor -- start 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7f19b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad105ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7f19b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad106220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7f19b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 0x55a9ad10d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7f19b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a9ad104370 con 0x55a9ad0fc2b0 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7f19b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a9ad01a8e0 con 0x55a9ad101420 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7f19b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a9ad1044f0 con 0x55a9ad0fb8c0 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7f19b740 10 client.0 _ll_register_callbacks cb 0x55a9acfe79c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7a231640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad106220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7a231640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad106220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39256/0 (socket says 172.21.15.12:39256) 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7a231640 1 -- 172.21.15.12:0/3291640069 learned_addr learned my addr 172.21.15.12:0/3291640069 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f79a30640 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad105ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f79a30640 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 0x55a9ad10d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7a231640 1 -- 172.21.15.12:0/3291640069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fb8c0 msgr2=0x55a9ad105ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7a231640 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad105ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7a231640 1 -- 172.21.15.12:0/3291640069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 msgr2=0x55a9ad10d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7a231640 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 0x55a9ad10d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f7a231640 1 -- 172.21.15.12:0/3291640069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9ace3d6b0 con 0x55a9ad0fc2b0 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.754+0000 7f0f79a30640 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 0x55a9ad10d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f7a231640 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad106220 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000be40 tx=0x7f0f70012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3d71eeef1205d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f5dffb640 1 -- 172.21.15.12:0/3291640069 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f700035f0 con 0x55a9ad0fc2b0 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f5dffb640 1 -- 172.21.15.12:0/3291640069 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f0f70003790 con 0x55a9ad0fc2b0 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f7f19b740 1 -- 172.21.15.12:0/3291640069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f64005020 con 0x55a9ad0fc2b0 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f5dffb640 1 -- 172.21.15.12:0/3291640069 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0f70003990 con 0x55a9ad0fc2b0 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f7f19b740 10 client.15429 fetch_fsmap learned FSMap version 28 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f7f19b740 1 -- 172.21.15.12:0/3291640069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a9ad110300 con 0x55a9ad0fc2b0 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f5dffb640 1 -- 172.21.15.12:0/3291640069 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0f64005020 con 0x55a9ad0fc2b0 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f5dffb640 1 -- 172.21.15.12:0/3291640069 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0f70019070 con 0x55a9ad0fc2b0 2024-05-09T09:25:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f7f19b740 10 client.15429 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f7f19b740 10 client.15429 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f7f19b740 20 client.15429 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55a9ad112980 0x55a9ad113df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f7f19b740 4 client.15429 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.755+0000 7f0f7f19b740 1 -- 172.21.15.12:0/3291640069 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9ad06cc00 con 0x55a9ad112980 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.756+0000 7f0f79a30640 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55a9ad112980 0x55a9ad113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.756+0000 7f0f79a30640 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55a9ad112980 0x55a9ad113df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=41851d7a8e917eb7 server_cookie=8bf3abe8a56a05ce in_seq=0 out_seq=0 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.756+0000 7f0f5dffb640 10 client.15429 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.756+0000 7f0f5dffb640 1 -- 172.21.15.12:0/3291640069 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9ad06cc00 con 0x55a9ad112980 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.756+0000 7f0f5dffb640 10 client.15429 handle_command_reply: tid=0 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.756+0000 7f0f7f19b740 1 client.15429 shutdown 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.757+0000 7f0f7f19b740 1 -- 172.21.15.12:0/3291640069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fc2b0 msgr2=0x55a9ad106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.757+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad106220 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000be40 tx=0x7f0f70012820 comp rx=0 tx=0).stop 2024-05-09T09:25:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.757+0000 7f0f7f19b740 1 -- 172.21.15.12:0/3291640069 shutdown_connections 2024-05-09T09:25:23.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.757+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55a9ad112980 0x55a9ad113df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.757+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 0x55a9ad10d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.757+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad106220 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.757+0000 7f0f5dffb640 0 client.15429 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:25:23.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.757+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/3291640069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad105ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.757+0000 7f0f7f19b740 1 -- 172.21.15.12:0/3291640069 >> 172.21.15.12:0/3291640069 conn(0x55a9ad0f6f30 msgr2=0x55a9ad0f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.757+0000 7f0f7f19b740 1 -- 172.21.15.12:0/3291640069 shutdown_connections 2024-05-09T09:25:23.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.757+0000 7f0f7f19b740 1 -- 172.21.15.12:0/3291640069 wait complete. 2024-05-09T09:25:23.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.757+0000 7f0f7f19b740 20 client.15429 trim_cache size 0 max 16384 2024-05-09T09:25:23.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.775+0000 7f0f7f19b740 1 Processor -- start 2024-05-09T09:25:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.775+0000 7f0f7f19b740 1 -- start start 2024-05-09T09:25:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.775+0000 7f0f7f19b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad0f9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.775+0000 7f0f7f19b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad10d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f7f19b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 0x55a9ad0fa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f7f19b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a9ad112130 con 0x55a9ad0fb8c0 2024-05-09T09:25:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f7f19b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a9ad112800 con 0x55a9ad101420 2024-05-09T09:25:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f7f19b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a9ad112ed0 con 0x55a9ad0fc2b0 2024-05-09T09:25:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f79a30640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad0f9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f79a30640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad0f9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39258/0 (socket says 172.21.15.12:39258) 2024-05-09T09:25:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f79a30640 1 -- 172.21.15.12:0/2857813651 learned_addr learned my addr 172.21.15.12:0/2857813651 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f79a30640 1 -- 172.21.15.12:0/2857813651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 msgr2=0x55a9ad10d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f79a30640 1 --2- 172.21.15.12:0/2857813651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad10d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f79a30640 1 -- 172.21.15.12:0/2857813651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 msgr2=0x55a9ad0fa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f79a30640 1 --2- 172.21.15.12:0/2857813651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 0x55a9ad0fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f79a30640 1 -- 172.21.15.12:0/2857813651 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9ad0f6ad0 con 0x55a9ad0fb8c0 2024-05-09T09:25:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f79a30640 1 --2- 172.21.15.12:0/2857813651 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad0f9fc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009900 tx=0x7f0f640058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285eaef09ee44fba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f5ffff640 1 -- 172.21.15.12:0/2857813651 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6404e020 con 0x55a9ad0fb8c0 2024-05-09T09:25:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f5ffff640 1 -- 172.21.15.12:0/2857813651 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0f6404f220 con 0x55a9ad0fb8c0 2024-05-09T09:25:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.776+0000 7f0f5ffff640 1 -- 172.21.15.12:0/2857813651 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64035040 con 0x55a9ad0fb8c0 2024-05-09T09:25:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 -- 172.21.15.12:0/2857813651 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 msgr2=0x55a9ad0f9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/2857813651 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad0f9fc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009900 tx=0x7f0f640058b0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 -- 172.21.15.12:0/2857813651 shutdown_connections 2024-05-09T09:25:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/2857813651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad101420 0x55a9ad0fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/2857813651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad10d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/2857813651 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad0f9fc0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 -- 172.21.15.12:0/2857813651 >> 172.21.15.12:0/2857813651 conn(0x55a9ad0f6f30 msgr2=0x55a9ad0f7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 -- 172.21.15.12:0/2857813651 shutdown_connections 2024-05-09T09:25:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 -- 172.21.15.12:0/2857813651 wait complete. 2024-05-09T09:25:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 -- start start 2024-05-09T09:25:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 Processor -- start 2024-05-09T09:25:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad174b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad1750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad101420 0x55a9ad17b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a9ad113c70 con 0x55a9ad101420 2024-05-09T09:25:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a9ad1135a0 con 0x55a9ad0fb8c0 2024-05-09T09:25:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a9ad114340 con 0x55a9ad0fc2b0 2024-05-09T09:25:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f7f19b740 10 client.0 _ll_register_callbacks cb 0x7f0f7000b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f79a30640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad101420 0x55a9ad17b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f79a30640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad101420 0x55a9ad17b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39266/0 (socket says 172.21.15.12:39266) 2024-05-09T09:25:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f79a30640 1 -- 172.21.15.12:0/2155187343 learned_addr learned my addr 172.21.15.12:0/2155187343 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.777+0000 7f0f79a30640 1 --2- 172.21.15.12:0/2155187343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad174b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f7a231640 1 --2- 172.21.15.12:0/2155187343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad1750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f79a30640 1 -- 172.21.15.12:0/2155187343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 msgr2=0x55a9ad1750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f79a30640 1 --2- 172.21.15.12:0/2155187343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad1750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f79a30640 1 -- 172.21.15.12:0/2155187343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad0fb8c0 msgr2=0x55a9ad174b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f79a30640 1 --2- 172.21.15.12:0/2155187343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad174b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f79a30640 1 -- 172.21.15.12:0/2155187343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9ad17bf50 con 0x55a9ad101420 2024-05-09T09:25:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f79a30640 1 --2- 172.21.15.12:0/2155187343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad101420 0x55a9ad17b990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0f64031b70 tx=0x7f0f6402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8eefe13a148c57 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f5dffb640 1 -- 172.21.15.12:0/2155187343 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64061060 con 0x55a9ad101420 2024-05-09T09:25:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f5dffb640 1 -- 172.21.15.12:0/2155187343 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f0f64005300 con 0x55a9ad101420 2024-05-09T09:25:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f7f19b740 1 -- 172.21.15.12:0/2155187343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f6404f220 con 0x55a9ad101420 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f5dffb640 1 -- 172.21.15.12:0/2155187343 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0f6404e040 con 0x55a9ad101420 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f7f19b740 10 client.15435 fetch_fsmap learned FSMap version 28 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f7f19b740 1 -- 172.21.15.12:0/2155187343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a9ad17d250 con 0x55a9ad101420 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.778+0000 7f0f5dffb640 1 -- 172.21.15.12:0/2155187343 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0f6404f220 con 0x55a9ad101420 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f5dffb640 1 -- 172.21.15.12:0/2155187343 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0f6405a040 con 0x55a9ad101420 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 10 client.15435 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 10 client.15435 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 20 client.15435 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/2155187343 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55a9ad181820 0x55a9ad17e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 4 client.15435 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 1 -- 172.21.15.12:0/2155187343 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "status"}) -- 0x55a9ad066f50 con 0x55a9ad181820 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7a231640 1 --2- 172.21.15.12:0/2155187343 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55a9ad181820 0x55a9ad17e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f5dffb640 10 client.15435 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:25:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7a231640 1 --2- 172.21.15.12:0/2155187343 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55a9ad181820 0x55a9ad17e330 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2455df3ef2106dc8 server_cookie=e45ed1130868e4e1 in_seq=0 out_seq=0 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f5dffb640 1 -- 172.21.15.12:0/2155187343 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a9ad066f50 con 0x55a9ad181820 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f5dffb640 10 client.15435 handle_command_reply: tid=0 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 1 client.15435 shutdown 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 1 -- 172.21.15.12:0/2155187343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad101420 msgr2=0x55a9ad17b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/2155187343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad101420 0x55a9ad17b990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0f64031b70 tx=0x7f0f6402a3f0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 1 -- 172.21.15.12:0/2155187343 shutdown_connections 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/2155187343 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55a9ad181820 0x55a9ad17e330 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/2155187343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9ad101420 0x55a9ad17b990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/2155187343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9ad0fc2b0 0x55a9ad1750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f5dffb640 0 client.15435 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 1 --2- 172.21.15.12:0/2155187343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9ad0fb8c0 0x55a9ad174b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.779+0000 7f0f7f19b740 1 -- 172.21.15.12:0/2155187343 >> 172.21.15.12:0/2155187343 conn(0x55a9ad0f6f30 msgr2=0x55a9ad0f9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7f19b740 1 -- 172.21.15.12:0/2155187343 shutdown_connections 2024-05-09T09:25:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7f19b740 1 -- 172.21.15.12:0/2155187343 wait complete. 2024-05-09T09:25:23.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7f19b740 20 client.15435 trim_cache size 0 max 16384 2024-05-09T09:25:23.787 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:25:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:25:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 4, 2024-05-09T09:25:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 14628, 2024-05-09T09:25:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-09T09:25:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-09T09:25:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-09T09:25:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 110.18828215400001, 2024-05-09T09:25:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-05-09T09:25:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 93, 2024-05-09T09:25:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T09:25:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 142.692283145 2024-05-09T09:25:23.788 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:25:23.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7c495640 1 -- 172.21.15.12:0/2183079350 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0f600a2b90 msgr2=0x7f0f600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7c495640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0f600a2b90 0x7f0f600a5050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c098490 tx=0x7f0f6c066c00 comp rx=0 tx=0).stop 2024-05-09T09:25:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7c495640 1 -- 172.21.15.12:0/2183079350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f74102320 msgr2=0x7f0f7419f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7c495640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f74102320 0x7f0f7419f120 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00cce0 tx=0x7f0f6c00a620 comp rx=0 tx=0).stop 2024-05-09T09:25:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7c495640 1 -- 172.21.15.12:0/2183079350 shutdown_connections 2024-05-09T09:25:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7c495640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0f600a2b90 0x7f0f600a5050 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7c495640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f74102320 0x7f0f7419f120 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7c495640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f74101960 0x7f0f741980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7c495640 1 --2- 172.21.15.12:0/2183079350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f74100770 0x7f0f74197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7c495640 1 -- 172.21.15.12:0/2183079350 >> 172.21.15.12:0/2183079350 conn(0x7f0f740fbce0 msgr2=0x7f0f740fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7c495640 1 -- 172.21.15.12:0/2183079350 shutdown_connections 2024-05-09T09:25:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.780+0000 7f0f7c495640 1 -- 172.21.15.12:0/2183079350 wait complete. 2024-05-09T09:25:23.791 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-05-09T09:25:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.897+0000 7f4a43f48640 1 -- 172.21.15.12:0/1469124917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a3c102a80 msgr2=0x7f4a3c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.897+0000 7f4a43f48640 1 --2- 172.21.15.12:0/1469124917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a3c102a80 0x7f4a3c102f00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009900 tx=0x7f4a2c02a410 comp rx=0 tx=0).stop 2024-05-09T09:25:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.897+0000 7f4a43f48640 1 -- 172.21.15.12:0/1469124917 shutdown_connections 2024-05-09T09:25:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.897+0000 7f4a43f48640 1 --2- 172.21.15.12:0/1469124917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c103440 0x7f4a3c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.897+0000 7f4a43f48640 1 --2- 172.21.15.12:0/1469124917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a3c102a80 0x7f4a3c102f00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.897+0000 7f4a43f48640 1 --2- 172.21.15.12:0/1469124917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c101890 0x7f4a3c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.897+0000 7f4a43f48640 1 -- 172.21.15.12:0/1469124917 >> 172.21.15.12:0/1469124917 conn(0x7f4a3c0fce00 msgr2=0x7f4a3c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.897+0000 7f4a43f48640 1 -- 172.21.15.12:0/1469124917 shutdown_connections 2024-05-09T09:25:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.897+0000 7f4a43f48640 1 -- 172.21.15.12:0/1469124917 wait complete. 2024-05-09T09:25:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a43f48640 1 Processor -- start 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a43f48640 1 -- start start 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a43f48640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c101890 0x7f4a3c0755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a43f48640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a3c102a80 0x7f4a3c073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a43f48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c103440 0x7f4a3c074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a43f48640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c10d4f0 con 0x7f4a3c102a80 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a43f48640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c10d370 con 0x7f4a3c103440 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a43f48640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c10d670 con 0x7f4a3c101890 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a414e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a3c102a80 0x7f4a3c073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a414e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a3c102a80 0x7f4a3c073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39278/0 (socket says 172.21.15.12:39278) 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a414e3640 1 -- 172.21.15.12:0/3647464968 learned_addr learned my addr 172.21.15.12:0/3647464968 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a41ce4640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c101890 0x7f4a3c0755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a424e5640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c103440 0x7f4a3c074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a414e3640 1 -- 172.21.15.12:0/3647464968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c101890 msgr2=0x7f4a3c0755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a414e3640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c101890 0x7f4a3c0755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a414e3640 1 -- 172.21.15.12:0/3647464968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c103440 msgr2=0x7f4a3c074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a414e3640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c103440 0x7f4a3c074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.898+0000 7f4a414e3640 1 -- 172.21.15.12:0/3647464968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c0749f0 con 0x7f4a3c102a80 2024-05-09T09:25:23.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.899+0000 7f4a414e3640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a3c102a80 0x7f4a3c073c20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009230 tx=0x7f4a2c031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.899+0000 7f4a32ffd640 1 -- 172.21.15.12:0/3647464968 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c03f030 con 0x7f4a3c102a80 2024-05-09T09:25:23.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.899+0000 7f4a43f48640 1 -- 172.21.15.12:0/3647464968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3c1a1c80 con 0x7f4a3c102a80 2024-05-09T09:25:23.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.899+0000 7f4a32ffd640 1 -- 172.21.15.12:0/3647464968 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4a2c003ac0 con 0x7f4a3c102a80 2024-05-09T09:25:23.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.899+0000 7f4a32ffd640 1 -- 172.21.15.12:0/3647464968 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c035070 con 0x7f4a3c102a80 2024-05-09T09:25:23.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.899+0000 7f4a43f48640 1 -- 172.21.15.12:0/3647464968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3c1a21d0 con 0x7f4a3c102a80 2024-05-09T09:25:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.901+0000 7f4a32ffd640 1 -- 172.21.15.12:0/3647464968 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4a2c042030 con 0x7f4a3c102a80 2024-05-09T09:25:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.902+0000 7f4a46c4b740 1 Processor -- start 2024-05-09T09:25:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.902+0000 7f4a46c4b740 1 -- start start 2024-05-09T09:25:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.902+0000 7f4a46c4b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 0x564500eb7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.902+0000 7f4a46c4b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb82b0 0x564500ebcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.902+0000 7f4a46c4b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500ebf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.902+0000 7f4a46c4b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564500e1b4d0 con 0x564500ebd420 2024-05-09T09:25:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.902+0000 7f4a46c4b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564500dc33e0 con 0x564500eb82b0 2024-05-09T09:25:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.902+0000 7f4a46c4b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564500e17490 con 0x564500eb78c0 2024-05-09T09:25:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.902+0000 7f4a414e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500ebf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.902+0000 7f4a41ce4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb82b0 0x564500ebcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.902+0000 7f4a414e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500ebf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39288/0 (socket says 172.21.15.12:39288) 2024-05-09T09:25:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.902+0000 7f4a414e3640 1 -- 172.21.15.12:0/2217942602 learned_addr learned my addr 172.21.15.12:0/2217942602 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.902+0000 7f4a424e5640 1 --2- 172.21.15.12:0/2217942602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 0x564500eb7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.903+0000 7f4a32ffd640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4a280a2b90 0x7f4a280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.903+0000 7f4a41ce4640 1 -- 172.21.15.12:0/2217942602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 msgr2=0x564500eb7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.903+0000 7f4a41ce4640 1 --2- 172.21.15.12:0/2217942602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 0x564500eb7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.903+0000 7f4a41ce4640 1 -- 172.21.15.12:0/2217942602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 msgr2=0x564500ebf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.903+0000 7f4a41ce4640 1 --2- 172.21.15.12:0/2217942602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500ebf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.903+0000 7f4a41ce4640 1 -- 172.21.15.12:0/2217942602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564500b2f2d0 con 0x564500eb82b0 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.903+0000 7f4a424e5640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4a280a2b90 0x7f4a280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.903+0000 7f4a32ffd640 1 -- 172.21.15.12:0/3647464968 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f4a2c0ef990 con 0x7f4a3c102a80 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.903+0000 7f4a41ce4640 1 --2- 172.21.15.12:0/2217942602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb82b0 0x564500ebcdb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009bd0 tx=0x7f4a38011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87bbe0e360658163 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.903+0000 7f4a27fff640 1 -- 172.21.15.12:0/2217942602 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38004230 con 0x564500eb82b0 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.903+0000 7f4a27fff640 1 -- 172.21.15.12:0/2217942602 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4a38009e40 con 0x564500eb82b0 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.903+0000 7f4a27fff640 1 -- 172.21.15.12:0/2217942602 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38019820 con 0x564500eb82b0 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.904+0000 7f4a46c4b740 1 -- 172.21.15.12:0/2217942602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb82b0 msgr2=0x564500ebcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.904+0000 7f4a424e5640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4a280a2b90 0x7f4a280a5050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x564500eb8b10 tx=0x7f4a34004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.904+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/2217942602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb82b0 0x564500ebcdb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009bd0 tx=0x7f4a38011810 comp rx=0 tx=0).stop 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.904+0000 7f4a46c4b740 1 -- 172.21.15.12:0/2217942602 shutdown_connections 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.904+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/2217942602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500ebf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.904+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/2217942602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb82b0 0x564500ebcdb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.904+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/2217942602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 0x564500eb7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.904+0000 7f4a46c4b740 1 -- 172.21.15.12:0/2217942602 >> 172.21.15.12:0/2217942602 conn(0x564500eb2f30 msgr2=0x564500eb3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.904+0000 7f4a46c4b740 1 -- 172.21.15.12:0/2217942602 shutdown_connections 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.904+0000 7f4a46c4b740 1 -- 172.21.15.12:0/2217942602 wait complete. 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.904+0000 7f4a46c4b740 1 -- start start 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.904+0000 7f4a46c4b740 1 Processor -- start 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a46c4b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 0x564500ec1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a46c4b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb82b0 0x564500ec2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a46c4b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500ec9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a46c4b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564500ec03d0 con 0x564500ebd420 2024-05-09T09:25:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a46c4b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564500dd68e0 con 0x564500eb82b0 2024-05-09T09:25:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a46c4b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564500ec0550 con 0x564500eb78c0 2024-05-09T09:25:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a46c4b740 10 client.0 _ll_register_callbacks cb 0x564500da39c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a424e5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500ec9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a424e5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500ec9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39304/0 (socket says 172.21.15.12:39304) 2024-05-09T09:25:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a424e5640 1 -- 172.21.15.12:0/1896420709 learned_addr learned my addr 172.21.15.12:0/1896420709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a41ce4640 1 --2- 172.21.15.12:0/1896420709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 0x564500ec1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a414e3640 1 --2- 172.21.15.12:0/1896420709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb82b0 0x564500ec2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a424e5640 1 -- 172.21.15.12:0/1896420709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 msgr2=0x564500ec1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a424e5640 1 --2- 172.21.15.12:0/1896420709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 0x564500ec1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a424e5640 1 -- 172.21.15.12:0/1896420709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb82b0 msgr2=0x564500ec2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a424e5640 1 --2- 172.21.15.12:0/1896420709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb82b0 0x564500ec2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a424e5640 1 -- 172.21.15.12:0/1896420709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564500bf96b0 con 0x564500ebd420 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a424e5640 1 --2- 172.21.15.12:0/1896420709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500ec9240 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4a34010a00 tx=0x7f4a34004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7591913cf967f379 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.905+0000 7f4a257fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a257fa640 1 -- 172.21.15.12:0/1896420709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3402b060 con 0x564500ebd420 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a46c4b740 1 -- 172.21.15.12:0/1896420709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564500e28c00 con 0x564500ebd420 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a257fa640 1 -- 172.21.15.12:0/1896420709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f4a34018070 con 0x564500ebd420 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a257fa640 1 -- 172.21.15.12:0/1896420709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a34032050 con 0x564500ebd420 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a46c4b740 10 client.15447 fetch_fsmap learned FSMap version 28 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a46c4b740 1 -- 172.21.15.12:0/1896420709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564500ecc2f0 con 0x564500ebd420 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a257fa640 1 -- 172.21.15.12:0/1896420709 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x564500e28c00 con 0x564500ebd420 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a257fa640 1 -- 172.21.15.12:0/1896420709 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a3400f040 con 0x564500ebd420 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a46c4b740 10 client.15447 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a46c4b740 10 client.15447 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T09:25:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a46c4b740 20 client.15447 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/1896420709 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x564500ece9a0 0x564500ecfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a46c4b740 4 client.15447 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a46c4b740 1 -- 172.21.15.12:0/1896420709 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564500e22f50 con 0x564500ece9a0 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.906+0000 7f4a41ce4640 1 --2- 172.21.15.12:0/1896420709 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x564500ece9a0 0x564500ecfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.907+0000 7f4a41ce4640 1 --2- 172.21.15.12:0/1896420709 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x564500ece9a0 0x564500ecfe10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=216b5b1418a4355c server_cookie=e7f2261d7583dd15 in_seq=0 out_seq=0 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.907+0000 7f4a257fa640 10 client.15447 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a257fa640 1 -- 172.21.15.12:0/1896420709 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564500e22f50 con 0x564500ece9a0 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a257fa640 10 client.15447 handle_command_reply: tid=0 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a46c4b740 1 client.15447 shutdown 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a46c4b740 1 -- 172.21.15.12:0/1896420709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 msgr2=0x564500ec9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/1896420709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500ec9240 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4a34010a00 tx=0x7f4a34004ab0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a46c4b740 1 -- 172.21.15.12:0/1896420709 shutdown_connections 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/1896420709 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x564500ece9a0 0x564500ecfe10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/1896420709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500ec9240 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a257fa640 0 client.15447 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:25:23.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/1896420709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb82b0 0x564500ec2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/1896420709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 0x564500ec1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a46c4b740 1 -- 172.21.15.12:0/1896420709 >> 172.21.15.12:0/1896420709 conn(0x564500eb2f30 msgr2=0x564500eb2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a46c4b740 1 -- 172.21.15.12:0/1896420709 shutdown_connections 2024-05-09T09:25:23.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a46c4b740 1 -- 172.21.15.12:0/1896420709 wait complete. 2024-05-09T09:25:23.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.908+0000 7f4a46c4b740 20 client.15447 trim_cache size 0 max 16384 2024-05-09T09:25:23.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.926+0000 7f4a46c4b740 1 Processor -- start 2024-05-09T09:25:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.926+0000 7f4a46c4b740 1 -- start start 2024-05-09T09:25:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.926+0000 7f4a46c4b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 0x564500ebcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a46c4b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500eb82b0 0x564500ebf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a46c4b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500ebd420 0x564500ec9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a46c4b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564500ece150 con 0x564500eb82b0 2024-05-09T09:25:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a46c4b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564500ece820 con 0x564500ebd420 2024-05-09T09:25:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a46c4b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564500eceef0 con 0x564500eb78c0 2024-05-09T09:25:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a414e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500eb82b0 0x564500ebf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a414e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500eb82b0 0x564500ebf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39320/0 (socket says 172.21.15.12:39320) 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a414e3640 1 -- 172.21.15.12:0/2187017021 learned_addr learned my addr 172.21.15.12:0/2187017021 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a414e3640 1 -- 172.21.15.12:0/2187017021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 msgr2=0x564500ebcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a414e3640 1 --2- 172.21.15.12:0/2187017021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 0x564500ebcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a414e3640 1 -- 172.21.15.12:0/2187017021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500ebd420 msgr2=0x564500ec9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a414e3640 1 --2- 172.21.15.12:0/2187017021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500ebd420 0x564500ec9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a414e3640 1 -- 172.21.15.12:0/2187017021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564500ec0710 con 0x564500eb82b0 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a414e3640 1 --2- 172.21.15.12:0/2187017021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500eb82b0 0x564500ebf810 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0b88a0 tx=0x7f4a2c0efb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9234ebf4580cb3b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a277fe640 1 -- 172.21.15.12:0/2187017021 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c049040 con 0x564500eb82b0 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a277fe640 1 -- 172.21.15.12:0/2187017021 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4a2c0bbed0 con 0x564500eb82b0 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.927+0000 7f4a277fe640 1 -- 172.21.15.12:0/2187017021 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0ba1d0 con 0x564500eb82b0 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.928+0000 7f4a46c4b740 1 -- 172.21.15.12:0/2187017021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500eb82b0 msgr2=0x564500ebf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.928+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/2187017021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500eb82b0 0x564500ebf810 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0b88a0 tx=0x7f4a2c0efb90 comp rx=0 tx=0).stop 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.928+0000 7f4a46c4b740 1 -- 172.21.15.12:0/2187017021 shutdown_connections 2024-05-09T09:25:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.928+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/2187017021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500ebd420 0x564500ec9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.928+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/2187017021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500eb82b0 0x564500ebf810 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.928+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/2187017021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb78c0 0x564500ebcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.928+0000 7f4a46c4b740 1 -- 172.21.15.12:0/2187017021 >> 172.21.15.12:0/2187017021 conn(0x564500eb2f30 msgr2=0x564500eb3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.928+0000 7f4a46c4b740 1 -- 172.21.15.12:0/2187017021 shutdown_connections 2024-05-09T09:25:23.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.928+0000 7f4a46c4b740 1 -- 172.21.15.12:0/2187017021 wait complete. 2024-05-09T09:25:23.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.928+0000 7f4a46c4b740 1 -- start start 2024-05-09T09:25:23.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.928+0000 7f4a46c4b740 1 Processor -- start 2024-05-09T09:25:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a46c4b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb78c0 0x564500ec4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a46c4b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb82b0 0x564500ec48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a46c4b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500f32940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a46c4b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564500ecfc90 con 0x564500ebd420 2024-05-09T09:25:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a46c4b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564500ecf5c0 con 0x564500eb78c0 2024-05-09T09:25:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a46c4b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564500ed0360 con 0x564500eb82b0 2024-05-09T09:25:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a46c4b740 10 client.0 _ll_register_callbacks cb 0x7f4a34009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a424e5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500f32940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a424e5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500f32940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39334/0 (socket says 172.21.15.12:39334) 2024-05-09T09:25:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a424e5640 1 -- 172.21.15.12:0/3979846344 learned_addr learned my addr 172.21.15.12:0/3979846344 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a414e3640 1 --2- 172.21.15.12:0/3979846344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb82b0 0x564500ec48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a424e5640 1 -- 172.21.15.12:0/3979846344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb82b0 msgr2=0x564500ec48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a424e5640 1 --2- 172.21.15.12:0/3979846344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb82b0 0x564500ec48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a41ce4640 1 --2- 172.21.15.12:0/3979846344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb78c0 0x564500ec4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a424e5640 1 -- 172.21.15.12:0/3979846344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb78c0 msgr2=0x564500ec4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a424e5640 1 --2- 172.21.15.12:0/3979846344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb78c0 0x564500ec4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a424e5640 1 -- 172.21.15.12:0/3979846344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564500f32f00 con 0x564500ebd420 2024-05-09T09:25:23.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a424e5640 1 --2- 172.21.15.12:0/3979846344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500f32940 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4a34004a10 tx=0x7f4a3401d3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a4b02209bfa7f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a257fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:23.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a257fa640 1 -- 172.21.15.12:0/3979846344 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34013040 con 0x564500ebd420 2024-05-09T09:25:23.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a46c4b740 1 -- 172.21.15.12:0/3979846344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a2c0bbed0 con 0x564500ebd420 2024-05-09T09:25:23.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.929+0000 7f4a257fa640 1 -- 172.21.15.12:0/3979846344 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f4a340131e0 con 0x564500ebd420 2024-05-09T09:25:23.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.930+0000 7f4a257fa640 1 -- 172.21.15.12:0/3979846344 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a3401b420 con 0x564500ebd420 2024-05-09T09:25:23.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.930+0000 7f4a46c4b740 10 client.15453 fetch_fsmap learned FSMap version 28 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.930+0000 7f4a46c4b740 1 -- 172.21.15.12:0/3979846344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564500f34200 con 0x564500ebd420 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.930+0000 7f4a257fa640 1 -- 172.21.15.12:0/3979846344 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4a2c0bbed0 con 0x564500ebd420 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.930+0000 7f4a257fa640 1 -- 172.21.15.12:0/3979846344 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a340126d0 con 0x564500ebd420 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.930+0000 7f4a46c4b740 10 client.15453 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.930+0000 7f4a46c4b740 10 client.15453 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.930+0000 7f4a46c4b740 20 client.15453 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.930+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/3979846344 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x564500f38e30 0x564500f37b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.930+0000 7f4a46c4b740 4 client.15453 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.930+0000 7f4a46c4b740 1 -- 172.21.15.12:0/3979846344 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "status"}) -- 0x564500da0b80 con 0x564500f38e30 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.930+0000 7f4a41ce4640 1 --2- 172.21.15.12:0/3979846344 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x564500f38e30 0x564500f37b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a257fa640 10 client.15453 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a41ce4640 1 --2- 172.21.15.12:0/3979846344 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x564500f38e30 0x564500f37b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a75c9b9cc468880 server_cookie=e02949a8eca43c46 in_seq=0 out_seq=0 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a257fa640 1 -- 172.21.15.12:0/3979846344 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564500da0b80 con 0x564500f38e30 2024-05-09T09:25:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a257fa640 10 client.15453 handle_command_reply: tid=0 2024-05-09T09:25:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a46c4b740 1 client.15453 shutdown 2024-05-09T09:25:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a46c4b740 1 -- 172.21.15.12:0/3979846344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 msgr2=0x564500f32940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/3979846344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500f32940 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4a34004a10 tx=0x7f4a3401d3d0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a46c4b740 1 -- 172.21.15.12:0/3979846344 shutdown_connections 2024-05-09T09:25:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/3979846344 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x564500f38e30 0x564500f37b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/3979846344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564500ebd420 0x564500f32940 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/3979846344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564500eb82b0 0x564500ec48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a257fa640 0 client.15453 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:25:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a46c4b740 1 --2- 172.21.15.12:0/3979846344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564500eb78c0 0x564500ec4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a46c4b740 1 -- 172.21.15.12:0/3979846344 >> 172.21.15.12:0/3979846344 conn(0x564500eb2f30 msgr2=0x564500ebc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a46c4b740 1 -- 172.21.15.12:0/3979846344 shutdown_connections 2024-05-09T09:25:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a46c4b740 1 -- 172.21.15.12:0/3979846344 wait complete. 2024-05-09T09:25:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.931+0000 7f4a46c4b740 20 client.15453 trim_cache size 0 max 16384 2024-05-09T09:25:23.938 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:25:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:25:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 3, 2024-05-09T09:25:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24484, 2024-05-09T09:25:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-09T09:25:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-09T09:25:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-09T09:25:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 111.26314450300001, 2024-05-09T09:25:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-05-09T09:25:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 93, 2024-05-09T09:25:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T09:25:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 157.30447027599999 2024-05-09T09:25:23.939 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:25:23.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.932+0000 7f4a43f48640 1 -- 172.21.15.12:0/3647464968 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4a280a2b90 msgr2=0x7f4a280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.932+0000 7f4a43f48640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4a280a2b90 0x7f4a280a5050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x564500eb8b10 tx=0x7f4a34004a40 comp rx=0 tx=0).stop 2024-05-09T09:25:23.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.932+0000 7f4a43f48640 1 -- 172.21.15.12:0/3647464968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a3c102a80 msgr2=0x7f4a3c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.932+0000 7f4a43f48640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a3c102a80 0x7f4a3c073c20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009230 tx=0x7f4a2c031ac0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.932+0000 7f4a43f48640 1 -- 172.21.15.12:0/3647464968 shutdown_connections 2024-05-09T09:25:23.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.932+0000 7f4a43f48640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4a280a2b90 0x7f4a280a5050 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.932+0000 7f4a43f48640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c103440 0x7f4a3c074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.932+0000 7f4a43f48640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a3c102a80 0x7f4a3c073c20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.932+0000 7f4a43f48640 1 --2- 172.21.15.12:0/3647464968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c101890 0x7f4a3c0755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.932+0000 7f4a43f48640 1 -- 172.21.15.12:0/3647464968 >> 172.21.15.12:0/3647464968 conn(0x7f4a3c0fce00 msgr2=0x7f4a3c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.932+0000 7f4a43f48640 1 -- 172.21.15.12:0/3647464968 shutdown_connections 2024-05-09T09:25:23.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:23.932+0000 7f4a43f48640 1 -- 172.21.15.12:0/3647464968 wait complete. 2024-05-09T09:25:23.943 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-09T09:25:24.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.048+0000 7f9afcde2640 1 -- 172.21.15.12:0/2147684013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9af81075f0 msgr2=0x7f9af81099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.048+0000 7f9afcde2640 1 --2- 172.21.15.12:0/2147684013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9af81075f0 0x7f9af81099e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009ca0 tx=0x7f9af002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.048+0000 7f9afcde2640 1 -- 172.21.15.12:0/2147684013 shutdown_connections 2024-05-09T09:25:24.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.048+0000 7f9afcde2640 1 --2- 172.21.15.12:0/2147684013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9af81075f0 0x7f9af81099e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.048+0000 7f9afcde2640 1 --2- 172.21.15.12:0/2147684013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af8104cc0 0x7f9af81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.048+0000 7f9afcde2640 1 --2- 172.21.15.12:0/2147684013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9af8102390 0x7f9af8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.048+0000 7f9afcde2640 1 -- 172.21.15.12:0/2147684013 >> 172.21.15.12:0/2147684013 conn(0x7f9af80fbd20 msgr2=0x7f9af80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.048+0000 7f9afcde2640 1 -- 172.21.15.12:0/2147684013 shutdown_connections 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.048+0000 7f9afcde2640 1 -- 172.21.15.12:0/2147684013 wait complete. 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.049+0000 7f9afcde2640 1 Processor -- start 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.049+0000 7f9afcde2640 1 -- start start 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.049+0000 7f9afcde2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9af8102390 0x7f9af8101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.049+0000 7f9afcde2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9af8104cc0 0x7f9af8100340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.049+0000 7f9afcde2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af81075f0 0x7f9af8100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.049+0000 7f9afcde2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9af810d510 con 0x7f9af8102390 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.049+0000 7f9afcde2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9af810d390 con 0x7f9af81075f0 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.049+0000 7f9afcde2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9af810d690 con 0x7f9af8104cc0 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.049+0000 7f9af659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9af8102390 0x7f9af8101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.049+0000 7f9af659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9af8102390 0x7f9af8101d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39342/0 (socket says 172.21.15.12:39342) 2024-05-09T09:25:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.049+0000 7f9af659c640 1 -- 172.21.15.12:0/2752874535 learned_addr learned my addr 172.21.15.12:0/2752874535 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.049+0000 7f9af6d9d640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af81075f0 0x7f9af8100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.049+0000 7f9aeffff640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9af8104cc0 0x7f9af8100340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.050+0000 7f9af659c640 1 -- 172.21.15.12:0/2752874535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9af8104cc0 msgr2=0x7f9af8100340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.050+0000 7f9af659c640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9af8104cc0 0x7f9af8100340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.050+0000 7f9af659c640 1 -- 172.21.15.12:0/2752874535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af81075f0 msgr2=0x7f9af8100880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.050+0000 7f9af659c640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af81075f0 0x7f9af8100880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.050+0000 7f9af659c640 1 -- 172.21.15.12:0/2752874535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af8101110 con 0x7f9af8102390 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.050+0000 7f9aeffff640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9af8104cc0 0x7f9af8100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.050+0000 7f9af659c640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9af8102390 0x7f9af8101d00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800d770 tx=0x7f9ae800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.050+0000 7f9aef7fe640 1 -- 172.21.15.12:0/2752874535 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8017070 con 0x7f9af8102390 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.050+0000 7f9afcde2640 1 -- 172.21.15.12:0/2752874535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af81a1c20 con 0x7f9af8102390 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.050+0000 7f9aef7fe640 1 -- 172.21.15.12:0/2752874535 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9ae80054f0 con 0x7f9af8102390 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.050+0000 7f9aef7fe640 1 -- 172.21.15.12:0/2752874535 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8004d60 con 0x7f9af8102390 2024-05-09T09:25:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.050+0000 7f9afcde2640 1 -- 172.21.15.12:0/2752874535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af81a2170 con 0x7f9af8102390 2024-05-09T09:25:24.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.053+0000 7f9aef7fe640 1 -- 172.21.15.12:0/2752874535 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9ae8004f00 con 0x7f9af8102390 2024-05-09T09:25:24.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.053+0000 7f9affbc5740 1 Processor -- start 2024-05-09T09:25:24.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.053+0000 7f9affbc5740 1 -- start start 2024-05-09T09:25:24.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.053+0000 7f9affbc5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 0x55993bc33ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.053+0000 7f9affbc5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc342b0 0x55993bc38db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.053+0000 7f9affbc5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc39420 0x55993bc3b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.053+0000 7f9affbc5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55993bb974d0 con 0x55993bc338c0 2024-05-09T09:25:24.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.053+0000 7f9affbc5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55993bb3f3e0 con 0x55993bc39420 2024-05-09T09:25:24.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.053+0000 7f9affbc5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55993bb93490 con 0x55993bc342b0 2024-05-09T09:25:24.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.053+0000 7f9af6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 0x55993bc33ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.053+0000 7f9af6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 0x55993bc33ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39344/0 (socket says 172.21.15.12:39344) 2024-05-09T09:25:24.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.053+0000 7f9af6d9d640 1 -- 172.21.15.12:0/379369087 learned_addr learned my addr 172.21.15.12:0/379369087 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.053+0000 7f9aeffff640 1 --2- 172.21.15.12:0/379369087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc39420 0x55993bc3b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9af659c640 1 --2- 172.21.15.12:0/379369087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc342b0 0x55993bc38db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9aeffff640 1 -- 172.21.15.12:0/379369087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc342b0 msgr2=0x55993bc38db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9aeffff640 1 --2- 172.21.15.12:0/379369087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc342b0 0x55993bc38db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9aeffff640 1 -- 172.21.15.12:0/379369087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 msgr2=0x55993bc33ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9aeffff640 1 --2- 172.21.15.12:0/379369087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 0x55993bc33ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9aeffff640 1 -- 172.21.15.12:0/379369087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55993b8ab2d0 con 0x55993bc39420 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9af6d9d640 1 --2- 172.21.15.12:0/379369087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 0x55993bc33ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9aef7fe640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9ae40a2b90 0x7f9ae40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9aeffff640 1 --2- 172.21.15.12:0/379369087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc39420 0x55993bc3b8b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9af8101a70 tx=0x7f9ae000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da197d40167964b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9af6d9d640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9ae40a2b90 0x7f9ae40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9aef7fe640 1 -- 172.21.15.12:0/2752874535 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f9ae80ceeb0 con 0x7f9af8102390 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9aecff9640 1 -- 172.21.15.12:0/379369087 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0014070 con 0x55993bc39420 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9aecff9640 1 -- 172.21.15.12:0/379369087 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9ae0008e50 con 0x55993bc39420 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.054+0000 7f9aecff9640 1 -- 172.21.15.12:0/379369087 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0010690 con 0x55993bc39420 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.055+0000 7f9affbc5740 1 -- 172.21.15.12:0/379369087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc39420 msgr2=0x55993bc3b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.055+0000 7f9af6d9d640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9ae40a2b90 0x7f9ae40a5050 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009a60 tx=0x7f9af0005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.055+0000 7f9affbc5740 1 --2- 172.21.15.12:0/379369087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc39420 0x55993bc3b8b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9af8101a70 tx=0x7f9ae000e720 comp rx=0 tx=0).stop 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.055+0000 7f9affbc5740 1 -- 172.21.15.12:0/379369087 shutdown_connections 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.055+0000 7f9affbc5740 1 --2- 172.21.15.12:0/379369087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc39420 0x55993bc3b8b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.055+0000 7f9affbc5740 1 --2- 172.21.15.12:0/379369087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc342b0 0x55993bc38db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.055+0000 7f9affbc5740 1 --2- 172.21.15.12:0/379369087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 0x55993bc33ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.055+0000 7f9affbc5740 1 -- 172.21.15.12:0/379369087 >> 172.21.15.12:0/379369087 conn(0x55993bc2ef30 msgr2=0x55993bc2f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.055+0000 7f9affbc5740 1 -- 172.21.15.12:0/379369087 shutdown_connections 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.055+0000 7f9affbc5740 1 -- 172.21.15.12:0/379369087 wait complete. 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.055+0000 7f9affbc5740 1 -- start start 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.055+0000 7f9affbc5740 1 Processor -- start 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9affbc5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc338c0 0x55993bc3dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9affbc5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 0x55993bc3e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9affbc5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc39420 0x55993bc45250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9affbc5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55993bc3c3d0 con 0x55993bc342b0 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9affbc5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55993bb528e0 con 0x55993bc338c0 2024-05-09T09:25:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9affbc5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55993bc3c550 con 0x55993bc39420 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9affbc5740 10 client.0 _ll_register_callbacks cb 0x55993bb1f9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9aeffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 0x55993bc3e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9aeffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 0x55993bc3e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39358/0 (socket says 172.21.15.12:39358) 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9aeffff640 1 -- 172.21.15.12:0/886959486 learned_addr learned my addr 172.21.15.12:0/886959486 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9af659c640 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc338c0 0x55993bc3dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9aeffff640 1 -- 172.21.15.12:0/886959486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc39420 msgr2=0x55993bc45250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9af6d9d640 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc39420 0x55993bc45250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9aeffff640 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc39420 0x55993bc45250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9aeffff640 1 -- 172.21.15.12:0/886959486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc338c0 msgr2=0x55993bc3dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9aeffff640 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc338c0 0x55993bc3dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9aeffff640 1 -- 172.21.15.12:0/886959486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55993b9756b0 con 0x55993bc342b0 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9af659c640 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc338c0 0x55993bc3dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9aeffff640 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 0x55993bc3e220 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0006010 tx=0x7f9ae0005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7a1a416cbb7945 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9addffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9addffb640 1 -- 172.21.15.12:0/886959486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0014070 con 0x55993bc342b0 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9addffb640 1 -- 172.21.15.12:0/886959486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f9ae00055d0 con 0x55993bc342b0 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.056+0000 7f9affbc5740 1 -- 172.21.15.12:0/886959486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae0008e50 con 0x55993bc342b0 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.057+0000 7f9addffb640 1 -- 172.21.15.12:0/886959486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ae002d050 con 0x55993bc342b0 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.057+0000 7f9affbc5740 10 client.15465 fetch_fsmap learned FSMap version 28 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.057+0000 7f9affbc5740 1 -- 172.21.15.12:0/886959486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55993bc48230 con 0x55993bc342b0 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.057+0000 7f9addffb640 1 -- 172.21.15.12:0/886959486 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9ae0008e50 con 0x55993bc342b0 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.057+0000 7f9addffb640 1 -- 172.21.15.12:0/886959486 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9ae0003ae0 con 0x55993bc342b0 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.057+0000 7f9affbc5740 10 client.15465 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.057+0000 7f9affbc5740 10 client.15465 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.057+0000 7f9affbc5740 20 client.15465 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.057+0000 7f9affbc5740 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55993bc4a4a0 0x55993bc4b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.057+0000 7f9affbc5740 4 client.15465 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.057+0000 7f9affbc5740 1 -- 172.21.15.12:0/886959486 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55993bba4c00 con 0x55993bc4a4a0 2024-05-09T09:25:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.057+0000 7f9af659c640 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55993bc4a4a0 0x55993bc4b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.058+0000 7f9af659c640 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55993bc4a4a0 0x55993bc4b910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10c972e670077d00 server_cookie=be49dc234d029d24 in_seq=0 out_seq=0 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.058+0000 7f9addffb640 10 client.15465 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.058+0000 7f9addffb640 1 -- 172.21.15.12:0/886959486 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55993bba4c00 con 0x55993bc4a4a0 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.058+0000 7f9addffb640 10 client.15465 handle_command_reply: tid=0 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.059+0000 7f9affbc5740 1 client.15465 shutdown 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.059+0000 7f9affbc5740 1 -- 172.21.15.12:0/886959486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 msgr2=0x55993bc3e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.059+0000 7f9affbc5740 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 0x55993bc3e220 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0006010 tx=0x7f9ae0005250 comp rx=0 tx=0).stop 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.059+0000 7f9affbc5740 1 -- 172.21.15.12:0/886959486 shutdown_connections 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.059+0000 7f9affbc5740 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55993bc4a4a0 0x55993bc4b910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.059+0000 7f9affbc5740 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc39420 0x55993bc45250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.059+0000 7f9affbc5740 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 0x55993bc3e220 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.059+0000 7f9affbc5740 1 --2- 172.21.15.12:0/886959486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc338c0 0x55993bc3dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.059+0000 7f9addffb640 0 client.15465 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:25:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.059+0000 7f9affbc5740 1 -- 172.21.15.12:0/886959486 >> 172.21.15.12:0/886959486 conn(0x55993bc2ef30 msgr2=0x55993bc2e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.059+0000 7f9affbc5740 1 -- 172.21.15.12:0/886959486 shutdown_connections 2024-05-09T09:25:24.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.059+0000 7f9affbc5740 1 -- 172.21.15.12:0/886959486 wait complete. 2024-05-09T09:25:24.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.059+0000 7f9affbc5740 20 client.15465 trim_cache size 0 max 16384 2024-05-09T09:25:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.077+0000 7f9affbc5740 1 Processor -- start 2024-05-09T09:25:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.077+0000 7f9affbc5740 1 -- start start 2024-05-09T09:25:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.077+0000 7f9affbc5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 0x55993bc3c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.077+0000 7f9affbc5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc342b0 0x55993bc3b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.077+0000 7f9affbc5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc39420 0x55993bc3d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.077+0000 7f9affbc5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55993bc49c50 con 0x55993bc338c0 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.077+0000 7f9affbc5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55993bc4a320 con 0x55993bc342b0 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.077+0000 7f9affbc5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55993bc4a9f0 con 0x55993bc39420 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.077+0000 7f9af659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 0x55993bc3c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.077+0000 7f9af659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 0x55993bc3c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39374/0 (socket says 172.21.15.12:39374) 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.077+0000 7f9af659c640 1 -- 172.21.15.12:0/3515254301 learned_addr learned my addr 172.21.15.12:0/3515254301 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.078+0000 7f9af6d9d640 1 --2- 172.21.15.12:0/3515254301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc39420 0x55993bc3d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.078+0000 7f9af659c640 1 -- 172.21.15.12:0/3515254301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc39420 msgr2=0x55993bc3d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.078+0000 7f9af659c640 1 --2- 172.21.15.12:0/3515254301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc39420 0x55993bc3d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.078+0000 7f9af659c640 1 -- 172.21.15.12:0/3515254301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc342b0 msgr2=0x55993bc3b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.078+0000 7f9af659c640 1 --2- 172.21.15.12:0/3515254301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc342b0 0x55993bc3b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.078+0000 7f9af659c640 1 -- 172.21.15.12:0/3515254301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55993bc34b40 con 0x55993bc338c0 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.078+0000 7f9af659c640 1 --2- 172.21.15.12:0/3515254301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 0x55993bc3c2b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9ae809b040 tx=0x7f9ae8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25171b618bebae4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.078+0000 7f9adffff640 1 -- 172.21.15.12:0/3515254301 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8020040 con 0x55993bc338c0 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.078+0000 7f9adffff640 1 -- 172.21.15.12:0/3515254301 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9ae8009cc0 con 0x55993bc338c0 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.078+0000 7f9adffff640 1 -- 172.21.15.12:0/3515254301 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80a3fb0 con 0x55993bc338c0 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.078+0000 7f9affbc5740 1 -- 172.21.15.12:0/3515254301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 msgr2=0x55993bc3c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.078+0000 7f9affbc5740 1 --2- 172.21.15.12:0/3515254301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 0x55993bc3c2b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9ae809b040 tx=0x7f9ae8009c30 comp rx=0 tx=0).stop 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 -- 172.21.15.12:0/3515254301 shutdown_connections 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 --2- 172.21.15.12:0/3515254301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc39420 0x55993bc3d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 --2- 172.21.15.12:0/3515254301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc342b0 0x55993bc3b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 --2- 172.21.15.12:0/3515254301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc338c0 0x55993bc3c2b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 -- 172.21.15.12:0/3515254301 >> 172.21.15.12:0/3515254301 conn(0x55993bc2ef30 msgr2=0x55993bc2f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 -- 172.21.15.12:0/3515254301 shutdown_connections 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 -- 172.21.15.12:0/3515254301 wait complete. 2024-05-09T09:25:24.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 -- start start 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 Processor -- start 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc338c0 0x55993bc37570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 0x55993bc37ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc39420 0x55993bc43680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55993bc4b790 con 0x55993bc342b0 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55993bc4b0c0 con 0x55993bc39420 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55993bc4be60 con 0x55993bc338c0 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9affbc5740 10 client.0 _ll_register_callbacks cb 0x7f9af0009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9aeffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 0x55993bc37ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9aeffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 0x55993bc37ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39384/0 (socket says 172.21.15.12:39384) 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9aeffff640 1 -- 172.21.15.12:0/367111007 learned_addr learned my addr 172.21.15.12:0/367111007 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9af6d9d640 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc39420 0x55993bc43680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.079+0000 7f9af659c640 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc338c0 0x55993bc37570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9aeffff640 1 -- 172.21.15.12:0/367111007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc338c0 msgr2=0x55993bc37570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9aeffff640 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc338c0 0x55993bc37570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9aeffff640 1 -- 172.21.15.12:0/367111007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc39420 msgr2=0x55993bc43680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9aeffff640 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc39420 0x55993bc43680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9aeffff640 1 -- 172.21.15.12:0/367111007 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55993bc43cc0 con 0x55993bc342b0 2024-05-09T09:25:24.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9af659c640 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc338c0 0x55993bc37570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:24.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9aeffff640 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 0x55993bc37ab0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000f010 tx=0x7f9ae0005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d50e5fa3f13c4c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9addffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:24.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9addffb640 1 -- 172.21.15.12:0/367111007 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0014040 con 0x55993bc342b0 2024-05-09T09:25:24.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9addffb640 1 -- 172.21.15.12:0/367111007 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f9ae0010550 con 0x55993bc342b0 2024-05-09T09:25:24.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9affbc5740 1 -- 172.21.15.12:0/367111007 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55993bb9ef50 con 0x55993bc342b0 2024-05-09T09:25:24.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9addffb640 1 -- 172.21.15.12:0/367111007 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ae0004ad0 con 0x55993bc342b0 2024-05-09T09:25:24.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9affbc5740 10 client.15471 fetch_fsmap learned FSMap version 28 2024-05-09T09:25:24.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9affbc5740 1 -- 172.21.15.12:0/367111007 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55993bcb0770 con 0x55993bc342b0 2024-05-09T09:25:24.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9addffb640 1 -- 172.21.15.12:0/367111007 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55993bb9ef50 con 0x55993bc342b0 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9addffb640 1 -- 172.21.15.12:0/367111007 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9ae0004000 con 0x55993bc342b0 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.080+0000 7f9affbc5740 10 client.15471 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9affbc5740 10 client.15471 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9affbc5740 20 client.15471 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9affbc5740 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55993bcb5360 0x55993bcb4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9affbc5740 4 client.15471 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9affbc5740 1 -- 172.21.15.12:0/367111007 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "status"}) -- 0x55993bb1cb80 con 0x55993bcb5360 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9af659c640 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55993bcb5360 0x55993bcb4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9af659c640 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55993bcb5360 0x55993bcb4070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d77db944a02908d4 server_cookie=b4d2ae94943f3b18 in_seq=0 out_seq=0 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9addffb640 10 client.15471 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9addffb640 1 -- 172.21.15.12:0/367111007 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55993bb1cb80 con 0x55993bcb5360 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9addffb640 10 client.15471 handle_command_reply: tid=0 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9affbc5740 1 client.15471 shutdown 2024-05-09T09:25:24.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9affbc5740 1 -- 172.21.15.12:0/367111007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 msgr2=0x55993bc37ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9affbc5740 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 0x55993bc37ab0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000f010 tx=0x7f9ae0005230 comp rx=0 tx=0).stop 2024-05-09T09:25:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9affbc5740 1 -- 172.21.15.12:0/367111007 shutdown_connections 2024-05-09T09:25:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9affbc5740 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55993bcb5360 0x55993bcb4070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.081+0000 7f9affbc5740 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55993bc39420 0x55993bc43680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9affbc5740 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55993bc342b0 0x55993bc37ab0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9affbc5740 1 --2- 172.21.15.12:0/367111007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993bc338c0 0x55993bc37570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9affbc5740 1 -- 172.21.15.12:0/367111007 >> 172.21.15.12:0/367111007 conn(0x55993bc2ef30 msgr2=0x55993bc38860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9addffb640 0 client.15471 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:25:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9affbc5740 1 -- 172.21.15.12:0/367111007 shutdown_connections 2024-05-09T09:25:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9affbc5740 1 -- 172.21.15.12:0/367111007 wait complete. 2024-05-09T09:25:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9affbc5740 20 client.15471 trim_cache size 0 max 16384 2024-05-09T09:25:24.089 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:25:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:25:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-05-09T09:25:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24502, 2024-05-09T09:25:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-09T09:25:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-09T09:25:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-09T09:25:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 112.42650705699999, 2024-05-09T09:25:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-05-09T09:25:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 93, 2024-05-09T09:25:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T09:25:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 154.616218723 2024-05-09T09:25:24.089 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:25:24.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9afcde2640 1 -- 172.21.15.12:0/2752874535 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9ae40a2b90 msgr2=0x7f9ae40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9afcde2640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9ae40a2b90 0x7f9ae40a5050 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009a60 tx=0x7f9af0005ef0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9afcde2640 1 -- 172.21.15.12:0/2752874535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9af8102390 msgr2=0x7f9af8101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9afcde2640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9af8102390 0x7f9af8101d00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800d770 tx=0x7f9ae800dc30 comp rx=0 tx=0).stop 2024-05-09T09:25:24.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9afcde2640 1 -- 172.21.15.12:0/2752874535 shutdown_connections 2024-05-09T09:25:24.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9afcde2640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9ae40a2b90 0x7f9ae40a5050 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9afcde2640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af81075f0 0x7f9af8100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9afcde2640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9af8104cc0 0x7f9af8100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9afcde2640 1 --2- 172.21.15.12:0/2752874535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9af8102390 0x7f9af8101d00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9afcde2640 1 -- 172.21.15.12:0/2752874535 >> 172.21.15.12:0/2752874535 conn(0x7f9af80fbd20 msgr2=0x7f9af8102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9afcde2640 1 -- 172.21.15.12:0/2752874535 shutdown_connections 2024-05-09T09:25:24.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.082+0000 7f9afcde2640 1 -- 172.21.15.12:0/2752874535 wait complete. 2024-05-09T09:25:24.094 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-09T09:25:24.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.200+0000 7f6c60fc1640 1 -- 172.21.15.12:0/721363137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c5c101960 msgr2=0x7f6c5c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.200+0000 7f6c60fc1640 1 --2- 172.21.15.12:0/721363137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c5c101960 0x7f6c5c101de0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009900 tx=0x7f6c5002a410 comp rx=0 tx=0).stop 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.201+0000 7f6c60fc1640 1 -- 172.21.15.12:0/721363137 shutdown_connections 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.201+0000 7f6c60fc1640 1 --2- 172.21.15.12:0/721363137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c5c102320 0x7f6c5c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.201+0000 7f6c60fc1640 1 --2- 172.21.15.12:0/721363137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c5c101960 0x7f6c5c101de0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.201+0000 7f6c60fc1640 1 --2- 172.21.15.12:0/721363137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c100770 0x7f6c5c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.201+0000 7f6c60fc1640 1 -- 172.21.15.12:0/721363137 >> 172.21.15.12:0/721363137 conn(0x7f6c5c0fbce0 msgr2=0x7f6c5c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.201+0000 7f6c60fc1640 1 -- 172.21.15.12:0/721363137 shutdown_connections 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.201+0000 7f6c60fc1640 1 -- 172.21.15.12:0/721363137 wait complete. 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.201+0000 7f6c60fc1640 1 Processor -- start 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.201+0000 7f6c60fc1640 1 -- start start 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.201+0000 7f6c60fc1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c5c100770 0x7f6c5c104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c60fc1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c101960 0x7f6c5c102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c60fc1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c5c102320 0x7f6c5c103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c60fc1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10c430 con 0x7f6c5c102320 2024-05-09T09:25:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c60fc1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10c2b0 con 0x7f6c5c101960 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c60fc1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10c5b0 con 0x7f6c5c100770 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c5ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c5c102320 0x7f6c5c103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c5ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c5c102320 0x7f6c5c103280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39400/0 (socket says 172.21.15.12:39400) 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c5ad9d640 1 -- 172.21.15.12:0/565362955 learned_addr learned my addr 172.21.15.12:0/565362955 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c101960 0x7f6c5c102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c5a59c640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c5c100770 0x7f6c5c104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c5ad9d640 1 -- 172.21.15.12:0/565362955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c5c100770 msgr2=0x7f6c5c104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c5ad9d640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c5c100770 0x7f6c5c104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c5ad9d640 1 -- 172.21.15.12:0/565362955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c101960 msgr2=0x7f6c5c102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c5ad9d640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c101960 0x7f6c5c102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c5ad9d640 1 -- 172.21.15.12:0/565362955 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c103b10 con 0x7f6c5c102320 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c101960 0x7f6c5c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c5ad9d640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c5c102320 0x7f6c5c103280 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009fd0 tx=0x7f6c5400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c4b7fe640 1 -- 172.21.15.12:0/565362955 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54011830 con 0x7f6c5c102320 2024-05-09T09:25:24.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c60fc1640 1 -- 172.21.15.12:0/565362955 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5c1a1ce0 con 0x7f6c5c102320 2024-05-09T09:25:24.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c4b7fe640 1 -- 172.21.15.12:0/565362955 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6c54016070 con 0x7f6c5c102320 2024-05-09T09:25:24.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c4b7fe640 1 -- 172.21.15.12:0/565362955 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5400f2e0 con 0x7f6c5c102320 2024-05-09T09:25:24.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.202+0000 7f6c60fc1640 1 -- 172.21.15.12:0/565362955 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5c1a2180 con 0x7f6c5c102320 2024-05-09T09:25:24.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.205+0000 7f6c63daf740 1 Processor -- start 2024-05-09T09:25:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.205+0000 7f6c4b7fe640 1 -- 172.21.15.12:0/565362955 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6c540119d0 con 0x7f6c5c102320 2024-05-09T09:25:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c63daf740 1 -- start start 2024-05-09T09:25:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c63daf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56be78c0 0x563d56be7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c63daf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56becdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c63daf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56bed420 0x563d56bef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c63daf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563d56b4b4d0 con 0x563d56be78c0 2024-05-09T09:25:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c63daf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d56af33e0 con 0x563d56be82b0 2024-05-09T09:25:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c63daf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563d56b47490 con 0x563d56bed420 2024-05-09T09:25:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c59d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56be78c0 0x563d56be7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c59d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56be78c0 0x563d56be7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39406/0 (socket says 172.21.15.12:39406) 2024-05-09T09:25:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c59d9b640 1 -- 172.21.15.12:0/981624599 learned_addr learned my addr 172.21.15.12:0/981624599 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c5a59c640 1 --2- 172.21.15.12:0/981624599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56becdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/981624599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56bed420 0x563d56bef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c4b7fe640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6c440a2b90 0x7f6c440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c59d9b640 1 -- 172.21.15.12:0/981624599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56bed420 msgr2=0x563d56bef8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/981624599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56bed420 0x563d56bef8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c59d9b640 1 -- 172.21.15.12:0/981624599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 msgr2=0x563d56becdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/981624599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56becdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c59d9b640 1 -- 172.21.15.12:0/981624599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d5685f2d0 con 0x563d56be78c0 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.206+0000 7f6c5a59c640 1 --2- 172.21.15.12:0/981624599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56becdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.207+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/981624599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56be78c0 0x563d56be7ce0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6c50005ab0 tx=0x7f6c50009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9ad51b09d282b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.207+0000 7f6c4b7fe640 1 -- 172.21.15.12:0/565362955 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f6c540cec90 con 0x7f6c5c102320 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.207+0000 7f6c5ad9d640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6c440a2b90 0x7f6c440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.207+0000 7f6c5ad9d640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6c440a2b90 0x7f6c440a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6c5409b050 tx=0x7f6c540047d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.207+0000 7f6c48ff9640 1 -- 172.21.15.12:0/981624599 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50035070 con 0x563d56be78c0 2024-05-09T09:25:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.207+0000 7f6c48ff9640 1 -- 172.21.15.12:0/981624599 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6c500318f0 con 0x563d56be78c0 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.207+0000 7f6c48ff9640 1 -- 172.21.15.12:0/981624599 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c500049a0 con 0x563d56be78c0 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.208+0000 7f6c63daf740 1 -- 172.21.15.12:0/981624599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56be78c0 msgr2=0x563d56be7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.208+0000 7f6c63daf740 1 --2- 172.21.15.12:0/981624599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56be78c0 0x563d56be7ce0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6c50005ab0 tx=0x7f6c50009560 comp rx=0 tx=0).stop 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.208+0000 7f6c63daf740 1 -- 172.21.15.12:0/981624599 shutdown_connections 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.208+0000 7f6c63daf740 1 --2- 172.21.15.12:0/981624599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56bed420 0x563d56bef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.208+0000 7f6c63daf740 1 --2- 172.21.15.12:0/981624599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56becdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.208+0000 7f6c63daf740 1 --2- 172.21.15.12:0/981624599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56be78c0 0x563d56be7ce0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.208+0000 7f6c63daf740 1 -- 172.21.15.12:0/981624599 >> 172.21.15.12:0/981624599 conn(0x563d56be2f30 msgr2=0x563d56be3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.208+0000 7f6c63daf740 1 -- 172.21.15.12:0/981624599 shutdown_connections 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.208+0000 7f6c63daf740 1 -- 172.21.15.12:0/981624599 wait complete. 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.208+0000 7f6c63daf740 1 -- start start 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.208+0000 7f6c63daf740 1 Processor -- start 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c63daf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56be78c0 0x563d56bf1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c63daf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56bf2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c63daf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bf9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c63daf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563d56bf0370 con 0x563d56bed420 2024-05-09T09:25:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c63daf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d56b068e0 con 0x563d56be82b0 2024-05-09T09:25:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c63daf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563d56bf04f0 con 0x563d56be78c0 2024-05-09T09:25:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c63daf740 10 client.0 _ll_register_callbacks cb 0x563d56ad39c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c59d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bf9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c59d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bf9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39418/0 (socket says 172.21.15.12:39418) 2024-05-09T09:25:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c59d9b640 1 -- 172.21.15.12:0/145782917 learned_addr learned my addr 172.21.15.12:0/145782917 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56be78c0 0x563d56bf1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c5a59c640 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56bf2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c59d9b640 1 -- 172.21.15.12:0/145782917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56be78c0 msgr2=0x563d56bf1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56be78c0 0x563d56bf1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c59d9b640 1 -- 172.21.15.12:0/145782917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 msgr2=0x563d56bf2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56bf2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c59d9b640 1 -- 172.21.15.12:0/145782917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d569296b0 con 0x563d56bed420 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c5a59c640 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56bf2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bf9250 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x563d56bf2b80 tx=0x7f6c5002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5922e2069dc3e43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.209+0000 7f6c41ffb640 1 -- 172.21.15.12:0/145782917 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50052060 con 0x563d56bed420 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c63daf740 1 -- 172.21.15.12:0/145782917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c500318f0 con 0x563d56bed420 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c41ffb640 1 -- 172.21.15.12:0/145782917 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f6c5004d900 con 0x563d56bed420 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c41ffb640 1 -- 172.21.15.12:0/145782917 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c50035040 con 0x563d56bed420 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c63daf740 10 client.15483 fetch_fsmap learned FSMap version 28 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c63daf740 1 -- 172.21.15.12:0/145782917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563d56bfc210 con 0x563d56bed420 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c41ffb640 1 -- 172.21.15.12:0/145782917 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6c500318f0 con 0x563d56bed420 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c41ffb640 1 -- 172.21.15.12:0/145782917 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6c5004b460 con 0x563d56bed420 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c63daf740 10 client.15483 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c63daf740 10 client.15483 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T09:25:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c63daf740 20 client.15483 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:24.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c63daf740 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563d56bfe3c0 0x563d56bff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c63daf740 4 client.15483 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:24.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c63daf740 1 -- 172.21.15.12:0/145782917 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d56b58c00 con 0x563d56bfe3c0 2024-05-09T09:25:24.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.210+0000 7f6c5a59c640 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563d56bfe3c0 0x563d56bff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.211+0000 7f6c5a59c640 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563d56bfe3c0 0x563d56bff830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e5150295f86c93f server_cookie=56ad0c0a9b303a0a in_seq=0 out_seq=0 2024-05-09T09:25:24.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.211+0000 7f6c41ffb640 10 client.15483 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:25:24.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c41ffb640 1 -- 172.21.15.12:0/145782917 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d56b58c00 con 0x563d56bfe3c0 2024-05-09T09:25:24.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c41ffb640 10 client.15483 handle_command_reply: tid=0 2024-05-09T09:25:24.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c63daf740 1 client.15483 shutdown 2024-05-09T09:25:24.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c63daf740 1 -- 172.21.15.12:0/145782917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 msgr2=0x563d56bf9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c63daf740 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bf9250 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x563d56bf2b80 tx=0x7f6c5002a410 comp rx=0 tx=0).stop 2024-05-09T09:25:24.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c63daf740 1 -- 172.21.15.12:0/145782917 shutdown_connections 2024-05-09T09:25:24.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c63daf740 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563d56bfe3c0 0x563d56bff830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c63daf740 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bf9250 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c63daf740 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56bf2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c63daf740 1 --2- 172.21.15.12:0/145782917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56be78c0 0x563d56bf1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c41ffb640 0 client.15483 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:25:24.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c63daf740 1 -- 172.21.15.12:0/145782917 >> 172.21.15.12:0/145782917 conn(0x563d56be2f30 msgr2=0x563d56be2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c63daf740 1 -- 172.21.15.12:0/145782917 shutdown_connections 2024-05-09T09:25:24.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c63daf740 1 -- 172.21.15.12:0/145782917 wait complete. 2024-05-09T09:25:24.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.212+0000 7f6c63daf740 20 client.15483 trim_cache size 0 max 16384 2024-05-09T09:25:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.230+0000 7f6c63daf740 1 Processor -- start 2024-05-09T09:25:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.230+0000 7f6c63daf740 1 -- start start 2024-05-09T09:25:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.230+0000 7f6c63daf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be78c0 0x563d56bef810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.230+0000 7f6c63daf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56be82b0 0x563d56bf9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.230+0000 7f6c63daf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bddd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.230+0000 7f6c63daf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563d56bfdb70 con 0x563d56bed420 2024-05-09T09:25:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.230+0000 7f6c63daf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d56bfe240 con 0x563d56be78c0 2024-05-09T09:25:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.230+0000 7f6c63daf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563d56bfe910 con 0x563d56be82b0 2024-05-09T09:25:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.230+0000 7f6c59d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.230+0000 7f6c59d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bddd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39434/0 (socket says 172.21.15.12:39434) 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.230+0000 7f6c59d9b640 1 -- 172.21.15.12:0/2560423657 learned_addr learned my addr 172.21.15.12:0/2560423657 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.231+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/2560423657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be78c0 0x563d56bef810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.231+0000 7f6c59d9b640 1 -- 172.21.15.12:0/2560423657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56be82b0 msgr2=0x563d56bf9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.231+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/2560423657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56be82b0 0x563d56bf9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.231+0000 7f6c59d9b640 1 -- 172.21.15.12:0/2560423657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be78c0 msgr2=0x563d56bef810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.231+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/2560423657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be78c0 0x563d56bef810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.231+0000 7f6c59d9b640 1 -- 172.21.15.12:0/2560423657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d56be3ca0 con 0x563d56bed420 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.231+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/2560423657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bddd10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6c50001eb0 tx=0x7f6c50040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df096e43cc2a20ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.231+0000 7f6c43fff640 1 -- 172.21.15.12:0/2560423657 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c500318f0 con 0x563d56bed420 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.231+0000 7f6c43fff640 1 -- 172.21.15.12:0/2560423657 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6c50049740 con 0x563d56bed420 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.231+0000 7f6c43fff640 1 -- 172.21.15.12:0/2560423657 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5004b6c0 con 0x563d56bed420 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 -- 172.21.15.12:0/2560423657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 msgr2=0x563d56bddd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 --2- 172.21.15.12:0/2560423657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bddd10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6c50001eb0 tx=0x7f6c50040040 comp rx=0 tx=0).stop 2024-05-09T09:25:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 -- 172.21.15.12:0/2560423657 shutdown_connections 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 --2- 172.21.15.12:0/2560423657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bddd10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 --2- 172.21.15.12:0/2560423657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56be82b0 0x563d56bf9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 --2- 172.21.15.12:0/2560423657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be78c0 0x563d56bef810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 -- 172.21.15.12:0/2560423657 >> 172.21.15.12:0/2560423657 conn(0x563d56be2f30 msgr2=0x563d56be3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 -- 172.21.15.12:0/2560423657 shutdown_connections 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 -- 172.21.15.12:0/2560423657 wait complete. 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 -- start start 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 Processor -- start 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56be78c0 0x563d56bf48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56bf4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bf7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563d56bff6b0 con 0x563d56bed420 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d56bfefe0 con 0x563d56be82b0 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563d56bffd80 con 0x563d56be78c0 2024-05-09T09:25:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c63daf740 10 client.0 _ll_register_callbacks cb 0x7f6c4c008530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c59d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bf7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c59d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bf7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39444/0 (socket says 172.21.15.12:39444) 2024-05-09T09:25:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c59d9b640 1 -- 172.21.15.12:0/599403625 learned_addr learned my addr 172.21.15.12:0/599403625 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.232+0000 7f6c5a59c640 1 --2- 172.21.15.12:0/599403625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56bf4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c59d9b640 1 -- 172.21.15.12:0/599403625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56be78c0 msgr2=0x563d56bf48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/599403625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56be78c0 0x563d56bf48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c59d9b640 1 -- 172.21.15.12:0/599403625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 msgr2=0x563d56bf4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/599403625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56bf4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c59d9b640 1 -- 172.21.15.12:0/599403625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d56bf83d0 con 0x563d56bed420 2024-05-09T09:25:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c59d9b640 1 --2- 172.21.15.12:0/599403625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bf7de0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6c5004ba70 tx=0x7f6c50005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f26bcdbb3f01b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c41ffb640 1 -- 172.21.15.12:0/599403625 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50060020 con 0x563d56bed420 2024-05-09T09:25:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c41ffb640 1 -- 172.21.15.12:0/599403625 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f6c5004d030 con 0x563d56bed420 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c63daf740 1 -- 172.21.15.12:0/599403625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c50049740 con 0x563d56bed420 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c41ffb640 1 -- 172.21.15.12:0/599403625 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c50052070 con 0x563d56bed420 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c63daf740 10 client.15489 fetch_fsmap learned FSMap version 28 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c63daf740 1 -- 172.21.15.12:0/599403625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563d56c64370 con 0x563d56bed420 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.233+0000 7f6c41ffb640 1 -- 172.21.15.12:0/599403625 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6c50049740 con 0x563d56bed420 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.234+0000 7f6c41ffb640 1 -- 172.21.15.12:0/599403625 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6c500498f0 con 0x563d56bed420 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.234+0000 7f6c63daf740 10 client.15489 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.234+0000 7f6c63daf740 10 client.15489 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.234+0000 7f6c63daf740 20 client.15489 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.234+0000 7f6c63daf740 1 --2- 172.21.15.12:0/599403625 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563d56c689d0 0x563d56c676e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.234+0000 7f6c63daf740 4 client.15489 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.234+0000 7f6c63daf740 1 -- 172.21.15.12:0/599403625 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "status"}) -- 0x563d56b52f50 con 0x563d56c689d0 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.234+0000 7f6c5a59c640 1 --2- 172.21.15.12:0/599403625 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563d56c689d0 0x563d56c676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.234+0000 7f6c41ffb640 10 client.15489 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:25:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.234+0000 7f6c5a59c640 1 --2- 172.21.15.12:0/599403625 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563d56c689d0 0x563d56c676e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d69cdf7c4fce884 server_cookie=460f26bf81368051 in_seq=0 out_seq=0 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.234+0000 7f6c41ffb640 1 -- 172.21.15.12:0/599403625 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563d56b52f50 con 0x563d56c689d0 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.234+0000 7f6c41ffb640 10 client.15489 handle_command_reply: tid=0 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.234+0000 7f6c63daf740 1 client.15489 shutdown 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c63daf740 1 -- 172.21.15.12:0/599403625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 msgr2=0x563d56bf7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c63daf740 1 --2- 172.21.15.12:0/599403625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bf7de0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6c5004ba70 tx=0x7f6c50005b50 comp rx=0 tx=0).stop 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c63daf740 1 -- 172.21.15.12:0/599403625 shutdown_connections 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c63daf740 1 --2- 172.21.15.12:0/599403625 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x563d56c689d0 0x563d56c676e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c63daf740 1 --2- 172.21.15.12:0/599403625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d56bed420 0x563d56bf7de0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c63daf740 1 --2- 172.21.15.12:0/599403625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d56be82b0 0x563d56bf4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c41ffb640 0 client.15489 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c63daf740 1 --2- 172.21.15.12:0/599403625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d56be78c0 0x563d56bf48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c63daf740 1 -- 172.21.15.12:0/599403625 >> 172.21.15.12:0/599403625 conn(0x563d56be2f30 msgr2=0x563d56be5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c63daf740 1 -- 172.21.15.12:0/599403625 shutdown_connections 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c63daf740 1 -- 172.21.15.12:0/599403625 wait complete. 2024-05-09T09:25:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c63daf740 20 client.15489 trim_cache size 0 max 16384 2024-05-09T09:25:24.241 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:25:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:25:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-05-09T09:25:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24515, 2024-05-09T09:25:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-09T09:25:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-09T09:25:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-09T09:25:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 113.592476013, 2024-05-09T09:25:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-05-09T09:25:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 93, 2024-05-09T09:25:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T09:25:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 140.13194577499999 2024-05-09T09:25:24.242 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:25:24.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c60fc1640 1 -- 172.21.15.12:0/565362955 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6c440a2b90 msgr2=0x7f6c440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c60fc1640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6c440a2b90 0x7f6c440a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6c5409b050 tx=0x7f6c540047d0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c60fc1640 1 -- 172.21.15.12:0/565362955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c5c102320 msgr2=0x7f6c5c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c60fc1640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c5c102320 0x7f6c5c103280 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009fd0 tx=0x7f6c5400b8e0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c60fc1640 1 -- 172.21.15.12:0/565362955 shutdown_connections 2024-05-09T09:25:24.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c60fc1640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6c440a2b90 0x7f6c440a5050 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c60fc1640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c5c102320 0x7f6c5c103280 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c60fc1640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c101960 0x7f6c5c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c60fc1640 1 --2- 172.21.15.12:0/565362955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c5c100770 0x7f6c5c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c60fc1640 1 -- 172.21.15.12:0/565362955 >> 172.21.15.12:0/565362955 conn(0x7f6c5c0fbce0 msgr2=0x7f6c5c0fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c60fc1640 1 -- 172.21.15.12:0/565362955 shutdown_connections 2024-05-09T09:25:24.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.235+0000 7f6c60fc1640 1 -- 172.21.15.12:0/565362955 wait complete. 2024-05-09T09:25:24.246 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-09T09:25:24.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.353+0000 7f25e3514640 1 -- 172.21.15.12:0/1073021773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25dc101400 msgr2=0x7f25dc101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.353+0000 7f25e3514640 1 --2- 172.21.15.12:0/1073021773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25dc101400 0x7f25dc101880 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009ca0 tx=0x7f25d402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.353+0000 7f25e3514640 1 -- 172.21.15.12:0/1073021773 shutdown_connections 2024-05-09T09:25:24.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.353+0000 7f25e3514640 1 --2- 172.21.15.12:0/1073021773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25dc101400 0x7f25dc101880 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.353+0000 7f25e3514640 1 --2- 172.21.15.12:0/1073021773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25dc100a60 0x7f25dc100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.353+0000 7f25e3514640 1 --2- 172.21.15.12:0/1073021773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25dc100100 0x7f25dc100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.353+0000 7f25e3514640 1 -- 172.21.15.12:0/1073021773 >> 172.21.15.12:0/1073021773 conn(0x7f25dc0fbd20 msgr2=0x7f25dc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.353+0000 7f25e3514640 1 -- 172.21.15.12:0/1073021773 shutdown_connections 2024-05-09T09:25:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.353+0000 7f25e3514640 1 -- 172.21.15.12:0/1073021773 wait complete. 2024-05-09T09:25:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e3514640 1 Processor -- start 2024-05-09T09:25:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e3514640 1 -- start start 2024-05-09T09:25:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e3514640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25dc100100 0x7f25dc199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e3514640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25dc100a60 0x7f25dc19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e3514640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25dc101400 0x7f25dc1a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e3514640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc10f700 con 0x7f25dc100100 2024-05-09T09:25:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e3514640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc10f580 con 0x7f25dc100a60 2024-05-09T09:25:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e3514640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc10f880 con 0x7f25dc101400 2024-05-09T09:25:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e12b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25dc100100 0x7f25dc199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e12b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25dc100100 0x7f25dc199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39454/0 (socket says 172.21.15.12:39454) 2024-05-09T09:25:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e12b0640 1 -- 172.21.15.12:0/1808069187 learned_addr learned my addr 172.21.15.12:0/1808069187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e1ab1640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25dc101400 0x7f25dc1a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e0aaf640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25dc100a60 0x7f25dc19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e12b0640 1 -- 172.21.15.12:0/1808069187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25dc101400 msgr2=0x7f25dc1a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e12b0640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25dc101400 0x7f25dc1a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e12b0640 1 -- 172.21.15.12:0/1808069187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25dc100a60 msgr2=0x7f25dc19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e12b0640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25dc100a60 0x7f25dc19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e12b0640 1 -- 172.21.15.12:0/1808069187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc1a1960 con 0x7f25dc100100 2024-05-09T09:25:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e1ab1640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25dc101400 0x7f25dc1a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e0aaf640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25dc100a60 0x7f25dc19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.354+0000 7f25e12b0640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25dc100100 0x7f25dc199d50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f25d800e9f0 tx=0x7f25d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.355+0000 7f25ce7fc640 1 -- 172.21.15.12:0/1808069187 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d800cd80 con 0x7f25dc100100 2024-05-09T09:25:24.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.355+0000 7f25ce7fc640 1 -- 172.21.15.12:0/1808069187 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f25d80054f0 con 0x7f25dc100100 2024-05-09T09:25:24.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.355+0000 7f25e3514640 1 -- 172.21.15.12:0/1808069187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25dc1a1c50 con 0x7f25dc100100 2024-05-09T09:25:24.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.355+0000 7f25ce7fc640 1 -- 172.21.15.12:0/1808069187 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8010690 con 0x7f25dc100100 2024-05-09T09:25:24.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.355+0000 7f25e3514640 1 -- 172.21.15.12:0/1808069187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25dc1a21a0 con 0x7f25dc100100 2024-05-09T09:25:24.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.357+0000 7f25ce7fc640 1 -- 172.21.15.12:0/1808069187 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f25d8004f20 con 0x7f25dc100100 2024-05-09T09:25:24.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.357+0000 7f25e6229740 1 Processor -- start 2024-05-09T09:25:24.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.357+0000 7f25e6229740 1 -- start start 2024-05-09T09:25:24.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e6229740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 0x56142be6dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e6229740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6e2b0 0x56142be72db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e6229740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be73420 0x56142be758b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e6229740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56142bdd14d0 con 0x56142be6d8c0 2024-05-09T09:25:24.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e6229740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56142bd793e0 con 0x56142be73420 2024-05-09T09:25:24.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e6229740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56142bdcd490 con 0x56142be6e2b0 2024-05-09T09:25:24.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e1ab1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 0x56142be6dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e1ab1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 0x56142be6dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39462/0 (socket says 172.21.15.12:39462) 2024-05-09T09:25:24.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e1ab1640 1 -- 172.21.15.12:0/3027764308 learned_addr learned my addr 172.21.15.12:0/3027764308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25ce7fc640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f25c80a2c10 0x7f25c80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e0aaf640 1 --2- 172.21.15.12:0/3027764308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be73420 0x56142be758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e12b0640 1 --2- 172.21.15.12:0/3027764308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6e2b0 0x56142be72db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e1ab1640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f25c80a2c10 0x7f25c80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25ce7fc640 1 -- 172.21.15.12:0/1808069187 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f25d80cfa20 con 0x7f25dc100100 2024-05-09T09:25:24.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e1ab1640 1 -- 172.21.15.12:0/3027764308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6e2b0 msgr2=0x56142be72db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e1ab1640 1 --2- 172.21.15.12:0/3027764308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6e2b0 0x56142be72db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e1ab1640 1 -- 172.21.15.12:0/3027764308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be73420 msgr2=0x56142be758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e1ab1640 1 --2- 172.21.15.12:0/3027764308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be73420 0x56142be758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e1ab1640 1 -- 172.21.15.12:0/3027764308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142bae52d0 con 0x56142be6d8c0 2024-05-09T09:25:24.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e0aaf640 1 --2- 172.21.15.12:0/3027764308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be73420 0x56142be758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:24.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.358+0000 7f25e1ab1640 1 --2- 172.21.15.12:0/3027764308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 0x56142be6dce0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f25d4005fe0 tx=0x7f25d40316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138f38a6f1642afd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.359+0000 7f25c77fe640 1 -- 172.21.15.12:0/3027764308 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4004d40 con 0x56142be6d8c0 2024-05-09T09:25:24.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.359+0000 7f25c77fe640 1 -- 172.21.15.12:0/3027764308 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f25d404e070 con 0x56142be6d8c0 2024-05-09T09:25:24.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.359+0000 7f25c77fe640 1 -- 172.21.15.12:0/3027764308 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d402faf0 con 0x56142be6d8c0 2024-05-09T09:25:24.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.359+0000 7f25e6229740 1 -- 172.21.15.12:0/3027764308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 msgr2=0x56142be6dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.359+0000 7f25e1ab1640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f25c80a2c10 0x7f25c80a50d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f25d402f500 tx=0x7f25d404b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.359+0000 7f25e6229740 1 --2- 172.21.15.12:0/3027764308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 0x56142be6dce0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f25d4005fe0 tx=0x7f25d40316e0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.359+0000 7f25e6229740 1 -- 172.21.15.12:0/3027764308 shutdown_connections 2024-05-09T09:25:24.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.359+0000 7f25e6229740 1 --2- 172.21.15.12:0/3027764308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be73420 0x56142be758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.359+0000 7f25e6229740 1 --2- 172.21.15.12:0/3027764308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6e2b0 0x56142be72db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.359+0000 7f25e6229740 1 --2- 172.21.15.12:0/3027764308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 0x56142be6dce0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.359+0000 7f25e6229740 1 -- 172.21.15.12:0/3027764308 >> 172.21.15.12:0/3027764308 conn(0x56142be68f30 msgr2=0x56142be69330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.359+0000 7f25e6229740 1 -- 172.21.15.12:0/3027764308 shutdown_connections 2024-05-09T09:25:24.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.359+0000 7f25e6229740 1 -- 172.21.15.12:0/3027764308 wait complete. 2024-05-09T09:25:24.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e6229740 1 -- start start 2024-05-09T09:25:24.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e6229740 1 Processor -- start 2024-05-09T09:25:24.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e6229740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 0x56142be77d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e6229740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be6e2b0 0x56142be78270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e6229740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be73420 0x56142be7f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e6229740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56142be763d0 con 0x56142be6d8c0 2024-05-09T09:25:24.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e6229740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56142bd8c8e0 con 0x56142be6e2b0 2024-05-09T09:25:24.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e6229740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56142be76550 con 0x56142be73420 2024-05-09T09:25:24.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e6229740 10 client.0 _ll_register_callbacks cb 0x56142bd599c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:24.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e12b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 0x56142be77d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e12b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 0x56142be77d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39468/0 (socket says 172.21.15.12:39468) 2024-05-09T09:25:24.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e12b0640 1 -- 172.21.15.12:0/1106214793 learned_addr learned my addr 172.21.15.12:0/1106214793 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e0aaf640 1 --2- 172.21.15.12:0/1106214793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be6e2b0 0x56142be78270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.360+0000 7f25e1ab1640 1 --2- 172.21.15.12:0/1106214793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be73420 0x56142be7f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25e12b0640 1 -- 172.21.15.12:0/1106214793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be73420 msgr2=0x56142be7f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25e12b0640 1 --2- 172.21.15.12:0/1106214793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be73420 0x56142be7f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25e12b0640 1 -- 172.21.15.12:0/1106214793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be6e2b0 msgr2=0x56142be78270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25e12b0640 1 --2- 172.21.15.12:0/1106214793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be6e2b0 0x56142be78270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25e12b0640 1 -- 172.21.15.12:0/1106214793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56142bbaf6b0 con 0x56142be6d8c0 2024-05-09T09:25:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25e12b0640 1 --2- 172.21.15.12:0/1106214793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 0x56142be77d30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f25d8098790 tx=0x7f25d809b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8ac516a0fb3370 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25c4ff9640 1 -- 172.21.15.12:0/1106214793 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d80a0200 con 0x56142be6d8c0 2024-05-09T09:25:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25c4ff9640 1 -- 172.21.15.12:0/1106214793 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f25d80046c0 con 0x56142be6d8c0 2024-05-09T09:25:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25e6229740 1 -- 172.21.15.12:0/1106214793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d404e070 con 0x56142be6d8c0 2024-05-09T09:25:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25c4ff9640 1 -- 172.21.15.12:0/1106214793 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f25d80a4ac0 con 0x56142be6d8c0 2024-05-09T09:25:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25e6229740 10 client.15501 fetch_fsmap learned FSMap version 28 2024-05-09T09:25:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25e6229740 1 -- 172.21.15.12:0/1106214793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56142be82390 con 0x56142be6d8c0 2024-05-09T09:25:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25c4ff9640 1 -- 172.21.15.12:0/1106214793 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f25d404e070 con 0x56142be6d8c0 2024-05-09T09:25:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.361+0000 7f25c4ff9640 1 -- 172.21.15.12:0/1106214793 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f25d8097b10 con 0x56142be6d8c0 2024-05-09T09:25:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.362+0000 7f25e6229740 10 client.15501 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:25:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.362+0000 7f25e6229740 10 client.15501 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T09:25:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.362+0000 7f25e6229740 20 client.15501 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.362+0000 7f25e6229740 1 --2- 172.21.15.12:0/1106214793 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56142be849b0 0x56142be85e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.362+0000 7f25e6229740 4 client.15501 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.362+0000 7f25e6229740 1 -- 172.21.15.12:0/1106214793 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56142bddec00 con 0x56142be849b0 2024-05-09T09:25:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.362+0000 7f25e0aaf640 1 --2- 172.21.15.12:0/1106214793 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56142be849b0 0x56142be85e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.362+0000 7f25e0aaf640 1 --2- 172.21.15.12:0/1106214793 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56142be849b0 0x56142be85e20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=805754dc49ad6ac8 server_cookie=688567fc7ac7f7f2 in_seq=0 out_seq=0 2024-05-09T09:25:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.362+0000 7f25c4ff9640 10 client.15501 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25c4ff9640 1 -- 172.21.15.12:0/1106214793 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56142bddec00 con 0x56142be849b0 2024-05-09T09:25:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25c4ff9640 10 client.15501 handle_command_reply: tid=0 2024-05-09T09:25:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25e6229740 1 client.15501 shutdown 2024-05-09T09:25:24.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25e6229740 1 -- 172.21.15.12:0/1106214793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 msgr2=0x56142be77d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25e6229740 1 --2- 172.21.15.12:0/1106214793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 0x56142be77d30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f25d8098790 tx=0x7f25d809b870 comp rx=0 tx=0).stop 2024-05-09T09:25:24.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25e6229740 1 -- 172.21.15.12:0/1106214793 shutdown_connections 2024-05-09T09:25:24.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25e6229740 1 --2- 172.21.15.12:0/1106214793 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56142be849b0 0x56142be85e20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25e6229740 1 --2- 172.21.15.12:0/1106214793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be73420 0x56142be7f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25e6229740 1 --2- 172.21.15.12:0/1106214793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be6e2b0 0x56142be78270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25c4ff9640 0 client.15501 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:24.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25e6229740 1 --2- 172.21.15.12:0/1106214793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6d8c0 0x56142be77d30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25e6229740 1 -- 172.21.15.12:0/1106214793 >> 172.21.15.12:0/1106214793 conn(0x56142be68f30 msgr2=0x56142be68000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25e6229740 1 -- 172.21.15.12:0/1106214793 shutdown_connections 2024-05-09T09:25:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25e6229740 1 -- 172.21.15.12:0/1106214793 wait complete. 2024-05-09T09:25:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.363+0000 7f25e6229740 20 client.15501 trim_cache size 0 max 16384 2024-05-09T09:25:24.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e6229740 1 Processor -- start 2024-05-09T09:25:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e6229740 1 -- start start 2024-05-09T09:25:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e6229740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6d8c0 0x56142be72ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e6229740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6e2b0 0x56142be7f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e6229740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be73420 0x56142be6f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e6229740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56142be84160 con 0x56142be6e2b0 2024-05-09T09:25:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e6229740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56142be84830 con 0x56142be73420 2024-05-09T09:25:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e6229740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56142be84f00 con 0x56142be6d8c0 2024-05-09T09:25:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e0aaf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6e2b0 0x56142be7f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e0aaf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6e2b0 0x56142be7f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39476/0 (socket says 172.21.15.12:39476) 2024-05-09T09:25:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e0aaf640 1 -- 172.21.15.12:0/1176917801 learned_addr learned my addr 172.21.15.12:0/1176917801 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e12b0640 1 --2- 172.21.15.12:0/1176917801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6d8c0 0x56142be72ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e1ab1640 1 --2- 172.21.15.12:0/1176917801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be73420 0x56142be6f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e0aaf640 1 -- 172.21.15.12:0/1176917801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6d8c0 msgr2=0x56142be72ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e0aaf640 1 --2- 172.21.15.12:0/1176917801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6d8c0 0x56142be72ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e0aaf640 1 -- 172.21.15.12:0/1176917801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be73420 msgr2=0x56142be6f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e0aaf640 1 --2- 172.21.15.12:0/1176917801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be73420 0x56142be6f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.382+0000 7f25e0aaf640 1 -- 172.21.15.12:0/1176917801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142be6d500 con 0x56142be6e2b0 2024-05-09T09:25:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.383+0000 7f25e0aaf640 1 --2- 172.21.15.12:0/1176917801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6e2b0 0x56142be7f230 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x56142be867b0 tx=0x7f25d00094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80afe48634caf047 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.383+0000 7f25c6ffd640 1 -- 172.21.15.12:0/1176917801 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00140d0 con 0x56142be6e2b0 2024-05-09T09:25:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.383+0000 7f25c6ffd640 1 -- 172.21.15.12:0/1176917801 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f25d0014270 con 0x56142be6e2b0 2024-05-09T09:25:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.383+0000 7f25c6ffd640 1 -- 172.21.15.12:0/1176917801 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0004730 con 0x56142be6e2b0 2024-05-09T09:25:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.383+0000 7f25e6229740 1 -- 172.21.15.12:0/1176917801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6e2b0 msgr2=0x56142be7f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.383+0000 7f25e6229740 1 --2- 172.21.15.12:0/1176917801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6e2b0 0x56142be7f230 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x56142be867b0 tx=0x7f25d00094a0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.383+0000 7f25e6229740 1 -- 172.21.15.12:0/1176917801 shutdown_connections 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.383+0000 7f25e6229740 1 --2- 172.21.15.12:0/1176917801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be73420 0x56142be6f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.383+0000 7f25e6229740 1 --2- 172.21.15.12:0/1176917801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be6e2b0 0x56142be7f230 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.383+0000 7f25e6229740 1 --2- 172.21.15.12:0/1176917801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6d8c0 0x56142be72ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.383+0000 7f25e6229740 1 -- 172.21.15.12:0/1176917801 >> 172.21.15.12:0/1176917801 conn(0x56142be68f30 msgr2=0x56142be69310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.383+0000 7f25e6229740 1 -- 172.21.15.12:0/1176917801 shutdown_connections 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.383+0000 7f25e6229740 1 -- 172.21.15.12:0/1176917801 wait complete. 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e6229740 1 -- start start 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e6229740 1 Processor -- start 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e6229740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6d8c0 0x56142be7d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e6229740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be6e2b0 0x56142bee6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e6229740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be73420 0x56142bee8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e6229740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56142be85ca0 con 0x56142be73420 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e6229740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56142be855d0 con 0x56142be6e2b0 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e6229740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56142be86370 con 0x56142be6d8c0 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e6229740 10 client.0 _ll_register_callbacks cb 0x7f25d8005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e1ab1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be73420 0x56142bee8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e1ab1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be73420 0x56142bee8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39488/0 (socket says 172.21.15.12:39488) 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e1ab1640 1 -- 172.21.15.12:0/3576895536 learned_addr learned my addr 172.21.15.12:0/3576895536 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e0aaf640 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be6e2b0 0x56142bee6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.384+0000 7f25e12b0640 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6d8c0 0x56142be7d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25e1ab1640 1 -- 172.21.15.12:0/3576895536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6d8c0 msgr2=0x56142be7d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25e1ab1640 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6d8c0 0x56142be7d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25e1ab1640 1 -- 172.21.15.12:0/3576895536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be6e2b0 msgr2=0x56142bee6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25e1ab1640 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be6e2b0 0x56142bee6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25e1ab1640 1 -- 172.21.15.12:0/3576895536 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56142bee9420 con 0x56142be73420 2024-05-09T09:25:24.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25e12b0640 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6d8c0 0x56142be7d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25e0aaf640 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be6e2b0 0x56142bee6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25e1ab1640 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be73420 0x56142bee8e90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f25d4004af0 tx=0x7f25d4004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190cdf3553f14150 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25c4ff9640 1 -- 172.21.15.12:0/3576895536 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4004610 con 0x56142be73420 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25e6229740 1 -- 172.21.15.12:0/3576895536 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d0014270 con 0x56142be73420 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25c4ff9640 1 -- 172.21.15.12:0/3576895536 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f25d40047b0 con 0x56142be73420 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25c4ff9640 1 -- 172.21.15.12:0/3576895536 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f25d406b050 con 0x56142be73420 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25e6229740 10 client.15507 fetch_fsmap learned FSMap version 28 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25e6229740 1 -- 172.21.15.12:0/3576895536 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56142beea720 con 0x56142be73420 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25c4ff9640 1 -- 172.21.15.12:0/3576895536 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f25d0014270 con 0x56142be73420 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25c4ff9640 1 -- 172.21.15.12:0/3576895536 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f25d40305c0 con 0x56142be73420 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.385+0000 7f25e6229740 10 client.15507 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25e6229740 10 client.15507 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T09:25:24.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25e6229740 20 client.15507 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25e6229740 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56142beeee50 0x56142beedb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25e6229740 4 client.15507 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25e6229740 1 -- 172.21.15.12:0/3576895536 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "status"}) -- 0x56142bdd8f50 con 0x56142beeee50 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25e12b0640 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56142beeee50 0x56142beedb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25e12b0640 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56142beeee50 0x56142beedb60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fcbed4eeb67371 server_cookie=1378505c31e86fd in_seq=0 out_seq=0 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25c4ff9640 10 client.15507 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25c4ff9640 1 -- 172.21.15.12:0/3576895536 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56142bdd8f50 con 0x56142beeee50 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25c4ff9640 10 client.15507 handle_command_reply: tid=0 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25e6229740 1 client.15507 shutdown 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25e6229740 1 -- 172.21.15.12:0/3576895536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be73420 msgr2=0x56142bee8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25e6229740 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be73420 0x56142bee8e90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f25d4004af0 tx=0x7f25d4004f90 comp rx=0 tx=0).stop 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.386+0000 7f25e6229740 1 -- 172.21.15.12:0/3576895536 shutdown_connections 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e6229740 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56142beeee50 0x56142beedb60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e6229740 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56142be73420 0x56142bee8e90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e6229740 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56142be6e2b0 0x56142bee6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25c4ff9640 0 client.15507 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:24.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e6229740 1 --2- 172.21.15.12:0/3576895536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56142be6d8c0 0x56142be7d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e6229740 1 -- 172.21.15.12:0/3576895536 >> 172.21.15.12:0/3576895536 conn(0x56142be68f30 msgr2=0x56142be76fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e6229740 1 -- 172.21.15.12:0/3576895536 shutdown_connections 2024-05-09T09:25:24.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e6229740 1 -- 172.21.15.12:0/3576895536 wait complete. 2024-05-09T09:25:24.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e6229740 20 client.15507 trim_cache size 0 max 16384 2024-05-09T09:25:24.394 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:25:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "770e12d0-0de4-11ef-bc98-c7b262605968", 2024-05-09T09:25:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-05-09T09:25:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24523, 2024-05-09T09:25:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-09T09:25:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-09T09:25:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-09T09:25:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 134.07771801300001, 2024-05-09T09:25:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-05-09T09:25:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 93, 2024-05-09T09:25:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T09:25:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 149.07481866200001 2024-05-09T09:25:24.396 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:25:24.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e3514640 1 -- 172.21.15.12:0/1808069187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f25c80a2c10 msgr2=0x7f25c80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e3514640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f25c80a2c10 0x7f25c80a50d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f25d402f500 tx=0x7f25d404b000 comp rx=0 tx=0).stop 2024-05-09T09:25:24.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e3514640 1 -- 172.21.15.12:0/1808069187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25dc100100 msgr2=0x7f25dc199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e3514640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25dc100100 0x7f25dc199d50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f25d800e9f0 tx=0x7f25d800edf0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e3514640 1 -- 172.21.15.12:0/1808069187 shutdown_connections 2024-05-09T09:25:24.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e3514640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f25c80a2c10 0x7f25c80a50d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e3514640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25dc101400 0x7f25dc1a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e3514640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25dc100a60 0x7f25dc19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e3514640 1 --2- 172.21.15.12:0/1808069187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25dc100100 0x7f25dc199d50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e3514640 1 -- 172.21.15.12:0/1808069187 >> 172.21.15.12:0/1808069187 conn(0x7f25dc0fbd20 msgr2=0x7f25dc0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e3514640 1 -- 172.21.15.12:0/1808069187 shutdown_connections 2024-05-09T09:25:24.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:24.387+0000 7f25e3514640 1 -- 172.21.15.12:0/1808069187 wait complete. 2024-05-09T09:25:24.399 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T09:25:24.412 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T09:25:24.412 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T09:25:24.413 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T09:25:24.413 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-09T09:25:24.435 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-09T09:25:24.435 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-09T09:25:24.435 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-05-09T09:25:24.435 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T09:25:24.435 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T09:25:24.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:24.436 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-05-09T09:25:24.452 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-09T09:25:24.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:24.453 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:25:24.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:24 smithi012 ceph-mon[30140]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:25:24.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:24 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2489923454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:24.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:24 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1435778221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:24.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:24 smithi170 ceph-mon[33632]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:25:24.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:24 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2489923454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:24.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:24 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1435778221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:24.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:24 smithi016 ceph-mon[33335]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:25:24.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:24 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2489923454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:24.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:24 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1435778221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:24.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:24.511 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-05-09T09:25:24.603 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-09T09:25:24.603 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-09T09:25:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:25:26.382 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:26 smithi012 ceph-mon[30140]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:25:26.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:26 smithi170 ceph-mon[33632]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:25:26.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:26 smithi016 ceph-mon[33335]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:25:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.639+0000 7f17d11a8700 1 -- 172.21.15.12:0/3301618591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17cc1083b0 msgr2=0x7f17cc108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.639+0000 7f17d11a8700 1 --2- 172.21.15.12:0/3301618591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17cc1083b0 0x7f17cc108790 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009990 tx=0x7f17bc009ca0 comp rx=0 tx=0).stop 2024-05-09T09:25:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.639+0000 7f17d11a8700 1 -- 172.21.15.12:0/3301618591 shutdown_connections 2024-05-09T09:25:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.639+0000 7f17d11a8700 1 --2- 172.21.15.12:0/3301618591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17cc1012b0 0x7f17cc10f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.639+0000 7f17d11a8700 1 --2- 172.21.15.12:0/3301618591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17cc1083b0 0x7f17cc108790 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.639+0000 7f17d11a8700 1 --2- 172.21.15.12:0/3301618591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17cc100910 0x7f17cc100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.639+0000 7f17d11a8700 1 -- 172.21.15.12:0/3301618591 >> 172.21.15.12:0/3301618591 conn(0x7f17cc077890 msgr2=0x7f17cc0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.639+0000 7f17d11a8700 1 -- 172.21.15.12:0/3301618591 shutdown_connections 2024-05-09T09:25:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.639+0000 7f17d11a8700 1 -- 172.21.15.12:0/3301618591 wait complete. 2024-05-09T09:25:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.640+0000 7f17d11a8700 1 Processor -- start 2024-05-09T09:25:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.640+0000 7f17d11a8700 1 -- start start 2024-05-09T09:25:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.640+0000 7f17d11a8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17cc100910 0x7f17cc19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.640+0000 7f17d11a8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17cc1012b0 0x7f17cc19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.640+0000 7f17d11a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17cc1083b0 0x7f17cc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.640+0000 7f17d11a8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1125e0 con 0x7f17cc100910 2024-05-09T09:25:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.640+0000 7f17d11a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc112460 con 0x7f17cc1083b0 2024-05-09T09:25:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.640+0000 7f17d11a8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc112760 con 0x7f17cc1012b0 2024-05-09T09:25:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.640+0000 7f17cad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17cc100910 0x7f17cc19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.640+0000 7f17cad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17cc100910 0x7f17cc19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39516/0 (socket says 172.21.15.12:39516) 2024-05-09T09:25:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.640+0000 7f17cad9d700 1 -- 172.21.15.12:0/4141270560 learned_addr learned my addr 172.21.15.12:0/4141270560 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.641+0000 7f17c3fff700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17cc1012b0 0x7f17cc19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.641+0000 7f17cb59e700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17cc1083b0 0x7f17cc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.641+0000 7f17cad9d700 1 -- 172.21.15.12:0/4141270560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17cc1012b0 msgr2=0x7f17cc19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.641+0000 7f17cad9d700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17cc1012b0 0x7f17cc19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.641+0000 7f17cad9d700 1 -- 172.21.15.12:0/4141270560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17cc1083b0 msgr2=0x7f17cc198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.641+0000 7f17cad9d700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17cc1083b0 0x7f17cc198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.641+0000 7f17cad9d700 1 -- 172.21.15.12:0/4141270560 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc1990c0 con 0x7f17cc100910 2024-05-09T09:25:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.641+0000 7f17cad9d700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17cc100910 0x7f17cc19e7a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0038b0 tx=0x7f17bc009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.641+0000 7f17c8d99700 1 -- 172.21.15.12:0/4141270560 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc027070 con 0x7f17cc100910 2024-05-09T09:25:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.641+0000 7f17d11a8700 1 -- 172.21.15.12:0/4141270560 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc199350 con 0x7f17cc100910 2024-05-09T09:25:26.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.641+0000 7f17c8d99700 1 -- 172.21.15.12:0/4141270560 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f17bc004040 con 0x7f17cc100910 2024-05-09T09:25:26.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.641+0000 7f17c8d99700 1 -- 172.21.15.12:0/4141270560 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc022c80 con 0x7f17cc100910 2024-05-09T09:25:26.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.642+0000 7f17d11a8700 1 -- 172.21.15.12:0/4141270560 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc1a5670 con 0x7f17cc100910 2024-05-09T09:25:26.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.642+0000 7f17d11a8700 1 -- 172.21.15.12:0/4141270560 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc103c30 con 0x7f17cc100910 2024-05-09T09:25:26.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.645+0000 7f17c8d99700 1 -- 172.21.15.12:0/4141270560 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f17bc02a070 con 0x7f17cc100910 2024-05-09T09:25:26.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.645+0000 7f17c8d99700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17b80a2c20 0x7f17b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:26.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.645+0000 7f17c3fff700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17b80a2c20 0x7f17b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:26.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.645+0000 7f17c8d99700 1 -- 172.21.15.12:0/4141270560 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f17bc0d0590 con 0x7f17cc100910 2024-05-09T09:25:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.646+0000 7f17c3fff700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17b80a2c20 0x7f17b80a50e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0fe8b0 tx=0x7f17b400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.649+0000 7f17c8d99700 1 -- 172.21.15.12:0/4141270560 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f17bc022e20 con 0x7f17cc100910 2024-05-09T09:25:26.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.904+0000 7f17d11a8700 1 -- 172.21.15.12:0/4141270560 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f17cc02db80 con 0x7f17cc100910 2024-05-09T09:25:26.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.905+0000 7f17c8d99700 1 -- 172.21.15.12:0/4141270560 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7f17bc013070 con 0x7f17cc100910 2024-05-09T09:25:26.904 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T09:25:26.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.907+0000 7f17d11a8700 1 -- 172.21.15.12:0/4141270560 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17b80a2c20 msgr2=0x7f17b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:26.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.907+0000 7f17d11a8700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17b80a2c20 0x7f17b80a50e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0fe8b0 tx=0x7f17b400b410 comp rx=0 tx=0).stop 2024-05-09T09:25:26.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.907+0000 7f17d11a8700 1 -- 172.21.15.12:0/4141270560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17cc100910 msgr2=0x7f17cc19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:26.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.907+0000 7f17d11a8700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17cc100910 0x7f17cc19e7a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0038b0 tx=0x7f17bc009ee0 comp rx=0 tx=0).stop 2024-05-09T09:25:26.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.907+0000 7f17d11a8700 1 -- 172.21.15.12:0/4141270560 shutdown_connections 2024-05-09T09:25:26.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.907+0000 7f17d11a8700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17b80a2c20 0x7f17b80a50e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:26.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.907+0000 7f17d11a8700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17cc1083b0 0x7f17cc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:26.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.907+0000 7f17d11a8700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17cc100910 0x7f17cc19e7a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:26.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.907+0000 7f17d11a8700 1 --2- 172.21.15.12:0/4141270560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17cc1012b0 0x7f17cc19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:26.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.907+0000 7f17d11a8700 1 -- 172.21.15.12:0/4141270560 >> 172.21.15.12:0/4141270560 conn(0x7f17cc077890 msgr2=0x7f17cc0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:26.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.907+0000 7f17d11a8700 1 -- 172.21.15.12:0/4141270560 shutdown_connections 2024-05-09T09:25:26.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:26.907+0000 7f17d11a8700 1 -- 172.21.15.12:0/4141270560 wait complete. 2024-05-09T09:25:27.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:27 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4141270560' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T09:25:27.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:27 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/4141270560' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T09:25:27.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:27 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/4141270560' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T09:25:27.604 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T09:25:27.604 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T09:25:27.604 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-05-09T09:25:27.605 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T09:25:27.605 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T09:25:27.605 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T09:25:27.605 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T09:25:27.605 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T09:25:27.605 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T09:25:27.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:27.605 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-05-09T09:25:27.622 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T09:25:27.622 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T09:25:27.622 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T09:25:27.622 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-09T09:25:27.623 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-05-09T09:25:27.623 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-09T09:25:27.623 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T09:25:27.623 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-05-09T09:25:27.623 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-05-09T09:25:27.623 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T09:25:27.623 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-05-09T09:25:27.623 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-05-09T09:25:27.623 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T09:25:27.623 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-05-09T09:25:27.623 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-05-09T09:25:27.623 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T09:25:27.623 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42099sec preferred_lft 42099sec 2024-05-09T09:25:27.624 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-05-09T09:25:27.624 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-09T09:25:27.624 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T09:25:27.624 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-05-09T09:25:27.624 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-05-09T09:25:27.624 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T09:25:27.624 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:25:27.624 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-09T09:25:27.624 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2024-05-09T09:25:27.624 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2024-05-09T09:25:27.624 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2024-05-09T09:25:27.625 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T09:25:27.625 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-09T09:25:27.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:25:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:25:27.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:27.769 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T09:25:27.796 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-05-09T09:25:27.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:27.798 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-05-09T09:25:27.855 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-05-09T09:25:27.855 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T09:25:27.855 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T09:25:27.855 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T09:25:27.855 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T09:25:27.855 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:25:27.856 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-09T09:25:27.856 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T09:25:27.856 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T09:25:27.856 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T09:25:27.856 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-09T09:25:27.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:25:27.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:25:27.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:27.986 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-05-09T09:25:28.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:28.003 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2024-05-09T09:25:28.059 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:25:28.059 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-09T09:25:28.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T09:25:28.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T09:25:28.060 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-09T09:25:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:25:28.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:25:28.180 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T09:25:28.180 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:25:28.181 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-09T09:25:28.181 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T09:25:28.181 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T09:25:28.181 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T09:25:28.181 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T09:25:28.181 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T09:25:28.181 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-09T09:25:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:25:28.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:25:28.300 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:25:28.300 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-09T09:25:28.300 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-05-09T09:25:28.300 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T09:25:28.300 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-09T09:25:28.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:25:28.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:25:28.380 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-05-09T09:25:28.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T09:25:28.380 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T09:25:28.397 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T09:25:28.397 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T09:25:28.397 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T09:25:28.450 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-05-09T09:25:28.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:28 smithi012 ceph-mon[30140]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:25:28.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:28 smithi170 ceph-mon[33632]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:25:28.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:28 smithi016 ceph-mon[33335]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:25:28.509 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T09:25:28.566 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-05-09T09:25:28.566 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-05-09T09:25:28.566 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-05-09T09:25:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e88de82f85e98df57e228d7fe5d2b464e921cd3fbb86b352dc1f8680310d2fc5/merged 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/47eb820a76be2ed00b0b2bc110d5ddcb3ba3cf21aeb046cba3fa97500223e232/merged 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/f53bf39be5eb886f0c543698bd5796a232fbe79c324e29998c6fce40c8424a3a/merged 2024-05-09T09:25:28.568 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/4c2954d02a4a8aae844dd087075ae7189c1efe2363e62e195ff316bd2b233f72/merged 2024-05-09T09:25:28.569 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e86ea0c2dbb511b70d72ff40ff8767c6e5c13ca820da6152cdf9dcc8bbeda890/merged 2024-05-09T09:25:28.569 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/d329092564ce4ad60562f3d9a0ca5a315c01c60bbd3fb776d3760e89f0c8e393/merged 2024-05-09T09:25:28.569 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/a76753f2da79e4dd54a726d9da6582f14c0f0ea3885cdd7abb165d85cf305e5c/merged 2024-05-09T09:25:28.569 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e0710f4ce6abaa0e66e2ffe05da939b27fb50b3ca2ed31ad8a5d5674efedec36/merged 2024-05-09T09:25:28.569 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e6f0b9765e9120f74f8a213a2b59acafc93d309a03fb4f9d4ae2cc3ca5db1398/merged 2024-05-09T09:25:28.569 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/ef4517c95f8187307254aa2de801d47b459dc6bd5826e09086e0e3ba1fda2f9e/merged 2024-05-09T09:25:28.569 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-05-09T09:25:28.569 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T09:25:28.569 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T09:25:28.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:28.569 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-05-09T09:25:28.622 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T09:25:28.622 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3 --client_fs=cephfs) 2024-05-09T09:25:28.665 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-05-09T09:25:28.691 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T09:25:28.748 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-05-09T09:25:28.748 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-05-09T09:25:28.748 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-05-09T09:25:28.748 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-05-09T09:25:28.748 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-05-09T09:25:28.748 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-05-09T09:25:28.748 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-05-09T09:25:28.749 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-05-09T09:25:28.749 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-05-09T09:25:28.749 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-05-09T09:25:28.749 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-05-09T09:25:28.749 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-05-09T09:25:28.749 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:25:28.749 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-05-09T09:25:28.749 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-05-09T09:25:28.749 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-05-09T09:25:28.749 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-05-09T09:25:28.749 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-05-09T09:25:28.749 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-05-09T09:25:28.749 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e88de82f85e98df57e228d7fe5d2b464e921cd3fbb86b352dc1f8680310d2fc5/merged 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/47eb820a76be2ed00b0b2bc110d5ddcb3ba3cf21aeb046cba3fa97500223e232/merged 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/f53bf39be5eb886f0c543698bd5796a232fbe79c324e29998c6fce40c8424a3a/merged 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/4c2954d02a4a8aae844dd087075ae7189c1efe2363e62e195ff316bd2b233f72/merged 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e86ea0c2dbb511b70d72ff40ff8767c6e5c13ca820da6152cdf9dcc8bbeda890/merged 2024-05-09T09:25:28.750 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/d329092564ce4ad60562f3d9a0ca5a315c01c60bbd3fb776d3760e89f0c8e393/merged 2024-05-09T09:25:28.751 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/a76753f2da79e4dd54a726d9da6582f14c0f0ea3885cdd7abb165d85cf305e5c/merged 2024-05-09T09:25:28.751 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e0710f4ce6abaa0e66e2ffe05da939b27fb50b3ca2ed31ad8a5d5674efedec36/merged 2024-05-09T09:25:28.751 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e6f0b9765e9120f74f8a213a2b59acafc93d309a03fb4f9d4ae2cc3ca5db1398/merged 2024-05-09T09:25:28.751 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/ef4517c95f8187307254aa2de801d47b459dc6bd5826e09086e0e3ba1fda2f9e/merged 2024-05-09T09:25:28.751 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-05-09T09:25:28.751 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T09:25:28.751 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T09:25:28.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:28.751 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-05-09T09:25:28.752 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi012.stderr:2024-05-09T09:25:28.751+0000 7f3089a0e480 -1 init, newargv = 0x55d7c80653a0 newargc=15 2024-05-09T09:25:28.752 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi012.stderr:ceph-fuse[85012]: starting ceph client 2024-05-09T09:25:28.773 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi012.stderr:ceph-fuse[85012]: starting fuse 2024-05-09T09:25:29.870 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-05-09T09:25:29.992 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T09:25:30.065 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-05-09T09:25:30.066 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-05-09T09:25:30.066 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-05-09T09:25:30.066 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-05-09T09:25:30.067 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-05-09T09:25:30.067 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-05-09T09:25:30.067 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-05-09T09:25:30.067 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-05-09T09:25:30.067 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-05-09T09:25:30.067 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-05-09T09:25:30.067 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-05-09T09:25:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-05-09T09:25:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:25:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-05-09T09:25:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-05-09T09:25:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-05-09T09:25:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-05-09T09:25:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-05-09T09:25:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-05-09T09:25:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T09:25:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T09:25:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T09:25:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T09:25:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-05-09T09:25:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-05-09T09:25:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-05-09T09:25:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e88de82f85e98df57e228d7fe5d2b464e921cd3fbb86b352dc1f8680310d2fc5/merged 2024-05-09T09:25:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/47eb820a76be2ed00b0b2bc110d5ddcb3ba3cf21aeb046cba3fa97500223e232/merged 2024-05-09T09:25:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-05-09T09:25:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:25:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/f53bf39be5eb886f0c543698bd5796a232fbe79c324e29998c6fce40c8424a3a/merged 2024-05-09T09:25:30.072 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/4c2954d02a4a8aae844dd087075ae7189c1efe2363e62e195ff316bd2b233f72/merged 2024-05-09T09:25:30.072 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e86ea0c2dbb511b70d72ff40ff8767c6e5c13ca820da6152cdf9dcc8bbeda890/merged 2024-05-09T09:25:30.072 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/d329092564ce4ad60562f3d9a0ca5a315c01c60bbd3fb776d3760e89f0c8e393/merged 2024-05-09T09:25:30.072 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/a76753f2da79e4dd54a726d9da6582f14c0f0ea3885cdd7abb165d85cf305e5c/merged 2024-05-09T09:25:30.072 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e0710f4ce6abaa0e66e2ffe05da939b27fb50b3ca2ed31ad8a5d5674efedec36/merged 2024-05-09T09:25:30.073 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e6f0b9765e9120f74f8a213a2b59acafc93d309a03fb4f9d4ae2cc3ca5db1398/merged 2024-05-09T09:25:30.073 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/ef4517c95f8187307254aa2de801d47b459dc6bd5826e09086e0e3ba1fda2f9e/merged 2024-05-09T09:25:30.073 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-05-09T09:25:30.073 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T09:25:30.073 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T09:25:30.073 INFO:teuthology.orchestra.run.smithi012.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-09T09:25:30.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:30.074 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-05-09T09:25:30.137 INFO:teuthology.orchestra.run.smithi012.stdout:114 2024-05-09T09:25:30.138 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-09T09:25:30.140 DEBUG:teuthology.orchestra.run.smithi012:> sudo stdin-killer -- python3 -c ' 2024-05-09T09:25:30.140 DEBUG:teuthology.orchestra.run.smithi012:> import glob 2024-05-09T09:25:30.140 DEBUG:teuthology.orchestra.run.smithi012:> import re 2024-05-09T09:25:30.141 DEBUG:teuthology.orchestra.run.smithi012:> import os 2024-05-09T09:25:30.141 DEBUG:teuthology.orchestra.run.smithi012:> import subprocess 2024-05-09T09:25:30.141 DEBUG:teuthology.orchestra.run.smithi012:> 2024-05-09T09:25:30.143 DEBUG:teuthology.orchestra.run.smithi012:> def _find_admin_socket(client_name): 2024-05-09T09:25:30.143 DEBUG:teuthology.orchestra.run.smithi012:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T09:25:30.143 DEBUG:teuthology.orchestra.run.smithi012:> files = glob.glob(asok_path) 2024-05-09T09:25:30.143 DEBUG:teuthology.orchestra.run.smithi012:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T09:25:30.143 DEBUG:teuthology.orchestra.run.smithi012:> 2024-05-09T09:25:30.143 DEBUG:teuthology.orchestra.run.smithi012:> # Given a non-glob path, it better be there 2024-05-09T09:25:30.143 DEBUG:teuthology.orchestra.run.smithi012:> if "*" not in asok_path: 2024-05-09T09:25:30.143 DEBUG:teuthology.orchestra.run.smithi012:> assert(len(files) == 1) 2024-05-09T09:25:30.143 DEBUG:teuthology.orchestra.run.smithi012:> return files[0] 2024-05-09T09:25:30.144 DEBUG:teuthology.orchestra.run.smithi012:> 2024-05-09T09:25:30.144 DEBUG:teuthology.orchestra.run.smithi012:> for f in files: 2024-05-09T09:25:30.144 DEBUG:teuthology.orchestra.run.smithi012:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T09:25:30.144 DEBUG:teuthology.orchestra.run.smithi012:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T09:25:30.144 DEBUG:teuthology.orchestra.run.smithi012:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T09:25:30.144 DEBUG:teuthology.orchestra.run.smithi012:> contents = proc_f.read() 2024-05-09T09:25:30.144 DEBUG:teuthology.orchestra.run.smithi012:> if mountpoint in contents: 2024-05-09T09:25:30.144 DEBUG:teuthology.orchestra.run.smithi012:> return f 2024-05-09T09:25:30.144 DEBUG:teuthology.orchestra.run.smithi012:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T09:25:30.144 DEBUG:teuthology.orchestra.run.smithi012:> 2024-05-09T09:25:30.144 DEBUG:teuthology.orchestra.run.smithi012:> print(_find_admin_socket("client.0")) 2024-05-09T09:25:30.144 DEBUG:teuthology.orchestra.run.smithi012:> ' 2024-05-09T09:25:30.274 INFO:teuthology.orchestra.run.smithi012.stdout:/var/run/ceph/ceph-client.0.85012.asok 2024-05-09T09:25:30.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:25:30.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:30 smithi012 ceph-mon[30140]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T09:25:30.286 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.85012.asok 2024-05-09T09:25:30.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:30.286 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.85012.asok status 2024-05-09T09:25:30.436 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:25:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "metadata": { 2024-05-09T09:25:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T09:25:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T09:25:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "entity_id": "0", 2024-05-09T09:25:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": "smithi012", 2024-05-09T09:25:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T09:25:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "pid": "85012", 2024-05-09T09:25:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "root": "/volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3" 2024-05-09T09:25:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:25:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_count": 0, 2024-05-09T09:25:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_pinned_count": 0, 2024-05-09T09:25:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 15519, 2024-05-09T09:25:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "inst": { 2024-05-09T09:25:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "name": { 2024-05-09T09:25:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "client", 2024-05-09T09:25:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "num": 15519 2024-05-09T09:25:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:25:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-05-09T09:25:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-05-09T09:25:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2024-05-09T09:25:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 832795604 2024-05-09T09:25:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:25:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:25:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-05-09T09:25:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-05-09T09:25:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2024-05-09T09:25:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 832795604 2024-05-09T09:25:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:25:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "inst_str": "client.15519 192.168.0.1:0/832795604", 2024-05-09T09:25:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "addr_str": "192.168.0.1:0/832795604", 2024-05-09T09:25:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "inode_count": 5, 2024-05-09T09:25:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_epoch": 28, 2024-05-09T09:25:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch": 93, 2024-05-09T09:25:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch_barrier": 0, 2024-05-09T09:25:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklisted": false, 2024-05-09T09:25:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs" 2024-05-09T09:25:30.440 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:25:30.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:30.449 DEBUG:teuthology.orchestra.run.smithi012:> stat --file-system '--printf=%T 2024-05-09T09:25:30.449 DEBUG:teuthology.orchestra.run.smithi012:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T09:25:30.466 INFO:teuthology.orchestra.run.smithi012.stdout:fuseblk 2024-05-09T09:25:30.466 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T09:25:30.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:30.467 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T09:25:30.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:30 smithi170 ceph-mon[33632]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T09:25:30.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:30 smithi016 ceph-mon[33335]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T09:25:30.542 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T09:25:31.053 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T09:25:31.054 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-09T09:25:31.054 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-09T09:25:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.172+0000 7f4c51838640 1 -- 172.21.15.12:0/2341078694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c102340 msgr2=0x7f4c4c108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.172+0000 7f4c51838640 1 --2- 172.21.15.12:0/2341078694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c102340 0x7f4c4c108bd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009ca0 tx=0x7f4c4402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.172+0000 7f4c51838640 1 -- 172.21.15.12:0/2341078694 shutdown_connections 2024-05-09T09:25:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.172+0000 7f4c51838640 1 --2- 172.21.15.12:0/2341078694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c102340 0x7f4c4c108bd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.172+0000 7f4c51838640 1 --2- 172.21.15.12:0/2341078694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c4c101980 0x7f4c4c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.172+0000 7f4c51838640 1 --2- 172.21.15.12:0/2341078694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c4c100790 0x7f4c4c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.172+0000 7f4c51838640 1 -- 172.21.15.12:0/2341078694 >> 172.21.15.12:0/2341078694 conn(0x7f4c4c0fbd00 msgr2=0x7f4c4c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.172+0000 7f4c51838640 1 -- 172.21.15.12:0/2341078694 shutdown_connections 2024-05-09T09:25:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.172+0000 7f4c51838640 1 -- 172.21.15.12:0/2341078694 wait complete. 2024-05-09T09:25:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.172+0000 7f4c51838640 1 Processor -- start 2024-05-09T09:25:31.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.172+0000 7f4c51838640 1 -- start start 2024-05-09T09:25:31.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c51838640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c100790 0x7f4c4c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c51838640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c4c101980 0x7f4c4c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c51838640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c4c102340 0x7f4c4c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c51838640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c10d500 con 0x7f4c4c100790 2024-05-09T09:25:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c51838640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c10d380 con 0x7f4c4c102340 2024-05-09T09:25:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c51838640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c10d680 con 0x7f4c4c101980 2024-05-09T09:25:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c4affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c100790 0x7f4c4c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c4affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c100790 0x7f4c4c197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39524/0 (socket says 172.21.15.12:39524) 2024-05-09T09:25:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c4affd640 1 -- 172.21.15.12:0/2808359683 learned_addr learned my addr 172.21.15.12:0/2808359683 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c4a7fc640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c4c101980 0x7f4c4c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c4b7fe640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c4c102340 0x7f4c4c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c4affd640 1 -- 172.21.15.12:0/2808359683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c4c101980 msgr2=0x7f4c4c198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c4affd640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c4c101980 0x7f4c4c198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c4affd640 1 -- 172.21.15.12:0/2808359683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c4c102340 msgr2=0x7f4c4c19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:31.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c4affd640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c4c102340 0x7f4c4c19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:31.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c4affd640 1 -- 172.21.15.12:0/2808359683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c19f700 con 0x7f4c4c100790 2024-05-09T09:25:31.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c4affd640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c100790 0x7f4c4c197ae0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00c970 tx=0x7f4c3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:31.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.173+0000 7f4c50836640 1 -- 172.21.15.12:0/2808359683 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c004220 con 0x7f4c4c100790 2024-05-09T09:25:31.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.174+0000 7f4c50836640 1 -- 172.21.15.12:0/2808359683 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4c3c0043c0 con 0x7f4c4c100790 2024-05-09T09:25:31.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.174+0000 7f4c51838640 1 -- 172.21.15.12:0/2808359683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4c19f9f0 con 0x7f4c4c100790 2024-05-09T09:25:31.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.174+0000 7f4c50836640 1 -- 172.21.15.12:0/2808359683 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00f450 con 0x7f4c4c100790 2024-05-09T09:25:31.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.174+0000 7f4c51838640 1 -- 172.21.15.12:0/2808359683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c4c19ff40 con 0x7f4c4c100790 2024-05-09T09:25:31.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.175+0000 7f4c51838640 1 -- 172.21.15.12:0/2808359683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4c107ab0 con 0x7f4c4c100790 2024-05-09T09:25:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.176+0000 7f4c50836640 1 -- 172.21.15.12:0/2808359683 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4c3c0079b0 con 0x7f4c4c100790 2024-05-09T09:25:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.176+0000 7f4c50836640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4c380a2c30 0x7f4c380a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.177+0000 7f4c4a7fc640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4c380a2c30 0x7f4c380a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.177+0000 7f4c50836640 1 -- 172.21.15.12:0/2808359683 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f4c3c0cf1b0 con 0x7f4c4c100790 2024-05-09T09:25:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.177+0000 7f4c4a7fc640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4c380a2c30 0x7f4c380a50f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c198ff0 tx=0x7f4c3400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:31.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.181+0000 7f4c50836640 1 -- 172.21.15.12:0/2808359683 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4c3c014030 con 0x7f4c4c100790 2024-05-09T09:25:31.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:31.409+0000 7f4c51838640 1 -- 172.21.15.12:0/2808359683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f4c4c02d1c0 con 0x7f4c4c100790 2024-05-09T09:25:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.219+0000 7f4c50836640 1 -- 172.21.15.12:0/2808359683 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7f4c3c097020 con 0x7f4c4c100790 2024-05-09T09:25:32.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.222+0000 7f4c51838640 1 -- 172.21.15.12:0/2808359683 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4c380a2c30 msgr2=0x7f4c380a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:32.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.222+0000 7f4c51838640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4c380a2c30 0x7f4c380a50f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c198ff0 tx=0x7f4c3400b390 comp rx=0 tx=0).stop 2024-05-09T09:25:32.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.222+0000 7f4c51838640 1 -- 172.21.15.12:0/2808359683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c100790 msgr2=0x7f4c4c197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:32.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.222+0000 7f4c51838640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c100790 0x7f4c4c197ae0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00c970 tx=0x7f4c3c00ce30 comp rx=0 tx=0).stop 2024-05-09T09:25:32.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.222+0000 7f4c51838640 1 -- 172.21.15.12:0/2808359683 shutdown_connections 2024-05-09T09:25:32.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.222+0000 7f4c51838640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4c380a2c30 0x7f4c380a50f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.222+0000 7f4c51838640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c4c102340 0x7f4c4c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.222+0000 7f4c51838640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c4c101980 0x7f4c4c198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.222+0000 7f4c51838640 1 --2- 172.21.15.12:0/2808359683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c100790 0x7f4c4c197ae0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.222+0000 7f4c51838640 1 -- 172.21.15.12:0/2808359683 >> 172.21.15.12:0/2808359683 conn(0x7f4c4c0fbd00 msgr2=0x7f4c4c104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:32.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.222+0000 7f4c51838640 1 -- 172.21.15.12:0/2808359683 shutdown_connections 2024-05-09T09:25:32.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.222+0000 7f4c51838640 1 -- 172.21.15.12:0/2808359683 wait complete. 2024-05-09T09:25:32.237 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-09T09:25:32.440 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-09T09:25:32.623 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:25:32.625 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:32 smithi016 ceph-mon[33335]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T09:25:32.625 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:32 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2808359683' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T09:25:32.626 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:32 smithi170 ceph-mon[33632]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T09:25:32.626 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:32 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2808359683' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T09:25:32.626 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:32 smithi012 ceph-mon[30140]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T09:25:32.626 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:32 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2808359683' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T09:25:32.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.745+0000 7f8cdea72640 1 -- 172.21.15.12:0/3844442823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cd8104c40 msgr2=0x7f8cd8107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:32.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.745+0000 7f8cdea72640 1 --2- 172.21.15.12:0/3844442823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cd8104c40 0x7f8cd8107030 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009ca0 tx=0x7f8cc802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.745+0000 7f8cdea72640 1 -- 172.21.15.12:0/3844442823 shutdown_connections 2024-05-09T09:25:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.745+0000 7f8cdea72640 1 --2- 172.21.15.12:0/3844442823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd8107570 0x7f8cd81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.745+0000 7f8cdea72640 1 --2- 172.21.15.12:0/3844442823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cd8104c40 0x7f8cd8107030 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.745+0000 7f8cdea72640 1 --2- 172.21.15.12:0/3844442823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd8102310 0x7f8cd8104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.745+0000 7f8cdea72640 1 -- 172.21.15.12:0/3844442823 >> 172.21.15.12:0/3844442823 conn(0x7f8cd80fbd20 msgr2=0x7f8cd80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.745+0000 7f8cdea72640 1 -- 172.21.15.12:0/3844442823 shutdown_connections 2024-05-09T09:25:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.745+0000 7f8cdea72640 1 -- 172.21.15.12:0/3844442823 wait complete. 2024-05-09T09:25:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.746+0000 7f8cdea72640 1 Processor -- start 2024-05-09T09:25:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.746+0000 7f8cdea72640 1 -- start start 2024-05-09T09:25:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.746+0000 7f8cdea72640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd8102310 0x7f8cd8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.746+0000 7f8cdea72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd8104c40 0x7f8cd8198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.746+0000 7f8cdea72640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cd8107570 0x7f8cd819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.746+0000 7f8cdea72640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd810d4f0 con 0x7f8cd8107570 2024-05-09T09:25:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.746+0000 7f8cdea72640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd810d370 con 0x7f8cd8104c40 2024-05-09T09:25:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.746+0000 7f8cdea72640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd810d670 con 0x7f8cd8102310 2024-05-09T09:25:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.746+0000 7f8cdd00f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cd8107570 0x7f8cd819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.746+0000 7f8cdd00f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cd8107570 0x7f8cd819f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33762/0 (socket says 172.21.15.12:33762) 2024-05-09T09:25:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.746+0000 7f8cdc80e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd8102310 0x7f8cd8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.746+0000 7f8cdd00f640 1 -- 172.21.15.12:0/2433564080 learned_addr learned my addr 172.21.15.12:0/2433564080 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.747+0000 7f8cd7fff640 1 --2- 172.21.15.12:0/2433564080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd8104c40 0x7f8cd8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.747+0000 7f8cdd00f640 1 -- 172.21.15.12:0/2433564080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd8102310 msgr2=0x7f8cd8197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.747+0000 7f8cdd00f640 1 --2- 172.21.15.12:0/2433564080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd8102310 0x7f8cd8197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.747+0000 7f8cdd00f640 1 -- 172.21.15.12:0/2433564080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd8104c40 msgr2=0x7f8cd8198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:32.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.747+0000 7f8cdd00f640 1 --2- 172.21.15.12:0/2433564080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd8104c40 0x7f8cd8198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.747+0000 7f8cdd00f640 1 -- 172.21.15.12:0/2433564080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd819f830 con 0x7f8cd8107570 2024-05-09T09:25:32.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.747+0000 7f8cdd00f640 1 --2- 172.21.15.12:0/2433564080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cd8107570 0x7f8cd819f190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000e970 tx=0x7f8cd000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:32.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.747+0000 7f8cd5ffb640 1 -- 172.21.15.12:0/2433564080 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd000ccc0 con 0x7f8cd8107570 2024-05-09T09:25:32.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.747+0000 7f8cdea72640 1 -- 172.21.15.12:0/2433564080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd819fb20 con 0x7f8cd8107570 2024-05-09T09:25:32.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.747+0000 7f8cd5ffb640 1 -- 172.21.15.12:0/2433564080 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8cd000ce60 con 0x7f8cd8107570 2024-05-09T09:25:32.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.747+0000 7f8cd5ffb640 1 -- 172.21.15.12:0/2433564080 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0010430 con 0x7f8cd8107570 2024-05-09T09:25:32.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.748+0000 7f8cdea72640 1 -- 172.21.15.12:0/2433564080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd81a0070 con 0x7f8cd8107570 2024-05-09T09:25:32.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.748+0000 7f8cdea72640 1 -- 172.21.15.12:0/2433564080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd80fd810 con 0x7f8cd8107570 2024-05-09T09:25:32.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.749+0000 7f8cd5ffb640 1 -- 172.21.15.12:0/2433564080 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8cd0005000 con 0x7f8cd8107570 2024-05-09T09:25:32.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.750+0000 7f8cd5ffb640 1 --2- 172.21.15.12:0/2433564080 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8cc40a2c10 0x7f8cc40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:32.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.750+0000 7f8cdc80e640 1 --2- 172.21.15.12:0/2433564080 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8cc40a2c10 0x7f8cc40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:32.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.750+0000 7f8cd5ffb640 1 -- 172.21.15.12:0/2433564080 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f8cd00cf810 con 0x7f8cd8107570 2024-05-09T09:25:32.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.751+0000 7f8cdc80e640 1 --2- 172.21.15.12:0/2433564080 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8cc40a2c10 0x7f8cc40a50d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8cd81990f0 tx=0x7f8cc000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:32.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.755+0000 7f8cd5ffb640 1 -- 172.21.15.12:0/2433564080 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8cd009c3a0 con 0x7f8cd8107570 2024-05-09T09:25:32.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.942+0000 7f8cdea72640 1 -- 172.21.15.12:0/2433564080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cd8062430 con 0x7f8cd8107570 2024-05-09T09:25:32.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.943+0000 7f8cd5ffb640 1 -- 172.21.15.12:0/2433564080 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24762 (secure 0 0 0) 0x7f8cd009bca0 con 0x7f8cd8107570 2024-05-09T09:25:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":93,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:25:06.152764+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:25:32.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.945+0000 7f8cdea72640 1 -- 172.21.15.12:0/2433564080 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8cc40a2c10 msgr2=0x7f8cc40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:32.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.945+0000 7f8cdea72640 1 --2- 172.21.15.12:0/2433564080 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8cc40a2c10 0x7f8cc40a50d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8cd81990f0 tx=0x7f8cc000b390 comp rx=0 tx=0).stop 2024-05-09T09:25:32.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.945+0000 7f8cdea72640 1 -- 172.21.15.12:0/2433564080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cd8107570 msgr2=0x7f8cd819f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:32.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.945+0000 7f8cdea72640 1 --2- 172.21.15.12:0/2433564080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cd8107570 0x7f8cd819f190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000e970 tx=0x7f8cd000ed70 comp rx=0 tx=0).stop 2024-05-09T09:25:32.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.945+0000 7f8cdea72640 1 -- 172.21.15.12:0/2433564080 shutdown_connections 2024-05-09T09:25:32.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.945+0000 7f8cdea72640 1 --2- 172.21.15.12:0/2433564080 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8cc40a2c10 0x7f8cc40a50d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.945+0000 7f8cdea72640 1 --2- 172.21.15.12:0/2433564080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cd8107570 0x7f8cd819f190 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.945+0000 7f8cdea72640 1 --2- 172.21.15.12:0/2433564080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd8104c40 0x7f8cd8198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.945+0000 7f8cdea72640 1 --2- 172.21.15.12:0/2433564080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd8102310 0x7f8cd8197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.945+0000 7f8cdea72640 1 -- 172.21.15.12:0/2433564080 >> 172.21.15.12:0/2433564080 conn(0x7f8cd80fbd20 msgr2=0x7f8cd8102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.945+0000 7f8cdea72640 1 -- 172.21.15.12:0/2433564080 shutdown_connections 2024-05-09T09:25:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:32.945+0000 7f8cdea72640 1 -- 172.21.15.12:0/2433564080 wait complete. 2024-05-09T09:25:32.956 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:23:08.167107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:23:09.065670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:23:12.773640+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:25:32.957 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:25:33.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.066+0000 7f6771e15640 1 -- 172.21.15.12:0/625045249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f676c101960 msgr2=0x7f676c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.066+0000 7f6771e15640 1 --2- 172.21.15.12:0/625045249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f676c101960 0x7f676c101de0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6754009900 tx=0x7f675402a410 comp rx=0 tx=0).stop 2024-05-09T09:25:33.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.066+0000 7f6771e15640 1 -- 172.21.15.12:0/625045249 shutdown_connections 2024-05-09T09:25:33.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.066+0000 7f6771e15640 1 --2- 172.21.15.12:0/625045249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c102320 0x7f676c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.066+0000 7f6771e15640 1 --2- 172.21.15.12:0/625045249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f676c101960 0x7f676c101de0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.066+0000 7f6771e15640 1 --2- 172.21.15.12:0/625045249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f676c100770 0x7f676c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.066+0000 7f6771e15640 1 -- 172.21.15.12:0/625045249 >> 172.21.15.12:0/625045249 conn(0x7f676c0fbd20 msgr2=0x7f676c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:33.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.066+0000 7f6771e15640 1 -- 172.21.15.12:0/625045249 shutdown_connections 2024-05-09T09:25:33.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.066+0000 7f6771e15640 1 -- 172.21.15.12:0/625045249 wait complete. 2024-05-09T09:25:33.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.067+0000 7f6771e15640 1 Processor -- start 2024-05-09T09:25:33.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.067+0000 7f6771e15640 1 -- start start 2024-05-09T09:25:33.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.067+0000 7f6771e15640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f676c100770 0x7f676c104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:33.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.067+0000 7f6771e15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c101960 0x7f676c102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:33.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.067+0000 7f6771e15640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f676c102320 0x7f676c103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:33.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.067+0000 7f6771e15640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f676c10c3d0 con 0x7f676c100770 2024-05-09T09:25:33.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.067+0000 7f6771e15640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f676c10c250 con 0x7f676c101960 2024-05-09T09:25:33.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.067+0000 7f6771e15640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f676c10c550 con 0x7f676c102320 2024-05-09T09:25:33.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.067+0000 7f676b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f676c100770 0x7f676c104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:33.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.067+0000 7f676b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f676c100770 0x7f676c104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33774/0 (socket says 172.21.15.12:33774) 2024-05-09T09:25:33.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.067+0000 7f676b7fe640 1 -- 172.21.15.12:0/383637778 learned_addr learned my addr 172.21.15.12:0/383637778 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:33.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.067+0000 7f6763fff640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c101960 0x7f676c102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:33.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.068+0000 7f676bfff640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f676c102320 0x7f676c103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:33.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.068+0000 7f676b7fe640 1 -- 172.21.15.12:0/383637778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f676c102320 msgr2=0x7f676c103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.068+0000 7f676b7fe640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f676c102320 0x7f676c103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.068+0000 7f676b7fe640 1 -- 172.21.15.12:0/383637778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c101960 msgr2=0x7f676c102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.068+0000 7f676b7fe640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c101960 0x7f676c102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.068+0000 7f676b7fe640 1 -- 172.21.15.12:0/383637778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c103b10 con 0x7f676c100770 2024-05-09T09:25:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.068+0000 7f676bfff640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f676c102320 0x7f676c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.068+0000 7f676b7fe640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f676c100770 0x7f676c104700 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f675c00d770 tx=0x7f675c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.068+0000 7f67697fa640 1 -- 172.21.15.12:0/383637778 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c017070 con 0x7f676c100770 2024-05-09T09:25:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.068+0000 7f6771e15640 1 -- 172.21.15.12:0/383637778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676c1a1cc0 con 0x7f676c100770 2024-05-09T09:25:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.068+0000 7f67697fa640 1 -- 172.21.15.12:0/383637778 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f675c009c30 con 0x7f676c100770 2024-05-09T09:25:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.068+0000 7f67697fa640 1 -- 172.21.15.12:0/383637778 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c003e10 con 0x7f676c100770 2024-05-09T09:25:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.068+0000 7f6771e15640 1 -- 172.21.15.12:0/383637778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f676c1a2190 con 0x7f676c100770 2024-05-09T09:25:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.069+0000 7f6771e15640 1 -- 172.21.15.12:0/383637778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f676c1a2360 con 0x7f676c100770 2024-05-09T09:25:33.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.070+0000 7f67697fa640 1 -- 172.21.15.12:0/383637778 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f675c003fb0 con 0x7f676c100770 2024-05-09T09:25:33.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.071+0000 7f67697fa640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67580a2b90 0x7f67580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:33.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.071+0000 7f6763fff640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67580a2b90 0x7f67580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:33.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.071+0000 7f67697fa640 1 -- 172.21.15.12:0/383637778 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f675c0cf1b0 con 0x7f676c100770 2024-05-09T09:25:33.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.072+0000 7f6763fff640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67580a2b90 0x7f67580a5050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6754000c00 tx=0x7f6754031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:33.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.075+0000 7f67697fa640 1 -- 172.21.15.12:0/383637778 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f675c09bd40 con 0x7f676c100770 2024-05-09T09:25:33.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.261+0000 7f6771e15640 1 -- 172.21.15.12:0/383637778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f676c1a2710 con 0x7f676c100770 2024-05-09T09:25:33.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.262+0000 7f67697fa640 1 -- 172.21.15.12:0/383637778 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f675c097030 con 0x7f676c100770 2024-05-09T09:25:33.261 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:25:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.263+0000 7f6771e15640 1 -- 172.21.15.12:0/383637778 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67580a2b90 msgr2=0x7f67580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.264+0000 7f6771e15640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67580a2b90 0x7f67580a5050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6754000c00 tx=0x7f6754031510 comp rx=0 tx=0).stop 2024-05-09T09:25:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.264+0000 7f6771e15640 1 -- 172.21.15.12:0/383637778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f676c100770 msgr2=0x7f676c104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.264+0000 7f6771e15640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f676c100770 0x7f676c104700 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f675c00d770 tx=0x7f675c00dc30 comp rx=0 tx=0).stop 2024-05-09T09:25:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.264+0000 7f6771e15640 1 -- 172.21.15.12:0/383637778 shutdown_connections 2024-05-09T09:25:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.264+0000 7f6771e15640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67580a2b90 0x7f67580a5050 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.264+0000 7f6771e15640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f676c102320 0x7f676c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.264+0000 7f6771e15640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c101960 0x7f676c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.264+0000 7f6771e15640 1 --2- 172.21.15.12:0/383637778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f676c100770 0x7f676c104700 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.264+0000 7f6771e15640 1 -- 172.21.15.12:0/383637778 >> 172.21.15.12:0/383637778 conn(0x7f676c0fbd20 msgr2=0x7f676c069e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.264+0000 7f6771e15640 1 -- 172.21.15.12:0/383637778 shutdown_connections 2024-05-09T09:25:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.264+0000 7f6771e15640 1 -- 172.21.15.12:0/383637778 wait complete. 2024-05-09T09:25:33.274 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:25:33.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.384+0000 7fe4f3970640 1 -- 172.21.15.12:0/1942173530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ec102a80 msgr2=0x7fe4ec102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.384+0000 7fe4f3970640 1 --2- 172.21.15.12:0/1942173530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ec102a80 0x7fe4ec102f00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009900 tx=0x7fe4dc02a410 comp rx=0 tx=0).stop 2024-05-09T09:25:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.384+0000 7fe4f3970640 1 -- 172.21.15.12:0/1942173530 shutdown_connections 2024-05-09T09:25:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.384+0000 7fe4f3970640 1 --2- 172.21.15.12:0/1942173530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ec103440 0x7fe4ec109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.384+0000 7fe4f3970640 1 --2- 172.21.15.12:0/1942173530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ec102a80 0x7fe4ec102f00 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.384+0000 7fe4f3970640 1 --2- 172.21.15.12:0/1942173530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4ec101890 0x7fe4ec101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.384+0000 7fe4f3970640 1 -- 172.21.15.12:0/1942173530 >> 172.21.15.12:0/1942173530 conn(0x7fe4ec0fce00 msgr2=0x7fe4ec0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.384+0000 7fe4f3970640 1 -- 172.21.15.12:0/1942173530 shutdown_connections 2024-05-09T09:25:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.384+0000 7fe4f3970640 1 -- 172.21.15.12:0/1942173530 wait complete. 2024-05-09T09:25:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f3970640 1 Processor -- start 2024-05-09T09:25:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f3970640 1 -- start start 2024-05-09T09:25:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f3970640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ec101890 0x7fe4ec197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:33.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f3970640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ec102a80 0x7fe4ec198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:33.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f3970640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4ec103440 0x7fe4ec19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:33.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f3970640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec10d550 con 0x7fe4ec101890 2024-05-09T09:25:33.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f3970640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec10d3d0 con 0x7fe4ec103440 2024-05-09T09:25:33.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f3970640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec10d6d0 con 0x7fe4ec102a80 2024-05-09T09:25:33.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f170c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ec101890 0x7fe4ec197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:33.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f170c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ec101890 0x7fe4ec197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33800/0 (socket says 172.21.15.12:33800) 2024-05-09T09:25:33.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f170c640 1 -- 172.21.15.12:0/774561065 learned_addr learned my addr 172.21.15.12:0/774561065 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:33.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f1f0d640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4ec103440 0x7fe4ec19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:33.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f170c640 1 -- 172.21.15.12:0/774561065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ec102a80 msgr2=0x7fe4ec198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f170c640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ec102a80 0x7fe4ec198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f170c640 1 -- 172.21.15.12:0/774561065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4ec103440 msgr2=0x7fe4ec19f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f170c640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4ec103440 0x7fe4ec19f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f170c640 1 -- 172.21.15.12:0/774561065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ec19f850 con 0x7fe4ec101890 2024-05-09T09:25:33.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.385+0000 7fe4f1f0d640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4ec103440 0x7fe4ec19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:33.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.386+0000 7fe4f170c640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ec101890 0x7fe4ec197c40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800d770 tx=0x7fe4e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:33.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.386+0000 7fe4e27fc640 1 -- 172.21.15.12:0/774561065 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8017070 con 0x7fe4ec101890 2024-05-09T09:25:33.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.386+0000 7fe4e27fc640 1 -- 172.21.15.12:0/774561065 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe4e8009c30 con 0x7fe4ec101890 2024-05-09T09:25:33.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.386+0000 7fe4f3970640 1 -- 172.21.15.12:0/774561065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ec19fb40 con 0x7fe4ec101890 2024-05-09T09:25:33.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.386+0000 7fe4e27fc640 1 -- 172.21.15.12:0/774561065 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8003e10 con 0x7fe4ec101890 2024-05-09T09:25:33.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.386+0000 7fe4f3970640 1 -- 172.21.15.12:0/774561065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ec1a0090 con 0x7fe4ec101890 2024-05-09T09:25:33.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.387+0000 7fe4f3970640 1 -- 172.21.15.12:0/774561065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4ec108c00 con 0x7fe4ec101890 2024-05-09T09:25:33.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.388+0000 7fe4e27fc640 1 -- 172.21.15.12:0/774561065 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe4e8004020 con 0x7fe4ec101890 2024-05-09T09:25:33.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.389+0000 7fe4e27fc640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe4d80a2b90 0x7fe4d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:33.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.389+0000 7fe4e27fc640 1 -- 172.21.15.12:0/774561065 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fe4e80cf360 con 0x7fe4ec101890 2024-05-09T09:25:33.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.389+0000 7fe4f0f0b640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe4d80a2b90 0x7fe4d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:33.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.389+0000 7fe4f0f0b640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe4d80a2b90 0x7fe4d80a5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc02a930 tx=0x7fe4dc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:33.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.392+0000 7fe4e27fc640 1 -- 172.21.15.12:0/774561065 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe4e809be90 con 0x7fe4ec101890 2024-05-09T09:25:33.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:33 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2808359683' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T09:25:33.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:33 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:25:33.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:33 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2433564080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:33.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:33 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2808359683' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T09:25:33.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:33 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:25:33.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:33 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2433564080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:33.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:33 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2808359683' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T09:25:33.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:33 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:25:33.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:33 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2433564080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:33.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.579+0000 7fe4f3970640 1 -- 172.21.15.12:0/774561065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe4ec02d1c0 con 0x7fe4ec101890 2024-05-09T09:25:33.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.579+0000 7fe4e27fc640 1 -- 172.21.15.12:0/774561065 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fe4e809b790 con 0x7fe4ec101890 2024-05-09T09:25:33.579 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:25:33.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.581+0000 7fe4f3970640 1 -- 172.21.15.12:0/774561065 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe4d80a2b90 msgr2=0x7fe4d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.581+0000 7fe4f3970640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe4d80a2b90 0x7fe4d80a5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc02a930 tx=0x7fe4dc000bc0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.581+0000 7fe4f3970640 1 -- 172.21.15.12:0/774561065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ec101890 msgr2=0x7fe4ec197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.581+0000 7fe4f3970640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ec101890 0x7fe4ec197c40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800d770 tx=0x7fe4e800dc30 comp rx=0 tx=0).stop 2024-05-09T09:25:33.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.581+0000 7fe4f3970640 1 -- 172.21.15.12:0/774561065 shutdown_connections 2024-05-09T09:25:33.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.581+0000 7fe4f3970640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe4d80a2b90 0x7fe4d80a5050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.581+0000 7fe4f3970640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4ec103440 0x7fe4ec19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.581+0000 7fe4f3970640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ec102a80 0x7fe4ec198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.581+0000 7fe4f3970640 1 --2- 172.21.15.12:0/774561065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ec101890 0x7fe4ec197c40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.581+0000 7fe4f3970640 1 -- 172.21.15.12:0/774561065 >> 172.21.15.12:0/774561065 conn(0x7fe4ec0fce00 msgr2=0x7fe4ec105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:33.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.581+0000 7fe4f3970640 1 -- 172.21.15.12:0/774561065 shutdown_connections 2024-05-09T09:25:33.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.581+0000 7fe4f3970640 1 -- 172.21.15.12:0/774561065 wait complete. 2024-05-09T09:25:33.592 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:25:33.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.701+0000 7fdfda95c640 1 -- 172.21.15.12:0/518463553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfd4101960 msgr2=0x7fdfd4101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.701+0000 7fdfda95c640 1 --2- 172.21.15.12:0/518463553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfd4101960 0x7fdfd4101de0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009900 tx=0x7fdfbc02a410 comp rx=0 tx=0).stop 2024-05-09T09:25:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.701+0000 7fdfda95c640 1 -- 172.21.15.12:0/518463553 shutdown_connections 2024-05-09T09:25:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.701+0000 7fdfda95c640 1 --2- 172.21.15.12:0/518463553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd4102320 0x7fdfd4108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.701+0000 7fdfda95c640 1 --2- 172.21.15.12:0/518463553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfd4101960 0x7fdfd4101de0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.701+0000 7fdfda95c640 1 --2- 172.21.15.12:0/518463553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4100770 0x7fdfd4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.701+0000 7fdfda95c640 1 -- 172.21.15.12:0/518463553 >> 172.21.15.12:0/518463553 conn(0x7fdfd40fbce0 msgr2=0x7fdfd40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.701+0000 7fdfda95c640 1 -- 172.21.15.12:0/518463553 shutdown_connections 2024-05-09T09:25:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.701+0000 7fdfda95c640 1 -- 172.21.15.12:0/518463553 wait complete. 2024-05-09T09:25:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.701+0000 7fdfda95c640 1 Processor -- start 2024-05-09T09:25:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.701+0000 7fdfda95c640 1 -- start start 2024-05-09T09:25:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfda95c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd4100770 0x7fdfd4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfda95c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfd4101960 0x7fdfd41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfda95c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4102320 0x7fdfd419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfda95c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd410c3d0 con 0x7fdfd4101960 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfda95c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd410c250 con 0x7fdfd4102320 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfda95c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd410c550 con 0x7fdfd4100770 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfcbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfd4101960 0x7fdfd41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfcbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfd4101960 0x7fdfd41980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33816/0 (socket says 172.21.15.12:33816) 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfcbfff640 1 -- 172.21.15.12:0/2695825064 learned_addr learned my addr 172.21.15.12:0/2695825064 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfd8ef9640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4102320 0x7fdfd419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfd3fff640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd4100770 0x7fdfd4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfcbfff640 1 -- 172.21.15.12:0/2695825064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd4100770 msgr2=0x7fdfd4197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfcbfff640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd4100770 0x7fdfd4197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfcbfff640 1 -- 172.21.15.12:0/2695825064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4102320 msgr2=0x7fdfd419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfcbfff640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4102320 0x7fdfd419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfcbfff640 1 -- 172.21.15.12:0/2695825064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd419f7a0 con 0x7fdfd4101960 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfd8ef9640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4102320 0x7fdfd419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfd3fff640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd4100770 0x7fdfd4197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfcbfff640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfd4101960 0x7fdfd41980c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009230 tx=0x7fdfbc031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:33.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfd1ffb640 1 -- 172.21.15.12:0/2695825064 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc03f030 con 0x7fdfd4101960 2024-05-09T09:25:33.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfda95c640 1 -- 172.21.15.12:0/2695825064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd419fa30 con 0x7fdfd4101960 2024-05-09T09:25:33.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfd1ffb640 1 -- 172.21.15.12:0/2695825064 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdfbc003ac0 con 0x7fdfd4101960 2024-05-09T09:25:33.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfd1ffb640 1 -- 172.21.15.12:0/2695825064 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc035070 con 0x7fdfd4101960 2024-05-09T09:25:33.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.702+0000 7fdfda95c640 1 -- 172.21.15.12:0/2695825064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd419ff20 con 0x7fdfd4101960 2024-05-09T09:25:33.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.703+0000 7fdfda95c640 1 -- 172.21.15.12:0/2695825064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd41a01b0 con 0x7fdfd4101960 2024-05-09T09:25:33.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.705+0000 7fdfd1ffb640 1 -- 172.21.15.12:0/2695825064 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdfbc042030 con 0x7fdfd4101960 2024-05-09T09:25:33.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.705+0000 7fdfd1ffb640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdfc00a2b90 0x7fdfc00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:33.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.705+0000 7fdfd3fff640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdfc00a2b90 0x7fdfc00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:33.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.705+0000 7fdfd1ffb640 1 -- 172.21.15.12:0/2695825064 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fdfbc0efaf0 con 0x7fdfd4101960 2024-05-09T09:25:33.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.706+0000 7fdfd3fff640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdfc00a2b90 0x7fdfc00a5050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdfd41017c0 tx=0x7fdfc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:33.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.710+0000 7fdfd1ffb640 1 -- 172.21.15.12:0/2695825064 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdfbc0bc680 con 0x7fdfd4101960 2024-05-09T09:25:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.900+0000 7fdfda95c640 1 -- 172.21.15.12:0/2695825064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdfd41a0540 con 0x7fdfd4101960 2024-05-09T09:25:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.900+0000 7fdfd1ffb640 1 -- 172.21.15.12:0/2695825064 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fdfbc0bbf80 con 0x7fdfd4101960 2024-05-09T09:25:33.900 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T09:25:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.902+0000 7fdfda95c640 1 -- 172.21.15.12:0/2695825064 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdfc00a2b90 msgr2=0x7fdfc00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.902+0000 7fdfda95c640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdfc00a2b90 0x7fdfc00a5050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdfd41017c0 tx=0x7fdfc4005f90 comp rx=0 tx=0).stop 2024-05-09T09:25:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.902+0000 7fdfda95c640 1 -- 172.21.15.12:0/2695825064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfd4101960 msgr2=0x7fdfd41980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.902+0000 7fdfda95c640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfd4101960 0x7fdfd41980c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009230 tx=0x7fdfbc031ac0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.902+0000 7fdfda95c640 1 -- 172.21.15.12:0/2695825064 shutdown_connections 2024-05-09T09:25:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.902+0000 7fdfda95c640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdfc00a2b90 0x7fdfc00a5050 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.902+0000 7fdfda95c640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4102320 0x7fdfd419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.902+0000 7fdfda95c640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfd4101960 0x7fdfd41980c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.902+0000 7fdfda95c640 1 --2- 172.21.15.12:0/2695825064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd4100770 0x7fdfd4197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.902+0000 7fdfda95c640 1 -- 172.21.15.12:0/2695825064 >> 172.21.15.12:0/2695825064 conn(0x7fdfd40fbce0 msgr2=0x7fdfd40689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.902+0000 7fdfda95c640 1 -- 172.21.15.12:0/2695825064 shutdown_connections 2024-05-09T09:25:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:33.902+0000 7fdfda95c640 1 -- 172.21.15.12:0/2695825064 wait complete. 2024-05-09T09:25:33.913 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:25:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.021+0000 7f98febe9640 1 -- 172.21.15.12:0/3455927049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98f8107610 msgr2=0x7f98f8109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.021+0000 7f98febe9640 1 --2- 172.21.15.12:0/3455927049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98f8107610 0x7f98f8109a00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009ca0 tx=0x7f98f002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.021+0000 7f98febe9640 1 -- 172.21.15.12:0/3455927049 shutdown_connections 2024-05-09T09:25:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.021+0000 7f98febe9640 1 --2- 172.21.15.12:0/3455927049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98f8107610 0x7f98f8109a00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.021+0000 7f98febe9640 1 --2- 172.21.15.12:0/3455927049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f8104ce0 0x7f98f81070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.021+0000 7f98febe9640 1 --2- 172.21.15.12:0/3455927049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f81023b0 0x7f98f81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.021+0000 7f98febe9640 1 -- 172.21.15.12:0/3455927049 >> 172.21.15.12:0/3455927049 conn(0x7f98f80fbd20 msgr2=0x7f98f80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.021+0000 7f98febe9640 1 -- 172.21.15.12:0/3455927049 shutdown_connections 2024-05-09T09:25:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.021+0000 7f98febe9640 1 -- 172.21.15.12:0/3455927049 wait complete. 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98febe9640 1 Processor -- start 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98febe9640 1 -- start start 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98febe9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f81023b0 0x7f98f8101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98febe9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98f8104ce0 0x7f98f8100360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98febe9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f8107610 0x7f98f81008a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98febe9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f98f810d510 con 0x7f98f8104ce0 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98febe9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98f810d390 con 0x7f98f81023b0 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98febe9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f98f810d690 con 0x7f98f8107610 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98f8104ce0 0x7f98f8100360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98f8104ce0 0x7f98f8100360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33840/0 (socket says 172.21.15.12:33840) 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98f7fff640 1 -- 172.21.15.12:0/2834150351 learned_addr learned my addr 172.21.15.12:0/2834150351 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98fd186640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f8107610 0x7f98f81008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98fc985640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f81023b0 0x7f98f8101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98f7fff640 1 -- 172.21.15.12:0/2834150351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f8107610 msgr2=0x7f98f81008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98f7fff640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f8107610 0x7f98f81008a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98f7fff640 1 -- 172.21.15.12:0/2834150351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f81023b0 msgr2=0x7f98f8101d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98f7fff640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f81023b0 0x7f98f8101d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98f7fff640 1 -- 172.21.15.12:0/2834150351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f8101130 con 0x7f98f8104ce0 2024-05-09T09:25:34.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98fc985640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f81023b0 0x7f98f8101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:34.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98fd186640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f8107610 0x7f98f81008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:34.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.022+0000 7f98f7fff640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98f8104ce0 0x7f98f8100360 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f98e400c970 tx=0x7f98e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:34.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.023+0000 7f98f5ffb640 1 -- 172.21.15.12:0/2834150351 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4007bb0 con 0x7f98f8104ce0 2024-05-09T09:25:34.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.023+0000 7f98febe9640 1 -- 172.21.15.12:0/2834150351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f81a1c70 con 0x7f98f8104ce0 2024-05-09T09:25:34.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.023+0000 7f98f5ffb640 1 -- 172.21.15.12:0/2834150351 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f98e4007d50 con 0x7f98f8104ce0 2024-05-09T09:25:34.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.023+0000 7f98f5ffb640 1 -- 172.21.15.12:0/2834150351 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e400f450 con 0x7f98f8104ce0 2024-05-09T09:25:34.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.023+0000 7f98febe9640 1 -- 172.21.15.12:0/2834150351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f81a21c0 con 0x7f98f8104ce0 2024-05-09T09:25:34.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.024+0000 7f98febe9640 1 -- 172.21.15.12:0/2834150351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f80fd810 con 0x7f98f8104ce0 2024-05-09T09:25:34.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.025+0000 7f98f5ffb640 1 -- 172.21.15.12:0/2834150351 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f98e4004f40 con 0x7f98f8104ce0 2024-05-09T09:25:34.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.026+0000 7f98f5ffb640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f98e80a2b90 0x7f98e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.026+0000 7f98fc985640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f98e80a2b90 0x7f98e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.026+0000 7f98f5ffb640 1 -- 172.21.15.12:0/2834150351 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f98e40cf290 con 0x7f98f8104ce0 2024-05-09T09:25:34.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.026+0000 7f98fc985640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f98e80a2b90 0x7f98e80a5050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009b40 tx=0x7f98ec005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:34.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.030+0000 7f98f5ffb640 1 -- 172.21.15.12:0/2834150351 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f98e4014030 con 0x7f98f8104ce0 2024-05-09T09:25:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.217+0000 7f98febe9640 1 -- 172.21.15.12:0/2834150351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f98f8062430 con 0x7f98f8104ce0 2024-05-09T09:25:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.218+0000 7f98f5ffb640 1 -- 172.21.15.12:0/2834150351 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f98e409bdc0 con 0x7f98f8104ce0 2024-05-09T09:25:34.217 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:25:34.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.220+0000 7f98febe9640 1 -- 172.21.15.12:0/2834150351 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f98e80a2b90 msgr2=0x7f98e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.220+0000 7f98febe9640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f98e80a2b90 0x7f98e80a5050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009b40 tx=0x7f98ec005e50 comp rx=0 tx=0).stop 2024-05-09T09:25:34.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.220+0000 7f98febe9640 1 -- 172.21.15.12:0/2834150351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98f8104ce0 msgr2=0x7f98f8100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.220+0000 7f98febe9640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98f8104ce0 0x7f98f8100360 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f98e400c970 tx=0x7f98e400ce30 comp rx=0 tx=0).stop 2024-05-09T09:25:34.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.220+0000 7f98febe9640 1 -- 172.21.15.12:0/2834150351 shutdown_connections 2024-05-09T09:25:34.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.220+0000 7f98febe9640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f98e80a2b90 0x7f98e80a5050 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.220+0000 7f98febe9640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f8107610 0x7f98f81008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.220+0000 7f98febe9640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98f8104ce0 0x7f98f8100360 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.220+0000 7f98febe9640 1 --2- 172.21.15.12:0/2834150351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f81023b0 0x7f98f8101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.220+0000 7f98febe9640 1 -- 172.21.15.12:0/2834150351 >> 172.21.15.12:0/2834150351 conn(0x7f98f80fbd20 msgr2=0x7f98f8102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:34.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.220+0000 7f98febe9640 1 -- 172.21.15.12:0/2834150351 shutdown_connections 2024-05-09T09:25:34.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.220+0000 7f98febe9640 1 -- 172.21.15.12:0/2834150351 wait complete. 2024-05-09T09:25:34.231 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:25:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.338+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/1214125896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e14102320 msgr2=0x7f6e14108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.338+0000 7f6e1b4a5640 1 --2- 172.21.15.12:0/1214125896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e14102320 0x7f6e14108bb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009ca0 tx=0x7f6e0c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.338+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/1214125896 shutdown_connections 2024-05-09T09:25:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.338+0000 7f6e1b4a5640 1 --2- 172.21.15.12:0/1214125896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e14102320 0x7f6e14108bb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.338+0000 7f6e1b4a5640 1 --2- 172.21.15.12:0/1214125896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14101960 0x7f6e14101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.338+0000 7f6e1b4a5640 1 --2- 172.21.15.12:0/1214125896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e14100770 0x7f6e14100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.338+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/1214125896 >> 172.21.15.12:0/1214125896 conn(0x7f6e140fbce0 msgr2=0x7f6e140fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.338+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/1214125896 shutdown_connections 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.338+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/1214125896 wait complete. 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.339+0000 7f6e1b4a5640 1 Processor -- start 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.339+0000 7f6e1b4a5640 1 -- start start 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.339+0000 7f6e1b4a5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e14100770 0x7f6e14199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.339+0000 7f6e1b4a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e14101960 0x7f6e1419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.339+0000 7f6e1b4a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14102320 0x7f6e141a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.339+0000 7f6e1b4a5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1410c3d0 con 0x7f6e14100770 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.339+0000 7f6e1b4a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1410c250 con 0x7f6e14102320 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.339+0000 7f6e1b4a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1410c550 con 0x7f6e14101960 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.339+0000 7f6e19241640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e14100770 0x7f6e14199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.339+0000 7f6e19241640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e14100770 0x7f6e14199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33860/0 (socket says 172.21.15.12:33860) 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.339+0000 7f6e19241640 1 -- 172.21.15.12:0/2837261331 learned_addr learned my addr 172.21.15.12:0/2837261331 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.339+0000 7f6e18a40640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e14101960 0x7f6e1419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.339+0000 7f6e19a42640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14102320 0x7f6e141a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.340+0000 7f6e19241640 1 -- 172.21.15.12:0/2837261331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e14101960 msgr2=0x7f6e1419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.340+0000 7f6e19241640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e14101960 0x7f6e1419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.340+0000 7f6e19241640 1 -- 172.21.15.12:0/2837261331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14102320 msgr2=0x7f6e141a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.340+0000 7f6e19241640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14102320 0x7f6e141a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.340+0000 7f6e19241640 1 -- 172.21.15.12:0/2837261331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e141a1980 con 0x7f6e14100770 2024-05-09T09:25:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.340+0000 7f6e19a42640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14102320 0x7f6e141a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.340+0000 7f6e19241640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e14100770 0x7f6e14199d60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000e9c0 tx=0x7f6e1000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.340+0000 7f6e0a7fc640 1 -- 172.21.15.12:0/2837261331 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1000cd80 con 0x7f6e14100770 2024-05-09T09:25:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.340+0000 7f6e0a7fc640 1 -- 172.21.15.12:0/2837261331 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6e100035f0 con 0x7f6e14100770 2024-05-09T09:25:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.340+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/2837261331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e141a1c70 con 0x7f6e14100770 2024-05-09T09:25:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.340+0000 7f6e0a7fc640 1 -- 172.21.15.12:0/2837261331 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10010690 con 0x7f6e14100770 2024-05-09T09:25:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.340+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/2837261331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e141a2170 con 0x7f6e14100770 2024-05-09T09:25:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.341+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/2837261331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e141a2340 con 0x7f6e14100770 2024-05-09T09:25:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.343+0000 7f6e0a7fc640 1 -- 172.21.15.12:0/2837261331 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6e100037a0 con 0x7f6e14100770 2024-05-09T09:25:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.343+0000 7f6e0a7fc640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6e000a2b90 0x7f6e000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.344+0000 7f6e18a40640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6e000a2b90 0x7f6e000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.344+0000 7f6e0a7fc640 1 -- 172.21.15.12:0/2837261331 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f6e100cf870 con 0x7f6e14100770 2024-05-09T09:25:34.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.344+0000 7f6e18a40640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6e000a2b90 0x7f6e000a5050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6e1419b270 tx=0x7f6e04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:34.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.348+0000 7f6e0a7fc640 1 -- 172.21.15.12:0/2837261331 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6e1009c400 con 0x7f6e14100770 2024-05-09T09:25:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:34 smithi012 ceph-mon[30140]: pgmap v367: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-09T09:25:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:34 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/383637778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:34 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/774561065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:34 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2695825064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:34 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2834150351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:34.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:34 smithi170 ceph-mon[33632]: pgmap v367: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-09T09:25:34.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:34 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/383637778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:34.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:34 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/774561065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:34.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:34 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2695825064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:34.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:34 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2834150351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:34.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:34 smithi016 ceph-mon[33335]: pgmap v367: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-09T09:25:34.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:34 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/383637778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:34.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:34 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/774561065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:34.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:34 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2695825064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:34.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:34 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2834150351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:34.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.538+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/2837261331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e141a2510 con 0x7f6e14100770 2024-05-09T09:25:34.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.540+0000 7f6e0a7fc640 1 -- 172.21.15.12:0/2837261331 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24762 (secure 0 0 0) 0x7f6e1009bd00 con 0x7f6e14100770 2024-05-09T09:25:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:34.539 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":93,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:25:06.152764+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:25:34.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.541+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/2837261331 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6e000a2b90 msgr2=0x7f6e000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.541+0000 7f6e1b4a5640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6e000a2b90 0x7f6e000a5050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6e1419b270 tx=0x7f6e04005f90 comp rx=0 tx=0).stop 2024-05-09T09:25:34.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.541+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/2837261331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e14100770 msgr2=0x7f6e14199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.541+0000 7f6e1b4a5640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e14100770 0x7f6e14199d60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000e9c0 tx=0x7f6e1000ee80 comp rx=0 tx=0).stop 2024-05-09T09:25:34.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.542+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/2837261331 shutdown_connections 2024-05-09T09:25:34.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.542+0000 7f6e1b4a5640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6e000a2b90 0x7f6e000a5050 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.542+0000 7f6e1b4a5640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14102320 0x7f6e141a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.542+0000 7f6e1b4a5640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e14101960 0x7f6e1419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.542+0000 7f6e1b4a5640 1 --2- 172.21.15.12:0/2837261331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e14100770 0x7f6e14199d60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.542+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/2837261331 >> 172.21.15.12:0/2837261331 conn(0x7f6e140fbce0 msgr2=0x7f6e140fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:34.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.542+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/2837261331 shutdown_connections 2024-05-09T09:25:34.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.542+0000 7f6e1b4a5640 1 -- 172.21.15.12:0/2837261331 wait complete. 2024-05-09T09:25:34.553 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:23:08.167107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:23:09.065670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:23:12.773640+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:25:34.553 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:25:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.662+0000 7fd4d53f9640 1 -- 172.21.15.12:0/3871649956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4d0103440 msgr2=0x7fd4d0109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.662+0000 7fd4d53f9640 1 --2- 172.21.15.12:0/3871649956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4d0103440 0x7fd4d0109cd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009ca0 tx=0x7fd4c802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.662+0000 7fd4d53f9640 1 -- 172.21.15.12:0/3871649956 shutdown_connections 2024-05-09T09:25:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.662+0000 7fd4d53f9640 1 --2- 172.21.15.12:0/3871649956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4d0103440 0x7fd4d0109cd0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.662+0000 7fd4d53f9640 1 --2- 172.21.15.12:0/3871649956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4d0102a80 0x7fd4d0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.662+0000 7fd4d53f9640 1 --2- 172.21.15.12:0/3871649956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d0101890 0x7fd4d0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.662+0000 7fd4d53f9640 1 -- 172.21.15.12:0/3871649956 >> 172.21.15.12:0/3871649956 conn(0x7fd4d00fce00 msgr2=0x7fd4d00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.662+0000 7fd4d53f9640 1 -- 172.21.15.12:0/3871649956 shutdown_connections 2024-05-09T09:25:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.662+0000 7fd4d53f9640 1 -- 172.21.15.12:0/3871649956 wait complete. 2024-05-09T09:25:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.663+0000 7fd4d53f9640 1 Processor -- start 2024-05-09T09:25:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.663+0000 7fd4d53f9640 1 -- start start 2024-05-09T09:25:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.663+0000 7fd4d53f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d0101890 0x7fd4d0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.663+0000 7fd4d53f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4d0102a80 0x7fd4d0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.663+0000 7fd4d53f9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4d0103440 0x7fd4d019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.663+0000 7fd4d53f9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d010d4f0 con 0x7fd4d0103440 2024-05-09T09:25:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.663+0000 7fd4d53f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d010d370 con 0x7fd4d0101890 2024-05-09T09:25:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.663+0000 7fd4d53f9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d010d670 con 0x7fd4d0102a80 2024-05-09T09:25:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.663+0000 7fd4cf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4d0103440 0x7fd4d019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.663+0000 7fd4ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d0101890 0x7fd4d0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.663+0000 7fd4cf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4d0103440 0x7fd4d019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33880/0 (socket says 172.21.15.12:33880) 2024-05-09T09:25:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.663+0000 7fd4cf7fe640 1 -- 172.21.15.12:0/2401117947 learned_addr learned my addr 172.21.15.12:0/2401117947 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.663+0000 7fd4ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d0101890 0x7fd4d0197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:43910/0 (socket says 172.21.15.12:43910) 2024-05-09T09:25:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.664+0000 7fd4ce7fc640 1 --2- 172.21.15.12:0/2401117947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4d0102a80 0x7fd4d0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.664+0000 7fd4cf7fe640 1 -- 172.21.15.12:0/2401117947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4d0102a80 msgr2=0x7fd4d0197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.664+0000 7fd4cf7fe640 1 --2- 172.21.15.12:0/2401117947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4d0102a80 0x7fd4d0197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.664+0000 7fd4cf7fe640 1 -- 172.21.15.12:0/2401117947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d0101890 msgr2=0x7fd4d0197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.664+0000 7fd4cf7fe640 1 --2- 172.21.15.12:0/2401117947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d0101890 0x7fd4d0197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.664+0000 7fd4cf7fe640 1 -- 172.21.15.12:0/2401117947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d019f640 con 0x7fd4d0103440 2024-05-09T09:25:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.664+0000 7fd4cf7fe640 1 --2- 172.21.15.12:0/2401117947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4d0103440 0x7fd4d019efd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd4c802f500 tx=0x7fd4c8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:34.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.664+0000 7fd4b7fff640 1 -- 172.21.15.12:0/2401117947 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8003e20 con 0x7fd4d0103440 2024-05-09T09:25:34.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.664+0000 7fd4b7fff640 1 -- 172.21.15.12:0/2401117947 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd4c8004510 con 0x7fd4d0103440 2024-05-09T09:25:34.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.664+0000 7fd4d53f9640 1 -- 172.21.15.12:0/2401117947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d019f8d0 con 0x7fd4d0103440 2024-05-09T09:25:34.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.664+0000 7fd4b7fff640 1 -- 172.21.15.12:0/2401117947 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c802fe50 con 0x7fd4d0103440 2024-05-09T09:25:34.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.664+0000 7fd4d53f9640 1 -- 172.21.15.12:0/2401117947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d019fe20 con 0x7fd4d0103440 2024-05-09T09:25:34.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.665+0000 7fd4d53f9640 1 -- 172.21.15.12:0/2401117947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d0108ae0 con 0x7fd4d0103440 2024-05-09T09:25:34.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.666+0000 7fd4b7fff640 1 -- 172.21.15.12:0/2401117947 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd4c8004040 con 0x7fd4d0103440 2024-05-09T09:25:34.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.667+0000 7fd4b7fff640 1 --2- 172.21.15.12:0/2401117947 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4b80a2b90 0x7fd4b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.667+0000 7fd4ceffd640 1 --2- 172.21.15.12:0/2401117947 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4b80a2b90 0x7fd4b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.667+0000 7fd4b7fff640 1 -- 172.21.15.12:0/2401117947 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd4c80f0cb0 con 0x7fd4d0103440 2024-05-09T09:25:34.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.668+0000 7fd4ceffd640 1 --2- 172.21.15.12:0/2401117947 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4b80a2b90 0x7fd4b80a5050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd4d01028e0 tx=0x7fd4c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:34.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.671+0000 7fd4b7fff640 1 -- 172.21.15.12:0/2401117947 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd4c8004330 con 0x7fd4d0103440 2024-05-09T09:25:34.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.857+0000 7fd4d53f9640 1 -- 172.21.15.12:0/2401117947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd4d0062430 con 0x7fd4d0103440 2024-05-09T09:25:34.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.857+0000 7fd4b7fff640 1 -- 172.21.15.12:0/2401117947 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fd4c80bd840 con 0x7fd4d0103440 2024-05-09T09:25:34.857 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:25:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.859+0000 7fd4d53f9640 1 -- 172.21.15.12:0/2401117947 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4b80a2b90 msgr2=0x7fd4b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.859+0000 7fd4d53f9640 1 --2- 172.21.15.12:0/2401117947 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4b80a2b90 0x7fd4b80a5050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd4d01028e0 tx=0x7fd4c0005f90 comp rx=0 tx=0).stop 2024-05-09T09:25:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.859+0000 7fd4d53f9640 1 -- 172.21.15.12:0/2401117947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4d0103440 msgr2=0x7fd4d019efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.859+0000 7fd4d53f9640 1 --2- 172.21.15.12:0/2401117947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4d0103440 0x7fd4d019efd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd4c802f500 tx=0x7fd4c8003970 comp rx=0 tx=0).stop 2024-05-09T09:25:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.859+0000 7fd4d53f9640 1 -- 172.21.15.12:0/2401117947 shutdown_connections 2024-05-09T09:25:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.859+0000 7fd4d53f9640 1 --2- 172.21.15.12:0/2401117947 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4b80a2b90 0x7fd4b80a5050 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.859+0000 7fd4d53f9640 1 --2- 172.21.15.12:0/2401117947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4d0103440 0x7fd4d019efd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.859+0000 7fd4d53f9640 1 --2- 172.21.15.12:0/2401117947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4d0102a80 0x7fd4d0197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.859+0000 7fd4d53f9640 1 --2- 172.21.15.12:0/2401117947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d0101890 0x7fd4d0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.859+0000 7fd4d53f9640 1 -- 172.21.15.12:0/2401117947 >> 172.21.15.12:0/2401117947 conn(0x7fd4d00fce00 msgr2=0x7fd4d0105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.860+0000 7fd4d53f9640 1 -- 172.21.15.12:0/2401117947 shutdown_connections 2024-05-09T09:25:34.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.860+0000 7fd4d53f9640 1 -- 172.21.15.12:0/2401117947 wait complete. 2024-05-09T09:25:34.870 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:25:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.980+0000 7f616951d640 1 -- 172.21.15.12:0/4171224646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6164102a80 msgr2=0x7f6164102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.980+0000 7f616951d640 1 --2- 172.21.15.12:0/4171224646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6164102a80 0x7f6164102f00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6158009900 tx=0x7f615802a410 comp rx=0 tx=0).stop 2024-05-09T09:25:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.980+0000 7f616951d640 1 -- 172.21.15.12:0/4171224646 shutdown_connections 2024-05-09T09:25:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.980+0000 7f616951d640 1 --2- 172.21.15.12:0/4171224646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6164103440 0x7f6164109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.980+0000 7f616951d640 1 --2- 172.21.15.12:0/4171224646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6164102a80 0x7f6164102f00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.980+0000 7f616951d640 1 --2- 172.21.15.12:0/4171224646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6164101890 0x7f6164101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.980+0000 7f616951d640 1 -- 172.21.15.12:0/4171224646 >> 172.21.15.12:0/4171224646 conn(0x7f61640fce00 msgr2=0x7f61640ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.980+0000 7f616951d640 1 -- 172.21.15.12:0/4171224646 shutdown_connections 2024-05-09T09:25:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.980+0000 7f616951d640 1 -- 172.21.15.12:0/4171224646 wait complete. 2024-05-09T09:25:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.980+0000 7f616951d640 1 Processor -- start 2024-05-09T09:25:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.980+0000 7f616951d640 1 -- start start 2024-05-09T09:25:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f616951d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6164101890 0x7f61640755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f616951d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6164102a80 0x7f6164073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f616951d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6164103440 0x7f6164074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f616951d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f616410d4f0 con 0x7f6164101890 2024-05-09T09:25:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f616951d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f616410d370 con 0x7f6164103440 2024-05-09T09:25:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f616951d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f616410d670 con 0x7f6164102a80 2024-05-09T09:25:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f6162ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6164101890 0x7f61640755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f6162ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6164101890 0x7f61640755e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33908/0 (socket says 172.21.15.12:33908) 2024-05-09T09:25:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f6162ffd640 1 -- 172.21.15.12:0/2790560857 learned_addr learned my addr 172.21.15.12:0/2790560857 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f61627fc640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6164102a80 0x7f6164073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f61637fe640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6164103440 0x7f6164074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f6162ffd640 1 -- 172.21.15.12:0/2790560857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6164102a80 msgr2=0x7f6164073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f6162ffd640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6164102a80 0x7f6164073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f6162ffd640 1 -- 172.21.15.12:0/2790560857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6164103440 msgr2=0x7f6164074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f6162ffd640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6164103440 0x7f6164074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f6162ffd640 1 -- 172.21.15.12:0/2790560857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61640749f0 con 0x7f6164101890 2024-05-09T09:25:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f6162ffd640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6164101890 0x7f61640755e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f615400d950 tx=0x7f615400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.981+0000 7f614bfff640 1 -- 172.21.15.12:0/2790560857 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154017070 con 0x7f6164101890 2024-05-09T09:25:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.982+0000 7f614bfff640 1 -- 172.21.15.12:0/2790560857 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6154009c30 con 0x7f6164101890 2024-05-09T09:25:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.982+0000 7f616951d640 1 -- 172.21.15.12:0/2790560857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61641a1cd0 con 0x7f6164101890 2024-05-09T09:25:34.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.982+0000 7f614bfff640 1 -- 172.21.15.12:0/2790560857 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615400f620 con 0x7f6164101890 2024-05-09T09:25:34.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.982+0000 7f616951d640 1 -- 172.21.15.12:0/2790560857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61641a21a0 con 0x7f6164101890 2024-05-09T09:25:34.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.983+0000 7f616951d640 1 -- 172.21.15.12:0/2790560857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6164108c80 con 0x7f6164101890 2024-05-09T09:25:34.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.984+0000 7f614bfff640 1 -- 172.21.15.12:0/2790560857 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f615401a070 con 0x7f6164101890 2024-05-09T09:25:34.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.984+0000 7f614bfff640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f614c0a2b90 0x7f614c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:34.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.985+0000 7f61627fc640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f614c0a2b90 0x7f614c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:34.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.985+0000 7f614bfff640 1 -- 172.21.15.12:0/2790560857 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f61540cede0 con 0x7f6164101890 2024-05-09T09:25:34.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.985+0000 7f61627fc640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f614c0a2b90 0x7f614c0a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c00 tx=0x7f6158031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:34.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:34.989+0000 7f614bfff640 1 -- 172.21.15.12:0/2790560857 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6154009de0 con 0x7f6164101890 2024-05-09T09:25:35.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.174+0000 7f616951d640 1 -- 172.21.15.12:0/2790560857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6164062460 con 0x7f6164101890 2024-05-09T09:25:35.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.175+0000 7f614bfff640 1 -- 172.21.15.12:0/2790560857 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f615409b320 con 0x7f6164101890 2024-05-09T09:25:35.175 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:25:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.176+0000 7f616951d640 1 -- 172.21.15.12:0/2790560857 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f614c0a2b90 msgr2=0x7f614c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.176+0000 7f616951d640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f614c0a2b90 0x7f614c0a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c00 tx=0x7f6158031510 comp rx=0 tx=0).stop 2024-05-09T09:25:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.176+0000 7f616951d640 1 -- 172.21.15.12:0/2790560857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6164101890 msgr2=0x7f61640755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.176+0000 7f616951d640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6164101890 0x7f61640755e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f615400d950 tx=0x7f615400de10 comp rx=0 tx=0).stop 2024-05-09T09:25:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.176+0000 7f616951d640 1 -- 172.21.15.12:0/2790560857 shutdown_connections 2024-05-09T09:25:35.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.176+0000 7f616951d640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f614c0a2b90 0x7f614c0a5050 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.176+0000 7f616951d640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6164103440 0x7f6164074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.176+0000 7f616951d640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6164102a80 0x7f6164073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.176+0000 7f616951d640 1 --2- 172.21.15.12:0/2790560857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6164101890 0x7f61640755e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.176+0000 7f616951d640 1 -- 172.21.15.12:0/2790560857 >> 172.21.15.12:0/2790560857 conn(0x7f61640fce00 msgr2=0x7f6164105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:35.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.177+0000 7f616951d640 1 -- 172.21.15.12:0/2790560857 shutdown_connections 2024-05-09T09:25:35.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.177+0000 7f616951d640 1 -- 172.21.15.12:0/2790560857 wait complete. 2024-05-09T09:25:35.188 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:25:35.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.297+0000 7f616f45d640 1 -- 172.21.15.12:0/1180934106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168074290 msgr2=0x7f6168074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.297+0000 7f616f45d640 1 --2- 172.21.15.12:0/1180934106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168074290 0x7f6168074710 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6158009930 tx=0x7f615802a420 comp rx=0 tx=0).stop 2024-05-09T09:25:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.297+0000 7f616f45d640 1 -- 172.21.15.12:0/1180934106 shutdown_connections 2024-05-09T09:25:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.297+0000 7f616f45d640 1 --2- 172.21.15.12:0/1180934106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f616810de50 0x7f6168110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.297+0000 7f616f45d640 1 --2- 172.21.15.12:0/1180934106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168074290 0x7f6168074710 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.297+0000 7f616f45d640 1 --2- 172.21.15.12:0/1180934106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6168075870 0x7f6168073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.297+0000 7f616f45d640 1 -- 172.21.15.12:0/1180934106 >> 172.21.15.12:0/1180934106 conn(0x7f61680fce10 msgr2=0x7f61680ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.297+0000 7f616f45d640 1 -- 172.21.15.12:0/1180934106 shutdown_connections 2024-05-09T09:25:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.297+0000 7f616f45d640 1 -- 172.21.15.12:0/1180934106 wait complete. 2024-05-09T09:25:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.297+0000 7f616f45d640 1 Processor -- start 2024-05-09T09:25:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.297+0000 7f616f45d640 1 -- start start 2024-05-09T09:25:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616f45d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168074290 0x7f616819e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616f45d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6168075870 0x7f616819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616f45d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f616810de50 0x7f61681a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616f45d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6168113ac0 con 0x7f6168074290 2024-05-09T09:25:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616f45d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6168113940 con 0x7f616810de50 2024-05-09T09:25:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616f45d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6168113c40 con 0x7f6168075870 2024-05-09T09:25:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616d1f9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168074290 0x7f616819e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616d1f9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168074290 0x7f616819e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33928/0 (socket says 172.21.15.12:33928) 2024-05-09T09:25:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616d1f9640 1 -- 172.21.15.12:0/3781049743 learned_addr learned my addr 172.21.15.12:0/3781049743 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616c9f8640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6168075870 0x7f616819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616d9fa640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f616810de50 0x7f61681a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616d1f9640 1 -- 172.21.15.12:0/3781049743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6168075870 msgr2=0x7f616819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616d1f9640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6168075870 0x7f616819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616d1f9640 1 -- 172.21.15.12:0/3781049743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f616810de50 msgr2=0x7f61681a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616d1f9640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f616810de50 0x7f61681a5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616d1f9640 1 -- 172.21.15.12:0/3781049743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61681a5de0 con 0x7f6168074290 2024-05-09T09:25:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616d9fa640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f616810de50 0x7f61681a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616d1f9640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168074290 0x7f616819e1c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f616400d950 tx=0x7f616400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f615e7fc640 1 -- 172.21.15.12:0/3781049743 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164017030 con 0x7f6168074290 2024-05-09T09:25:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f615e7fc640 1 -- 172.21.15.12:0/3781049743 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6164004020 con 0x7f6168074290 2024-05-09T09:25:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616f45d640 1 -- 172.21.15.12:0/3781049743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61681a60d0 con 0x7f6168074290 2024-05-09T09:25:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f615e7fc640 1 -- 172.21.15.12:0/3781049743 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61640047a0 con 0x7f6168074290 2024-05-09T09:25:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.298+0000 7f616f45d640 1 -- 172.21.15.12:0/3781049743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61681a6640 con 0x7f6168074290 2024-05-09T09:25:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.299+0000 7f616f45d640 1 -- 172.21.15.12:0/3781049743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6168103800 con 0x7f6168074290 2024-05-09T09:25:35.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.302+0000 7f615e7fc640 1 -- 172.21.15.12:0/3781049743 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f61640041d0 con 0x7f6168074290 2024-05-09T09:25:35.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.302+0000 7f615e7fc640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f61540a2c10 0x7f61540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:35.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.302+0000 7f616c9f8640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f61540a2c10 0x7f61540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:35.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.302+0000 7f615e7fc640 1 -- 172.21.15.12:0/3781049743 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f61640cf490 con 0x7f6168074290 2024-05-09T09:25:35.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.303+0000 7f616c9f8640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f61540a2c10 0x7f61540a50d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6158006010 tx=0x7f6158031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.306+0000 7f615e7fc640 1 -- 172.21.15.12:0/3781049743 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f616409c0a0 con 0x7f6168074290 2024-05-09T09:25:35.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:35 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2837261331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:35.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:35 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2401117947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:35.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:35 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2790560857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:35.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:35 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2837261331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:35.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:35 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2401117947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:35.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:35 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2790560857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:35.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.494+0000 7f616f45d640 1 -- 172.21.15.12:0/3781049743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6168062430 con 0x7f6168074290 2024-05-09T09:25:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.494+0000 7f615e7fc640 1 -- 172.21.15.12:0/3781049743 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f616409b9a0 con 0x7f6168074290 2024-05-09T09:25:35.494 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T09:25:35.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.496+0000 7f616f45d640 1 -- 172.21.15.12:0/3781049743 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f61540a2c10 msgr2=0x7f61540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.496+0000 7f616f45d640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f61540a2c10 0x7f61540a50d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6158006010 tx=0x7f6158031510 comp rx=0 tx=0).stop 2024-05-09T09:25:35.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.496+0000 7f616f45d640 1 -- 172.21.15.12:0/3781049743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168074290 msgr2=0x7f616819e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.496+0000 7f616f45d640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168074290 0x7f616819e1c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f616400d950 tx=0x7f616400de10 comp rx=0 tx=0).stop 2024-05-09T09:25:35.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.496+0000 7f616f45d640 1 -- 172.21.15.12:0/3781049743 shutdown_connections 2024-05-09T09:25:35.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.496+0000 7f616f45d640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f61540a2c10 0x7f61540a50d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.496+0000 7f616f45d640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f616810de50 0x7f61681a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.496+0000 7f616f45d640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6168075870 0x7f616819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.496+0000 7f616f45d640 1 --2- 172.21.15.12:0/3781049743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168074290 0x7f616819e1c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.496+0000 7f616f45d640 1 -- 172.21.15.12:0/3781049743 >> 172.21.15.12:0/3781049743 conn(0x7f61680fce10 msgr2=0x7f61680ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:35.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.496+0000 7f616f45d640 1 -- 172.21.15.12:0/3781049743 shutdown_connections 2024-05-09T09:25:35.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.497+0000 7f616f45d640 1 -- 172.21.15.12:0/3781049743 wait complete. 2024-05-09T09:25:35.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:35 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2837261331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:35.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:35 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2401117947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:35.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:35 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2790560857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:35.507 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:25:35.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.616+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1369572304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb64102b10 msgr2=0x7feb64102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.616+0000 7feb6c4fc640 1 --2- 172.21.15.12:0/1369572304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb64102b10 0x7feb64102f90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feb54009930 tx=0x7feb5402a420 comp rx=0 tx=0).stop 2024-05-09T09:25:35.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.616+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1369572304 shutdown_connections 2024-05-09T09:25:35.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.616+0000 7feb6c4fc640 1 --2- 172.21.15.12:0/1369572304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb641034d0 0x7feb64109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.616+0000 7feb6c4fc640 1 --2- 172.21.15.12:0/1369572304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb64102b10 0x7feb64102f90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.616+0000 7feb6c4fc640 1 --2- 172.21.15.12:0/1369572304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb640ffb40 0x7feb640fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.616+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1369572304 >> 172.21.15.12:0/1369572304 conn(0x7feb640fb470 msgr2=0x7feb640fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.616+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1369572304 shutdown_connections 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.616+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1369572304 wait complete. 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6c4fc640 1 Processor -- start 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6c4fc640 1 -- start start 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6c4fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb640ffb40 0x7feb64102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6c4fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64102b10 0x7feb64100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6c4fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb641034d0 0x7feb64101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6c4fc640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7feb6410d500 con 0x7feb641034d0 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6c4fc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb6410d380 con 0x7feb64102b10 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6c4fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feb6410d680 con 0x7feb640ffb40 2024-05-09T09:25:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6aa99640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb641034d0 0x7feb64101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6aa99640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb641034d0 0x7feb64101000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33948/0 (socket says 172.21.15.12:33948) 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6aa99640 1 -- 172.21.15.12:0/1501593942 learned_addr learned my addr 172.21.15.12:0/1501593942 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb69a97640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64102b10 0x7feb64100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6a298640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb640ffb40 0x7feb64102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6aa99640 1 -- 172.21.15.12:0/1501593942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb640ffb40 msgr2=0x7feb64102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6aa99640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb640ffb40 0x7feb64102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6aa99640 1 -- 172.21.15.12:0/1501593942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64102b10 msgr2=0x7feb64100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6aa99640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64102b10 0x7feb64100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb6aa99640 1 -- 172.21.15.12:0/1501593942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb64101890 con 0x7feb641034d0 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.617+0000 7feb69a97640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64102b10 0x7feb64100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.618+0000 7feb6aa99640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb641034d0 0x7feb64101000 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00eaa0 tx=0x7feb5c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.618+0000 7feb5b7fe640 1 -- 172.21.15.12:0/1501593942 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c00cdb0 con 0x7feb641034d0 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.618+0000 7feb5b7fe640 1 -- 172.21.15.12:0/1501593942 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7feb5c005020 con 0x7feb641034d0 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.618+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1501593942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb641a1cd0 con 0x7feb641034d0 2024-05-09T09:25:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.618+0000 7feb5b7fe640 1 -- 172.21.15.12:0/1501593942 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c0116b0 con 0x7feb641034d0 2024-05-09T09:25:35.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.618+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1501593942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb641a21d0 con 0x7feb641034d0 2024-05-09T09:25:35.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.619+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1501593942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb64108c40 con 0x7feb641034d0 2024-05-09T09:25:35.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.620+0000 7feb5b7fe640 1 -- 172.21.15.12:0/1501593942 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feb5c0051d0 con 0x7feb641034d0 2024-05-09T09:25:35.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.621+0000 7feb5b7fe640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7feb500a2b90 0x7feb500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:35.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.621+0000 7feb6a298640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7feb500a2b90 0x7feb500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:35.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.621+0000 7feb5b7fe640 1 -- 172.21.15.12:0/1501593942 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7feb5c0cf6f0 con 0x7feb641034d0 2024-05-09T09:25:35.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.621+0000 7feb6a298640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7feb500a2b90 0x7feb500a5050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb60005fd0 tx=0x7feb60005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:35.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.625+0000 7feb5b7fe640 1 -- 172.21.15.12:0/1501593942 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feb5c016030 con 0x7feb641034d0 2024-05-09T09:25:35.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.813+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1501593942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7feb64062430 con 0x7feb641034d0 2024-05-09T09:25:35.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.814+0000 7feb5b7fe640 1 -- 172.21.15.12:0/1501593942 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7feb5c09c280 con 0x7feb641034d0 2024-05-09T09:25:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:25:35.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.816+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1501593942 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7feb500a2b90 msgr2=0x7feb500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.816+0000 7feb6c4fc640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7feb500a2b90 0x7feb500a5050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb60005fd0 tx=0x7feb60005c70 comp rx=0 tx=0).stop 2024-05-09T09:25:35.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.816+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1501593942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb641034d0 msgr2=0x7feb64101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.816+0000 7feb6c4fc640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb641034d0 0x7feb64101000 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00eaa0 tx=0x7feb5c00ef60 comp rx=0 tx=0).stop 2024-05-09T09:25:35.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.816+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1501593942 shutdown_connections 2024-05-09T09:25:35.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.816+0000 7feb6c4fc640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7feb500a2b90 0x7feb500a5050 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.816+0000 7feb6c4fc640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb641034d0 0x7feb64101000 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.816+0000 7feb6c4fc640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb64102b10 0x7feb64100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.816+0000 7feb6c4fc640 1 --2- 172.21.15.12:0/1501593942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb640ffb40 0x7feb64102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.816+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1501593942 >> 172.21.15.12:0/1501593942 conn(0x7feb640fb470 msgr2=0x7feb64105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:35.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.816+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1501593942 shutdown_connections 2024-05-09T09:25:35.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.816+0000 7feb6c4fc640 1 -- 172.21.15.12:0/1501593942 wait complete. 2024-05-09T09:25:35.827 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-09T09:25:35.827 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:25:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.933+0000 7f65adc12640 1 -- 172.21.15.12:0/3312254779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a8100aa0 msgr2=0x7f65a8100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.933+0000 7f65adc12640 1 --2- 172.21.15.12:0/3312254779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a8100aa0 0x7f65a8100f00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f659c009900 tx=0x7f659c02a410 comp rx=0 tx=0).stop 2024-05-09T09:25:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.933+0000 7f65adc12640 1 -- 172.21.15.12:0/3312254779 shutdown_connections 2024-05-09T09:25:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.933+0000 7f65adc12640 1 --2- 172.21.15.12:0/3312254779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8101440 0x7f65a81018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.933+0000 7f65adc12640 1 --2- 172.21.15.12:0/3312254779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a8100aa0 0x7f65a8100f00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.933+0000 7f65adc12640 1 --2- 172.21.15.12:0/3312254779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65a8100140 0x7f65a8100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.933+0000 7f65adc12640 1 -- 172.21.15.12:0/3312254779 >> 172.21.15.12:0/3312254779 conn(0x7f65a80fbce0 msgr2=0x7f65a80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.933+0000 7f65adc12640 1 -- 172.21.15.12:0/3312254779 shutdown_connections 2024-05-09T09:25:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.933+0000 7f65adc12640 1 -- 172.21.15.12:0/3312254779 wait complete. 2024-05-09T09:25:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.934+0000 7f65adc12640 1 Processor -- start 2024-05-09T09:25:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.934+0000 7f65adc12640 1 -- start start 2024-05-09T09:25:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.934+0000 7f65adc12640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a8100140 0x7f65a8199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.934+0000 7f65adc12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8100aa0 0x7f65a819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.934+0000 7f65adc12640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65a8101440 0x7f65a81a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.934+0000 7f65adc12640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f65a810f740 con 0x7f65a8100140 2024-05-09T09:25:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.934+0000 7f65adc12640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65a810f5c0 con 0x7f65a8100aa0 2024-05-09T09:25:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.934+0000 7f65adc12640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f65a810f8c0 con 0x7f65a8101440 2024-05-09T09:25:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.934+0000 7f65a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a8100140 0x7f65a8199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.934+0000 7f65a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a8100140 0x7f65a8199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33968/0 (socket says 172.21.15.12:33968) 2024-05-09T09:25:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.934+0000 7f65a77fe640 1 -- 172.21.15.12:0/2982099792 learned_addr learned my addr 172.21.15.12:0/2982099792 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.934+0000 7f65a6ffd640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8100aa0 0x7f65a819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65a7fff640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65a8101440 0x7f65a81a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65a77fe640 1 -- 172.21.15.12:0/2982099792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65a8101440 msgr2=0x7f65a81a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65a77fe640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65a8101440 0x7f65a81a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65a77fe640 1 -- 172.21.15.12:0/2982099792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8100aa0 msgr2=0x7f65a819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65a77fe640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8100aa0 0x7f65a819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65a77fe640 1 -- 172.21.15.12:0/2982099792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a81a19e0 con 0x7f65a8100140 2024-05-09T09:25:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65a6ffd640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8100aa0 0x7f65a819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65a7fff640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65a8101440 0x7f65a81a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65a77fe640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a8100140 0x7f65a8199dc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f659800d770 tx=0x7f659800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65a4ff9640 1 -- 172.21.15.12:0/2982099792 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598017070 con 0x7f65a8100140 2024-05-09T09:25:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65a4ff9640 1 -- 172.21.15.12:0/2982099792 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f65980054f0 con 0x7f65a8100140 2024-05-09T09:25:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65adc12640 1 -- 172.21.15.12:0/2982099792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a81a1cd0 con 0x7f65a8100140 2024-05-09T09:25:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65a4ff9640 1 -- 172.21.15.12:0/2982099792 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598004d60 con 0x7f65a8100140 2024-05-09T09:25:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.935+0000 7f65adc12640 1 -- 172.21.15.12:0/2982099792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a81a21d0 con 0x7f65a8100140 2024-05-09T09:25:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.936+0000 7f65adc12640 1 -- 172.21.15.12:0/2982099792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a810b040 con 0x7f65a8100140 2024-05-09T09:25:35.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.937+0000 7f65a4ff9640 1 -- 172.21.15.12:0/2982099792 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6598004f00 con 0x7f65a8100140 2024-05-09T09:25:35.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.938+0000 7f65a4ff9640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f65900a2c10 0x7f65900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:35.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.938+0000 7f65a6ffd640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f65900a2c10 0x7f65900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:35.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.938+0000 7f65a4ff9640 1 -- 172.21.15.12:0/2982099792 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f65980cef30 con 0x7f65a8100140 2024-05-09T09:25:35.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.939+0000 7f65a6ffd640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f65900a2c10 0x7f65900a50d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f65a819b2d0 tx=0x7f659c031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:35.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:35.943+0000 7f65a4ff9640 1 -- 172.21.15.12:0/2982099792 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f659809ba60 con 0x7f65a8100140 2024-05-09T09:25:36.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.168+0000 7f65adc12640 1 -- 172.21.15.12:0/2982099792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65a80624d0 con 0x7f65a8100140 2024-05-09T09:25:36.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.169+0000 7f65a4ff9640 1 -- 172.21.15.12:0/2982099792 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11931 (secure 0 0 0) 0x7f659809b360 con 0x7f65a8100140 2024-05-09T09:25:36.168 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:36.169 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:25:31.411273+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[14628,24484,24515,24523,24502]},"id":1}]} 2024-05-09T09:25:36.169 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-05-09T09:25:36.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.171+0000 7f65adc12640 1 -- 172.21.15.12:0/2982099792 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f65900a2c10 msgr2=0x7f65900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:36.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.171+0000 7f65adc12640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f65900a2c10 0x7f65900a50d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f65a819b2d0 tx=0x7f659c031510 comp rx=0 tx=0).stop 2024-05-09T09:25:36.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.171+0000 7f65adc12640 1 -- 172.21.15.12:0/2982099792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a8100140 msgr2=0x7f65a8199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:36.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.171+0000 7f65adc12640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a8100140 0x7f65a8199dc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f659800d770 tx=0x7f659800dc30 comp rx=0 tx=0).stop 2024-05-09T09:25:36.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.171+0000 7f65adc12640 1 -- 172.21.15.12:0/2982099792 shutdown_connections 2024-05-09T09:25:36.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.171+0000 7f65adc12640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f65900a2c10 0x7f65900a50d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.171+0000 7f65adc12640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65a8101440 0x7f65a81a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.171+0000 7f65adc12640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8100aa0 0x7f65a819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.171+0000 7f65adc12640 1 --2- 172.21.15.12:0/2982099792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a8100140 0x7f65a8199dc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.171+0000 7f65adc12640 1 -- 172.21.15.12:0/2982099792 >> 172.21.15.12:0/2982099792 conn(0x7f65a80fbce0 msgr2=0x7f65a80fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:36.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.171+0000 7f65adc12640 1 -- 172.21.15.12:0/2982099792 shutdown_connections 2024-05-09T09:25:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.171+0000 7f65adc12640 1 -- 172.21.15.12:0/2982099792 wait complete. 2024-05-09T09:25:36.182 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-09T09:25:36.182 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-09T09:25:36.182 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T09:25:36.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.288+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1249962799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc102340 msgr2=0x7f5ccc108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:36.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.288+0000 7f5cd0d53640 1 --2- 172.21.15.12:0/1249962799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc102340 0x7f5ccc108bd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009ca0 tx=0x7f5cc402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.288+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1249962799 shutdown_connections 2024-05-09T09:25:36.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.288+0000 7f5cd0d53640 1 --2- 172.21.15.12:0/1249962799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc102340 0x7f5ccc108bd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.288+0000 7f5cd0d53640 1 --2- 172.21.15.12:0/1249962799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ccc101980 0x7f5ccc101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.288+0000 7f5cd0d53640 1 --2- 172.21.15.12:0/1249962799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ccc100790 0x7f5ccc100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.288+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1249962799 >> 172.21.15.12:0/1249962799 conn(0x7f5ccc0fbd20 msgr2=0x7f5ccc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:36.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.288+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1249962799 shutdown_connections 2024-05-09T09:25:36.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.288+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1249962799 wait complete. 2024-05-09T09:25:36.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.289+0000 7f5cd0d53640 1 Processor -- start 2024-05-09T09:25:36.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.289+0000 7f5cd0d53640 1 -- start start 2024-05-09T09:25:36.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.289+0000 7f5cd0d53640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc100790 0x7f5ccc10ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:36.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.289+0000 7f5cd0d53640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ccc101980 0x7f5ccc109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:36.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.289+0000 7f5cd0d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ccc102340 0x7f5ccc109990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:36.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.289+0000 7f5cd0d53640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc10d4f0 con 0x7f5ccc100790 2024-05-09T09:25:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.289+0000 7f5cd0d53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc10d370 con 0x7f5ccc102340 2024-05-09T09:25:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.289+0000 7f5cd0d53640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc10d670 con 0x7f5ccc101980 2024-05-09T09:25:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.289+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc100790 0x7f5ccc10ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc100790 0x7f5ccc10ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33972/0 (socket says 172.21.15.12:33972) 2024-05-09T09:25:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cca59c640 1 -- 172.21.15.12:0/1438282880 learned_addr learned my addr 172.21.15.12:0/1438282880 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5ccad9d640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ccc102340 0x7f5ccc109990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cc9d9b640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ccc101980 0x7f5ccc109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cca59c640 1 -- 172.21.15.12:0/1438282880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ccc101980 msgr2=0x7f5ccc109450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cca59c640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ccc101980 0x7f5ccc109450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cca59c640 1 -- 172.21.15.12:0/1438282880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ccc102340 msgr2=0x7f5ccc109990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cca59c640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ccc102340 0x7f5ccc109990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cca59c640 1 -- 172.21.15.12:0/1438282880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc10a220 con 0x7f5ccc100790 2024-05-09T09:25:36.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cca59c640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc100790 0x7f5ccc10ae10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00c970 tx=0x7f5cbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:36.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1438282880 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc004220 con 0x7f5ccc100790 2024-05-09T09:25:36.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1438282880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc1a1c80 con 0x7f5ccc100790 2024-05-09T09:25:36.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1438282880 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5cbc0043c0 con 0x7f5ccc100790 2024-05-09T09:25:36.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1438282880 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc004a00 con 0x7f5ccc100790 2024-05-09T09:25:36.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.290+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1438282880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc1a21d0 con 0x7f5ccc100790 2024-05-09T09:25:36.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.291+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1438282880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ccc107ab0 con 0x7f5ccc100790 2024-05-09T09:25:36.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.293+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1438282880 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5cbc004ba0 con 0x7f5ccc100790 2024-05-09T09:25:36.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.293+0000 7f5cbb7fe640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5cb40a2b90 0x7f5cb40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:36.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.293+0000 7f5cc9d9b640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5cb40a2b90 0x7f5cb40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:36.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.293+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1438282880 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f5cbc0cf180 con 0x7f5ccc100790 2024-05-09T09:25:36.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.294+0000 7f5cc9d9b640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5cb40a2b90 0x7f5cb40a5050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc10ab80 tx=0x7f5cc0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:36.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.298+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1438282880 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5cbc014030 con 0x7f5ccc100790 2024-05-09T09:25:36.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.479+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1438282880 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ccc061c70 con 0x7f5cb40a2b90 2024-05-09T09:25:36.483 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:36 smithi012 ceph-mon[30140]: pgmap v368: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-05-09T09:25:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:36 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3781049743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:36 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1501593942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:36 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2982099792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:36.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.489+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1438282880 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340543 (secure 0 0 0) 0x7f5ccc061c70 con 0x7f5cb40a2b90 2024-05-09T09:25:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:36.495 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-09T09:25:36.496 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:36 smithi170 ceph-mon[33632]: pgmap v368: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-05-09T09:25:36.497 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:36 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3781049743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:36.497 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:36 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1501593942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:36.497 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:36 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2982099792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:36.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.492+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1438282880 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5cb40a2b90 msgr2=0x7f5cb40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:36.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.492+0000 7f5cd0d53640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5cb40a2b90 0x7f5cb40a5050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc10ab80 tx=0x7f5cc0005f90 comp rx=0 tx=0).stop 2024-05-09T09:25:36.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.492+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1438282880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc100790 msgr2=0x7f5ccc10ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:36.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.492+0000 7f5cd0d53640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc100790 0x7f5ccc10ae10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00c970 tx=0x7f5cbc00ce30 comp rx=0 tx=0).stop 2024-05-09T09:25:36.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.492+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1438282880 shutdown_connections 2024-05-09T09:25:36.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.492+0000 7f5cd0d53640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5cb40a2b90 0x7f5cb40a5050 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.492+0000 7f5cd0d53640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ccc102340 0x7f5ccc109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.492+0000 7f5cd0d53640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ccc101980 0x7f5ccc109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.492+0000 7f5cd0d53640 1 --2- 172.21.15.12:0/1438282880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc100790 0x7f5ccc10ae10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.492+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1438282880 >> 172.21.15.12:0/1438282880 conn(0x7f5ccc0fbd20 msgr2=0x7f5ccc104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:36.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.492+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1438282880 shutdown_connections 2024-05-09T09:25:36.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.492+0000 7f5cd0d53640 1 -- 172.21.15.12:0/1438282880 wait complete. 2024-05-09T09:25:36.503 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-05-09T09:25:34.590637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663626,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1365,"num_read_kb":810,"num_write":448,"num_write_kb":3098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":868,"ondisk_log_size":868,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392904,"kb_used_data":14624,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124548920,"statfs":{"total":1151940427776,"available":1151538094080,"internally_reserved":0,"allocated":14974976,"data_stored":8199620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":74,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003478"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160720+0000","last_change":"2024-05-09T09:23:10.334738+0000","last_active":"2024-05-09T09:25:06.160720+0000","last_peered":"2024-05-09T09:25:06.160720+0000","last_clean":"2024-05-09T09:25:06.160720+0000","last_became_active":"2024-05-09T09:23:10.333406+0000","last_became_peered":"2024-05-09T09:23:10.333406+0000","last_unstale":"2024-05-09T09:25:06.160720+0000","last_undegraded":"2024-05-09T09:25:06.160720+0000","last_fullsized":"2024-05-09T09:25:06.160720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:44.957557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165459+0000","last_change":"2024-05-09T09:23:10.331860+0000","last_active":"2024-05-09T09:25:06.165459+0000","last_peered":"2024-05-09T09:25:06.165459+0000","last_clean":"2024-05-09T09:25:06.165459+0000","last_became_active":"2024-05-09T09:23:10.331396+0000","last_became_peered":"2024-05-09T09:23:10.331396+0000","last_unstale":"2024-05-09T09:25:06.165459+0000","last_undegraded":"2024-05-09T09:25:06.165459+0000","last_fullsized":"2024-05-09T09:25:06.165459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:47.676399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165707+0000","last_change":"2024-05-09T09:23:10.336543+0000","last_active":"2024-05-09T09:25:06.165707+0000","last_peered":"2024-05-09T09:25:06.165707+0000","last_clean":"2024-05-09T09:25:06.165707+0000","last_became_active":"2024-05-09T09:23:10.336189+0000","last_became_peered":"2024-05-09T09:23:10.336189+0000","last_unstale":"2024-05-09T09:25:06.165707+0000","last_undegraded":"2024-05-09T09:25:06.165707+0000","last_fullsized":"2024-05-09T09:25:06.165707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:06.419177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157538+0000","last_change":"2024-05-09T09:23:10.331739+0000","last_active":"2024-05-09T09:25:06.157538+0000","last_peered":"2024-05-09T09:25:06.157538+0000","last_clean":"2024-05-09T09:25:06.157538+0000","last_became_active":"2024-05-09T09:23:10.331333+0000","last_became_peered":"2024-05-09T09:23:10.331333+0000","last_unstale":"2024-05-09T09:25:06.157538+0000","last_undegraded":"2024-05-09T09:25:06.157538+0000","last_fullsized":"2024-05-09T09:25:06.157538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:04.925697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160387+0000","last_change":"2024-05-09T09:23:10.345294+0000","last_active":"2024-05-09T09:25:06.160387+0000","last_peered":"2024-05-09T09:25:06.160387+0000","last_clean":"2024-05-09T09:25:06.160387+0000","last_became_active":"2024-05-09T09:23:10.344940+0000","last_became_peered":"2024-05-09T09:23:10.344940+0000","last_unstale":"2024-05-09T09:25:06.160387+0000","last_undegraded":"2024-05-09T09:25:06.160387+0000","last_fullsized":"2024-05-09T09:25:06.160387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:41.150284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256906+0000","last_change":"2024-05-09T09:23:10.329143+0000","last_active":"2024-05-09T09:25:06.256906+0000","last_peered":"2024-05-09T09:25:06.256906+0000","last_clean":"2024-05-09T09:25:06.256906+0000","last_became_active":"2024-05-09T09:23:10.328712+0000","last_became_peered":"2024-05-09T09:23:10.328712+0000","last_unstale":"2024-05-09T09:25:06.256906+0000","last_undegraded":"2024-05-09T09:25:06.256906+0000","last_fullsized":"2024-05-09T09:25:06.256906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:10:35.085759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161026+0000","last_change":"2024-05-09T09:23:10.338176+0000","last_active":"2024-05-09T09:25:06.161026+0000","last_peered":"2024-05-09T09:25:06.161026+0000","last_clean":"2024-05-09T09:25:06.161026+0000","last_became_active":"2024-05-09T09:23:10.337845+0000","last_became_peered":"2024-05-09T09:23:10.337845+0000","last_unstale":"2024-05-09T09:25:06.161026+0000","last_undegraded":"2024-05-09T09:25:06.161026+0000","last_fullsized":"2024-05-09T09:25:06.161026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:34:21.783295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160809+0000","last_change":"2024-05-09T09:23:10.342473+0000","last_active":"2024-05-09T09:25:06.160809+0000","last_peered":"2024-05-09T09:25:06.160809+0000","last_clean":"2024-05-09T09:25:06.160809+0000","last_became_active":"2024-05-09T09:23:10.340682+0000","last_became_peered":"2024-05-09T09:23:10.340682+0000","last_unstale":"2024-05-09T09:25:06.160809+0000","last_undegraded":"2024-05-09T09:25:06.160809+0000","last_fullsized":"2024-05-09T09:25:06.160809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:56.730332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257607+0000","last_change":"2024-05-09T09:23:10.339888+0000","last_active":"2024-05-09T09:25:06.257607+0000","last_peered":"2024-05-09T09:25:06.257607+0000","last_clean":"2024-05-09T09:25:06.257607+0000","last_became_active":"2024-05-09T09:23:10.337849+0000","last_became_peered":"2024-05-09T09:23:10.337849+0000","last_unstale":"2024-05-09T09:25:06.257607+0000","last_undegraded":"2024-05-09T09:25:06.257607+0000","last_fullsized":"2024-05-09T09:25:06.257607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:48.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257800+0000","last_change":"2024-05-09T09:23:10.332395+0000","last_active":"2024-05-09T09:25:06.257800+0000","last_peered":"2024-05-09T09:25:06.257800+0000","last_clean":"2024-05-09T09:25:06.257800+0000","last_became_active":"2024-05-09T09:23:10.332040+0000","last_became_peered":"2024-05-09T09:23:10.332040+0000","last_unstale":"2024-05-09T09:25:06.257800+0000","last_undegraded":"2024-05-09T09:25:06.257800+0000","last_fullsized":"2024-05-09T09:25:06.257800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:04.036871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161769+0000","last_change":"2024-05-09T09:23:10.341148+0000","last_active":"2024-05-09T09:25:06.161769+0000","last_peered":"2024-05-09T09:25:06.161769+0000","last_clean":"2024-05-09T09:25:06.161769+0000","last_became_active":"2024-05-09T09:23:10.340799+0000","last_became_peered":"2024-05-09T09:23:10.340799+0000","last_unstale":"2024-05-09T09:25:06.161769+0000","last_undegraded":"2024-05-09T09:25:06.161769+0000","last_fullsized":"2024-05-09T09:25:06.161769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:31.020659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157535+0000","last_change":"2024-05-09T09:23:10.336191+0000","last_active":"2024-05-09T09:25:06.157535+0000","last_peered":"2024-05-09T09:25:06.157535+0000","last_clean":"2024-05-09T09:25:06.157535+0000","last_became_active":"2024-05-09T09:23:10.333591+0000","last_became_peered":"2024-05-09T09:23:10.333591+0000","last_unstale":"2024-05-09T09:25:06.157535+0000","last_undegraded":"2024-05-09T09:25:06.157535+0000","last_fullsized":"2024-05-09T09:25:06.157535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:24.836954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156200+0000","last_change":"2024-05-09T09:23:10.314506+0000","last_active":"2024-05-09T09:25:06.156200+0000","last_peered":"2024-05-09T09:25:06.156200+0000","last_clean":"2024-05-09T09:25:06.156200+0000","last_became_active":"2024-05-09T09:23:10.314171+0000","last_became_peered":"2024-05-09T09:23:10.314171+0000","last_unstale":"2024-05-09T09:25:06.156200+0000","last_undegraded":"2024-05-09T09:25:06.156200+0000","last_fullsized":"2024-05-09T09:25:06.156200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:51.088814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165407+0000","last_change":"2024-05-09T09:23:10.341670+0000","last_active":"2024-05-09T09:25:06.165407+0000","last_peered":"2024-05-09T09:25:06.165407+0000","last_clean":"2024-05-09T09:25:06.165407+0000","last_became_active":"2024-05-09T09:23:10.341155+0000","last_became_peered":"2024-05-09T09:23:10.341155+0000","last_unstale":"2024-05-09T09:25:06.165407+0000","last_undegraded":"2024-05-09T09:25:06.165407+0000","last_fullsized":"2024-05-09T09:25:06.165407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:14:32.734302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165578+0000","last_change":"2024-05-09T09:24:05.742117+0000","last_active":"2024-05-09T09:25:06.165578+0000","last_peered":"2024-05-09T09:25:06.165578+0000","last_clean":"2024-05-09T09:25:06.165578+0000","last_became_active":"2024-05-09T09:24:05.741655+0000","last_became_peered":"2024-05-09T09:24:05.741655+0000","last_unstale":"2024-05-09T09:25:06.165578+0000","last_undegraded":"2024-05-09T09:25:06.165578+0000","last_fullsized":"2024-05-09T09:25:06.165578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:11.468527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256668+0000","last_change":"2024-05-09T09:23:10.337367+0000","last_active":"2024-05-09T09:25:06.256668+0000","last_peered":"2024-05-09T09:25:06.256668+0000","last_clean":"2024-05-09T09:25:06.256668+0000","last_became_active":"2024-05-09T09:23:10.337000+0000","last_became_peered":"2024-05-09T09:23:10.337000+0000","last_unstale":"2024-05-09T09:25:06.256668+0000","last_undegraded":"2024-05-09T09:25:06.256668+0000","last_fullsized":"2024-05-09T09:25:06.256668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:54.431820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160278+0000","last_change":"2024-05-09T09:23:10.334293+0000","last_active":"2024-05-09T09:25:06.160278+0000","last_peered":"2024-05-09T09:25:06.160278+0000","last_clean":"2024-05-09T09:25:06.160278+0000","last_became_active":"2024-05-09T09:23:10.333938+0000","last_became_peered":"2024-05-09T09:23:10.333938+0000","last_unstale":"2024-05-09T09:25:06.160278+0000","last_undegraded":"2024-05-09T09:25:06.160278+0000","last_fullsized":"2024-05-09T09:25:06.160278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:56:09.048848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159638+0000","last_change":"2024-05-09T09:23:10.332394+0000","last_active":"2024-05-09T09:25:06.159638+0000","last_peered":"2024-05-09T09:25:06.159638+0000","last_clean":"2024-05-09T09:25:06.159638+0000","last_became_active":"2024-05-09T09:23:10.332054+0000","last_became_peered":"2024-05-09T09:23:10.332054+0000","last_unstale":"2024-05-09T09:25:06.159638+0000","last_undegraded":"2024-05-09T09:25:06.159638+0000","last_fullsized":"2024-05-09T09:25:06.159638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:05.628740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164212+0000","last_change":"2024-05-09T09:23:10.336189+0000","last_active":"2024-05-09T09:25:06.164212+0000","last_peered":"2024-05-09T09:25:06.164212+0000","last_clean":"2024-05-09T09:25:06.164212+0000","last_became_active":"2024-05-09T09:23:10.335698+0000","last_became_peered":"2024-05-09T09:23:10.335698+0000","last_unstale":"2024-05-09T09:25:06.164212+0000","last_undegraded":"2024-05-09T09:25:06.164212+0000","last_fullsized":"2024-05-09T09:25:06.164212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:01.403918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162866+0000","last_change":"2024-05-09T09:23:14.372134+0000","last_active":"2024-05-09T09:25:06.162866+0000","last_peered":"2024-05-09T09:25:06.162866+0000","last_clean":"2024-05-09T09:25:06.162866+0000","last_became_active":"2024-05-09T09:23:14.371466+0000","last_became_peered":"2024-05-09T09:23:14.371466+0000","last_unstale":"2024-05-09T09:25:06.162866+0000","last_undegraded":"2024-05-09T09:25:06.162866+0000","last_fullsized":"2024-05-09T09:25:06.162866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:22.048565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:31.760845+0000","last_change":"2024-05-09T09:23:09.304421+0000","last_active":"2024-05-09T09:25:31.760845+0000","last_peered":"2024-05-09T09:25:31.760845+0000","last_clean":"2024-05-09T09:25:31.760845+0000","last_became_active":"2024-05-09T09:23:09.304109+0000","last_became_peered":"2024-05-09T09:23:09.304109+0000","last_unstale":"2024-05-09T09:25:31.760845+0000","last_undegraded":"2024-05-09T09:25:31.760845+0000","last_fullsized":"2024-05-09T09:25:31.760845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:36.796458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162409+0000","last_change":"2024-05-09T09:23:10.341875+0000","last_active":"2024-05-09T09:25:06.162409+0000","last_peered":"2024-05-09T09:25:06.162409+0000","last_clean":"2024-05-09T09:25:06.162409+0000","last_became_active":"2024-05-09T09:23:10.341385+0000","last_became_peered":"2024-05-09T09:23:10.341385+0000","last_unstale":"2024-05-09T09:25:06.162409+0000","last_undegraded":"2024-05-09T09:25:06.162409+0000","last_fullsized":"2024-05-09T09:25:06.162409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:44:33.939722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156972+0000","last_change":"2024-05-09T09:23:14.371120+0000","last_active":"2024-05-09T09:25:06.156972+0000","last_peered":"2024-05-09T09:25:06.156972+0000","last_clean":"2024-05-09T09:25:06.156972+0000","last_became_active":"2024-05-09T09:23:14.370663+0000","last_became_peered":"2024-05-09T09:23:14.370663+0000","last_unstale":"2024-05-09T09:25:06.156972+0000","last_undegraded":"2024-05-09T09:25:06.156972+0000","last_fullsized":"2024-05-09T09:25:06.156972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:50:06.553518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162781+0000","last_change":"2024-05-09T09:23:09.304357+0000","last_active":"2024-05-09T09:25:06.162781+0000","last_peered":"2024-05-09T09:25:06.162781+0000","last_clean":"2024-05-09T09:25:06.162781+0000","last_became_active":"2024-05-09T09:23:09.304007+0000","last_became_peered":"2024-05-09T09:23:09.304007+0000","last_unstale":"2024-05-09T09:25:06.162781+0000","last_undegraded":"2024-05-09T09:25:06.162781+0000","last_fullsized":"2024-05-09T09:25:06.162781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:32:46.132287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165838+0000","last_change":"2024-05-09T09:23:10.339533+0000","last_active":"2024-05-09T09:25:06.165838+0000","last_peered":"2024-05-09T09:25:06.165838+0000","last_clean":"2024-05-09T09:25:06.165838+0000","last_became_active":"2024-05-09T09:23:10.337142+0000","last_became_peered":"2024-05-09T09:23:10.337142+0000","last_unstale":"2024-05-09T09:25:06.165838+0000","last_undegraded":"2024-05-09T09:25:06.165838+0000","last_fullsized":"2024-05-09T09:25:06.165838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:11:11.921491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157173+0000","last_change":"2024-05-09T09:24:05.744009+0000","last_active":"2024-05-09T09:25:06.157173+0000","last_peered":"2024-05-09T09:25:06.157173+0000","last_clean":"2024-05-09T09:25:06.157173+0000","last_became_active":"2024-05-09T09:24:05.743611+0000","last_became_peered":"2024-05-09T09:24:05.743611+0000","last_unstale":"2024-05-09T09:25:06.157173+0000","last_undegraded":"2024-05-09T09:25:06.157173+0000","last_fullsized":"2024-05-09T09:25:06.157173+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:55.775740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160457+0000","last_change":"2024-05-09T09:23:09.300754+0000","last_active":"2024-05-09T09:25:06.160457+0000","last_peered":"2024-05-09T09:25:06.160457+0000","last_clean":"2024-05-09T09:25:06.160457+0000","last_became_active":"2024-05-09T09:23:09.300419+0000","last_became_peered":"2024-05-09T09:23:09.300419+0000","last_unstale":"2024-05-09T09:25:06.160457+0000","last_undegraded":"2024-05-09T09:25:06.160457+0000","last_fullsized":"2024-05-09T09:25:06.160457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:06.205582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257117+0000","last_change":"2024-05-09T09:23:10.331952+0000","last_active":"2024-05-09T09:25:06.257117+0000","last_peered":"2024-05-09T09:25:06.257117+0000","last_clean":"2024-05-09T09:25:06.257117+0000","last_became_active":"2024-05-09T09:23:10.331515+0000","last_became_peered":"2024-05-09T09:23:10.331515+0000","last_unstale":"2024-05-09T09:25:06.257117+0000","last_undegraded":"2024-05-09T09:25:06.257117+0000","last_fullsized":"2024-05-09T09:25:06.257117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:23.809672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156952+0000","last_change":"2024-05-09T09:23:14.368518+0000","last_active":"2024-05-09T09:25:06.156952+0000","last_peered":"2024-05-09T09:25:06.156952+0000","last_clean":"2024-05-09T09:25:06.156952+0000","last_became_active":"2024-05-09T09:23:14.367995+0000","last_became_peered":"2024-05-09T09:23:14.367995+0000","last_unstale":"2024-05-09T09:25:06.156952+0000","last_undegraded":"2024-05-09T09:25:06.156952+0000","last_fullsized":"2024-05-09T09:25:06.156952+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:11.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160465+0000","last_change":"2024-05-09T09:23:09.299656+0000","last_active":"2024-05-09T09:25:06.160465+0000","last_peered":"2024-05-09T09:25:06.160465+0000","last_clean":"2024-05-09T09:25:06.160465+0000","last_became_active":"2024-05-09T09:23:09.299374+0000","last_became_peered":"2024-05-09T09:23:09.299374+0000","last_unstale":"2024-05-09T09:25:06.160465+0000","last_undegraded":"2024-05-09T09:25:06.160465+0000","last_fullsized":"2024-05-09T09:25:06.160465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:05.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157723+0000","last_change":"2024-05-09T09:23:10.341933+0000","last_active":"2024-05-09T09:25:06.157723+0000","last_peered":"2024-05-09T09:25:06.157723+0000","last_clean":"2024-05-09T09:25:06.157723+0000","last_became_active":"2024-05-09T09:23:10.341621+0000","last_became_peered":"2024-05-09T09:23:10.341621+0000","last_unstale":"2024-05-09T09:25:06.157723+0000","last_undegraded":"2024-05-09T09:25:06.157723+0000","last_fullsized":"2024-05-09T09:25:06.157723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:31.938768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.163011+0000","last_change":"2024-05-09T09:23:14.364783+0000","last_active":"2024-05-09T09:25:06.163011+0000","last_peered":"2024-05-09T09:25:06.163011+0000","last_clean":"2024-05-09T09:25:06.163011+0000","last_became_active":"2024-05-09T09:23:14.364309+0000","last_became_peered":"2024-05-09T09:23:14.364309+0000","last_unstale":"2024-05-09T09:25:06.163011+0000","last_undegraded":"2024-05-09T09:25:06.163011+0000","last_fullsized":"2024-05-09T09:25:06.163011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:44.252111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'6","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:32.763334+0000","last_change":"2024-05-09T09:23:09.297913+0000","last_active":"2024-05-09T09:25:32.763334+0000","last_peered":"2024-05-09T09:25:32.763334+0000","last_clean":"2024-05-09T09:25:32.763334+0000","last_became_active":"2024-05-09T09:23:09.297612+0000","last_became_peered":"2024-05-09T09:23:09.297612+0000","last_unstale":"2024-05-09T09:25:32.763334+0000","last_undegraded":"2024-05-09T09:25:32.763334+0000","last_fullsized":"2024-05-09T09:25:32.763334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:21.808703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.163905+0000","last_change":"2024-05-09T09:25:06.163905+0000","last_active":"2024-05-09T09:25:06.163905+0000","last_peered":"2024-05-09T09:25:06.163905+0000","last_clean":"2024-05-09T09:25:06.163905+0000","last_became_active":"2024-05-09T09:25:06.163454+0000","last_became_peered":"2024-05-09T09:25:06.163454+0000","last_unstale":"2024-05-09T09:25:06.163905+0000","last_undegraded":"2024-05-09T09:25:06.163905+0000","last_fullsized":"2024-05-09T09:25:06.163905+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:49:05.654792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162936+0000","last_change":"2024-05-09T09:23:14.371781+0000","last_active":"2024-05-09T09:25:06.162936+0000","last_peered":"2024-05-09T09:25:06.162936+0000","last_clean":"2024-05-09T09:25:06.162936+0000","last_became_active":"2024-05-09T09:23:14.371286+0000","last_became_peered":"2024-05-09T09:23:14.371286+0000","last_unstale":"2024-05-09T09:25:06.162936+0000","last_undegraded":"2024-05-09T09:25:06.162936+0000","last_fullsized":"2024-05-09T09:25:06.162936+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:26:29.185064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256852+0000","last_change":"2024-05-09T09:23:09.303786+0000","last_active":"2024-05-09T09:25:06.256852+0000","last_peered":"2024-05-09T09:25:06.256852+0000","last_clean":"2024-05-09T09:25:06.256852+0000","last_became_active":"2024-05-09T09:23:09.303475+0000","last_became_peered":"2024-05-09T09:23:09.303475+0000","last_unstale":"2024-05-09T09:25:06.256852+0000","last_undegraded":"2024-05-09T09:25:06.256852+0000","last_fullsized":"2024-05-09T09:25:06.256852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:28.833055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157664+0000","last_change":"2024-05-09T09:23:10.344643+0000","last_active":"2024-05-09T09:25:06.157664+0000","last_peered":"2024-05-09T09:25:06.157664+0000","last_clean":"2024-05-09T09:25:06.157664+0000","last_became_active":"2024-05-09T09:23:10.344314+0000","last_became_peered":"2024-05-09T09:23:10.344314+0000","last_unstale":"2024-05-09T09:25:06.157664+0000","last_undegraded":"2024-05-09T09:25:06.157664+0000","last_fullsized":"2024-05-09T09:25:06.157664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:15.147107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160150+0000","last_change":"2024-05-09T09:23:14.361170+0000","last_active":"2024-05-09T09:25:06.160150+0000","last_peered":"2024-05-09T09:25:06.160150+0000","last_clean":"2024-05-09T09:25:06.160150+0000","last_became_active":"2024-05-09T09:23:14.360767+0000","last_became_peered":"2024-05-09T09:23:14.360767+0000","last_unstale":"2024-05-09T09:25:06.160150+0000","last_undegraded":"2024-05-09T09:25:06.160150+0000","last_fullsized":"2024-05-09T09:25:06.160150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:59:21.393148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156023+0000","last_change":"2024-05-09T09:23:09.300757+0000","last_active":"2024-05-09T09:25:06.156023+0000","last_peered":"2024-05-09T09:25:06.156023+0000","last_clean":"2024-05-09T09:25:06.156023+0000","last_became_active":"2024-05-09T09:23:09.300328+0000","last_became_peered":"2024-05-09T09:23:09.300328+0000","last_unstale":"2024-05-09T09:25:06.156023+0000","last_undegraded":"2024-05-09T09:25:06.156023+0000","last_fullsized":"2024-05-09T09:25:06.156023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.553878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161040+0000","last_change":"2024-05-09T09:23:10.336077+0000","last_active":"2024-05-09T09:25:06.161040+0000","last_peered":"2024-05-09T09:25:06.161040+0000","last_clean":"2024-05-09T09:25:06.161040+0000","last_became_active":"2024-05-09T09:23:10.335762+0000","last_became_peered":"2024-05-09T09:23:10.335762+0000","last_unstale":"2024-05-09T09:25:06.161040+0000","last_undegraded":"2024-05-09T09:25:06.161040+0000","last_fullsized":"2024-05-09T09:25:06.161040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:48.063385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165335+0000","last_change":"2024-05-09T09:23:14.371413+0000","last_active":"2024-05-09T09:25:06.165335+0000","last_peered":"2024-05-09T09:25:06.165335+0000","last_clean":"2024-05-09T09:25:06.165335+0000","last_became_active":"2024-05-09T09:23:14.370018+0000","last_became_peered":"2024-05-09T09:23:14.370018+0000","last_unstale":"2024-05-09T09:25:06.165335+0000","last_undegraded":"2024-05-09T09:25:06.165335+0000","last_fullsized":"2024-05-09T09:25:06.165335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:18.475976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157139+0000","last_change":"2024-05-09T09:23:09.297328+0000","last_active":"2024-05-09T09:25:06.157139+0000","last_peered":"2024-05-09T09:25:06.157139+0000","last_clean":"2024-05-09T09:25:06.157139+0000","last_became_active":"2024-05-09T09:23:09.297001+0000","last_became_peered":"2024-05-09T09:23:09.297001+0000","last_unstale":"2024-05-09T09:25:06.157139+0000","last_undegraded":"2024-05-09T09:25:06.157139+0000","last_fullsized":"2024-05-09T09:25:06.157139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:19.241362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159057+0000","last_change":"2024-05-09T09:23:10.338342+0000","last_active":"2024-05-09T09:25:06.159057+0000","last_peered":"2024-05-09T09:25:06.159057+0000","last_clean":"2024-05-09T09:25:06.159057+0000","last_became_active":"2024-05-09T09:23:10.337995+0000","last_became_peered":"2024-05-09T09:23:10.337995+0000","last_unstale":"2024-05-09T09:25:06.159057+0000","last_undegraded":"2024-05-09T09:25:06.159057+0000","last_fullsized":"2024-05-09T09:25:06.159057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.691862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161348+0000","last_change":"2024-05-09T09:23:14.366237+0000","last_active":"2024-05-09T09:25:06.161348+0000","last_peered":"2024-05-09T09:25:06.161348+0000","last_clean":"2024-05-09T09:25:06.161348+0000","last_became_active":"2024-05-09T09:23:14.365809+0000","last_became_peered":"2024-05-09T09:23:14.365809+0000","last_unstale":"2024-05-09T09:25:06.161348+0000","last_undegraded":"2024-05-09T09:25:06.161348+0000","last_fullsized":"2024-05-09T09:25:06.161348+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:28.813467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":279,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:33.764231+0000","last_change":"2024-05-09T09:23:09.298658+0000","last_active":"2024-05-09T09:25:33.764231+0000","last_peered":"2024-05-09T09:25:33.764231+0000","last_clean":"2024-05-09T09:25:33.764231+0000","last_became_active":"2024-05-09T09:23:09.298325+0000","last_became_peered":"2024-05-09T09:23:09.298325+0000","last_unstale":"2024-05-09T09:25:33.764231+0000","last_undegraded":"2024-05-09T09:25:33.764231+0000","last_fullsized":"2024-05-09T09:25:33.764231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:22.108202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":242,"num_read_kb":242,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257446+0000","last_change":"2024-05-09T09:23:10.326865+0000","last_active":"2024-05-09T09:25:06.257446+0000","last_peered":"2024-05-09T09:25:06.257446+0000","last_clean":"2024-05-09T09:25:06.257446+0000","last_became_active":"2024-05-09T09:23:10.326478+0000","last_became_peered":"2024-05-09T09:23:10.326478+0000","last_unstale":"2024-05-09T09:25:06.257446+0000","last_undegraded":"2024-05-09T09:25:06.257446+0000","last_fullsized":"2024-05-09T09:25:06.257446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:54.521226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.158982+0000","last_change":"2024-05-09T09:23:14.373601+0000","last_active":"2024-05-09T09:25:06.158982+0000","last_peered":"2024-05-09T09:25:06.158982+0000","last_clean":"2024-05-09T09:25:06.158982+0000","last_became_active":"2024-05-09T09:23:14.372764+0000","last_became_peered":"2024-05-09T09:23:14.372764+0000","last_unstale":"2024-05-09T09:25:06.158982+0000","last_undegraded":"2024-05-09T09:25:06.158982+0000","last_fullsized":"2024-05-09T09:25:06.158982+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:59.496755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161999+0000","last_change":"2024-05-09T09:23:09.302293+0000","last_active":"2024-05-09T09:25:06.161999+0000","last_peered":"2024-05-09T09:25:06.161999+0000","last_clean":"2024-05-09T09:25:06.161999+0000","last_became_active":"2024-05-09T09:23:09.301216+0000","last_became_peered":"2024-05-09T09:23:09.301216+0000","last_unstale":"2024-05-09T09:25:06.161999+0000","last_undegraded":"2024-05-09T09:25:06.161999+0000","last_fullsized":"2024-05-09T09:25:06.161999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:35.146860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165710+0000","last_change":"2024-05-09T09:23:10.339827+0000","last_active":"2024-05-09T09:25:06.165710+0000","last_peered":"2024-05-09T09:25:06.165710+0000","last_clean":"2024-05-09T09:25:06.165710+0000","last_became_active":"2024-05-09T09:23:10.339415+0000","last_became_peered":"2024-05-09T09:23:10.339415+0000","last_unstale":"2024-05-09T09:25:06.165710+0000","last_undegraded":"2024-05-09T09:25:06.165710+0000","last_fullsized":"2024-05-09T09:25:06.165710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:45.250126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157306+0000","last_change":"2024-05-09T09:23:14.377001+0000","last_active":"2024-05-09T09:25:06.157306+0000","last_peered":"2024-05-09T09:25:06.157306+0000","last_clean":"2024-05-09T09:25:06.157306+0000","last_became_active":"2024-05-09T09:23:14.376402+0000","last_became_peered":"2024-05-09T09:23:14.376402+0000","last_unstale":"2024-05-09T09:25:06.157306+0000","last_undegraded":"2024-05-09T09:25:06.157306+0000","last_fullsized":"2024-05-09T09:25:06.157306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:08.142495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.158215+0000","last_change":"2024-05-09T09:23:09.299331+0000","last_active":"2024-05-09T09:25:06.158215+0000","last_peered":"2024-05-09T09:25:06.158215+0000","last_clean":"2024-05-09T09:25:06.158215+0000","last_became_active":"2024-05-09T09:23:09.299017+0000","last_became_peered":"2024-05-09T09:23:09.299017+0000","last_unstale":"2024-05-09T09:25:06.158215+0000","last_undegraded":"2024-05-09T09:25:06.158215+0000","last_fullsized":"2024-05-09T09:25:06.158215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:28.328704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257442+0000","last_change":"2024-05-09T09:24:05.743586+0000","last_active":"2024-05-09T09:25:06.257442+0000","last_peered":"2024-05-09T09:25:06.257442+0000","last_clean":"2024-05-09T09:25:06.257442+0000","last_became_active":"2024-05-09T09:24:05.742610+0000","last_became_peered":"2024-05-09T09:24:05.742610+0000","last_unstale":"2024-05-09T09:25:06.257442+0000","last_undegraded":"2024-05-09T09:25:06.257442+0000","last_fullsized":"2024-05-09T09:25:06.257442+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:05.586985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159196+0000","last_change":"2024-05-09T09:23:14.363362+0000","last_active":"2024-05-09T09:25:06.159196+0000","last_peered":"2024-05-09T09:25:06.159196+0000","last_clean":"2024-05-09T09:25:06.159196+0000","last_became_active":"2024-05-09T09:23:14.362926+0000","last_became_peered":"2024-05-09T09:23:14.362926+0000","last_unstale":"2024-05-09T09:25:06.159196+0000","last_undegraded":"2024-05-09T09:25:06.159196+0000","last_fullsized":"2024-05-09T09:25:06.159196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:25.339727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":321,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:33.763965+0000","last_change":"2024-05-09T09:23:09.297814+0000","last_active":"2024-05-09T09:25:33.763965+0000","last_peered":"2024-05-09T09:25:33.763965+0000","last_clean":"2024-05-09T09:25:33.763965+0000","last_became_active":"2024-05-09T09:23:09.297005+0000","last_became_peered":"2024-05-09T09:23:09.297005+0000","last_unstale":"2024-05-09T09:25:33.763965+0000","last_undegraded":"2024-05-09T09:25:33.763965+0000","last_fullsized":"2024-05-09T09:25:33.763965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:39.240217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64098,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":251,"num_read_kb":188,"num_write":41,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160129+0000","last_change":"2024-05-09T09:23:10.333013+0000","last_active":"2024-05-09T09:25:06.160129+0000","last_peered":"2024-05-09T09:25:06.160129+0000","last_clean":"2024-05-09T09:25:06.160129+0000","last_became_active":"2024-05-09T09:23:10.332666+0000","last_became_peered":"2024-05-09T09:23:10.332666+0000","last_unstale":"2024-05-09T09:25:06.160129+0000","last_undegraded":"2024-05-09T09:25:06.160129+0000","last_fullsized":"2024-05-09T09:25:06.160129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:28:50.306551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.158019+0000","last_change":"2024-05-09T09:23:14.367715+0000","last_active":"2024-05-09T09:25:06.158019+0000","last_peered":"2024-05-09T09:25:06.158019+0000","last_clean":"2024-05-09T09:25:06.158019+0000","last_became_active":"2024-05-09T09:23:14.367307+0000","last_became_peered":"2024-05-09T09:23:14.367307+0000","last_unstale":"2024-05-09T09:25:06.158019+0000","last_undegraded":"2024-05-09T09:25:06.158019+0000","last_fullsized":"2024-05-09T09:25:06.158019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:09.075572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164381+0000","last_change":"2024-05-09T09:23:09.306302+0000","last_active":"2024-05-09T09:25:06.164381+0000","last_peered":"2024-05-09T09:25:06.164381+0000","last_clean":"2024-05-09T09:25:06.164381+0000","last_became_active":"2024-05-09T09:23:09.306074+0000","last_became_peered":"2024-05-09T09:23:09.306074+0000","last_unstale":"2024-05-09T09:25:06.164381+0000","last_undegraded":"2024-05-09T09:25:06.164381+0000","last_fullsized":"2024-05-09T09:25:06.164381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:18.434899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156080+0000","last_change":"2024-05-09T09:23:10.331201+0000","last_active":"2024-05-09T09:25:06.156080+0000","last_peered":"2024-05-09T09:25:06.156080+0000","last_clean":"2024-05-09T09:25:06.156080+0000","last_became_active":"2024-05-09T09:23:10.329830+0000","last_became_peered":"2024-05-09T09:23:10.329830+0000","last_unstale":"2024-05-09T09:25:06.156080+0000","last_undegraded":"2024-05-09T09:25:06.156080+0000","last_fullsized":"2024-05-09T09:25:06.156080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:40.199622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159955+0000","last_change":"2024-05-09T09:23:14.359095+0000","last_active":"2024-05-09T09:25:06.159955+0000","last_peered":"2024-05-09T09:25:06.159955+0000","last_clean":"2024-05-09T09:25:06.159955+0000","last_became_active":"2024-05-09T09:23:14.358717+0000","last_became_peered":"2024-05-09T09:23:14.358717+0000","last_unstale":"2024-05-09T09:25:06.159955+0000","last_undegraded":"2024-05-09T09:25:06.159955+0000","last_fullsized":"2024-05-09T09:25:06.159955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:16:09.874767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157568+0000","last_change":"2024-05-09T09:23:09.305370+0000","last_active":"2024-05-09T09:25:06.157568+0000","last_peered":"2024-05-09T09:25:06.157568+0000","last_clean":"2024-05-09T09:25:06.157568+0000","last_became_active":"2024-05-09T09:23:09.305054+0000","last_became_peered":"2024-05-09T09:23:09.305054+0000","last_unstale":"2024-05-09T09:25:06.157568+0000","last_undegraded":"2024-05-09T09:25:06.157568+0000","last_fullsized":"2024-05-09T09:25:06.157568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:28.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161625+0000","last_change":"2024-05-09T09:23:10.332541+0000","last_active":"2024-05-09T09:25:06.161625+0000","last_peered":"2024-05-09T09:25:06.161625+0000","last_clean":"2024-05-09T09:25:06.161625+0000","last_became_active":"2024-05-09T09:23:10.332155+0000","last_became_peered":"2024-05-09T09:23:10.332155+0000","last_unstale":"2024-05-09T09:25:06.161625+0000","last_undegraded":"2024-05-09T09:25:06.161625+0000","last_fullsized":"2024-05-09T09:25:06.161625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:39.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159430+0000","last_change":"2024-05-09T09:23:14.374499+0000","last_active":"2024-05-09T09:25:06.159430+0000","last_peered":"2024-05-09T09:25:06.159430+0000","last_clean":"2024-05-09T09:25:06.159430+0000","last_became_active":"2024-05-09T09:23:14.370688+0000","last_became_peered":"2024-05-09T09:23:14.370688+0000","last_unstale":"2024-05-09T09:25:06.159430+0000","last_undegraded":"2024-05-09T09:25:06.159430+0000","last_fullsized":"2024-05-09T09:25:06.159430+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:53.367237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'8","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:28.763416+0000","last_change":"2024-05-09T09:23:09.297776+0000","last_active":"2024-05-09T09:25:28.763416+0000","last_peered":"2024-05-09T09:25:28.763416+0000","last_clean":"2024-05-09T09:25:28.763416+0000","last_became_active":"2024-05-09T09:23:09.297407+0000","last_became_peered":"2024-05-09T09:23:09.297407+0000","last_unstale":"2024-05-09T09:25:28.763416+0000","last_undegraded":"2024-05-09T09:25:28.763416+0000","last_fullsized":"2024-05-09T09:25:28.763416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:21.762941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161199+0000","last_change":"2024-05-09T09:23:10.339127+0000","last_active":"2024-05-09T09:25:06.161199+0000","last_peered":"2024-05-09T09:25:06.161199+0000","last_clean":"2024-05-09T09:25:06.161199+0000","last_became_active":"2024-05-09T09:23:10.338790+0000","last_became_peered":"2024-05-09T09:23:10.338790+0000","last_unstale":"2024-05-09T09:25:06.161199+0000","last_undegraded":"2024-05-09T09:25:06.161199+0000","last_fullsized":"2024-05-09T09:25:06.161199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:36.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156425+0000","last_change":"2024-05-09T09:23:14.373873+0000","last_active":"2024-05-09T09:25:06.156425+0000","last_peered":"2024-05-09T09:25:06.156425+0000","last_clean":"2024-05-09T09:25:06.156425+0000","last_became_active":"2024-05-09T09:23:14.371453+0000","last_became_peered":"2024-05-09T09:23:14.371453+0000","last_unstale":"2024-05-09T09:25:06.156425+0000","last_undegraded":"2024-05-09T09:25:06.156425+0000","last_fullsized":"2024-05-09T09:25:06.156425+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:26.885938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159355+0000","last_change":"2024-05-09T09:23:09.299373+0000","last_active":"2024-05-09T09:25:06.159355+0000","last_peered":"2024-05-09T09:25:06.159355+0000","last_clean":"2024-05-09T09:25:06.159355+0000","last_became_active":"2024-05-09T09:23:09.298926+0000","last_became_peered":"2024-05-09T09:23:09.298926+0000","last_unstale":"2024-05-09T09:25:06.159355+0000","last_undegraded":"2024-05-09T09:25:06.159355+0000","last_fullsized":"2024-05-09T09:25:06.159355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:35.629929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157309+0000","last_change":"2024-05-09T09:23:10.343911+0000","last_active":"2024-05-09T09:25:06.157309+0000","last_peered":"2024-05-09T09:25:06.157309+0000","last_clean":"2024-05-09T09:25:06.157309+0000","last_became_active":"2024-05-09T09:23:10.343568+0000","last_became_peered":"2024-05-09T09:23:10.343568+0000","last_unstale":"2024-05-09T09:25:06.157309+0000","last_undegraded":"2024-05-09T09:25:06.157309+0000","last_fullsized":"2024-05-09T09:25:06.157309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:07.311554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157224+0000","last_change":"2024-05-09T09:23:14.368544+0000","last_active":"2024-05-09T09:25:06.157224+0000","last_peered":"2024-05-09T09:25:06.157224+0000","last_clean":"2024-05-09T09:25:06.157224+0000","last_became_active":"2024-05-09T09:23:14.368140+0000","last_became_peered":"2024-05-09T09:23:14.368140+0000","last_unstale":"2024-05-09T09:25:06.157224+0000","last_undegraded":"2024-05-09T09:25:06.157224+0000","last_fullsized":"2024-05-09T09:25:06.157224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.091768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:27.142002+0000","last_change":"2024-05-09T09:23:09.297612+0000","last_active":"2024-05-09T09:25:27.142002+0000","last_peered":"2024-05-09T09:25:27.142002+0000","last_clean":"2024-05-09T09:25:27.142002+0000","last_became_active":"2024-05-09T09:23:09.297216+0000","last_became_peered":"2024-05-09T09:23:09.297216+0000","last_unstale":"2024-05-09T09:25:27.142002+0000","last_undegraded":"2024-05-09T09:25:27.142002+0000","last_fullsized":"2024-05-09T09:25:27.142002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:09:45.108082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164210+0000","last_change":"2024-05-09T09:25:06.163625+0000","last_active":"2024-05-09T09:25:06.164210+0000","last_peered":"2024-05-09T09:25:06.164210+0000","last_clean":"2024-05-09T09:25:06.164210+0000","last_became_active":"2024-05-09T09:25:06.163198+0000","last_became_peered":"2024-05-09T09:25:06.163198+0000","last_unstale":"2024-05-09T09:25:06.164210+0000","last_undegraded":"2024-05-09T09:25:06.164210+0000","last_fullsized":"2024-05-09T09:25:06.164210+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:35.223030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160161+0000","last_change":"2024-05-09T09:23:14.366119+0000","last_active":"2024-05-09T09:25:06.160161+0000","last_peered":"2024-05-09T09:25:06.160161+0000","last_clean":"2024-05-09T09:25:06.160161+0000","last_became_active":"2024-05-09T09:23:14.365659+0000","last_became_peered":"2024-05-09T09:23:14.365659+0000","last_unstale":"2024-05-09T09:25:06.160161+0000","last_undegraded":"2024-05-09T09:25:06.160161+0000","last_fullsized":"2024-05-09T09:25:06.160161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:23.312632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159327+0000","last_change":"2024-05-09T09:24:05.740866+0000","last_active":"2024-05-09T09:25:06.159327+0000","last_peered":"2024-05-09T09:25:06.159327+0000","last_clean":"2024-05-09T09:25:06.159327+0000","last_became_active":"2024-05-09T09:24:05.740418+0000","last_became_peered":"2024-05-09T09:24:05.740418+0000","last_unstale":"2024-05-09T09:25:06.159327+0000","last_undegraded":"2024-05-09T09:25:06.159327+0000","last_fullsized":"2024-05-09T09:25:06.159327+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:45.946040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257359+0000","last_change":"2024-05-09T09:23:10.339788+0000","last_active":"2024-05-09T09:25:06.257359+0000","last_peered":"2024-05-09T09:25:06.257359+0000","last_clean":"2024-05-09T09:25:06.257359+0000","last_became_active":"2024-05-09T09:23:10.339430+0000","last_became_peered":"2024-05-09T09:23:10.339430+0000","last_unstale":"2024-05-09T09:25:06.257359+0000","last_undegraded":"2024-05-09T09:25:06.257359+0000","last_fullsized":"2024-05-09T09:25:06.257359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:22.467287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156237+0000","last_change":"2024-05-09T09:23:14.368710+0000","last_active":"2024-05-09T09:25:06.156237+0000","last_peered":"2024-05-09T09:25:06.156237+0000","last_clean":"2024-05-09T09:25:06.156237+0000","last_became_active":"2024-05-09T09:23:14.368288+0000","last_became_peered":"2024-05-09T09:23:14.368288+0000","last_unstale":"2024-05-09T09:25:06.156237+0000","last_undegraded":"2024-05-09T09:25:06.156237+0000","last_fullsized":"2024-05-09T09:25:06.156237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:40.941273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":156,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:29.758648+0000","last_change":"2024-05-09T09:23:09.301976+0000","last_active":"2024-05-09T09:25:29.758648+0000","last_peered":"2024-05-09T09:25:29.758648+0000","last_clean":"2024-05-09T09:25:29.758648+0000","last_became_active":"2024-05-09T09:23:09.301494+0000","last_became_peered":"2024-05-09T09:23:09.301494+0000","last_unstale":"2024-05-09T09:25:29.758648+0000","last_undegraded":"2024-05-09T09:25:29.758648+0000","last_fullsized":"2024-05-09T09:25:29.758648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.448246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256796+0000","last_change":"2024-05-09T09:23:10.331691+0000","last_active":"2024-05-09T09:25:06.256796+0000","last_peered":"2024-05-09T09:25:06.256796+0000","last_clean":"2024-05-09T09:25:06.256796+0000","last_became_active":"2024-05-09T09:23:10.331332+0000","last_became_peered":"2024-05-09T09:23:10.331332+0000","last_unstale":"2024-05-09T09:25:06.256796+0000","last_undegraded":"2024-05-09T09:25:06.256796+0000","last_fullsized":"2024-05-09T09:25:06.256796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:51.643695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156538+0000","last_change":"2024-05-09T09:23:14.367539+0000","last_active":"2024-05-09T09:25:06.156538+0000","last_peered":"2024-05-09T09:25:06.156538+0000","last_clean":"2024-05-09T09:25:06.156538+0000","last_became_active":"2024-05-09T09:23:14.366806+0000","last_became_peered":"2024-05-09T09:23:14.366806+0000","last_unstale":"2024-05-09T09:25:06.156538+0000","last_undegraded":"2024-05-09T09:25:06.156538+0000","last_fullsized":"2024-05-09T09:25:06.156538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:58.679095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157306+0000","last_change":"2024-05-09T09:23:10.340849+0000","last_active":"2024-05-09T09:25:06.157306+0000","last_peered":"2024-05-09T09:25:06.157306+0000","last_clean":"2024-05-09T09:25:06.157306+0000","last_became_active":"2024-05-09T09:23:10.340430+0000","last_became_peered":"2024-05-09T09:23:10.340430+0000","last_unstale":"2024-05-09T09:25:06.157306+0000","last_undegraded":"2024-05-09T09:25:06.157306+0000","last_fullsized":"2024-05-09T09:25:06.157306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:33:47.827659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'7","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:29.758396+0000","last_change":"2024-05-09T09:23:09.308544+0000","last_active":"2024-05-09T09:25:29.758396+0000","last_peered":"2024-05-09T09:25:29.758396+0000","last_clean":"2024-05-09T09:25:29.758396+0000","last_became_active":"2024-05-09T09:23:09.308195+0000","last_became_peered":"2024-05-09T09:23:09.308195+0000","last_unstale":"2024-05-09T09:25:29.758396+0000","last_undegraded":"2024-05-09T09:25:29.758396+0000","last_fullsized":"2024-05-09T09:25:29.758396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:17:34.884385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161350+0000","last_change":"2024-05-09T09:24:05.755846+0000","last_active":"2024-05-09T09:25:06.161350+0000","last_peered":"2024-05-09T09:25:06.161350+0000","last_clean":"2024-05-09T09:25:06.161350+0000","last_became_active":"2024-05-09T09:24:05.747246+0000","last_became_peered":"2024-05-09T09:24:05.747246+0000","last_unstale":"2024-05-09T09:25:06.161350+0000","last_undegraded":"2024-05-09T09:25:06.161350+0000","last_fullsized":"2024-05-09T09:25:06.161350+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:23:40.544770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161873+0000","last_change":"2024-05-09T09:23:10.341320+0000","last_active":"2024-05-09T09:25:06.161873+0000","last_peered":"2024-05-09T09:25:06.161873+0000","last_clean":"2024-05-09T09:25:06.161873+0000","last_became_active":"2024-05-09T09:23:10.340868+0000","last_became_peered":"2024-05-09T09:23:10.340868+0000","last_unstale":"2024-05-09T09:25:06.161873+0000","last_undegraded":"2024-05-09T09:25:06.161873+0000","last_fullsized":"2024-05-09T09:25:06.161873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:55.846642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157331+0000","last_change":"2024-05-09T09:23:09.299574+0000","last_active":"2024-05-09T09:25:06.157331+0000","last_peered":"2024-05-09T09:25:06.157331+0000","last_clean":"2024-05-09T09:25:06.157331+0000","last_became_active":"2024-05-09T09:23:09.298878+0000","last_became_peered":"2024-05-09T09:23:09.298878+0000","last_unstale":"2024-05-09T09:25:06.157331+0000","last_undegraded":"2024-05-09T09:25:06.157331+0000","last_fullsized":"2024-05-09T09:25:06.157331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.275907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":352,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160988+0000","last_change":"2024-05-09T09:21:57.576132+0000","last_active":"2024-05-09T09:25:06.160988+0000","last_peered":"2024-05-09T09:25:06.160988+0000","last_clean":"2024-05-09T09:25:06.160988+0000","last_became_active":"2024-05-09T09:21:57.575602+0000","last_became_peered":"2024-05-09T09:21:57.575602+0000","last_unstale":"2024-05-09T09:25:06.160988+0000","last_undegraded":"2024-05-09T09:25:06.160988+0000","last_fullsized":"2024-05-09T09:25:06.160988+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_clean_scrub_stamp":"2024-05-09T09:18:31.335063+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:07.639787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157785+0000","last_change":"2024-05-09T09:23:14.371184+0000","last_active":"2024-05-09T09:25:06.157785+0000","last_peered":"2024-05-09T09:25:06.157785+0000","last_clean":"2024-05-09T09:25:06.157785+0000","last_became_active":"2024-05-09T09:23:14.370780+0000","last_became_peered":"2024-05-09T09:23:14.370780+0000","last_unstale":"2024-05-09T09:25:06.157785+0000","last_undegraded":"2024-05-09T09:25:06.157785+0000","last_fullsized":"2024-05-09T09:25:06.157785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:39.361947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.155976+0000","last_change":"2024-05-09T09:23:10.330915+0000","last_active":"2024-05-09T09:25:06.155976+0000","last_peered":"2024-05-09T09:25:06.155976+0000","last_clean":"2024-05-09T09:25:06.155976+0000","last_became_active":"2024-05-09T09:23:10.330458+0000","last_became_peered":"2024-05-09T09:23:10.330458+0000","last_unstale":"2024-05-09T09:25:06.155976+0000","last_undegraded":"2024-05-09T09:25:06.155976+0000","last_fullsized":"2024-05-09T09:25:06.155976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:10.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161228+0000","last_change":"2024-05-09T09:23:09.303895+0000","last_active":"2024-05-09T09:25:06.161228+0000","last_peered":"2024-05-09T09:25:06.161228+0000","last_clean":"2024-05-09T09:25:06.161228+0000","last_became_active":"2024-05-09T09:23:09.303555+0000","last_became_peered":"2024-05-09T09:23:09.303555+0000","last_unstale":"2024-05-09T09:25:06.161228+0000","last_undegraded":"2024-05-09T09:25:06.161228+0000","last_fullsized":"2024-05-09T09:25:06.161228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:04.350161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159708+0000","last_change":"2024-05-09T09:23:14.359390+0000","last_active":"2024-05-09T09:25:06.159708+0000","last_peered":"2024-05-09T09:25:06.159708+0000","last_clean":"2024-05-09T09:25:06.159708+0000","last_became_active":"2024-05-09T09:23:14.359000+0000","last_became_peered":"2024-05-09T09:23:14.359000+0000","last_unstale":"2024-05-09T09:25:06.159708+0000","last_undegraded":"2024-05-09T09:25:06.159708+0000","last_fullsized":"2024-05-09T09:25:06.159708+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:25.077835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160314+0000","last_change":"2024-05-09T09:23:09.304535+0000","last_active":"2024-05-09T09:25:06.160314+0000","last_peered":"2024-05-09T09:25:06.160314+0000","last_clean":"2024-05-09T09:25:06.160314+0000","last_became_active":"2024-05-09T09:23:09.304166+0000","last_became_peered":"2024-05-09T09:23:09.304166+0000","last_unstale":"2024-05-09T09:25:06.160314+0000","last_undegraded":"2024-05-09T09:25:06.160314+0000","last_fullsized":"2024-05-09T09:25:06.160314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:03:20.281620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156507+0000","last_change":"2024-05-09T09:23:10.335657+0000","last_active":"2024-05-09T09:25:06.156507+0000","last_peered":"2024-05-09T09:25:06.156507+0000","last_clean":"2024-05-09T09:25:06.156507+0000","last_became_active":"2024-05-09T09:23:10.335231+0000","last_became_peered":"2024-05-09T09:23:10.335231+0000","last_unstale":"2024-05-09T09:25:06.156507+0000","last_undegraded":"2024-05-09T09:25:06.156507+0000","last_fullsized":"2024-05-09T09:25:06.156507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:07.990165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161193+0000","last_change":"2024-05-09T09:23:14.366988+0000","last_active":"2024-05-09T09:25:06.161193+0000","last_peered":"2024-05-09T09:25:06.161193+0000","last_clean":"2024-05-09T09:25:06.161193+0000","last_became_active":"2024-05-09T09:23:14.366562+0000","last_became_peered":"2024-05-09T09:23:14.366562+0000","last_unstale":"2024-05-09T09:25:06.161193+0000","last_undegraded":"2024-05-09T09:25:06.161193+0000","last_fullsized":"2024-05-09T09:25:06.161193+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:42.670036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'15","reported_seq":62,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:18.524078+0000","last_change":"2024-05-09T09:24:05.752748+0000","last_active":"2024-05-09T09:25:18.524078+0000","last_peered":"2024-05-09T09:25:18.524078+0000","last_clean":"2024-05-09T09:25:18.524078+0000","last_became_active":"2024-05-09T09:24:05.743638+0000","last_became_peered":"2024-05-09T09:24:05.743638+0000","last_unstale":"2024-05-09T09:25:18.524078+0000","last_undegraded":"2024-05-09T09:25:18.524078+0000","last_fullsized":"2024-05-09T09:25:18.524078+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:01.834934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161808+0000","last_change":"2024-05-09T09:23:10.345840+0000","last_active":"2024-05-09T09:25:06.161808+0000","last_peered":"2024-05-09T09:25:06.161808+0000","last_clean":"2024-05-09T09:25:06.161808+0000","last_became_active":"2024-05-09T09:23:10.345487+0000","last_became_peered":"2024-05-09T09:23:10.345487+0000","last_unstale":"2024-05-09T09:25:06.161808+0000","last_undegraded":"2024-05-09T09:25:06.161808+0000","last_fullsized":"2024-05-09T09:25:06.161808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:47.723429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162727+0000","last_change":"2024-05-09T09:23:14.365485+0000","last_active":"2024-05-09T09:25:06.162727+0000","last_peered":"2024-05-09T09:25:06.162727+0000","last_clean":"2024-05-09T09:25:06.162727+0000","last_became_active":"2024-05-09T09:23:14.365018+0000","last_became_peered":"2024-05-09T09:23:14.365018+0000","last_unstale":"2024-05-09T09:25:06.162727+0000","last_undegraded":"2024-05-09T09:25:06.162727+0000","last_fullsized":"2024-05-09T09:25:06.162727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:54:04.712920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157667+0000","last_change":"2024-05-09T09:23:09.302365+0000","last_active":"2024-05-09T09:25:06.157667+0000","last_peered":"2024-05-09T09:25:06.157667+0000","last_clean":"2024-05-09T09:25:06.157667+0000","last_became_active":"2024-05-09T09:23:09.302008+0000","last_became_peered":"2024-05-09T09:23:09.302008+0000","last_unstale":"2024-05-09T09:25:06.157667+0000","last_undegraded":"2024-05-09T09:25:06.157667+0000","last_fullsized":"2024-05-09T09:25:06.157667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:48.829940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165695+0000","last_change":"2024-05-09T09:23:10.339170+0000","last_active":"2024-05-09T09:25:06.165695+0000","last_peered":"2024-05-09T09:25:06.165695+0000","last_clean":"2024-05-09T09:25:06.165695+0000","last_became_active":"2024-05-09T09:23:10.338829+0000","last_became_peered":"2024-05-09T09:23:10.338829+0000","last_unstale":"2024-05-09T09:25:06.165695+0000","last_undegraded":"2024-05-09T09:25:06.165695+0000","last_fullsized":"2024-05-09T09:25:06.165695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:04.510495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157578+0000","last_change":"2024-05-09T09:24:05.747227+0000","last_active":"2024-05-09T09:25:06.157578+0000","last_peered":"2024-05-09T09:25:06.157578+0000","last_clean":"2024-05-09T09:25:06.157578+0000","last_became_active":"2024-05-09T09:24:05.746338+0000","last_became_peered":"2024-05-09T09:24:05.746338+0000","last_unstale":"2024-05-09T09:25:06.157578+0000","last_undegraded":"2024-05-09T09:25:06.157578+0000","last_fullsized":"2024-05-09T09:25:06.157578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:20.899664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256404+0000","last_change":"2024-05-09T09:23:09.297418+0000","last_active":"2024-05-09T09:25:06.256404+0000","last_peered":"2024-05-09T09:25:06.256404+0000","last_clean":"2024-05-09T09:25:06.256404+0000","last_became_active":"2024-05-09T09:23:09.296963+0000","last_became_peered":"2024-05-09T09:23:09.296963+0000","last_unstale":"2024-05-09T09:25:06.256404+0000","last_undegraded":"2024-05-09T09:25:06.256404+0000","last_fullsized":"2024-05-09T09:25:06.256404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:54.552857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159481+0000","last_change":"2024-05-09T09:23:10.325882+0000","last_active":"2024-05-09T09:25:06.159481+0000","last_peered":"2024-05-09T09:25:06.159481+0000","last_clean":"2024-05-09T09:25:06.159481+0000","last_became_active":"2024-05-09T09:23:10.325494+0000","last_became_peered":"2024-05-09T09:23:10.325494+0000","last_unstale":"2024-05-09T09:25:06.159481+0000","last_undegraded":"2024-05-09T09:25:06.159481+0000","last_fullsized":"2024-05-09T09:25:06.159481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:27.191310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159617+0000","last_change":"2024-05-09T09:23:14.375572+0000","last_active":"2024-05-09T09:25:06.159617+0000","last_peered":"2024-05-09T09:25:06.159617+0000","last_clean":"2024-05-09T09:25:06.159617+0000","last_became_active":"2024-05-09T09:23:14.374069+0000","last_became_peered":"2024-05-09T09:23:14.374069+0000","last_unstale":"2024-05-09T09:25:06.159617+0000","last_undegraded":"2024-05-09T09:25:06.159617+0000","last_fullsized":"2024-05-09T09:25:06.159617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:44.045012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161469+0000","last_change":"2024-05-09T09:23:09.304137+0000","last_active":"2024-05-09T09:25:06.161469+0000","last_peered":"2024-05-09T09:25:06.161469+0000","last_clean":"2024-05-09T09:25:06.161469+0000","last_became_active":"2024-05-09T09:23:09.303762+0000","last_became_peered":"2024-05-09T09:23:09.303762+0000","last_unstale":"2024-05-09T09:25:06.161469+0000","last_undegraded":"2024-05-09T09:25:06.161469+0000","last_fullsized":"2024-05-09T09:25:06.161469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:56.811194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159736+0000","last_change":"2024-05-09T09:23:10.331998+0000","last_active":"2024-05-09T09:25:06.159736+0000","last_peered":"2024-05-09T09:25:06.159736+0000","last_clean":"2024-05-09T09:25:06.159736+0000","last_became_active":"2024-05-09T09:23:10.331642+0000","last_became_peered":"2024-05-09T09:23:10.331642+0000","last_unstale":"2024-05-09T09:25:06.159736+0000","last_undegraded":"2024-05-09T09:25:06.159736+0000","last_fullsized":"2024-05-09T09:25:06.159736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:41.569300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157348+0000","last_change":"2024-05-09T09:23:14.377444+0000","last_active":"2024-05-09T09:25:06.157348+0000","last_peered":"2024-05-09T09:25:06.157348+0000","last_clean":"2024-05-09T09:25:06.157348+0000","last_became_active":"2024-05-09T09:23:14.376189+0000","last_became_peered":"2024-05-09T09:23:14.376189+0000","last_unstale":"2024-05-09T09:25:06.157348+0000","last_undegraded":"2024-05-09T09:25:06.157348+0000","last_fullsized":"2024-05-09T09:25:06.157348+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:06.973535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:19.360581+0000","last_change":"2024-05-09T09:23:09.304101+0000","last_active":"2024-05-09T09:25:19.360581+0000","last_peered":"2024-05-09T09:25:19.360581+0000","last_clean":"2024-05-09T09:25:19.360581+0000","last_became_active":"2024-05-09T09:23:09.303743+0000","last_became_peered":"2024-05-09T09:23:09.303743+0000","last_unstale":"2024-05-09T09:25:19.360581+0000","last_undegraded":"2024-05-09T09:25:19.360581+0000","last_fullsized":"2024-05-09T09:25:19.360581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:39.339815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162614+0000","last_change":"2024-05-09T09:23:10.345938+0000","last_active":"2024-05-09T09:25:06.162614+0000","last_peered":"2024-05-09T09:25:06.162614+0000","last_clean":"2024-05-09T09:25:06.162614+0000","last_became_active":"2024-05-09T09:23:10.345576+0000","last_became_peered":"2024-05-09T09:23:10.345576+0000","last_unstale":"2024-05-09T09:25:06.162614+0000","last_undegraded":"2024-05-09T09:25:06.162614+0000","last_fullsized":"2024-05-09T09:25:06.162614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:08.571900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157437+0000","last_change":"2024-05-09T09:23:14.371549+0000","last_active":"2024-05-09T09:25:06.157437+0000","last_peered":"2024-05-09T09:25:06.157437+0000","last_clean":"2024-05-09T09:25:06.157437+0000","last_became_active":"2024-05-09T09:23:14.369931+0000","last_became_peered":"2024-05-09T09:23:14.369931+0000","last_unstale":"2024-05-09T09:25:06.157437+0000","last_undegraded":"2024-05-09T09:25:06.157437+0000","last_fullsized":"2024-05-09T09:25:06.157437+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:20.777172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159654+0000","last_change":"2024-05-09T09:23:09.302909+0000","last_active":"2024-05-09T09:25:06.159654+0000","last_peered":"2024-05-09T09:25:06.159654+0000","last_clean":"2024-05-09T09:25:06.159654+0000","last_became_active":"2024-05-09T09:23:09.302570+0000","last_became_peered":"2024-05-09T09:23:09.302570+0000","last_unstale":"2024-05-09T09:25:06.159654+0000","last_undegraded":"2024-05-09T09:25:06.159654+0000","last_fullsized":"2024-05-09T09:25:06.159654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:23.856755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161777+0000","last_change":"2024-05-09T09:23:10.342676+0000","last_active":"2024-05-09T09:25:06.161777+0000","last_peered":"2024-05-09T09:25:06.161777+0000","last_clean":"2024-05-09T09:25:06.161777+0000","last_became_active":"2024-05-09T09:23:10.341906+0000","last_became_peered":"2024-05-09T09:23:10.341906+0000","last_unstale":"2024-05-09T09:25:06.161777+0000","last_undegraded":"2024-05-09T09:25:06.161777+0000","last_fullsized":"2024-05-09T09:25:06.161777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:59:11.137648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256737+0000","last_change":"2024-05-09T09:23:14.363779+0000","last_active":"2024-05-09T09:25:06.256737+0000","last_peered":"2024-05-09T09:25:06.256737+0000","last_clean":"2024-05-09T09:25:06.256737+0000","last_became_active":"2024-05-09T09:23:14.363307+0000","last_became_peered":"2024-05-09T09:23:14.363307+0000","last_unstale":"2024-05-09T09:25:06.256737+0000","last_undegraded":"2024-05-09T09:25:06.256737+0000","last_fullsized":"2024-05-09T09:25:06.256737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:46.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"93'6","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:32.763660+0000","last_change":"2024-05-09T09:23:09.298094+0000","last_active":"2024-05-09T09:25:32.763660+0000","last_peered":"2024-05-09T09:25:32.763660+0000","last_clean":"2024-05-09T09:25:32.763660+0000","last_became_active":"2024-05-09T09:23:09.297777+0000","last_became_peered":"2024-05-09T09:23:09.297777+0000","last_unstale":"2024-05-09T09:25:32.763660+0000","last_undegraded":"2024-05-09T09:25:32.763660+0000","last_fullsized":"2024-05-09T09:25:32.763660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:25.099193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161899+0000","last_change":"2024-05-09T09:23:10.340120+0000","last_active":"2024-05-09T09:25:06.161899+0000","last_peered":"2024-05-09T09:25:06.161899+0000","last_clean":"2024-05-09T09:25:06.161899+0000","last_became_active":"2024-05-09T09:23:10.339690+0000","last_became_peered":"2024-05-09T09:23:10.339690+0000","last_unstale":"2024-05-09T09:25:06.161899+0000","last_undegraded":"2024-05-09T09:25:06.161899+0000","last_fullsized":"2024-05-09T09:25:06.161899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:36.783411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165251+0000","last_change":"2024-05-09T09:24:05.744892+0000","last_active":"2024-05-09T09:25:06.165251+0000","last_peered":"2024-05-09T09:25:06.165251+0000","last_clean":"2024-05-09T09:25:06.165251+0000","last_became_active":"2024-05-09T09:24:05.744437+0000","last_became_peered":"2024-05-09T09:24:05.744437+0000","last_unstale":"2024-05-09T09:25:06.165251+0000","last_undegraded":"2024-05-09T09:25:06.165251+0000","last_fullsized":"2024-05-09T09:25:06.165251+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:38:33.709821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157886+0000","last_change":"2024-05-09T09:23:09.307592+0000","last_active":"2024-05-09T09:25:06.157886+0000","last_peered":"2024-05-09T09:25:06.157886+0000","last_clean":"2024-05-09T09:25:06.157886+0000","last_became_active":"2024-05-09T09:23:09.307341+0000","last_became_peered":"2024-05-09T09:23:09.307341+0000","last_unstale":"2024-05-09T09:25:06.157886+0000","last_undegraded":"2024-05-09T09:25:06.157886+0000","last_fullsized":"2024-05-09T09:25:06.157886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:21.790980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161686+0000","last_change":"2024-05-09T09:23:10.349217+0000","last_active":"2024-05-09T09:25:06.161686+0000","last_peered":"2024-05-09T09:25:06.161686+0000","last_clean":"2024-05-09T09:25:06.161686+0000","last_became_active":"2024-05-09T09:23:10.348861+0000","last_became_peered":"2024-05-09T09:23:10.348861+0000","last_unstale":"2024-05-09T09:25:06.161686+0000","last_undegraded":"2024-05-09T09:25:06.161686+0000","last_fullsized":"2024-05-09T09:25:06.161686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:51.621045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157038+0000","last_change":"2024-05-09T09:23:14.363773+0000","last_active":"2024-05-09T09:25:06.157038+0000","last_peered":"2024-05-09T09:25:06.157038+0000","last_clean":"2024-05-09T09:25:06.157038+0000","last_became_active":"2024-05-09T09:23:14.363393+0000","last_became_peered":"2024-05-09T09:23:14.363393+0000","last_unstale":"2024-05-09T09:25:06.157038+0000","last_undegraded":"2024-05-09T09:25:06.157038+0000","last_fullsized":"2024-05-09T09:25:06.157038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:57.322985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156417+0000","last_change":"2024-05-09T09:23:09.308723+0000","last_active":"2024-05-09T09:25:06.156417+0000","last_peered":"2024-05-09T09:25:06.156417+0000","last_clean":"2024-05-09T09:25:06.156417+0000","last_became_active":"2024-05-09T09:23:09.308267+0000","last_became_peered":"2024-05-09T09:23:09.308267+0000","last_unstale":"2024-05-09T09:25:06.156417+0000","last_undegraded":"2024-05-09T09:25:06.156417+0000","last_fullsized":"2024-05-09T09:25:06.156417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:09:25.036421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162140+0000","last_change":"2024-05-09T09:23:10.342246+0000","last_active":"2024-05-09T09:25:06.162140+0000","last_peered":"2024-05-09T09:25:06.162140+0000","last_clean":"2024-05-09T09:25:06.162140+0000","last_became_active":"2024-05-09T09:23:10.341927+0000","last_became_peered":"2024-05-09T09:23:10.341927+0000","last_unstale":"2024-05-09T09:25:06.162140+0000","last_undegraded":"2024-05-09T09:25:06.162140+0000","last_fullsized":"2024-05-09T09:25:06.162140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:01.182226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159866+0000","last_change":"2024-05-09T09:23:10.332768+0000","last_active":"2024-05-09T09:25:06.159866+0000","last_peered":"2024-05-09T09:25:06.159866+0000","last_clean":"2024-05-09T09:25:06.159866+0000","last_became_active":"2024-05-09T09:23:10.332441+0000","last_became_peered":"2024-05-09T09:23:10.332441+0000","last_unstale":"2024-05-09T09:25:06.159866+0000","last_undegraded":"2024-05-09T09:25:06.159866+0000","last_fullsized":"2024-05-09T09:25:06.159866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:53.377058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164248+0000","last_change":"2024-05-09T09:23:10.337944+0000","last_active":"2024-05-09T09:25:06.164248+0000","last_peered":"2024-05-09T09:25:06.164248+0000","last_clean":"2024-05-09T09:25:06.164248+0000","last_became_active":"2024-05-09T09:23:10.337593+0000","last_became_peered":"2024-05-09T09:23:10.337593+0000","last_unstale":"2024-05-09T09:25:06.164248+0000","last_undegraded":"2024-05-09T09:25:06.164248+0000","last_fullsized":"2024-05-09T09:25:06.164248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:20.381543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.167531+0000","last_change":"2024-05-09T09:25:06.167114+0000","last_active":"2024-05-09T09:25:06.167531+0000","last_peered":"2024-05-09T09:25:06.167531+0000","last_clean":"2024-05-09T09:25:06.167531+0000","last_became_active":"2024-05-09T09:25:06.166712+0000","last_became_peered":"2024-05-09T09:25:06.166712+0000","last_unstale":"2024-05-09T09:25:06.167531+0000","last_undegraded":"2024-05-09T09:25:06.167531+0000","last_fullsized":"2024-05-09T09:25:06.167531+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:00.253528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165435+0000","last_change":"2024-05-09T09:23:10.340243+0000","last_active":"2024-05-09T09:25:06.165435+0000","last_peered":"2024-05-09T09:25:06.165435+0000","last_clean":"2024-05-09T09:25:06.165435+0000","last_became_active":"2024-05-09T09:23:10.339919+0000","last_became_peered":"2024-05-09T09:23:10.339919+0000","last_unstale":"2024-05-09T09:25:06.165435+0000","last_undegraded":"2024-05-09T09:25:06.165435+0000","last_fullsized":"2024-05-09T09:25:06.165435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:21.460928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156424+0000","last_change":"2024-05-09T09:23:10.333136+0000","last_active":"2024-05-09T09:25:06.156424+0000","last_peered":"2024-05-09T09:25:06.156424+0000","last_clean":"2024-05-09T09:25:06.156424+0000","last_became_active":"2024-05-09T09:23:10.332799+0000","last_became_peered":"2024-05-09T09:23:10.332799+0000","last_unstale":"2024-05-09T09:25:06.156424+0000","last_undegraded":"2024-05-09T09:25:06.156424+0000","last_fullsized":"2024-05-09T09:25:06.156424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:50.682747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160593+0000","last_change":"2024-05-09T09:24:05.745143+0000","last_active":"2024-05-09T09:25:06.160593+0000","last_peered":"2024-05-09T09:25:06.160593+0000","last_clean":"2024-05-09T09:25:06.160593+0000","last_became_active":"2024-05-09T09:24:05.744117+0000","last_became_peered":"2024-05-09T09:24:05.744117+0000","last_unstale":"2024-05-09T09:25:06.160593+0000","last_undegraded":"2024-05-09T09:25:06.160593+0000","last_fullsized":"2024-05-09T09:25:06.160593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:25:09.853820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164776+0000","last_change":"2024-05-09T09:23:10.341776+0000","last_active":"2024-05-09T09:25:06.164776+0000","last_peered":"2024-05-09T09:25:06.164776+0000","last_clean":"2024-05-09T09:25:06.164776+0000","last_became_active":"2024-05-09T09:23:10.341436+0000","last_became_peered":"2024-05-09T09:23:10.341436+0000","last_unstale":"2024-05-09T09:25:06.164776+0000","last_undegraded":"2024-05-09T09:25:06.164776+0000","last_fullsized":"2024-05-09T09:25:06.164776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:42.178856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161898+0000","last_change":"2024-05-09T09:24:05.745324+0000","last_active":"2024-05-09T09:25:06.161898+0000","last_peered":"2024-05-09T09:25:06.161898+0000","last_clean":"2024-05-09T09:25:06.161898+0000","last_became_active":"2024-05-09T09:24:05.744899+0000","last_became_peered":"2024-05-09T09:24:05.744899+0000","last_unstale":"2024-05-09T09:25:06.161898+0000","last_undegraded":"2024-05-09T09:25:06.161898+0000","last_fullsized":"2024-05-09T09:25:06.161898+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:59.372367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157167+0000","last_change":"2024-05-09T09:23:10.337117+0000","last_active":"2024-05-09T09:25:06.157167+0000","last_peered":"2024-05-09T09:25:06.157167+0000","last_clean":"2024-05-09T09:25:06.157167+0000","last_became_active":"2024-05-09T09:23:10.336672+0000","last_became_peered":"2024-05-09T09:23:10.336672+0000","last_unstale":"2024-05-09T09:25:06.157167+0000","last_undegraded":"2024-05-09T09:25:06.157167+0000","last_fullsized":"2024-05-09T09:25:06.157167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:59:20.723742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161586+0000","last_change":"2024-05-09T09:23:10.337733+0000","last_active":"2024-05-09T09:25:06.161586+0000","last_peered":"2024-05-09T09:25:06.161586+0000","last_clean":"2024-05-09T09:25:06.161586+0000","last_became_active":"2024-05-09T09:23:10.337343+0000","last_became_peered":"2024-05-09T09:23:10.337343+0000","last_unstale":"2024-05-09T09:25:06.161586+0000","last_undegraded":"2024-05-09T09:25:06.161586+0000","last_fullsized":"2024-05-09T09:25:06.161586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:53.447911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164449+0000","last_change":"2024-05-09T09:25:06.163856+0000","last_active":"2024-05-09T09:25:06.164449+0000","last_peered":"2024-05-09T09:25:06.164449+0000","last_clean":"2024-05-09T09:25:06.164449+0000","last_became_active":"2024-05-09T09:25:06.163367+0000","last_became_peered":"2024-05-09T09:25:06.163367+0000","last_unstale":"2024-05-09T09:25:06.164449+0000","last_undegraded":"2024-05-09T09:25:06.164449+0000","last_fullsized":"2024-05-09T09:25:06.164449+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:59.920586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,3],"acting":[9,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156375+0000","last_change":"2024-05-09T09:23:10.332271+0000","last_active":"2024-05-09T09:25:06.156375+0000","last_peered":"2024-05-09T09:25:06.156375+0000","last_clean":"2024-05-09T09:25:06.156375+0000","last_became_active":"2024-05-09T09:23:10.331941+0000","last_became_peered":"2024-05-09T09:23:10.331941+0000","last_unstale":"2024-05-09T09:25:06.156375+0000","last_undegraded":"2024-05-09T09:25:06.156375+0000","last_fullsized":"2024-05-09T09:25:06.156375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:27.707554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160211+0000","last_change":"2024-05-09T09:23:10.328965+0000","last_active":"2024-05-09T09:25:06.160211+0000","last_peered":"2024-05-09T09:25:06.160211+0000","last_clean":"2024-05-09T09:25:06.160211+0000","last_became_active":"2024-05-09T09:23:10.328618+0000","last_became_peered":"2024-05-09T09:23:10.328618+0000","last_unstale":"2024-05-09T09:25:06.160211+0000","last_undegraded":"2024-05-09T09:25:06.160211+0000","last_fullsized":"2024-05-09T09:25:06.160211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:16:02.003378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73024,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1239,"num_read_kb":701,"num_write":265,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":219072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":520504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28740,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716412,"statfs":{"total":95995035648,"available":95965605888,"internally_reserved":0,"allocated":1765376,"data_stored":1176046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":519941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":520319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1683456,"data_stored":1118436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":515938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":581066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":517394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40624,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704528,"statfs":{"total":95995035648,"available":95953436672,"internally_reserved":0,"allocated":1155072,"data_stored":581779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":515153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":520544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1112500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:25:36.510 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T09:25:36.511 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:36 smithi016 ceph-mon[33335]: pgmap v368: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-05-09T09:25:36.511 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:36 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3781049743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:36.511 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:36 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1501593942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:36.511 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:36 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2982099792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:36.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.649+0000 7f2f0b961640 1 -- 172.21.15.12:0/3005183148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f04101270 msgr2=0x7f2f04101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:36.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.649+0000 7f2f0b961640 1 --2- 172.21.15.12:0/3005183148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f04101270 0x7f2f04101af0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009ca0 tx=0x7f2efc02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.649+0000 7f2f0b961640 1 -- 172.21.15.12:0/3005183148 shutdown_connections 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.649+0000 7f2f0b961640 1 --2- 172.21.15.12:0/3005183148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f04101270 0x7f2f04101af0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.649+0000 7f2f0b961640 1 --2- 172.21.15.12:0/3005183148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f041008b0 0x7f2f04100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.649+0000 7f2f0b961640 1 --2- 172.21.15.12:0/3005183148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f040ffb50 0x7f2f040fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.649+0000 7f2f0b961640 1 -- 172.21.15.12:0/3005183148 >> 172.21.15.12:0/3005183148 conn(0x7f2f040fb4a0 msgr2=0x7f2f040fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.649+0000 7f2f0b961640 1 -- 172.21.15.12:0/3005183148 shutdown_connections 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.649+0000 7f2f0b961640 1 -- 172.21.15.12:0/3005183148 wait complete. 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.650+0000 7f2f0b961640 1 Processor -- start 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.650+0000 7f2f0b961640 1 -- start start 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.650+0000 7f2f0b961640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f040ffb50 0x7f2f04199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.650+0000 7f2f0b961640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f041008b0 0x7f2f0419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.650+0000 7f2f0b961640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f04101270 0x7f2f041a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.650+0000 7f2f0b961640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0410f740 con 0x7f2f040ffb50 2024-05-09T09:25:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.650+0000 7f2f0b961640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0410f5c0 con 0x7f2f041008b0 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.650+0000 7f2f0b961640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0410f8c0 con 0x7f2f04101270 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.650+0000 7f2f096fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f040ffb50 0x7f2f04199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.650+0000 7f2f096fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f040ffb50 0x7f2f04199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33978/0 (socket says 172.21.15.12:33978) 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.650+0000 7f2f096fd640 1 -- 172.21.15.12:0/1524582542 learned_addr learned my addr 172.21.15.12:0/1524582542 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.650+0000 7f2f08efc640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f041008b0 0x7f2f0419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.651+0000 7f2f09efe640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f04101270 0x7f2f041a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.651+0000 7f2f096fd640 1 -- 172.21.15.12:0/1524582542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f04101270 msgr2=0x7f2f041a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.651+0000 7f2f096fd640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f04101270 0x7f2f041a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.651+0000 7f2f096fd640 1 -- 172.21.15.12:0/1524582542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f041008b0 msgr2=0x7f2f0419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.651+0000 7f2f096fd640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f041008b0 0x7f2f0419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.651+0000 7f2f096fd640 1 -- 172.21.15.12:0/1524582542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f041a1980 con 0x7f2f040ffb50 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.651+0000 7f2f096fd640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f040ffb50 0x7f2f04199d60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000e9f0 tx=0x7f2f0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.651+0000 7f2efa7fc640 1 -- 172.21.15.12:0/1524582542 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000cd80 con 0x7f2f040ffb50 2024-05-09T09:25:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.651+0000 7f2efa7fc640 1 -- 172.21.15.12:0/1524582542 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2f00010900 con 0x7f2f040ffb50 2024-05-09T09:25:36.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.651+0000 7f2f0b961640 1 -- 172.21.15.12:0/1524582542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f041a1c70 con 0x7f2f040ffb50 2024-05-09T09:25:36.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.651+0000 7f2efa7fc640 1 -- 172.21.15.12:0/1524582542 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f000189e0 con 0x7f2f040ffb50 2024-05-09T09:25:36.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.651+0000 7f2f0b961640 1 -- 172.21.15.12:0/1524582542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f041a21c0 con 0x7f2f040ffb50 2024-05-09T09:25:36.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.652+0000 7f2f0b961640 1 -- 172.21.15.12:0/1524582542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0410b660 con 0x7f2f040ffb50 2024-05-09T09:25:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.653+0000 7f2efa7fc640 1 -- 172.21.15.12:0/1524582542 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2f00010ab0 con 0x7f2f040ffb50 2024-05-09T09:25:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.654+0000 7f2efa7fc640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ef00a2c10 0x7f2ef00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.654+0000 7f2f08efc640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ef00a2c10 0x7f2ef00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.654+0000 7f2efa7fc640 1 -- 172.21.15.12:0/1524582542 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f2f000cfc00 con 0x7f2f040ffb50 2024-05-09T09:25:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.655+0000 7f2f08efc640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ef00a2c10 0x7f2ef00a50d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2f0419b270 tx=0x7f2ef4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:36.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.657+0000 7f2efa7fc640 1 -- 172.21.15.12:0/1524582542 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2f0009c790 con 0x7f2f040ffb50 2024-05-09T09:25:36.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.842+0000 7f2f0b961640 1 -- 172.21.15.12:0/1524582542 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2f04061c70 con 0x7f2ef00a2c10 2024-05-09T09:25:36.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.852+0000 7f2efa7fc640 1 -- 172.21.15.12:0/1524582542 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340543 (secure 0 0 0) 0x7f2f04061c70 con 0x7f2ef00a2c10 2024-05-09T09:25:36.855 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:36.857 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-09T09:25:36.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.855+0000 7f2f0b961640 1 -- 172.21.15.12:0/1524582542 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ef00a2c10 msgr2=0x7f2ef00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:36.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.855+0000 7f2f0b961640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ef00a2c10 0x7f2ef00a50d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2f0419b270 tx=0x7f2ef4005f90 comp rx=0 tx=0).stop 2024-05-09T09:25:36.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.855+0000 7f2f0b961640 1 -- 172.21.15.12:0/1524582542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f040ffb50 msgr2=0x7f2f04199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:36.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.855+0000 7f2f0b961640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f040ffb50 0x7f2f04199d60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000e9f0 tx=0x7f2f0000eeb0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.855+0000 7f2f0b961640 1 -- 172.21.15.12:0/1524582542 shutdown_connections 2024-05-09T09:25:36.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.855+0000 7f2f0b961640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ef00a2c10 0x7f2ef00a50d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.855+0000 7f2f0b961640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f04101270 0x7f2f041a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.855+0000 7f2f0b961640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f041008b0 0x7f2f0419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.856+0000 7f2f0b961640 1 --2- 172.21.15.12:0/1524582542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f040ffb50 0x7f2f04199d60 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:36.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.856+0000 7f2f0b961640 1 -- 172.21.15.12:0/1524582542 >> 172.21.15.12:0/1524582542 conn(0x7f2f040fb4a0 msgr2=0x7f2f040fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:36.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.856+0000 7f2f0b961640 1 -- 172.21.15.12:0/1524582542 shutdown_connections 2024-05-09T09:25:36.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:36.856+0000 7f2f0b961640 1 -- 172.21.15.12:0/1524582542 wait complete. 2024-05-09T09:25:36.866 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-05-09T09:25:36.591134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663626,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1381,"num_read_kb":816,"num_write":448,"num_write_kb":3098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":868,"ondisk_log_size":868,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392900,"kb_used_data":14628,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124548924,"statfs":{"total":1151940427776,"available":1151538098176,"internally_reserved":0,"allocated":14979072,"data_stored":8203934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":60,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003534"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160720+0000","last_change":"2024-05-09T09:23:10.334738+0000","last_active":"2024-05-09T09:25:06.160720+0000","last_peered":"2024-05-09T09:25:06.160720+0000","last_clean":"2024-05-09T09:25:06.160720+0000","last_became_active":"2024-05-09T09:23:10.333406+0000","last_became_peered":"2024-05-09T09:23:10.333406+0000","last_unstale":"2024-05-09T09:25:06.160720+0000","last_undegraded":"2024-05-09T09:25:06.160720+0000","last_fullsized":"2024-05-09T09:25:06.160720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:44.957557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165459+0000","last_change":"2024-05-09T09:23:10.331860+0000","last_active":"2024-05-09T09:25:06.165459+0000","last_peered":"2024-05-09T09:25:06.165459+0000","last_clean":"2024-05-09T09:25:06.165459+0000","last_became_active":"2024-05-09T09:23:10.331396+0000","last_became_peered":"2024-05-09T09:23:10.331396+0000","last_unstale":"2024-05-09T09:25:06.165459+0000","last_undegraded":"2024-05-09T09:25:06.165459+0000","last_fullsized":"2024-05-09T09:25:06.165459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:47.676399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165707+0000","last_change":"2024-05-09T09:23:10.336543+0000","last_active":"2024-05-09T09:25:06.165707+0000","last_peered":"2024-05-09T09:25:06.165707+0000","last_clean":"2024-05-09T09:25:06.165707+0000","last_became_active":"2024-05-09T09:23:10.336189+0000","last_became_peered":"2024-05-09T09:23:10.336189+0000","last_unstale":"2024-05-09T09:25:06.165707+0000","last_undegraded":"2024-05-09T09:25:06.165707+0000","last_fullsized":"2024-05-09T09:25:06.165707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:06.419177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157538+0000","last_change":"2024-05-09T09:23:10.331739+0000","last_active":"2024-05-09T09:25:06.157538+0000","last_peered":"2024-05-09T09:25:06.157538+0000","last_clean":"2024-05-09T09:25:06.157538+0000","last_became_active":"2024-05-09T09:23:10.331333+0000","last_became_peered":"2024-05-09T09:23:10.331333+0000","last_unstale":"2024-05-09T09:25:06.157538+0000","last_undegraded":"2024-05-09T09:25:06.157538+0000","last_fullsized":"2024-05-09T09:25:06.157538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:04.925697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160387+0000","last_change":"2024-05-09T09:23:10.345294+0000","last_active":"2024-05-09T09:25:06.160387+0000","last_peered":"2024-05-09T09:25:06.160387+0000","last_clean":"2024-05-09T09:25:06.160387+0000","last_became_active":"2024-05-09T09:23:10.344940+0000","last_became_peered":"2024-05-09T09:23:10.344940+0000","last_unstale":"2024-05-09T09:25:06.160387+0000","last_undegraded":"2024-05-09T09:25:06.160387+0000","last_fullsized":"2024-05-09T09:25:06.160387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:41.150284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256906+0000","last_change":"2024-05-09T09:23:10.329143+0000","last_active":"2024-05-09T09:25:06.256906+0000","last_peered":"2024-05-09T09:25:06.256906+0000","last_clean":"2024-05-09T09:25:06.256906+0000","last_became_active":"2024-05-09T09:23:10.328712+0000","last_became_peered":"2024-05-09T09:23:10.328712+0000","last_unstale":"2024-05-09T09:25:06.256906+0000","last_undegraded":"2024-05-09T09:25:06.256906+0000","last_fullsized":"2024-05-09T09:25:06.256906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:10:35.085759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161026+0000","last_change":"2024-05-09T09:23:10.338176+0000","last_active":"2024-05-09T09:25:06.161026+0000","last_peered":"2024-05-09T09:25:06.161026+0000","last_clean":"2024-05-09T09:25:06.161026+0000","last_became_active":"2024-05-09T09:23:10.337845+0000","last_became_peered":"2024-05-09T09:23:10.337845+0000","last_unstale":"2024-05-09T09:25:06.161026+0000","last_undegraded":"2024-05-09T09:25:06.161026+0000","last_fullsized":"2024-05-09T09:25:06.161026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:34:21.783295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160809+0000","last_change":"2024-05-09T09:23:10.342473+0000","last_active":"2024-05-09T09:25:06.160809+0000","last_peered":"2024-05-09T09:25:06.160809+0000","last_clean":"2024-05-09T09:25:06.160809+0000","last_became_active":"2024-05-09T09:23:10.340682+0000","last_became_peered":"2024-05-09T09:23:10.340682+0000","last_unstale":"2024-05-09T09:25:06.160809+0000","last_undegraded":"2024-05-09T09:25:06.160809+0000","last_fullsized":"2024-05-09T09:25:06.160809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:56.730332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257663+0000","last_change":"2024-05-09T09:23:10.339888+0000","last_active":"2024-05-09T09:25:06.257663+0000","last_peered":"2024-05-09T09:25:06.257663+0000","last_clean":"2024-05-09T09:25:06.257663+0000","last_became_active":"2024-05-09T09:23:10.337849+0000","last_became_peered":"2024-05-09T09:23:10.337849+0000","last_unstale":"2024-05-09T09:25:06.257663+0000","last_undegraded":"2024-05-09T09:25:06.257663+0000","last_fullsized":"2024-05-09T09:25:06.257663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:48.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257800+0000","last_change":"2024-05-09T09:23:10.332395+0000","last_active":"2024-05-09T09:25:06.257800+0000","last_peered":"2024-05-09T09:25:06.257800+0000","last_clean":"2024-05-09T09:25:06.257800+0000","last_became_active":"2024-05-09T09:23:10.332040+0000","last_became_peered":"2024-05-09T09:23:10.332040+0000","last_unstale":"2024-05-09T09:25:06.257800+0000","last_undegraded":"2024-05-09T09:25:06.257800+0000","last_fullsized":"2024-05-09T09:25:06.257800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:04.036871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161826+0000","last_change":"2024-05-09T09:23:10.341148+0000","last_active":"2024-05-09T09:25:06.161826+0000","last_peered":"2024-05-09T09:25:06.161826+0000","last_clean":"2024-05-09T09:25:06.161826+0000","last_became_active":"2024-05-09T09:23:10.340799+0000","last_became_peered":"2024-05-09T09:23:10.340799+0000","last_unstale":"2024-05-09T09:25:06.161826+0000","last_undegraded":"2024-05-09T09:25:06.161826+0000","last_fullsized":"2024-05-09T09:25:06.161826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:31.020659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157609+0000","last_change":"2024-05-09T09:23:10.336191+0000","last_active":"2024-05-09T09:25:06.157609+0000","last_peered":"2024-05-09T09:25:06.157609+0000","last_clean":"2024-05-09T09:25:06.157609+0000","last_became_active":"2024-05-09T09:23:10.333591+0000","last_became_peered":"2024-05-09T09:23:10.333591+0000","last_unstale":"2024-05-09T09:25:06.157609+0000","last_undegraded":"2024-05-09T09:25:06.157609+0000","last_fullsized":"2024-05-09T09:25:06.157609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:24.836954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156258+0000","last_change":"2024-05-09T09:23:10.314506+0000","last_active":"2024-05-09T09:25:06.156258+0000","last_peered":"2024-05-09T09:25:06.156258+0000","last_clean":"2024-05-09T09:25:06.156258+0000","last_became_active":"2024-05-09T09:23:10.314171+0000","last_became_peered":"2024-05-09T09:23:10.314171+0000","last_unstale":"2024-05-09T09:25:06.156258+0000","last_undegraded":"2024-05-09T09:25:06.156258+0000","last_fullsized":"2024-05-09T09:25:06.156258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:51.088814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165407+0000","last_change":"2024-05-09T09:23:10.341670+0000","last_active":"2024-05-09T09:25:06.165407+0000","last_peered":"2024-05-09T09:25:06.165407+0000","last_clean":"2024-05-09T09:25:06.165407+0000","last_became_active":"2024-05-09T09:23:10.341155+0000","last_became_peered":"2024-05-09T09:23:10.341155+0000","last_unstale":"2024-05-09T09:25:06.165407+0000","last_undegraded":"2024-05-09T09:25:06.165407+0000","last_fullsized":"2024-05-09T09:25:06.165407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:14:32.734302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165578+0000","last_change":"2024-05-09T09:24:05.742117+0000","last_active":"2024-05-09T09:25:06.165578+0000","last_peered":"2024-05-09T09:25:06.165578+0000","last_clean":"2024-05-09T09:25:06.165578+0000","last_became_active":"2024-05-09T09:24:05.741655+0000","last_became_peered":"2024-05-09T09:24:05.741655+0000","last_unstale":"2024-05-09T09:25:06.165578+0000","last_undegraded":"2024-05-09T09:25:06.165578+0000","last_fullsized":"2024-05-09T09:25:06.165578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:11.468527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256668+0000","last_change":"2024-05-09T09:23:10.337367+0000","last_active":"2024-05-09T09:25:06.256668+0000","last_peered":"2024-05-09T09:25:06.256668+0000","last_clean":"2024-05-09T09:25:06.256668+0000","last_became_active":"2024-05-09T09:23:10.337000+0000","last_became_peered":"2024-05-09T09:23:10.337000+0000","last_unstale":"2024-05-09T09:25:06.256668+0000","last_undegraded":"2024-05-09T09:25:06.256668+0000","last_fullsized":"2024-05-09T09:25:06.256668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:54.431820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160278+0000","last_change":"2024-05-09T09:23:10.334293+0000","last_active":"2024-05-09T09:25:06.160278+0000","last_peered":"2024-05-09T09:25:06.160278+0000","last_clean":"2024-05-09T09:25:06.160278+0000","last_became_active":"2024-05-09T09:23:10.333938+0000","last_became_peered":"2024-05-09T09:23:10.333938+0000","last_unstale":"2024-05-09T09:25:06.160278+0000","last_undegraded":"2024-05-09T09:25:06.160278+0000","last_fullsized":"2024-05-09T09:25:06.160278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:56:09.048848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159638+0000","last_change":"2024-05-09T09:23:10.332394+0000","last_active":"2024-05-09T09:25:06.159638+0000","last_peered":"2024-05-09T09:25:06.159638+0000","last_clean":"2024-05-09T09:25:06.159638+0000","last_became_active":"2024-05-09T09:23:10.332054+0000","last_became_peered":"2024-05-09T09:23:10.332054+0000","last_unstale":"2024-05-09T09:25:06.159638+0000","last_undegraded":"2024-05-09T09:25:06.159638+0000","last_fullsized":"2024-05-09T09:25:06.159638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:05.628740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164212+0000","last_change":"2024-05-09T09:23:10.336189+0000","last_active":"2024-05-09T09:25:06.164212+0000","last_peered":"2024-05-09T09:25:06.164212+0000","last_clean":"2024-05-09T09:25:06.164212+0000","last_became_active":"2024-05-09T09:23:10.335698+0000","last_became_peered":"2024-05-09T09:23:10.335698+0000","last_unstale":"2024-05-09T09:25:06.164212+0000","last_undegraded":"2024-05-09T09:25:06.164212+0000","last_fullsized":"2024-05-09T09:25:06.164212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:01.403918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162866+0000","last_change":"2024-05-09T09:23:14.372134+0000","last_active":"2024-05-09T09:25:06.162866+0000","last_peered":"2024-05-09T09:25:06.162866+0000","last_clean":"2024-05-09T09:25:06.162866+0000","last_became_active":"2024-05-09T09:23:14.371466+0000","last_became_peered":"2024-05-09T09:23:14.371466+0000","last_unstale":"2024-05-09T09:25:06.162866+0000","last_undegraded":"2024-05-09T09:25:06.162866+0000","last_fullsized":"2024-05-09T09:25:06.162866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:22.048565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:31.760845+0000","last_change":"2024-05-09T09:23:09.304421+0000","last_active":"2024-05-09T09:25:31.760845+0000","last_peered":"2024-05-09T09:25:31.760845+0000","last_clean":"2024-05-09T09:25:31.760845+0000","last_became_active":"2024-05-09T09:23:09.304109+0000","last_became_peered":"2024-05-09T09:23:09.304109+0000","last_unstale":"2024-05-09T09:25:31.760845+0000","last_undegraded":"2024-05-09T09:25:31.760845+0000","last_fullsized":"2024-05-09T09:25:31.760845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:36.796458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162409+0000","last_change":"2024-05-09T09:23:10.341875+0000","last_active":"2024-05-09T09:25:06.162409+0000","last_peered":"2024-05-09T09:25:06.162409+0000","last_clean":"2024-05-09T09:25:06.162409+0000","last_became_active":"2024-05-09T09:23:10.341385+0000","last_became_peered":"2024-05-09T09:23:10.341385+0000","last_unstale":"2024-05-09T09:25:06.162409+0000","last_undegraded":"2024-05-09T09:25:06.162409+0000","last_fullsized":"2024-05-09T09:25:06.162409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:44:33.939722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156972+0000","last_change":"2024-05-09T09:23:14.371120+0000","last_active":"2024-05-09T09:25:06.156972+0000","last_peered":"2024-05-09T09:25:06.156972+0000","last_clean":"2024-05-09T09:25:06.156972+0000","last_became_active":"2024-05-09T09:23:14.370663+0000","last_became_peered":"2024-05-09T09:23:14.370663+0000","last_unstale":"2024-05-09T09:25:06.156972+0000","last_undegraded":"2024-05-09T09:25:06.156972+0000","last_fullsized":"2024-05-09T09:25:06.156972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:50:06.553518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162781+0000","last_change":"2024-05-09T09:23:09.304357+0000","last_active":"2024-05-09T09:25:06.162781+0000","last_peered":"2024-05-09T09:25:06.162781+0000","last_clean":"2024-05-09T09:25:06.162781+0000","last_became_active":"2024-05-09T09:23:09.304007+0000","last_became_peered":"2024-05-09T09:23:09.304007+0000","last_unstale":"2024-05-09T09:25:06.162781+0000","last_undegraded":"2024-05-09T09:25:06.162781+0000","last_fullsized":"2024-05-09T09:25:06.162781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:32:46.132287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165838+0000","last_change":"2024-05-09T09:23:10.339533+0000","last_active":"2024-05-09T09:25:06.165838+0000","last_peered":"2024-05-09T09:25:06.165838+0000","last_clean":"2024-05-09T09:25:06.165838+0000","last_became_active":"2024-05-09T09:23:10.337142+0000","last_became_peered":"2024-05-09T09:23:10.337142+0000","last_unstale":"2024-05-09T09:25:06.165838+0000","last_undegraded":"2024-05-09T09:25:06.165838+0000","last_fullsized":"2024-05-09T09:25:06.165838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:11:11.921491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157173+0000","last_change":"2024-05-09T09:24:05.744009+0000","last_active":"2024-05-09T09:25:06.157173+0000","last_peered":"2024-05-09T09:25:06.157173+0000","last_clean":"2024-05-09T09:25:06.157173+0000","last_became_active":"2024-05-09T09:24:05.743611+0000","last_became_peered":"2024-05-09T09:24:05.743611+0000","last_unstale":"2024-05-09T09:25:06.157173+0000","last_undegraded":"2024-05-09T09:25:06.157173+0000","last_fullsized":"2024-05-09T09:25:06.157173+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:55.775740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160457+0000","last_change":"2024-05-09T09:23:09.300754+0000","last_active":"2024-05-09T09:25:06.160457+0000","last_peered":"2024-05-09T09:25:06.160457+0000","last_clean":"2024-05-09T09:25:06.160457+0000","last_became_active":"2024-05-09T09:23:09.300419+0000","last_became_peered":"2024-05-09T09:23:09.300419+0000","last_unstale":"2024-05-09T09:25:06.160457+0000","last_undegraded":"2024-05-09T09:25:06.160457+0000","last_fullsized":"2024-05-09T09:25:06.160457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:06.205582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257117+0000","last_change":"2024-05-09T09:23:10.331952+0000","last_active":"2024-05-09T09:25:06.257117+0000","last_peered":"2024-05-09T09:25:06.257117+0000","last_clean":"2024-05-09T09:25:06.257117+0000","last_became_active":"2024-05-09T09:23:10.331515+0000","last_became_peered":"2024-05-09T09:23:10.331515+0000","last_unstale":"2024-05-09T09:25:06.257117+0000","last_undegraded":"2024-05-09T09:25:06.257117+0000","last_fullsized":"2024-05-09T09:25:06.257117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:23.809672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156952+0000","last_change":"2024-05-09T09:23:14.368518+0000","last_active":"2024-05-09T09:25:06.156952+0000","last_peered":"2024-05-09T09:25:06.156952+0000","last_clean":"2024-05-09T09:25:06.156952+0000","last_became_active":"2024-05-09T09:23:14.367995+0000","last_became_peered":"2024-05-09T09:23:14.367995+0000","last_unstale":"2024-05-09T09:25:06.156952+0000","last_undegraded":"2024-05-09T09:25:06.156952+0000","last_fullsized":"2024-05-09T09:25:06.156952+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:11.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160465+0000","last_change":"2024-05-09T09:23:09.299656+0000","last_active":"2024-05-09T09:25:06.160465+0000","last_peered":"2024-05-09T09:25:06.160465+0000","last_clean":"2024-05-09T09:25:06.160465+0000","last_became_active":"2024-05-09T09:23:09.299374+0000","last_became_peered":"2024-05-09T09:23:09.299374+0000","last_unstale":"2024-05-09T09:25:06.160465+0000","last_undegraded":"2024-05-09T09:25:06.160465+0000","last_fullsized":"2024-05-09T09:25:06.160465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:05.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157723+0000","last_change":"2024-05-09T09:23:10.341933+0000","last_active":"2024-05-09T09:25:06.157723+0000","last_peered":"2024-05-09T09:25:06.157723+0000","last_clean":"2024-05-09T09:25:06.157723+0000","last_became_active":"2024-05-09T09:23:10.341621+0000","last_became_peered":"2024-05-09T09:23:10.341621+0000","last_unstale":"2024-05-09T09:25:06.157723+0000","last_undegraded":"2024-05-09T09:25:06.157723+0000","last_fullsized":"2024-05-09T09:25:06.157723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:31.938768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.163011+0000","last_change":"2024-05-09T09:23:14.364783+0000","last_active":"2024-05-09T09:25:06.163011+0000","last_peered":"2024-05-09T09:25:06.163011+0000","last_clean":"2024-05-09T09:25:06.163011+0000","last_became_active":"2024-05-09T09:23:14.364309+0000","last_became_peered":"2024-05-09T09:23:14.364309+0000","last_unstale":"2024-05-09T09:25:06.163011+0000","last_undegraded":"2024-05-09T09:25:06.163011+0000","last_fullsized":"2024-05-09T09:25:06.163011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:44.252111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'6","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:32.763334+0000","last_change":"2024-05-09T09:23:09.297913+0000","last_active":"2024-05-09T09:25:32.763334+0000","last_peered":"2024-05-09T09:25:32.763334+0000","last_clean":"2024-05-09T09:25:32.763334+0000","last_became_active":"2024-05-09T09:23:09.297612+0000","last_became_peered":"2024-05-09T09:23:09.297612+0000","last_unstale":"2024-05-09T09:25:32.763334+0000","last_undegraded":"2024-05-09T09:25:32.763334+0000","last_fullsized":"2024-05-09T09:25:32.763334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:21.808703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.163905+0000","last_change":"2024-05-09T09:25:06.163905+0000","last_active":"2024-05-09T09:25:06.163905+0000","last_peered":"2024-05-09T09:25:06.163905+0000","last_clean":"2024-05-09T09:25:06.163905+0000","last_became_active":"2024-05-09T09:25:06.163454+0000","last_became_peered":"2024-05-09T09:25:06.163454+0000","last_unstale":"2024-05-09T09:25:06.163905+0000","last_undegraded":"2024-05-09T09:25:06.163905+0000","last_fullsized":"2024-05-09T09:25:06.163905+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:49:05.654792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162936+0000","last_change":"2024-05-09T09:23:14.371781+0000","last_active":"2024-05-09T09:25:06.162936+0000","last_peered":"2024-05-09T09:25:06.162936+0000","last_clean":"2024-05-09T09:25:06.162936+0000","last_became_active":"2024-05-09T09:23:14.371286+0000","last_became_peered":"2024-05-09T09:23:14.371286+0000","last_unstale":"2024-05-09T09:25:06.162936+0000","last_undegraded":"2024-05-09T09:25:06.162936+0000","last_fullsized":"2024-05-09T09:25:06.162936+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:26:29.185064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256852+0000","last_change":"2024-05-09T09:23:09.303786+0000","last_active":"2024-05-09T09:25:06.256852+0000","last_peered":"2024-05-09T09:25:06.256852+0000","last_clean":"2024-05-09T09:25:06.256852+0000","last_became_active":"2024-05-09T09:23:09.303475+0000","last_became_peered":"2024-05-09T09:23:09.303475+0000","last_unstale":"2024-05-09T09:25:06.256852+0000","last_undegraded":"2024-05-09T09:25:06.256852+0000","last_fullsized":"2024-05-09T09:25:06.256852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:28.833055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157664+0000","last_change":"2024-05-09T09:23:10.344643+0000","last_active":"2024-05-09T09:25:06.157664+0000","last_peered":"2024-05-09T09:25:06.157664+0000","last_clean":"2024-05-09T09:25:06.157664+0000","last_became_active":"2024-05-09T09:23:10.344314+0000","last_became_peered":"2024-05-09T09:23:10.344314+0000","last_unstale":"2024-05-09T09:25:06.157664+0000","last_undegraded":"2024-05-09T09:25:06.157664+0000","last_fullsized":"2024-05-09T09:25:06.157664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:15.147107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160150+0000","last_change":"2024-05-09T09:23:14.361170+0000","last_active":"2024-05-09T09:25:06.160150+0000","last_peered":"2024-05-09T09:25:06.160150+0000","last_clean":"2024-05-09T09:25:06.160150+0000","last_became_active":"2024-05-09T09:23:14.360767+0000","last_became_peered":"2024-05-09T09:23:14.360767+0000","last_unstale":"2024-05-09T09:25:06.160150+0000","last_undegraded":"2024-05-09T09:25:06.160150+0000","last_fullsized":"2024-05-09T09:25:06.160150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:59:21.393148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156074+0000","last_change":"2024-05-09T09:23:09.300757+0000","last_active":"2024-05-09T09:25:06.156074+0000","last_peered":"2024-05-09T09:25:06.156074+0000","last_clean":"2024-05-09T09:25:06.156074+0000","last_became_active":"2024-05-09T09:23:09.300328+0000","last_became_peered":"2024-05-09T09:23:09.300328+0000","last_unstale":"2024-05-09T09:25:06.156074+0000","last_undegraded":"2024-05-09T09:25:06.156074+0000","last_fullsized":"2024-05-09T09:25:06.156074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.553878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161040+0000","last_change":"2024-05-09T09:23:10.336077+0000","last_active":"2024-05-09T09:25:06.161040+0000","last_peered":"2024-05-09T09:25:06.161040+0000","last_clean":"2024-05-09T09:25:06.161040+0000","last_became_active":"2024-05-09T09:23:10.335762+0000","last_became_peered":"2024-05-09T09:23:10.335762+0000","last_unstale":"2024-05-09T09:25:06.161040+0000","last_undegraded":"2024-05-09T09:25:06.161040+0000","last_fullsized":"2024-05-09T09:25:06.161040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:48.063385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165335+0000","last_change":"2024-05-09T09:23:14.371413+0000","last_active":"2024-05-09T09:25:06.165335+0000","last_peered":"2024-05-09T09:25:06.165335+0000","last_clean":"2024-05-09T09:25:06.165335+0000","last_became_active":"2024-05-09T09:23:14.370018+0000","last_became_peered":"2024-05-09T09:23:14.370018+0000","last_unstale":"2024-05-09T09:25:06.165335+0000","last_undegraded":"2024-05-09T09:25:06.165335+0000","last_fullsized":"2024-05-09T09:25:06.165335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:18.475976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157139+0000","last_change":"2024-05-09T09:23:09.297328+0000","last_active":"2024-05-09T09:25:06.157139+0000","last_peered":"2024-05-09T09:25:06.157139+0000","last_clean":"2024-05-09T09:25:06.157139+0000","last_became_active":"2024-05-09T09:23:09.297001+0000","last_became_peered":"2024-05-09T09:23:09.297001+0000","last_unstale":"2024-05-09T09:25:06.157139+0000","last_undegraded":"2024-05-09T09:25:06.157139+0000","last_fullsized":"2024-05-09T09:25:06.157139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:19.241362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159057+0000","last_change":"2024-05-09T09:23:10.338342+0000","last_active":"2024-05-09T09:25:06.159057+0000","last_peered":"2024-05-09T09:25:06.159057+0000","last_clean":"2024-05-09T09:25:06.159057+0000","last_became_active":"2024-05-09T09:23:10.337995+0000","last_became_peered":"2024-05-09T09:23:10.337995+0000","last_unstale":"2024-05-09T09:25:06.159057+0000","last_undegraded":"2024-05-09T09:25:06.159057+0000","last_fullsized":"2024-05-09T09:25:06.159057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.691862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161348+0000","last_change":"2024-05-09T09:23:14.366237+0000","last_active":"2024-05-09T09:25:06.161348+0000","last_peered":"2024-05-09T09:25:06.161348+0000","last_clean":"2024-05-09T09:25:06.161348+0000","last_became_active":"2024-05-09T09:23:14.365809+0000","last_became_peered":"2024-05-09T09:23:14.365809+0000","last_unstale":"2024-05-09T09:25:06.161348+0000","last_undegraded":"2024-05-09T09:25:06.161348+0000","last_fullsized":"2024-05-09T09:25:06.161348+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:28.813467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":279,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:33.764231+0000","last_change":"2024-05-09T09:23:09.298658+0000","last_active":"2024-05-09T09:25:33.764231+0000","last_peered":"2024-05-09T09:25:33.764231+0000","last_clean":"2024-05-09T09:25:33.764231+0000","last_became_active":"2024-05-09T09:23:09.298325+0000","last_became_peered":"2024-05-09T09:23:09.298325+0000","last_unstale":"2024-05-09T09:25:33.764231+0000","last_undegraded":"2024-05-09T09:25:33.764231+0000","last_fullsized":"2024-05-09T09:25:33.764231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:22.108202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":242,"num_read_kb":242,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257522+0000","last_change":"2024-05-09T09:23:10.326865+0000","last_active":"2024-05-09T09:25:06.257522+0000","last_peered":"2024-05-09T09:25:06.257522+0000","last_clean":"2024-05-09T09:25:06.257522+0000","last_became_active":"2024-05-09T09:23:10.326478+0000","last_became_peered":"2024-05-09T09:23:10.326478+0000","last_unstale":"2024-05-09T09:25:06.257522+0000","last_undegraded":"2024-05-09T09:25:06.257522+0000","last_fullsized":"2024-05-09T09:25:06.257522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:54.521226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.158982+0000","last_change":"2024-05-09T09:23:14.373601+0000","last_active":"2024-05-09T09:25:06.158982+0000","last_peered":"2024-05-09T09:25:06.158982+0000","last_clean":"2024-05-09T09:25:06.158982+0000","last_became_active":"2024-05-09T09:23:14.372764+0000","last_became_peered":"2024-05-09T09:23:14.372764+0000","last_unstale":"2024-05-09T09:25:06.158982+0000","last_undegraded":"2024-05-09T09:25:06.158982+0000","last_fullsized":"2024-05-09T09:25:06.158982+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:59.496755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162066+0000","last_change":"2024-05-09T09:23:09.302293+0000","last_active":"2024-05-09T09:25:06.162066+0000","last_peered":"2024-05-09T09:25:06.162066+0000","last_clean":"2024-05-09T09:25:06.162066+0000","last_became_active":"2024-05-09T09:23:09.301216+0000","last_became_peered":"2024-05-09T09:23:09.301216+0000","last_unstale":"2024-05-09T09:25:06.162066+0000","last_undegraded":"2024-05-09T09:25:06.162066+0000","last_fullsized":"2024-05-09T09:25:06.162066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:35.146860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165710+0000","last_change":"2024-05-09T09:23:10.339827+0000","last_active":"2024-05-09T09:25:06.165710+0000","last_peered":"2024-05-09T09:25:06.165710+0000","last_clean":"2024-05-09T09:25:06.165710+0000","last_became_active":"2024-05-09T09:23:10.339415+0000","last_became_peered":"2024-05-09T09:23:10.339415+0000","last_unstale":"2024-05-09T09:25:06.165710+0000","last_undegraded":"2024-05-09T09:25:06.165710+0000","last_fullsized":"2024-05-09T09:25:06.165710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:45.250126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157361+0000","last_change":"2024-05-09T09:23:14.377001+0000","last_active":"2024-05-09T09:25:06.157361+0000","last_peered":"2024-05-09T09:25:06.157361+0000","last_clean":"2024-05-09T09:25:06.157361+0000","last_became_active":"2024-05-09T09:23:14.376402+0000","last_became_peered":"2024-05-09T09:23:14.376402+0000","last_unstale":"2024-05-09T09:25:06.157361+0000","last_undegraded":"2024-05-09T09:25:06.157361+0000","last_fullsized":"2024-05-09T09:25:06.157361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:08.142495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.158215+0000","last_change":"2024-05-09T09:23:09.299331+0000","last_active":"2024-05-09T09:25:06.158215+0000","last_peered":"2024-05-09T09:25:06.158215+0000","last_clean":"2024-05-09T09:25:06.158215+0000","last_became_active":"2024-05-09T09:23:09.299017+0000","last_became_peered":"2024-05-09T09:23:09.299017+0000","last_unstale":"2024-05-09T09:25:06.158215+0000","last_undegraded":"2024-05-09T09:25:06.158215+0000","last_fullsized":"2024-05-09T09:25:06.158215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:28.328704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257442+0000","last_change":"2024-05-09T09:24:05.743586+0000","last_active":"2024-05-09T09:25:06.257442+0000","last_peered":"2024-05-09T09:25:06.257442+0000","last_clean":"2024-05-09T09:25:06.257442+0000","last_became_active":"2024-05-09T09:24:05.742610+0000","last_became_peered":"2024-05-09T09:24:05.742610+0000","last_unstale":"2024-05-09T09:25:06.257442+0000","last_undegraded":"2024-05-09T09:25:06.257442+0000","last_fullsized":"2024-05-09T09:25:06.257442+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:05.586985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159196+0000","last_change":"2024-05-09T09:23:14.363362+0000","last_active":"2024-05-09T09:25:06.159196+0000","last_peered":"2024-05-09T09:25:06.159196+0000","last_clean":"2024-05-09T09:25:06.159196+0000","last_became_active":"2024-05-09T09:23:14.362926+0000","last_became_peered":"2024-05-09T09:23:14.362926+0000","last_unstale":"2024-05-09T09:25:06.159196+0000","last_undegraded":"2024-05-09T09:25:06.159196+0000","last_fullsized":"2024-05-09T09:25:06.159196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:25.339727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":321,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:33.763965+0000","last_change":"2024-05-09T09:23:09.297814+0000","last_active":"2024-05-09T09:25:33.763965+0000","last_peered":"2024-05-09T09:25:33.763965+0000","last_clean":"2024-05-09T09:25:33.763965+0000","last_became_active":"2024-05-09T09:23:09.297005+0000","last_became_peered":"2024-05-09T09:23:09.297005+0000","last_unstale":"2024-05-09T09:25:33.763965+0000","last_undegraded":"2024-05-09T09:25:33.763965+0000","last_fullsized":"2024-05-09T09:25:33.763965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:39.240217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64098,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":251,"num_read_kb":188,"num_write":41,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160129+0000","last_change":"2024-05-09T09:23:10.333013+0000","last_active":"2024-05-09T09:25:06.160129+0000","last_peered":"2024-05-09T09:25:06.160129+0000","last_clean":"2024-05-09T09:25:06.160129+0000","last_became_active":"2024-05-09T09:23:10.332666+0000","last_became_peered":"2024-05-09T09:23:10.332666+0000","last_unstale":"2024-05-09T09:25:06.160129+0000","last_undegraded":"2024-05-09T09:25:06.160129+0000","last_fullsized":"2024-05-09T09:25:06.160129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:28:50.306551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.158019+0000","last_change":"2024-05-09T09:23:14.367715+0000","last_active":"2024-05-09T09:25:06.158019+0000","last_peered":"2024-05-09T09:25:06.158019+0000","last_clean":"2024-05-09T09:25:06.158019+0000","last_became_active":"2024-05-09T09:23:14.367307+0000","last_became_peered":"2024-05-09T09:23:14.367307+0000","last_unstale":"2024-05-09T09:25:06.158019+0000","last_undegraded":"2024-05-09T09:25:06.158019+0000","last_fullsized":"2024-05-09T09:25:06.158019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:09.075572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164381+0000","last_change":"2024-05-09T09:23:09.306302+0000","last_active":"2024-05-09T09:25:06.164381+0000","last_peered":"2024-05-09T09:25:06.164381+0000","last_clean":"2024-05-09T09:25:06.164381+0000","last_became_active":"2024-05-09T09:23:09.306074+0000","last_became_peered":"2024-05-09T09:23:09.306074+0000","last_unstale":"2024-05-09T09:25:06.164381+0000","last_undegraded":"2024-05-09T09:25:06.164381+0000","last_fullsized":"2024-05-09T09:25:06.164381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:18.434899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156080+0000","last_change":"2024-05-09T09:23:10.331201+0000","last_active":"2024-05-09T09:25:06.156080+0000","last_peered":"2024-05-09T09:25:06.156080+0000","last_clean":"2024-05-09T09:25:06.156080+0000","last_became_active":"2024-05-09T09:23:10.329830+0000","last_became_peered":"2024-05-09T09:23:10.329830+0000","last_unstale":"2024-05-09T09:25:06.156080+0000","last_undegraded":"2024-05-09T09:25:06.156080+0000","last_fullsized":"2024-05-09T09:25:06.156080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:40.199622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160009+0000","last_change":"2024-05-09T09:23:14.359095+0000","last_active":"2024-05-09T09:25:06.160009+0000","last_peered":"2024-05-09T09:25:06.160009+0000","last_clean":"2024-05-09T09:25:06.160009+0000","last_became_active":"2024-05-09T09:23:14.358717+0000","last_became_peered":"2024-05-09T09:23:14.358717+0000","last_unstale":"2024-05-09T09:25:06.160009+0000","last_undegraded":"2024-05-09T09:25:06.160009+0000","last_fullsized":"2024-05-09T09:25:06.160009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:16:09.874767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157568+0000","last_change":"2024-05-09T09:23:09.305370+0000","last_active":"2024-05-09T09:25:06.157568+0000","last_peered":"2024-05-09T09:25:06.157568+0000","last_clean":"2024-05-09T09:25:06.157568+0000","last_became_active":"2024-05-09T09:23:09.305054+0000","last_became_peered":"2024-05-09T09:23:09.305054+0000","last_unstale":"2024-05-09T09:25:06.157568+0000","last_undegraded":"2024-05-09T09:25:06.157568+0000","last_fullsized":"2024-05-09T09:25:06.157568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:28.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161625+0000","last_change":"2024-05-09T09:23:10.332541+0000","last_active":"2024-05-09T09:25:06.161625+0000","last_peered":"2024-05-09T09:25:06.161625+0000","last_clean":"2024-05-09T09:25:06.161625+0000","last_became_active":"2024-05-09T09:23:10.332155+0000","last_became_peered":"2024-05-09T09:23:10.332155+0000","last_unstale":"2024-05-09T09:25:06.161625+0000","last_undegraded":"2024-05-09T09:25:06.161625+0000","last_fullsized":"2024-05-09T09:25:06.161625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:39.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159430+0000","last_change":"2024-05-09T09:23:14.374499+0000","last_active":"2024-05-09T09:25:06.159430+0000","last_peered":"2024-05-09T09:25:06.159430+0000","last_clean":"2024-05-09T09:25:06.159430+0000","last_became_active":"2024-05-09T09:23:14.370688+0000","last_became_peered":"2024-05-09T09:23:14.370688+0000","last_unstale":"2024-05-09T09:25:06.159430+0000","last_undegraded":"2024-05-09T09:25:06.159430+0000","last_fullsized":"2024-05-09T09:25:06.159430+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:53.367237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'8","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:32.763278+0000","last_change":"2024-05-09T09:23:09.297776+0000","last_active":"2024-05-09T09:25:32.763278+0000","last_peered":"2024-05-09T09:25:32.763278+0000","last_clean":"2024-05-09T09:25:32.763278+0000","last_became_active":"2024-05-09T09:23:09.297407+0000","last_became_peered":"2024-05-09T09:23:09.297407+0000","last_unstale":"2024-05-09T09:25:32.763278+0000","last_undegraded":"2024-05-09T09:25:32.763278+0000","last_fullsized":"2024-05-09T09:25:32.763278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:21.762941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161199+0000","last_change":"2024-05-09T09:23:10.339127+0000","last_active":"2024-05-09T09:25:06.161199+0000","last_peered":"2024-05-09T09:25:06.161199+0000","last_clean":"2024-05-09T09:25:06.161199+0000","last_became_active":"2024-05-09T09:23:10.338790+0000","last_became_peered":"2024-05-09T09:23:10.338790+0000","last_unstale":"2024-05-09T09:25:06.161199+0000","last_undegraded":"2024-05-09T09:25:06.161199+0000","last_fullsized":"2024-05-09T09:25:06.161199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:36.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156425+0000","last_change":"2024-05-09T09:23:14.373873+0000","last_active":"2024-05-09T09:25:06.156425+0000","last_peered":"2024-05-09T09:25:06.156425+0000","last_clean":"2024-05-09T09:25:06.156425+0000","last_became_active":"2024-05-09T09:23:14.371453+0000","last_became_peered":"2024-05-09T09:23:14.371453+0000","last_unstale":"2024-05-09T09:25:06.156425+0000","last_undegraded":"2024-05-09T09:25:06.156425+0000","last_fullsized":"2024-05-09T09:25:06.156425+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:26.885938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159355+0000","last_change":"2024-05-09T09:23:09.299373+0000","last_active":"2024-05-09T09:25:06.159355+0000","last_peered":"2024-05-09T09:25:06.159355+0000","last_clean":"2024-05-09T09:25:06.159355+0000","last_became_active":"2024-05-09T09:23:09.298926+0000","last_became_peered":"2024-05-09T09:23:09.298926+0000","last_unstale":"2024-05-09T09:25:06.159355+0000","last_undegraded":"2024-05-09T09:25:06.159355+0000","last_fullsized":"2024-05-09T09:25:06.159355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:35.629929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157309+0000","last_change":"2024-05-09T09:23:10.343911+0000","last_active":"2024-05-09T09:25:06.157309+0000","last_peered":"2024-05-09T09:25:06.157309+0000","last_clean":"2024-05-09T09:25:06.157309+0000","last_became_active":"2024-05-09T09:23:10.343568+0000","last_became_peered":"2024-05-09T09:23:10.343568+0000","last_unstale":"2024-05-09T09:25:06.157309+0000","last_undegraded":"2024-05-09T09:25:06.157309+0000","last_fullsized":"2024-05-09T09:25:06.157309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:07.311554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157224+0000","last_change":"2024-05-09T09:23:14.368544+0000","last_active":"2024-05-09T09:25:06.157224+0000","last_peered":"2024-05-09T09:25:06.157224+0000","last_clean":"2024-05-09T09:25:06.157224+0000","last_became_active":"2024-05-09T09:23:14.368140+0000","last_became_peered":"2024-05-09T09:23:14.368140+0000","last_unstale":"2024-05-09T09:25:06.157224+0000","last_undegraded":"2024-05-09T09:25:06.157224+0000","last_fullsized":"2024-05-09T09:25:06.157224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.091768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:27.142002+0000","last_change":"2024-05-09T09:23:09.297612+0000","last_active":"2024-05-09T09:25:27.142002+0000","last_peered":"2024-05-09T09:25:27.142002+0000","last_clean":"2024-05-09T09:25:27.142002+0000","last_became_active":"2024-05-09T09:23:09.297216+0000","last_became_peered":"2024-05-09T09:23:09.297216+0000","last_unstale":"2024-05-09T09:25:27.142002+0000","last_undegraded":"2024-05-09T09:25:27.142002+0000","last_fullsized":"2024-05-09T09:25:27.142002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:09:45.108082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164210+0000","last_change":"2024-05-09T09:25:06.163625+0000","last_active":"2024-05-09T09:25:06.164210+0000","last_peered":"2024-05-09T09:25:06.164210+0000","last_clean":"2024-05-09T09:25:06.164210+0000","last_became_active":"2024-05-09T09:25:06.163198+0000","last_became_peered":"2024-05-09T09:25:06.163198+0000","last_unstale":"2024-05-09T09:25:06.164210+0000","last_undegraded":"2024-05-09T09:25:06.164210+0000","last_fullsized":"2024-05-09T09:25:06.164210+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:35.223030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160161+0000","last_change":"2024-05-09T09:23:14.366119+0000","last_active":"2024-05-09T09:25:06.160161+0000","last_peered":"2024-05-09T09:25:06.160161+0000","last_clean":"2024-05-09T09:25:06.160161+0000","last_became_active":"2024-05-09T09:23:14.365659+0000","last_became_peered":"2024-05-09T09:23:14.365659+0000","last_unstale":"2024-05-09T09:25:06.160161+0000","last_undegraded":"2024-05-09T09:25:06.160161+0000","last_fullsized":"2024-05-09T09:25:06.160161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:23.312632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159327+0000","last_change":"2024-05-09T09:24:05.740866+0000","last_active":"2024-05-09T09:25:06.159327+0000","last_peered":"2024-05-09T09:25:06.159327+0000","last_clean":"2024-05-09T09:25:06.159327+0000","last_became_active":"2024-05-09T09:24:05.740418+0000","last_became_peered":"2024-05-09T09:24:05.740418+0000","last_unstale":"2024-05-09T09:25:06.159327+0000","last_undegraded":"2024-05-09T09:25:06.159327+0000","last_fullsized":"2024-05-09T09:25:06.159327+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:45.946040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257359+0000","last_change":"2024-05-09T09:23:10.339788+0000","last_active":"2024-05-09T09:25:06.257359+0000","last_peered":"2024-05-09T09:25:06.257359+0000","last_clean":"2024-05-09T09:25:06.257359+0000","last_became_active":"2024-05-09T09:23:10.339430+0000","last_became_peered":"2024-05-09T09:23:10.339430+0000","last_unstale":"2024-05-09T09:25:06.257359+0000","last_undegraded":"2024-05-09T09:25:06.257359+0000","last_fullsized":"2024-05-09T09:25:06.257359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:22.467287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156237+0000","last_change":"2024-05-09T09:23:14.368710+0000","last_active":"2024-05-09T09:25:06.156237+0000","last_peered":"2024-05-09T09:25:06.156237+0000","last_clean":"2024-05-09T09:25:06.156237+0000","last_became_active":"2024-05-09T09:23:14.368288+0000","last_became_peered":"2024-05-09T09:23:14.368288+0000","last_unstale":"2024-05-09T09:25:06.156237+0000","last_undegraded":"2024-05-09T09:25:06.156237+0000","last_fullsized":"2024-05-09T09:25:06.156237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:40.941273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:34.765316+0000","last_change":"2024-05-09T09:23:09.301976+0000","last_active":"2024-05-09T09:25:34.765316+0000","last_peered":"2024-05-09T09:25:34.765316+0000","last_clean":"2024-05-09T09:25:34.765316+0000","last_became_active":"2024-05-09T09:23:09.301494+0000","last_became_peered":"2024-05-09T09:23:09.301494+0000","last_unstale":"2024-05-09T09:25:34.765316+0000","last_undegraded":"2024-05-09T09:25:34.765316+0000","last_fullsized":"2024-05-09T09:25:34.765316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.448246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256796+0000","last_change":"2024-05-09T09:23:10.331691+0000","last_active":"2024-05-09T09:25:06.256796+0000","last_peered":"2024-05-09T09:25:06.256796+0000","last_clean":"2024-05-09T09:25:06.256796+0000","last_became_active":"2024-05-09T09:23:10.331332+0000","last_became_peered":"2024-05-09T09:23:10.331332+0000","last_unstale":"2024-05-09T09:25:06.256796+0000","last_undegraded":"2024-05-09T09:25:06.256796+0000","last_fullsized":"2024-05-09T09:25:06.256796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:51.643695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156538+0000","last_change":"2024-05-09T09:23:14.367539+0000","last_active":"2024-05-09T09:25:06.156538+0000","last_peered":"2024-05-09T09:25:06.156538+0000","last_clean":"2024-05-09T09:25:06.156538+0000","last_became_active":"2024-05-09T09:23:14.366806+0000","last_became_peered":"2024-05-09T09:23:14.366806+0000","last_unstale":"2024-05-09T09:25:06.156538+0000","last_undegraded":"2024-05-09T09:25:06.156538+0000","last_fullsized":"2024-05-09T09:25:06.156538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:58.679095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157306+0000","last_change":"2024-05-09T09:23:10.340849+0000","last_active":"2024-05-09T09:25:06.157306+0000","last_peered":"2024-05-09T09:25:06.157306+0000","last_clean":"2024-05-09T09:25:06.157306+0000","last_became_active":"2024-05-09T09:23:10.340430+0000","last_became_peered":"2024-05-09T09:23:10.340430+0000","last_unstale":"2024-05-09T09:25:06.157306+0000","last_undegraded":"2024-05-09T09:25:06.157306+0000","last_fullsized":"2024-05-09T09:25:06.157306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:33:47.827659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'7","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:34.765356+0000","last_change":"2024-05-09T09:23:09.308544+0000","last_active":"2024-05-09T09:25:34.765356+0000","last_peered":"2024-05-09T09:25:34.765356+0000","last_clean":"2024-05-09T09:25:34.765356+0000","last_became_active":"2024-05-09T09:23:09.308195+0000","last_became_peered":"2024-05-09T09:23:09.308195+0000","last_unstale":"2024-05-09T09:25:34.765356+0000","last_undegraded":"2024-05-09T09:25:34.765356+0000","last_fullsized":"2024-05-09T09:25:34.765356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:17:34.884385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161350+0000","last_change":"2024-05-09T09:24:05.755846+0000","last_active":"2024-05-09T09:25:06.161350+0000","last_peered":"2024-05-09T09:25:06.161350+0000","last_clean":"2024-05-09T09:25:06.161350+0000","last_became_active":"2024-05-09T09:24:05.747246+0000","last_became_peered":"2024-05-09T09:24:05.747246+0000","last_unstale":"2024-05-09T09:25:06.161350+0000","last_undegraded":"2024-05-09T09:25:06.161350+0000","last_fullsized":"2024-05-09T09:25:06.161350+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:23:40.544770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161873+0000","last_change":"2024-05-09T09:23:10.341320+0000","last_active":"2024-05-09T09:25:06.161873+0000","last_peered":"2024-05-09T09:25:06.161873+0000","last_clean":"2024-05-09T09:25:06.161873+0000","last_became_active":"2024-05-09T09:23:10.340868+0000","last_became_peered":"2024-05-09T09:23:10.340868+0000","last_unstale":"2024-05-09T09:25:06.161873+0000","last_undegraded":"2024-05-09T09:25:06.161873+0000","last_fullsized":"2024-05-09T09:25:06.161873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:55.846642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157331+0000","last_change":"2024-05-09T09:23:09.299574+0000","last_active":"2024-05-09T09:25:06.157331+0000","last_peered":"2024-05-09T09:25:06.157331+0000","last_clean":"2024-05-09T09:25:06.157331+0000","last_became_active":"2024-05-09T09:23:09.298878+0000","last_became_peered":"2024-05-09T09:23:09.298878+0000","last_unstale":"2024-05-09T09:25:06.157331+0000","last_undegraded":"2024-05-09T09:25:06.157331+0000","last_fullsized":"2024-05-09T09:25:06.157331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.275907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":352,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160988+0000","last_change":"2024-05-09T09:21:57.576132+0000","last_active":"2024-05-09T09:25:06.160988+0000","last_peered":"2024-05-09T09:25:06.160988+0000","last_clean":"2024-05-09T09:25:06.160988+0000","last_became_active":"2024-05-09T09:21:57.575602+0000","last_became_peered":"2024-05-09T09:21:57.575602+0000","last_unstale":"2024-05-09T09:25:06.160988+0000","last_undegraded":"2024-05-09T09:25:06.160988+0000","last_fullsized":"2024-05-09T09:25:06.160988+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_clean_scrub_stamp":"2024-05-09T09:18:31.335063+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:07.639787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157785+0000","last_change":"2024-05-09T09:23:14.371184+0000","last_active":"2024-05-09T09:25:06.157785+0000","last_peered":"2024-05-09T09:25:06.157785+0000","last_clean":"2024-05-09T09:25:06.157785+0000","last_became_active":"2024-05-09T09:23:14.370780+0000","last_became_peered":"2024-05-09T09:23:14.370780+0000","last_unstale":"2024-05-09T09:25:06.157785+0000","last_undegraded":"2024-05-09T09:25:06.157785+0000","last_fullsized":"2024-05-09T09:25:06.157785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:39.361947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.155976+0000","last_change":"2024-05-09T09:23:10.330915+0000","last_active":"2024-05-09T09:25:06.155976+0000","last_peered":"2024-05-09T09:25:06.155976+0000","last_clean":"2024-05-09T09:25:06.155976+0000","last_became_active":"2024-05-09T09:23:10.330458+0000","last_became_peered":"2024-05-09T09:23:10.330458+0000","last_unstale":"2024-05-09T09:25:06.155976+0000","last_undegraded":"2024-05-09T09:25:06.155976+0000","last_fullsized":"2024-05-09T09:25:06.155976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:10.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161228+0000","last_change":"2024-05-09T09:23:09.303895+0000","last_active":"2024-05-09T09:25:06.161228+0000","last_peered":"2024-05-09T09:25:06.161228+0000","last_clean":"2024-05-09T09:25:06.161228+0000","last_became_active":"2024-05-09T09:23:09.303555+0000","last_became_peered":"2024-05-09T09:23:09.303555+0000","last_unstale":"2024-05-09T09:25:06.161228+0000","last_undegraded":"2024-05-09T09:25:06.161228+0000","last_fullsized":"2024-05-09T09:25:06.161228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:04.350161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159708+0000","last_change":"2024-05-09T09:23:14.359390+0000","last_active":"2024-05-09T09:25:06.159708+0000","last_peered":"2024-05-09T09:25:06.159708+0000","last_clean":"2024-05-09T09:25:06.159708+0000","last_became_active":"2024-05-09T09:23:14.359000+0000","last_became_peered":"2024-05-09T09:23:14.359000+0000","last_unstale":"2024-05-09T09:25:06.159708+0000","last_undegraded":"2024-05-09T09:25:06.159708+0000","last_fullsized":"2024-05-09T09:25:06.159708+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:25.077835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160314+0000","last_change":"2024-05-09T09:23:09.304535+0000","last_active":"2024-05-09T09:25:06.160314+0000","last_peered":"2024-05-09T09:25:06.160314+0000","last_clean":"2024-05-09T09:25:06.160314+0000","last_became_active":"2024-05-09T09:23:09.304166+0000","last_became_peered":"2024-05-09T09:23:09.304166+0000","last_unstale":"2024-05-09T09:25:06.160314+0000","last_undegraded":"2024-05-09T09:25:06.160314+0000","last_fullsized":"2024-05-09T09:25:06.160314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:03:20.281620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156507+0000","last_change":"2024-05-09T09:23:10.335657+0000","last_active":"2024-05-09T09:25:06.156507+0000","last_peered":"2024-05-09T09:25:06.156507+0000","last_clean":"2024-05-09T09:25:06.156507+0000","last_became_active":"2024-05-09T09:23:10.335231+0000","last_became_peered":"2024-05-09T09:23:10.335231+0000","last_unstale":"2024-05-09T09:25:06.156507+0000","last_undegraded":"2024-05-09T09:25:06.156507+0000","last_fullsized":"2024-05-09T09:25:06.156507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:07.990165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161193+0000","last_change":"2024-05-09T09:23:14.366988+0000","last_active":"2024-05-09T09:25:06.161193+0000","last_peered":"2024-05-09T09:25:06.161193+0000","last_clean":"2024-05-09T09:25:06.161193+0000","last_became_active":"2024-05-09T09:23:14.366562+0000","last_became_peered":"2024-05-09T09:23:14.366562+0000","last_unstale":"2024-05-09T09:25:06.161193+0000","last_undegraded":"2024-05-09T09:25:06.161193+0000","last_fullsized":"2024-05-09T09:25:06.161193+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:42.670036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'15","reported_seq":62,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:18.524078+0000","last_change":"2024-05-09T09:24:05.752748+0000","last_active":"2024-05-09T09:25:18.524078+0000","last_peered":"2024-05-09T09:25:18.524078+0000","last_clean":"2024-05-09T09:25:18.524078+0000","last_became_active":"2024-05-09T09:24:05.743638+0000","last_became_peered":"2024-05-09T09:24:05.743638+0000","last_unstale":"2024-05-09T09:25:18.524078+0000","last_undegraded":"2024-05-09T09:25:18.524078+0000","last_fullsized":"2024-05-09T09:25:18.524078+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:01.834934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161808+0000","last_change":"2024-05-09T09:23:10.345840+0000","last_active":"2024-05-09T09:25:06.161808+0000","last_peered":"2024-05-09T09:25:06.161808+0000","last_clean":"2024-05-09T09:25:06.161808+0000","last_became_active":"2024-05-09T09:23:10.345487+0000","last_became_peered":"2024-05-09T09:23:10.345487+0000","last_unstale":"2024-05-09T09:25:06.161808+0000","last_undegraded":"2024-05-09T09:25:06.161808+0000","last_fullsized":"2024-05-09T09:25:06.161808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:47.723429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162727+0000","last_change":"2024-05-09T09:23:14.365485+0000","last_active":"2024-05-09T09:25:06.162727+0000","last_peered":"2024-05-09T09:25:06.162727+0000","last_clean":"2024-05-09T09:25:06.162727+0000","last_became_active":"2024-05-09T09:23:14.365018+0000","last_became_peered":"2024-05-09T09:23:14.365018+0000","last_unstale":"2024-05-09T09:25:06.162727+0000","last_undegraded":"2024-05-09T09:25:06.162727+0000","last_fullsized":"2024-05-09T09:25:06.162727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:54:04.712920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157667+0000","last_change":"2024-05-09T09:23:09.302365+0000","last_active":"2024-05-09T09:25:06.157667+0000","last_peered":"2024-05-09T09:25:06.157667+0000","last_clean":"2024-05-09T09:25:06.157667+0000","last_became_active":"2024-05-09T09:23:09.302008+0000","last_became_peered":"2024-05-09T09:23:09.302008+0000","last_unstale":"2024-05-09T09:25:06.157667+0000","last_undegraded":"2024-05-09T09:25:06.157667+0000","last_fullsized":"2024-05-09T09:25:06.157667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:48.829940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165695+0000","last_change":"2024-05-09T09:23:10.339170+0000","last_active":"2024-05-09T09:25:06.165695+0000","last_peered":"2024-05-09T09:25:06.165695+0000","last_clean":"2024-05-09T09:25:06.165695+0000","last_became_active":"2024-05-09T09:23:10.338829+0000","last_became_peered":"2024-05-09T09:23:10.338829+0000","last_unstale":"2024-05-09T09:25:06.165695+0000","last_undegraded":"2024-05-09T09:25:06.165695+0000","last_fullsized":"2024-05-09T09:25:06.165695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:04.510495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157578+0000","last_change":"2024-05-09T09:24:05.747227+0000","last_active":"2024-05-09T09:25:06.157578+0000","last_peered":"2024-05-09T09:25:06.157578+0000","last_clean":"2024-05-09T09:25:06.157578+0000","last_became_active":"2024-05-09T09:24:05.746338+0000","last_became_peered":"2024-05-09T09:24:05.746338+0000","last_unstale":"2024-05-09T09:25:06.157578+0000","last_undegraded":"2024-05-09T09:25:06.157578+0000","last_fullsized":"2024-05-09T09:25:06.157578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:20.899664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256404+0000","last_change":"2024-05-09T09:23:09.297418+0000","last_active":"2024-05-09T09:25:06.256404+0000","last_peered":"2024-05-09T09:25:06.256404+0000","last_clean":"2024-05-09T09:25:06.256404+0000","last_became_active":"2024-05-09T09:23:09.296963+0000","last_became_peered":"2024-05-09T09:23:09.296963+0000","last_unstale":"2024-05-09T09:25:06.256404+0000","last_undegraded":"2024-05-09T09:25:06.256404+0000","last_fullsized":"2024-05-09T09:25:06.256404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:54.552857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159481+0000","last_change":"2024-05-09T09:23:10.325882+0000","last_active":"2024-05-09T09:25:06.159481+0000","last_peered":"2024-05-09T09:25:06.159481+0000","last_clean":"2024-05-09T09:25:06.159481+0000","last_became_active":"2024-05-09T09:23:10.325494+0000","last_became_peered":"2024-05-09T09:23:10.325494+0000","last_unstale":"2024-05-09T09:25:06.159481+0000","last_undegraded":"2024-05-09T09:25:06.159481+0000","last_fullsized":"2024-05-09T09:25:06.159481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:27.191310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159617+0000","last_change":"2024-05-09T09:23:14.375572+0000","last_active":"2024-05-09T09:25:06.159617+0000","last_peered":"2024-05-09T09:25:06.159617+0000","last_clean":"2024-05-09T09:25:06.159617+0000","last_became_active":"2024-05-09T09:23:14.374069+0000","last_became_peered":"2024-05-09T09:23:14.374069+0000","last_unstale":"2024-05-09T09:25:06.159617+0000","last_undegraded":"2024-05-09T09:25:06.159617+0000","last_fullsized":"2024-05-09T09:25:06.159617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:44.045012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161469+0000","last_change":"2024-05-09T09:23:09.304137+0000","last_active":"2024-05-09T09:25:06.161469+0000","last_peered":"2024-05-09T09:25:06.161469+0000","last_clean":"2024-05-09T09:25:06.161469+0000","last_became_active":"2024-05-09T09:23:09.303762+0000","last_became_peered":"2024-05-09T09:23:09.303762+0000","last_unstale":"2024-05-09T09:25:06.161469+0000","last_undegraded":"2024-05-09T09:25:06.161469+0000","last_fullsized":"2024-05-09T09:25:06.161469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:56.811194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159736+0000","last_change":"2024-05-09T09:23:10.331998+0000","last_active":"2024-05-09T09:25:06.159736+0000","last_peered":"2024-05-09T09:25:06.159736+0000","last_clean":"2024-05-09T09:25:06.159736+0000","last_became_active":"2024-05-09T09:23:10.331642+0000","last_became_peered":"2024-05-09T09:23:10.331642+0000","last_unstale":"2024-05-09T09:25:06.159736+0000","last_undegraded":"2024-05-09T09:25:06.159736+0000","last_fullsized":"2024-05-09T09:25:06.159736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:41.569300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157348+0000","last_change":"2024-05-09T09:23:14.377444+0000","last_active":"2024-05-09T09:25:06.157348+0000","last_peered":"2024-05-09T09:25:06.157348+0000","last_clean":"2024-05-09T09:25:06.157348+0000","last_became_active":"2024-05-09T09:23:14.376189+0000","last_became_peered":"2024-05-09T09:23:14.376189+0000","last_unstale":"2024-05-09T09:25:06.157348+0000","last_undegraded":"2024-05-09T09:25:06.157348+0000","last_fullsized":"2024-05-09T09:25:06.157348+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:06.973535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:19.360581+0000","last_change":"2024-05-09T09:23:09.304101+0000","last_active":"2024-05-09T09:25:19.360581+0000","last_peered":"2024-05-09T09:25:19.360581+0000","last_clean":"2024-05-09T09:25:19.360581+0000","last_became_active":"2024-05-09T09:23:09.303743+0000","last_became_peered":"2024-05-09T09:23:09.303743+0000","last_unstale":"2024-05-09T09:25:19.360581+0000","last_undegraded":"2024-05-09T09:25:19.360581+0000","last_fullsized":"2024-05-09T09:25:19.360581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:39.339815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162614+0000","last_change":"2024-05-09T09:23:10.345938+0000","last_active":"2024-05-09T09:25:06.162614+0000","last_peered":"2024-05-09T09:25:06.162614+0000","last_clean":"2024-05-09T09:25:06.162614+0000","last_became_active":"2024-05-09T09:23:10.345576+0000","last_became_peered":"2024-05-09T09:23:10.345576+0000","last_unstale":"2024-05-09T09:25:06.162614+0000","last_undegraded":"2024-05-09T09:25:06.162614+0000","last_fullsized":"2024-05-09T09:25:06.162614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:08.571900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157437+0000","last_change":"2024-05-09T09:23:14.371549+0000","last_active":"2024-05-09T09:25:06.157437+0000","last_peered":"2024-05-09T09:25:06.157437+0000","last_clean":"2024-05-09T09:25:06.157437+0000","last_became_active":"2024-05-09T09:23:14.369931+0000","last_became_peered":"2024-05-09T09:23:14.369931+0000","last_unstale":"2024-05-09T09:25:06.157437+0000","last_undegraded":"2024-05-09T09:25:06.157437+0000","last_fullsized":"2024-05-09T09:25:06.157437+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:20.777172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159654+0000","last_change":"2024-05-09T09:23:09.302909+0000","last_active":"2024-05-09T09:25:06.159654+0000","last_peered":"2024-05-09T09:25:06.159654+0000","last_clean":"2024-05-09T09:25:06.159654+0000","last_became_active":"2024-05-09T09:23:09.302570+0000","last_became_peered":"2024-05-09T09:23:09.302570+0000","last_unstale":"2024-05-09T09:25:06.159654+0000","last_undegraded":"2024-05-09T09:25:06.159654+0000","last_fullsized":"2024-05-09T09:25:06.159654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:23.856755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161777+0000","last_change":"2024-05-09T09:23:10.342676+0000","last_active":"2024-05-09T09:25:06.161777+0000","last_peered":"2024-05-09T09:25:06.161777+0000","last_clean":"2024-05-09T09:25:06.161777+0000","last_became_active":"2024-05-09T09:23:10.341906+0000","last_became_peered":"2024-05-09T09:23:10.341906+0000","last_unstale":"2024-05-09T09:25:06.161777+0000","last_undegraded":"2024-05-09T09:25:06.161777+0000","last_fullsized":"2024-05-09T09:25:06.161777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:59:11.137648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256737+0000","last_change":"2024-05-09T09:23:14.363779+0000","last_active":"2024-05-09T09:25:06.256737+0000","last_peered":"2024-05-09T09:25:06.256737+0000","last_clean":"2024-05-09T09:25:06.256737+0000","last_became_active":"2024-05-09T09:23:14.363307+0000","last_became_peered":"2024-05-09T09:23:14.363307+0000","last_unstale":"2024-05-09T09:25:06.256737+0000","last_undegraded":"2024-05-09T09:25:06.256737+0000","last_fullsized":"2024-05-09T09:25:06.256737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:46.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"93'6","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:32.763660+0000","last_change":"2024-05-09T09:23:09.298094+0000","last_active":"2024-05-09T09:25:32.763660+0000","last_peered":"2024-05-09T09:25:32.763660+0000","last_clean":"2024-05-09T09:25:32.763660+0000","last_became_active":"2024-05-09T09:23:09.297777+0000","last_became_peered":"2024-05-09T09:23:09.297777+0000","last_unstale":"2024-05-09T09:25:32.763660+0000","last_undegraded":"2024-05-09T09:25:32.763660+0000","last_fullsized":"2024-05-09T09:25:32.763660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:25.099193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161899+0000","last_change":"2024-05-09T09:23:10.340120+0000","last_active":"2024-05-09T09:25:06.161899+0000","last_peered":"2024-05-09T09:25:06.161899+0000","last_clean":"2024-05-09T09:25:06.161899+0000","last_became_active":"2024-05-09T09:23:10.339690+0000","last_became_peered":"2024-05-09T09:23:10.339690+0000","last_unstale":"2024-05-09T09:25:06.161899+0000","last_undegraded":"2024-05-09T09:25:06.161899+0000","last_fullsized":"2024-05-09T09:25:06.161899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:36.783411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165251+0000","last_change":"2024-05-09T09:24:05.744892+0000","last_active":"2024-05-09T09:25:06.165251+0000","last_peered":"2024-05-09T09:25:06.165251+0000","last_clean":"2024-05-09T09:25:06.165251+0000","last_became_active":"2024-05-09T09:24:05.744437+0000","last_became_peered":"2024-05-09T09:24:05.744437+0000","last_unstale":"2024-05-09T09:25:06.165251+0000","last_undegraded":"2024-05-09T09:25:06.165251+0000","last_fullsized":"2024-05-09T09:25:06.165251+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:38:33.709821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157886+0000","last_change":"2024-05-09T09:23:09.307592+0000","last_active":"2024-05-09T09:25:06.157886+0000","last_peered":"2024-05-09T09:25:06.157886+0000","last_clean":"2024-05-09T09:25:06.157886+0000","last_became_active":"2024-05-09T09:23:09.307341+0000","last_became_peered":"2024-05-09T09:23:09.307341+0000","last_unstale":"2024-05-09T09:25:06.157886+0000","last_undegraded":"2024-05-09T09:25:06.157886+0000","last_fullsized":"2024-05-09T09:25:06.157886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:21.790980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161686+0000","last_change":"2024-05-09T09:23:10.349217+0000","last_active":"2024-05-09T09:25:06.161686+0000","last_peered":"2024-05-09T09:25:06.161686+0000","last_clean":"2024-05-09T09:25:06.161686+0000","last_became_active":"2024-05-09T09:23:10.348861+0000","last_became_peered":"2024-05-09T09:23:10.348861+0000","last_unstale":"2024-05-09T09:25:06.161686+0000","last_undegraded":"2024-05-09T09:25:06.161686+0000","last_fullsized":"2024-05-09T09:25:06.161686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:51.621045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157038+0000","last_change":"2024-05-09T09:23:14.363773+0000","last_active":"2024-05-09T09:25:06.157038+0000","last_peered":"2024-05-09T09:25:06.157038+0000","last_clean":"2024-05-09T09:25:06.157038+0000","last_became_active":"2024-05-09T09:23:14.363393+0000","last_became_peered":"2024-05-09T09:23:14.363393+0000","last_unstale":"2024-05-09T09:25:06.157038+0000","last_undegraded":"2024-05-09T09:25:06.157038+0000","last_fullsized":"2024-05-09T09:25:06.157038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:57.322985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156417+0000","last_change":"2024-05-09T09:23:09.308723+0000","last_active":"2024-05-09T09:25:06.156417+0000","last_peered":"2024-05-09T09:25:06.156417+0000","last_clean":"2024-05-09T09:25:06.156417+0000","last_became_active":"2024-05-09T09:23:09.308267+0000","last_became_peered":"2024-05-09T09:23:09.308267+0000","last_unstale":"2024-05-09T09:25:06.156417+0000","last_undegraded":"2024-05-09T09:25:06.156417+0000","last_fullsized":"2024-05-09T09:25:06.156417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:09:25.036421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162140+0000","last_change":"2024-05-09T09:23:10.342246+0000","last_active":"2024-05-09T09:25:06.162140+0000","last_peered":"2024-05-09T09:25:06.162140+0000","last_clean":"2024-05-09T09:25:06.162140+0000","last_became_active":"2024-05-09T09:23:10.341927+0000","last_became_peered":"2024-05-09T09:23:10.341927+0000","last_unstale":"2024-05-09T09:25:06.162140+0000","last_undegraded":"2024-05-09T09:25:06.162140+0000","last_fullsized":"2024-05-09T09:25:06.162140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:01.182226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159866+0000","last_change":"2024-05-09T09:23:10.332768+0000","last_active":"2024-05-09T09:25:06.159866+0000","last_peered":"2024-05-09T09:25:06.159866+0000","last_clean":"2024-05-09T09:25:06.159866+0000","last_became_active":"2024-05-09T09:23:10.332441+0000","last_became_peered":"2024-05-09T09:23:10.332441+0000","last_unstale":"2024-05-09T09:25:06.159866+0000","last_undegraded":"2024-05-09T09:25:06.159866+0000","last_fullsized":"2024-05-09T09:25:06.159866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:53.377058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164248+0000","last_change":"2024-05-09T09:23:10.337944+0000","last_active":"2024-05-09T09:25:06.164248+0000","last_peered":"2024-05-09T09:25:06.164248+0000","last_clean":"2024-05-09T09:25:06.164248+0000","last_became_active":"2024-05-09T09:23:10.337593+0000","last_became_peered":"2024-05-09T09:23:10.337593+0000","last_unstale":"2024-05-09T09:25:06.164248+0000","last_undegraded":"2024-05-09T09:25:06.164248+0000","last_fullsized":"2024-05-09T09:25:06.164248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:20.381543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.167531+0000","last_change":"2024-05-09T09:25:06.167114+0000","last_active":"2024-05-09T09:25:06.167531+0000","last_peered":"2024-05-09T09:25:06.167531+0000","last_clean":"2024-05-09T09:25:06.167531+0000","last_became_active":"2024-05-09T09:25:06.166712+0000","last_became_peered":"2024-05-09T09:25:06.166712+0000","last_unstale":"2024-05-09T09:25:06.167531+0000","last_undegraded":"2024-05-09T09:25:06.167531+0000","last_fullsized":"2024-05-09T09:25:06.167531+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:00.253528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165435+0000","last_change":"2024-05-09T09:23:10.340243+0000","last_active":"2024-05-09T09:25:06.165435+0000","last_peered":"2024-05-09T09:25:06.165435+0000","last_clean":"2024-05-09T09:25:06.165435+0000","last_became_active":"2024-05-09T09:23:10.339919+0000","last_became_peered":"2024-05-09T09:23:10.339919+0000","last_unstale":"2024-05-09T09:25:06.165435+0000","last_undegraded":"2024-05-09T09:25:06.165435+0000","last_fullsized":"2024-05-09T09:25:06.165435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:21.460928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156424+0000","last_change":"2024-05-09T09:23:10.333136+0000","last_active":"2024-05-09T09:25:06.156424+0000","last_peered":"2024-05-09T09:25:06.156424+0000","last_clean":"2024-05-09T09:25:06.156424+0000","last_became_active":"2024-05-09T09:23:10.332799+0000","last_became_peered":"2024-05-09T09:23:10.332799+0000","last_unstale":"2024-05-09T09:25:06.156424+0000","last_undegraded":"2024-05-09T09:25:06.156424+0000","last_fullsized":"2024-05-09T09:25:06.156424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:50.682747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160593+0000","last_change":"2024-05-09T09:24:05.745143+0000","last_active":"2024-05-09T09:25:06.160593+0000","last_peered":"2024-05-09T09:25:06.160593+0000","last_clean":"2024-05-09T09:25:06.160593+0000","last_became_active":"2024-05-09T09:24:05.744117+0000","last_became_peered":"2024-05-09T09:24:05.744117+0000","last_unstale":"2024-05-09T09:25:06.160593+0000","last_undegraded":"2024-05-09T09:25:06.160593+0000","last_fullsized":"2024-05-09T09:25:06.160593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:25:09.853820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164776+0000","last_change":"2024-05-09T09:23:10.341776+0000","last_active":"2024-05-09T09:25:06.164776+0000","last_peered":"2024-05-09T09:25:06.164776+0000","last_clean":"2024-05-09T09:25:06.164776+0000","last_became_active":"2024-05-09T09:23:10.341436+0000","last_became_peered":"2024-05-09T09:23:10.341436+0000","last_unstale":"2024-05-09T09:25:06.164776+0000","last_undegraded":"2024-05-09T09:25:06.164776+0000","last_fullsized":"2024-05-09T09:25:06.164776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:42.178856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161898+0000","last_change":"2024-05-09T09:24:05.745324+0000","last_active":"2024-05-09T09:25:06.161898+0000","last_peered":"2024-05-09T09:25:06.161898+0000","last_clean":"2024-05-09T09:25:06.161898+0000","last_became_active":"2024-05-09T09:24:05.744899+0000","last_became_peered":"2024-05-09T09:24:05.744899+0000","last_unstale":"2024-05-09T09:25:06.161898+0000","last_undegraded":"2024-05-09T09:25:06.161898+0000","last_fullsized":"2024-05-09T09:25:06.161898+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:59.372367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157167+0000","last_change":"2024-05-09T09:23:10.337117+0000","last_active":"2024-05-09T09:25:06.157167+0000","last_peered":"2024-05-09T09:25:06.157167+0000","last_clean":"2024-05-09T09:25:06.157167+0000","last_became_active":"2024-05-09T09:23:10.336672+0000","last_became_peered":"2024-05-09T09:23:10.336672+0000","last_unstale":"2024-05-09T09:25:06.157167+0000","last_undegraded":"2024-05-09T09:25:06.157167+0000","last_fullsized":"2024-05-09T09:25:06.157167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:59:20.723742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161586+0000","last_change":"2024-05-09T09:23:10.337733+0000","last_active":"2024-05-09T09:25:06.161586+0000","last_peered":"2024-05-09T09:25:06.161586+0000","last_clean":"2024-05-09T09:25:06.161586+0000","last_became_active":"2024-05-09T09:23:10.337343+0000","last_became_peered":"2024-05-09T09:23:10.337343+0000","last_unstale":"2024-05-09T09:25:06.161586+0000","last_undegraded":"2024-05-09T09:25:06.161586+0000","last_fullsized":"2024-05-09T09:25:06.161586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:53.447911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164449+0000","last_change":"2024-05-09T09:25:06.163856+0000","last_active":"2024-05-09T09:25:06.164449+0000","last_peered":"2024-05-09T09:25:06.164449+0000","last_clean":"2024-05-09T09:25:06.164449+0000","last_became_active":"2024-05-09T09:25:06.163367+0000","last_became_peered":"2024-05-09T09:25:06.163367+0000","last_unstale":"2024-05-09T09:25:06.164449+0000","last_undegraded":"2024-05-09T09:25:06.164449+0000","last_fullsized":"2024-05-09T09:25:06.164449+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:59.920586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,3],"acting":[9,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156375+0000","last_change":"2024-05-09T09:23:10.332271+0000","last_active":"2024-05-09T09:25:06.156375+0000","last_peered":"2024-05-09T09:25:06.156375+0000","last_clean":"2024-05-09T09:25:06.156375+0000","last_became_active":"2024-05-09T09:23:10.331941+0000","last_became_peered":"2024-05-09T09:23:10.331941+0000","last_unstale":"2024-05-09T09:25:06.156375+0000","last_undegraded":"2024-05-09T09:25:06.156375+0000","last_fullsized":"2024-05-09T09:25:06.156375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:27.707554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160211+0000","last_change":"2024-05-09T09:23:10.328965+0000","last_active":"2024-05-09T09:25:06.160211+0000","last_peered":"2024-05-09T09:25:06.160211+0000","last_clean":"2024-05-09T09:25:06.160211+0000","last_became_active":"2024-05-09T09:23:10.328618+0000","last_became_peered":"2024-05-09T09:23:10.328618+0000","last_unstale":"2024-05-09T09:25:06.160211+0000","last_undegraded":"2024-05-09T09:25:06.160211+0000","last_fullsized":"2024-05-09T09:25:06.160211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:16:02.003378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73024,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1255,"num_read_kb":707,"num_write":265,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":223386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":520504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28740,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716412,"statfs":{"total":95995035648,"available":95965605888,"internally_reserved":0,"allocated":1765376,"data_stored":1176046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":519941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":520319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1683456,"data_stored":1118436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":515938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":585380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":517394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40624,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704528,"statfs":{"total":95995035648,"available":95953436672,"internally_reserved":0,"allocated":1155072,"data_stored":581779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":515153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":520544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1112500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:25:36.871 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-09T09:25:36.871 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T09:25:37.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.012+0000 7f4f0a109640 1 -- 172.21.15.12:0/1316497332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f04101880 msgr2=0x7f4f04101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.012+0000 7f4f0a109640 1 --2- 172.21.15.12:0/1316497332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f04101880 0x7f4f04101ca0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009900 tx=0x7f4ef402a410 comp rx=0 tx=0).stop 2024-05-09T09:25:37.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.012+0000 7f4f0a109640 1 -- 172.21.15.12:0/1316497332 shutdown_connections 2024-05-09T09:25:37.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.012+0000 7f4f0a109640 1 --2- 172.21.15.12:0/1316497332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f04103430 0x7f4f04109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.012+0000 7f4f0a109640 1 --2- 172.21.15.12:0/1316497332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04102a70 0x7f4f04102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.012+0000 7f4f0a109640 1 --2- 172.21.15.12:0/1316497332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f04101880 0x7f4f04101ca0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.012+0000 7f4f0a109640 1 -- 172.21.15.12:0/1316497332 >> 172.21.15.12:0/1316497332 conn(0x7f4f040fce10 msgr2=0x7f4f040ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:37.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.012+0000 7f4f0a109640 1 -- 172.21.15.12:0/1316497332 shutdown_connections 2024-05-09T09:25:37.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.012+0000 7f4f0a109640 1 -- 172.21.15.12:0/1316497332 wait complete. 2024-05-09T09:25:37.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f0a109640 1 Processor -- start 2024-05-09T09:25:37.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f0a109640 1 -- start start 2024-05-09T09:25:37.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f0a109640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04101880 0x7f4f040755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:37.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f0a109640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f04102a70 0x7f4f04073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:37.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f0a109640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f04103430 0x7f4f04074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:37.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f0a109640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0410d4d0 con 0x7f4f04102a70 2024-05-09T09:25:37.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f0a109640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0410d350 con 0x7f4f04101880 2024-05-09T09:25:37.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f0a109640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0410d650 con 0x7f4f04103430 2024-05-09T09:25:37.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f02ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f04102a70 0x7f4f04073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:37.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f02ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f04102a70 0x7f4f04073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34000/0 (socket says 172.21.15.12:34000) 2024-05-09T09:25:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f02ffd640 1 -- 172.21.15.12:0/2536456711 learned_addr learned my addr 172.21.15.12:0/2536456711 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f03fff640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f04103430 0x7f4f04074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f037fe640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04101880 0x7f4f040755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f02ffd640 1 -- 172.21.15.12:0/2536456711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f04103430 msgr2=0x7f4f04074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f02ffd640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f04103430 0x7f4f04074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f02ffd640 1 -- 172.21.15.12:0/2536456711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04101880 msgr2=0x7f4f040755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f02ffd640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04101880 0x7f4f040755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.013+0000 7f4f02ffd640 1 -- 172.21.15.12:0/2536456711 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f040749f0 con 0x7f4f04102a70 2024-05-09T09:25:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.014+0000 7f4f02ffd640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f04102a70 0x7f4f04073c20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800d950 tx=0x7f4ef800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.014+0000 7f4f00ff9640 1 -- 172.21.15.12:0/2536456711 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8017070 con 0x7f4f04102a70 2024-05-09T09:25:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.014+0000 7f4f00ff9640 1 -- 172.21.15.12:0/2536456711 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4ef8009c30 con 0x7f4f04102a70 2024-05-09T09:25:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.014+0000 7f4f0a109640 1 -- 172.21.15.12:0/2536456711 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f041a1cf0 con 0x7f4f04102a70 2024-05-09T09:25:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.014+0000 7f4f00ff9640 1 -- 172.21.15.12:0/2536456711 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800f6b0 con 0x7f4f04102a70 2024-05-09T09:25:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.014+0000 7f4f0a109640 1 -- 172.21.15.12:0/2536456711 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f041a2190 con 0x7f4f04102a70 2024-05-09T09:25:37.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.015+0000 7f4f0a109640 1 -- 172.21.15.12:0/2536456711 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f04108c20 con 0x7f4f04102a70 2024-05-09T09:25:37.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.017+0000 7f4f00ff9640 1 -- 172.21.15.12:0/2536456711 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4ef8010a50 con 0x7f4f04102a70 2024-05-09T09:25:37.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.018+0000 7f4f00ff9640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4eec0a2b90 0x7f4eec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:37.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.018+0000 7f4f037fe640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4eec0a2b90 0x7f4eec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:37.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.018+0000 7f4f00ff9640 1 -- 172.21.15.12:0/2536456711 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f4ef80cf080 con 0x7f4f04102a70 2024-05-09T09:25:37.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.018+0000 7f4f037fe640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4eec0a2b90 0x7f4eec0a5050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4000c00 tx=0x7f4ef4031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:37.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.023+0000 7f4f00ff9640 1 -- 172.21.15.12:0/2536456711 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4ef8009de0 con 0x7f4f04102a70 2024-05-09T09:25:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.207+0000 7f4f0a109640 1 -- 172.21.15.12:0/2536456711 --> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f04061c70 con 0x7f4eec0a2b90 2024-05-09T09:25:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.216+0000 7f4f00ff9640 1 -- 172.21.15.12:0/2536456711 <== mgr.14150 v2:172.21.15.12:6800/3809515815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340543 (secure 0 0 0) 0x7f4f04061c70 con 0x7f4eec0a2b90 2024-05-09T09:25:37.220 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:37.222 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-09T09:25:37.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.220+0000 7f4f0a109640 1 -- 172.21.15.12:0/2536456711 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4eec0a2b90 msgr2=0x7f4eec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.220+0000 7f4f0a109640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4eec0a2b90 0x7f4eec0a5050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4000c00 tx=0x7f4ef4031510 comp rx=0 tx=0).stop 2024-05-09T09:25:37.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.220+0000 7f4f0a109640 1 -- 172.21.15.12:0/2536456711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f04102a70 msgr2=0x7f4f04073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.220+0000 7f4f0a109640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f04102a70 0x7f4f04073c20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800d950 tx=0x7f4ef800de10 comp rx=0 tx=0).stop 2024-05-09T09:25:37.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.220+0000 7f4f0a109640 1 -- 172.21.15.12:0/2536456711 shutdown_connections 2024-05-09T09:25:37.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.220+0000 7f4f0a109640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4eec0a2b90 0x7f4eec0a5050 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.220+0000 7f4f0a109640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f04103430 0x7f4f04074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.220+0000 7f4f0a109640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f04102a70 0x7f4f04073c20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.220+0000 7f4f0a109640 1 --2- 172.21.15.12:0/2536456711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04101880 0x7f4f040755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.220+0000 7f4f0a109640 1 -- 172.21.15.12:0/2536456711 >> 172.21.15.12:0/2536456711 conn(0x7f4f040fce10 msgr2=0x7f4f04105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:37.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.220+0000 7f4f0a109640 1 -- 172.21.15.12:0/2536456711 shutdown_connections 2024-05-09T09:25:37.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.220+0000 7f4f0a109640 1 -- 172.21.15.12:0/2536456711 wait complete. 2024-05-09T09:25:37.231 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-05-09T09:25:36.591134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663626,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1381,"num_read_kb":816,"num_write":448,"num_write_kb":3098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":868,"ondisk_log_size":868,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392900,"kb_used_data":14628,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124548924,"statfs":{"total":1151940427776,"available":1151538098176,"internally_reserved":0,"allocated":14979072,"data_stored":8203934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":60,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003534"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160720+0000","last_change":"2024-05-09T09:23:10.334738+0000","last_active":"2024-05-09T09:25:06.160720+0000","last_peered":"2024-05-09T09:25:06.160720+0000","last_clean":"2024-05-09T09:25:06.160720+0000","last_became_active":"2024-05-09T09:23:10.333406+0000","last_became_peered":"2024-05-09T09:23:10.333406+0000","last_unstale":"2024-05-09T09:25:06.160720+0000","last_undegraded":"2024-05-09T09:25:06.160720+0000","last_fullsized":"2024-05-09T09:25:06.160720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:44.957557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165459+0000","last_change":"2024-05-09T09:23:10.331860+0000","last_active":"2024-05-09T09:25:06.165459+0000","last_peered":"2024-05-09T09:25:06.165459+0000","last_clean":"2024-05-09T09:25:06.165459+0000","last_became_active":"2024-05-09T09:23:10.331396+0000","last_became_peered":"2024-05-09T09:23:10.331396+0000","last_unstale":"2024-05-09T09:25:06.165459+0000","last_undegraded":"2024-05-09T09:25:06.165459+0000","last_fullsized":"2024-05-09T09:25:06.165459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:47.676399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165707+0000","last_change":"2024-05-09T09:23:10.336543+0000","last_active":"2024-05-09T09:25:06.165707+0000","last_peered":"2024-05-09T09:25:06.165707+0000","last_clean":"2024-05-09T09:25:06.165707+0000","last_became_active":"2024-05-09T09:23:10.336189+0000","last_became_peered":"2024-05-09T09:23:10.336189+0000","last_unstale":"2024-05-09T09:25:06.165707+0000","last_undegraded":"2024-05-09T09:25:06.165707+0000","last_fullsized":"2024-05-09T09:25:06.165707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:06.419177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157538+0000","last_change":"2024-05-09T09:23:10.331739+0000","last_active":"2024-05-09T09:25:06.157538+0000","last_peered":"2024-05-09T09:25:06.157538+0000","last_clean":"2024-05-09T09:25:06.157538+0000","last_became_active":"2024-05-09T09:23:10.331333+0000","last_became_peered":"2024-05-09T09:23:10.331333+0000","last_unstale":"2024-05-09T09:25:06.157538+0000","last_undegraded":"2024-05-09T09:25:06.157538+0000","last_fullsized":"2024-05-09T09:25:06.157538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:04.925697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160387+0000","last_change":"2024-05-09T09:23:10.345294+0000","last_active":"2024-05-09T09:25:06.160387+0000","last_peered":"2024-05-09T09:25:06.160387+0000","last_clean":"2024-05-09T09:25:06.160387+0000","last_became_active":"2024-05-09T09:23:10.344940+0000","last_became_peered":"2024-05-09T09:23:10.344940+0000","last_unstale":"2024-05-09T09:25:06.160387+0000","last_undegraded":"2024-05-09T09:25:06.160387+0000","last_fullsized":"2024-05-09T09:25:06.160387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:41.150284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256906+0000","last_change":"2024-05-09T09:23:10.329143+0000","last_active":"2024-05-09T09:25:06.256906+0000","last_peered":"2024-05-09T09:25:06.256906+0000","last_clean":"2024-05-09T09:25:06.256906+0000","last_became_active":"2024-05-09T09:23:10.328712+0000","last_became_peered":"2024-05-09T09:23:10.328712+0000","last_unstale":"2024-05-09T09:25:06.256906+0000","last_undegraded":"2024-05-09T09:25:06.256906+0000","last_fullsized":"2024-05-09T09:25:06.256906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:10:35.085759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161026+0000","last_change":"2024-05-09T09:23:10.338176+0000","last_active":"2024-05-09T09:25:06.161026+0000","last_peered":"2024-05-09T09:25:06.161026+0000","last_clean":"2024-05-09T09:25:06.161026+0000","last_became_active":"2024-05-09T09:23:10.337845+0000","last_became_peered":"2024-05-09T09:23:10.337845+0000","last_unstale":"2024-05-09T09:25:06.161026+0000","last_undegraded":"2024-05-09T09:25:06.161026+0000","last_fullsized":"2024-05-09T09:25:06.161026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:34:21.783295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160809+0000","last_change":"2024-05-09T09:23:10.342473+0000","last_active":"2024-05-09T09:25:06.160809+0000","last_peered":"2024-05-09T09:25:06.160809+0000","last_clean":"2024-05-09T09:25:06.160809+0000","last_became_active":"2024-05-09T09:23:10.340682+0000","last_became_peered":"2024-05-09T09:23:10.340682+0000","last_unstale":"2024-05-09T09:25:06.160809+0000","last_undegraded":"2024-05-09T09:25:06.160809+0000","last_fullsized":"2024-05-09T09:25:06.160809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:56.730332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257663+0000","last_change":"2024-05-09T09:23:10.339888+0000","last_active":"2024-05-09T09:25:06.257663+0000","last_peered":"2024-05-09T09:25:06.257663+0000","last_clean":"2024-05-09T09:25:06.257663+0000","last_became_active":"2024-05-09T09:23:10.337849+0000","last_became_peered":"2024-05-09T09:23:10.337849+0000","last_unstale":"2024-05-09T09:25:06.257663+0000","last_undegraded":"2024-05-09T09:25:06.257663+0000","last_fullsized":"2024-05-09T09:25:06.257663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:48.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257800+0000","last_change":"2024-05-09T09:23:10.332395+0000","last_active":"2024-05-09T09:25:06.257800+0000","last_peered":"2024-05-09T09:25:06.257800+0000","last_clean":"2024-05-09T09:25:06.257800+0000","last_became_active":"2024-05-09T09:23:10.332040+0000","last_became_peered":"2024-05-09T09:23:10.332040+0000","last_unstale":"2024-05-09T09:25:06.257800+0000","last_undegraded":"2024-05-09T09:25:06.257800+0000","last_fullsized":"2024-05-09T09:25:06.257800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:04.036871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161826+0000","last_change":"2024-05-09T09:23:10.341148+0000","last_active":"2024-05-09T09:25:06.161826+0000","last_peered":"2024-05-09T09:25:06.161826+0000","last_clean":"2024-05-09T09:25:06.161826+0000","last_became_active":"2024-05-09T09:23:10.340799+0000","last_became_peered":"2024-05-09T09:23:10.340799+0000","last_unstale":"2024-05-09T09:25:06.161826+0000","last_undegraded":"2024-05-09T09:25:06.161826+0000","last_fullsized":"2024-05-09T09:25:06.161826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:31.020659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157609+0000","last_change":"2024-05-09T09:23:10.336191+0000","last_active":"2024-05-09T09:25:06.157609+0000","last_peered":"2024-05-09T09:25:06.157609+0000","last_clean":"2024-05-09T09:25:06.157609+0000","last_became_active":"2024-05-09T09:23:10.333591+0000","last_became_peered":"2024-05-09T09:23:10.333591+0000","last_unstale":"2024-05-09T09:25:06.157609+0000","last_undegraded":"2024-05-09T09:25:06.157609+0000","last_fullsized":"2024-05-09T09:25:06.157609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:24.836954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156258+0000","last_change":"2024-05-09T09:23:10.314506+0000","last_active":"2024-05-09T09:25:06.156258+0000","last_peered":"2024-05-09T09:25:06.156258+0000","last_clean":"2024-05-09T09:25:06.156258+0000","last_became_active":"2024-05-09T09:23:10.314171+0000","last_became_peered":"2024-05-09T09:23:10.314171+0000","last_unstale":"2024-05-09T09:25:06.156258+0000","last_undegraded":"2024-05-09T09:25:06.156258+0000","last_fullsized":"2024-05-09T09:25:06.156258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:51.088814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165407+0000","last_change":"2024-05-09T09:23:10.341670+0000","last_active":"2024-05-09T09:25:06.165407+0000","last_peered":"2024-05-09T09:25:06.165407+0000","last_clean":"2024-05-09T09:25:06.165407+0000","last_became_active":"2024-05-09T09:23:10.341155+0000","last_became_peered":"2024-05-09T09:23:10.341155+0000","last_unstale":"2024-05-09T09:25:06.165407+0000","last_undegraded":"2024-05-09T09:25:06.165407+0000","last_fullsized":"2024-05-09T09:25:06.165407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:14:32.734302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165578+0000","last_change":"2024-05-09T09:24:05.742117+0000","last_active":"2024-05-09T09:25:06.165578+0000","last_peered":"2024-05-09T09:25:06.165578+0000","last_clean":"2024-05-09T09:25:06.165578+0000","last_became_active":"2024-05-09T09:24:05.741655+0000","last_became_peered":"2024-05-09T09:24:05.741655+0000","last_unstale":"2024-05-09T09:25:06.165578+0000","last_undegraded":"2024-05-09T09:25:06.165578+0000","last_fullsized":"2024-05-09T09:25:06.165578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:11.468527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256668+0000","last_change":"2024-05-09T09:23:10.337367+0000","last_active":"2024-05-09T09:25:06.256668+0000","last_peered":"2024-05-09T09:25:06.256668+0000","last_clean":"2024-05-09T09:25:06.256668+0000","last_became_active":"2024-05-09T09:23:10.337000+0000","last_became_peered":"2024-05-09T09:23:10.337000+0000","last_unstale":"2024-05-09T09:25:06.256668+0000","last_undegraded":"2024-05-09T09:25:06.256668+0000","last_fullsized":"2024-05-09T09:25:06.256668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:54.431820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160278+0000","last_change":"2024-05-09T09:23:10.334293+0000","last_active":"2024-05-09T09:25:06.160278+0000","last_peered":"2024-05-09T09:25:06.160278+0000","last_clean":"2024-05-09T09:25:06.160278+0000","last_became_active":"2024-05-09T09:23:10.333938+0000","last_became_peered":"2024-05-09T09:23:10.333938+0000","last_unstale":"2024-05-09T09:25:06.160278+0000","last_undegraded":"2024-05-09T09:25:06.160278+0000","last_fullsized":"2024-05-09T09:25:06.160278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:56:09.048848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159638+0000","last_change":"2024-05-09T09:23:10.332394+0000","last_active":"2024-05-09T09:25:06.159638+0000","last_peered":"2024-05-09T09:25:06.159638+0000","last_clean":"2024-05-09T09:25:06.159638+0000","last_became_active":"2024-05-09T09:23:10.332054+0000","last_became_peered":"2024-05-09T09:23:10.332054+0000","last_unstale":"2024-05-09T09:25:06.159638+0000","last_undegraded":"2024-05-09T09:25:06.159638+0000","last_fullsized":"2024-05-09T09:25:06.159638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:05.628740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164212+0000","last_change":"2024-05-09T09:23:10.336189+0000","last_active":"2024-05-09T09:25:06.164212+0000","last_peered":"2024-05-09T09:25:06.164212+0000","last_clean":"2024-05-09T09:25:06.164212+0000","last_became_active":"2024-05-09T09:23:10.335698+0000","last_became_peered":"2024-05-09T09:23:10.335698+0000","last_unstale":"2024-05-09T09:25:06.164212+0000","last_undegraded":"2024-05-09T09:25:06.164212+0000","last_fullsized":"2024-05-09T09:25:06.164212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:01.403918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162866+0000","last_change":"2024-05-09T09:23:14.372134+0000","last_active":"2024-05-09T09:25:06.162866+0000","last_peered":"2024-05-09T09:25:06.162866+0000","last_clean":"2024-05-09T09:25:06.162866+0000","last_became_active":"2024-05-09T09:23:14.371466+0000","last_became_peered":"2024-05-09T09:23:14.371466+0000","last_unstale":"2024-05-09T09:25:06.162866+0000","last_undegraded":"2024-05-09T09:25:06.162866+0000","last_fullsized":"2024-05-09T09:25:06.162866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:22.048565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:31.760845+0000","last_change":"2024-05-09T09:23:09.304421+0000","last_active":"2024-05-09T09:25:31.760845+0000","last_peered":"2024-05-09T09:25:31.760845+0000","last_clean":"2024-05-09T09:25:31.760845+0000","last_became_active":"2024-05-09T09:23:09.304109+0000","last_became_peered":"2024-05-09T09:23:09.304109+0000","last_unstale":"2024-05-09T09:25:31.760845+0000","last_undegraded":"2024-05-09T09:25:31.760845+0000","last_fullsized":"2024-05-09T09:25:31.760845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:36.796458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162409+0000","last_change":"2024-05-09T09:23:10.341875+0000","last_active":"2024-05-09T09:25:06.162409+0000","last_peered":"2024-05-09T09:25:06.162409+0000","last_clean":"2024-05-09T09:25:06.162409+0000","last_became_active":"2024-05-09T09:23:10.341385+0000","last_became_peered":"2024-05-09T09:23:10.341385+0000","last_unstale":"2024-05-09T09:25:06.162409+0000","last_undegraded":"2024-05-09T09:25:06.162409+0000","last_fullsized":"2024-05-09T09:25:06.162409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:44:33.939722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156972+0000","last_change":"2024-05-09T09:23:14.371120+0000","last_active":"2024-05-09T09:25:06.156972+0000","last_peered":"2024-05-09T09:25:06.156972+0000","last_clean":"2024-05-09T09:25:06.156972+0000","last_became_active":"2024-05-09T09:23:14.370663+0000","last_became_peered":"2024-05-09T09:23:14.370663+0000","last_unstale":"2024-05-09T09:25:06.156972+0000","last_undegraded":"2024-05-09T09:25:06.156972+0000","last_fullsized":"2024-05-09T09:25:06.156972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:50:06.553518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162781+0000","last_change":"2024-05-09T09:23:09.304357+0000","last_active":"2024-05-09T09:25:06.162781+0000","last_peered":"2024-05-09T09:25:06.162781+0000","last_clean":"2024-05-09T09:25:06.162781+0000","last_became_active":"2024-05-09T09:23:09.304007+0000","last_became_peered":"2024-05-09T09:23:09.304007+0000","last_unstale":"2024-05-09T09:25:06.162781+0000","last_undegraded":"2024-05-09T09:25:06.162781+0000","last_fullsized":"2024-05-09T09:25:06.162781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:32:46.132287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165838+0000","last_change":"2024-05-09T09:23:10.339533+0000","last_active":"2024-05-09T09:25:06.165838+0000","last_peered":"2024-05-09T09:25:06.165838+0000","last_clean":"2024-05-09T09:25:06.165838+0000","last_became_active":"2024-05-09T09:23:10.337142+0000","last_became_peered":"2024-05-09T09:23:10.337142+0000","last_unstale":"2024-05-09T09:25:06.165838+0000","last_undegraded":"2024-05-09T09:25:06.165838+0000","last_fullsized":"2024-05-09T09:25:06.165838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:11:11.921491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157173+0000","last_change":"2024-05-09T09:24:05.744009+0000","last_active":"2024-05-09T09:25:06.157173+0000","last_peered":"2024-05-09T09:25:06.157173+0000","last_clean":"2024-05-09T09:25:06.157173+0000","last_became_active":"2024-05-09T09:24:05.743611+0000","last_became_peered":"2024-05-09T09:24:05.743611+0000","last_unstale":"2024-05-09T09:25:06.157173+0000","last_undegraded":"2024-05-09T09:25:06.157173+0000","last_fullsized":"2024-05-09T09:25:06.157173+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:55.775740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160457+0000","last_change":"2024-05-09T09:23:09.300754+0000","last_active":"2024-05-09T09:25:06.160457+0000","last_peered":"2024-05-09T09:25:06.160457+0000","last_clean":"2024-05-09T09:25:06.160457+0000","last_became_active":"2024-05-09T09:23:09.300419+0000","last_became_peered":"2024-05-09T09:23:09.300419+0000","last_unstale":"2024-05-09T09:25:06.160457+0000","last_undegraded":"2024-05-09T09:25:06.160457+0000","last_fullsized":"2024-05-09T09:25:06.160457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:06.205582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257117+0000","last_change":"2024-05-09T09:23:10.331952+0000","last_active":"2024-05-09T09:25:06.257117+0000","last_peered":"2024-05-09T09:25:06.257117+0000","last_clean":"2024-05-09T09:25:06.257117+0000","last_became_active":"2024-05-09T09:23:10.331515+0000","last_became_peered":"2024-05-09T09:23:10.331515+0000","last_unstale":"2024-05-09T09:25:06.257117+0000","last_undegraded":"2024-05-09T09:25:06.257117+0000","last_fullsized":"2024-05-09T09:25:06.257117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:23.809672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156952+0000","last_change":"2024-05-09T09:23:14.368518+0000","last_active":"2024-05-09T09:25:06.156952+0000","last_peered":"2024-05-09T09:25:06.156952+0000","last_clean":"2024-05-09T09:25:06.156952+0000","last_became_active":"2024-05-09T09:23:14.367995+0000","last_became_peered":"2024-05-09T09:23:14.367995+0000","last_unstale":"2024-05-09T09:25:06.156952+0000","last_undegraded":"2024-05-09T09:25:06.156952+0000","last_fullsized":"2024-05-09T09:25:06.156952+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:11.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160465+0000","last_change":"2024-05-09T09:23:09.299656+0000","last_active":"2024-05-09T09:25:06.160465+0000","last_peered":"2024-05-09T09:25:06.160465+0000","last_clean":"2024-05-09T09:25:06.160465+0000","last_became_active":"2024-05-09T09:23:09.299374+0000","last_became_peered":"2024-05-09T09:23:09.299374+0000","last_unstale":"2024-05-09T09:25:06.160465+0000","last_undegraded":"2024-05-09T09:25:06.160465+0000","last_fullsized":"2024-05-09T09:25:06.160465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:05.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157723+0000","last_change":"2024-05-09T09:23:10.341933+0000","last_active":"2024-05-09T09:25:06.157723+0000","last_peered":"2024-05-09T09:25:06.157723+0000","last_clean":"2024-05-09T09:25:06.157723+0000","last_became_active":"2024-05-09T09:23:10.341621+0000","last_became_peered":"2024-05-09T09:23:10.341621+0000","last_unstale":"2024-05-09T09:25:06.157723+0000","last_undegraded":"2024-05-09T09:25:06.157723+0000","last_fullsized":"2024-05-09T09:25:06.157723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:31.938768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.163011+0000","last_change":"2024-05-09T09:23:14.364783+0000","last_active":"2024-05-09T09:25:06.163011+0000","last_peered":"2024-05-09T09:25:06.163011+0000","last_clean":"2024-05-09T09:25:06.163011+0000","last_became_active":"2024-05-09T09:23:14.364309+0000","last_became_peered":"2024-05-09T09:23:14.364309+0000","last_unstale":"2024-05-09T09:25:06.163011+0000","last_undegraded":"2024-05-09T09:25:06.163011+0000","last_fullsized":"2024-05-09T09:25:06.163011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:44.252111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'6","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:32.763334+0000","last_change":"2024-05-09T09:23:09.297913+0000","last_active":"2024-05-09T09:25:32.763334+0000","last_peered":"2024-05-09T09:25:32.763334+0000","last_clean":"2024-05-09T09:25:32.763334+0000","last_became_active":"2024-05-09T09:23:09.297612+0000","last_became_peered":"2024-05-09T09:23:09.297612+0000","last_unstale":"2024-05-09T09:25:32.763334+0000","last_undegraded":"2024-05-09T09:25:32.763334+0000","last_fullsized":"2024-05-09T09:25:32.763334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:21.808703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.163905+0000","last_change":"2024-05-09T09:25:06.163905+0000","last_active":"2024-05-09T09:25:06.163905+0000","last_peered":"2024-05-09T09:25:06.163905+0000","last_clean":"2024-05-09T09:25:06.163905+0000","last_became_active":"2024-05-09T09:25:06.163454+0000","last_became_peered":"2024-05-09T09:25:06.163454+0000","last_unstale":"2024-05-09T09:25:06.163905+0000","last_undegraded":"2024-05-09T09:25:06.163905+0000","last_fullsized":"2024-05-09T09:25:06.163905+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:49:05.654792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162936+0000","last_change":"2024-05-09T09:23:14.371781+0000","last_active":"2024-05-09T09:25:06.162936+0000","last_peered":"2024-05-09T09:25:06.162936+0000","last_clean":"2024-05-09T09:25:06.162936+0000","last_became_active":"2024-05-09T09:23:14.371286+0000","last_became_peered":"2024-05-09T09:23:14.371286+0000","last_unstale":"2024-05-09T09:25:06.162936+0000","last_undegraded":"2024-05-09T09:25:06.162936+0000","last_fullsized":"2024-05-09T09:25:06.162936+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:26:29.185064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256852+0000","last_change":"2024-05-09T09:23:09.303786+0000","last_active":"2024-05-09T09:25:06.256852+0000","last_peered":"2024-05-09T09:25:06.256852+0000","last_clean":"2024-05-09T09:25:06.256852+0000","last_became_active":"2024-05-09T09:23:09.303475+0000","last_became_peered":"2024-05-09T09:23:09.303475+0000","last_unstale":"2024-05-09T09:25:06.256852+0000","last_undegraded":"2024-05-09T09:25:06.256852+0000","last_fullsized":"2024-05-09T09:25:06.256852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:28.833055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157664+0000","last_change":"2024-05-09T09:23:10.344643+0000","last_active":"2024-05-09T09:25:06.157664+0000","last_peered":"2024-05-09T09:25:06.157664+0000","last_clean":"2024-05-09T09:25:06.157664+0000","last_became_active":"2024-05-09T09:23:10.344314+0000","last_became_peered":"2024-05-09T09:23:10.344314+0000","last_unstale":"2024-05-09T09:25:06.157664+0000","last_undegraded":"2024-05-09T09:25:06.157664+0000","last_fullsized":"2024-05-09T09:25:06.157664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:15.147107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160150+0000","last_change":"2024-05-09T09:23:14.361170+0000","last_active":"2024-05-09T09:25:06.160150+0000","last_peered":"2024-05-09T09:25:06.160150+0000","last_clean":"2024-05-09T09:25:06.160150+0000","last_became_active":"2024-05-09T09:23:14.360767+0000","last_became_peered":"2024-05-09T09:23:14.360767+0000","last_unstale":"2024-05-09T09:25:06.160150+0000","last_undegraded":"2024-05-09T09:25:06.160150+0000","last_fullsized":"2024-05-09T09:25:06.160150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:59:21.393148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156074+0000","last_change":"2024-05-09T09:23:09.300757+0000","last_active":"2024-05-09T09:25:06.156074+0000","last_peered":"2024-05-09T09:25:06.156074+0000","last_clean":"2024-05-09T09:25:06.156074+0000","last_became_active":"2024-05-09T09:23:09.300328+0000","last_became_peered":"2024-05-09T09:23:09.300328+0000","last_unstale":"2024-05-09T09:25:06.156074+0000","last_undegraded":"2024-05-09T09:25:06.156074+0000","last_fullsized":"2024-05-09T09:25:06.156074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.553878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161040+0000","last_change":"2024-05-09T09:23:10.336077+0000","last_active":"2024-05-09T09:25:06.161040+0000","last_peered":"2024-05-09T09:25:06.161040+0000","last_clean":"2024-05-09T09:25:06.161040+0000","last_became_active":"2024-05-09T09:23:10.335762+0000","last_became_peered":"2024-05-09T09:23:10.335762+0000","last_unstale":"2024-05-09T09:25:06.161040+0000","last_undegraded":"2024-05-09T09:25:06.161040+0000","last_fullsized":"2024-05-09T09:25:06.161040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:48.063385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165335+0000","last_change":"2024-05-09T09:23:14.371413+0000","last_active":"2024-05-09T09:25:06.165335+0000","last_peered":"2024-05-09T09:25:06.165335+0000","last_clean":"2024-05-09T09:25:06.165335+0000","last_became_active":"2024-05-09T09:23:14.370018+0000","last_became_peered":"2024-05-09T09:23:14.370018+0000","last_unstale":"2024-05-09T09:25:06.165335+0000","last_undegraded":"2024-05-09T09:25:06.165335+0000","last_fullsized":"2024-05-09T09:25:06.165335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:18.475976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157139+0000","last_change":"2024-05-09T09:23:09.297328+0000","last_active":"2024-05-09T09:25:06.157139+0000","last_peered":"2024-05-09T09:25:06.157139+0000","last_clean":"2024-05-09T09:25:06.157139+0000","last_became_active":"2024-05-09T09:23:09.297001+0000","last_became_peered":"2024-05-09T09:23:09.297001+0000","last_unstale":"2024-05-09T09:25:06.157139+0000","last_undegraded":"2024-05-09T09:25:06.157139+0000","last_fullsized":"2024-05-09T09:25:06.157139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:19.241362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159057+0000","last_change":"2024-05-09T09:23:10.338342+0000","last_active":"2024-05-09T09:25:06.159057+0000","last_peered":"2024-05-09T09:25:06.159057+0000","last_clean":"2024-05-09T09:25:06.159057+0000","last_became_active":"2024-05-09T09:23:10.337995+0000","last_became_peered":"2024-05-09T09:23:10.337995+0000","last_unstale":"2024-05-09T09:25:06.159057+0000","last_undegraded":"2024-05-09T09:25:06.159057+0000","last_fullsized":"2024-05-09T09:25:06.159057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.691862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161348+0000","last_change":"2024-05-09T09:23:14.366237+0000","last_active":"2024-05-09T09:25:06.161348+0000","last_peered":"2024-05-09T09:25:06.161348+0000","last_clean":"2024-05-09T09:25:06.161348+0000","last_became_active":"2024-05-09T09:23:14.365809+0000","last_became_peered":"2024-05-09T09:23:14.365809+0000","last_unstale":"2024-05-09T09:25:06.161348+0000","last_undegraded":"2024-05-09T09:25:06.161348+0000","last_fullsized":"2024-05-09T09:25:06.161348+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:28.813467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":279,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:33.764231+0000","last_change":"2024-05-09T09:23:09.298658+0000","last_active":"2024-05-09T09:25:33.764231+0000","last_peered":"2024-05-09T09:25:33.764231+0000","last_clean":"2024-05-09T09:25:33.764231+0000","last_became_active":"2024-05-09T09:23:09.298325+0000","last_became_peered":"2024-05-09T09:23:09.298325+0000","last_unstale":"2024-05-09T09:25:33.764231+0000","last_undegraded":"2024-05-09T09:25:33.764231+0000","last_fullsized":"2024-05-09T09:25:33.764231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:22.108202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":242,"num_read_kb":242,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257522+0000","last_change":"2024-05-09T09:23:10.326865+0000","last_active":"2024-05-09T09:25:06.257522+0000","last_peered":"2024-05-09T09:25:06.257522+0000","last_clean":"2024-05-09T09:25:06.257522+0000","last_became_active":"2024-05-09T09:23:10.326478+0000","last_became_peered":"2024-05-09T09:23:10.326478+0000","last_unstale":"2024-05-09T09:25:06.257522+0000","last_undegraded":"2024-05-09T09:25:06.257522+0000","last_fullsized":"2024-05-09T09:25:06.257522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:54.521226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.158982+0000","last_change":"2024-05-09T09:23:14.373601+0000","last_active":"2024-05-09T09:25:06.158982+0000","last_peered":"2024-05-09T09:25:06.158982+0000","last_clean":"2024-05-09T09:25:06.158982+0000","last_became_active":"2024-05-09T09:23:14.372764+0000","last_became_peered":"2024-05-09T09:23:14.372764+0000","last_unstale":"2024-05-09T09:25:06.158982+0000","last_undegraded":"2024-05-09T09:25:06.158982+0000","last_fullsized":"2024-05-09T09:25:06.158982+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:59.496755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162066+0000","last_change":"2024-05-09T09:23:09.302293+0000","last_active":"2024-05-09T09:25:06.162066+0000","last_peered":"2024-05-09T09:25:06.162066+0000","last_clean":"2024-05-09T09:25:06.162066+0000","last_became_active":"2024-05-09T09:23:09.301216+0000","last_became_peered":"2024-05-09T09:23:09.301216+0000","last_unstale":"2024-05-09T09:25:06.162066+0000","last_undegraded":"2024-05-09T09:25:06.162066+0000","last_fullsized":"2024-05-09T09:25:06.162066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:35.146860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165710+0000","last_change":"2024-05-09T09:23:10.339827+0000","last_active":"2024-05-09T09:25:06.165710+0000","last_peered":"2024-05-09T09:25:06.165710+0000","last_clean":"2024-05-09T09:25:06.165710+0000","last_became_active":"2024-05-09T09:23:10.339415+0000","last_became_peered":"2024-05-09T09:23:10.339415+0000","last_unstale":"2024-05-09T09:25:06.165710+0000","last_undegraded":"2024-05-09T09:25:06.165710+0000","last_fullsized":"2024-05-09T09:25:06.165710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:45.250126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157361+0000","last_change":"2024-05-09T09:23:14.377001+0000","last_active":"2024-05-09T09:25:06.157361+0000","last_peered":"2024-05-09T09:25:06.157361+0000","last_clean":"2024-05-09T09:25:06.157361+0000","last_became_active":"2024-05-09T09:23:14.376402+0000","last_became_peered":"2024-05-09T09:23:14.376402+0000","last_unstale":"2024-05-09T09:25:06.157361+0000","last_undegraded":"2024-05-09T09:25:06.157361+0000","last_fullsized":"2024-05-09T09:25:06.157361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:08.142495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.158215+0000","last_change":"2024-05-09T09:23:09.299331+0000","last_active":"2024-05-09T09:25:06.158215+0000","last_peered":"2024-05-09T09:25:06.158215+0000","last_clean":"2024-05-09T09:25:06.158215+0000","last_became_active":"2024-05-09T09:23:09.299017+0000","last_became_peered":"2024-05-09T09:23:09.299017+0000","last_unstale":"2024-05-09T09:25:06.158215+0000","last_undegraded":"2024-05-09T09:25:06.158215+0000","last_fullsized":"2024-05-09T09:25:06.158215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:28.328704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257442+0000","last_change":"2024-05-09T09:24:05.743586+0000","last_active":"2024-05-09T09:25:06.257442+0000","last_peered":"2024-05-09T09:25:06.257442+0000","last_clean":"2024-05-09T09:25:06.257442+0000","last_became_active":"2024-05-09T09:24:05.742610+0000","last_became_peered":"2024-05-09T09:24:05.742610+0000","last_unstale":"2024-05-09T09:25:06.257442+0000","last_undegraded":"2024-05-09T09:25:06.257442+0000","last_fullsized":"2024-05-09T09:25:06.257442+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:05.586985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159196+0000","last_change":"2024-05-09T09:23:14.363362+0000","last_active":"2024-05-09T09:25:06.159196+0000","last_peered":"2024-05-09T09:25:06.159196+0000","last_clean":"2024-05-09T09:25:06.159196+0000","last_became_active":"2024-05-09T09:23:14.362926+0000","last_became_peered":"2024-05-09T09:23:14.362926+0000","last_unstale":"2024-05-09T09:25:06.159196+0000","last_undegraded":"2024-05-09T09:25:06.159196+0000","last_fullsized":"2024-05-09T09:25:06.159196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:25.339727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":321,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:33.763965+0000","last_change":"2024-05-09T09:23:09.297814+0000","last_active":"2024-05-09T09:25:33.763965+0000","last_peered":"2024-05-09T09:25:33.763965+0000","last_clean":"2024-05-09T09:25:33.763965+0000","last_became_active":"2024-05-09T09:23:09.297005+0000","last_became_peered":"2024-05-09T09:23:09.297005+0000","last_unstale":"2024-05-09T09:25:33.763965+0000","last_undegraded":"2024-05-09T09:25:33.763965+0000","last_fullsized":"2024-05-09T09:25:33.763965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:39.240217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64098,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":251,"num_read_kb":188,"num_write":41,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160129+0000","last_change":"2024-05-09T09:23:10.333013+0000","last_active":"2024-05-09T09:25:06.160129+0000","last_peered":"2024-05-09T09:25:06.160129+0000","last_clean":"2024-05-09T09:25:06.160129+0000","last_became_active":"2024-05-09T09:23:10.332666+0000","last_became_peered":"2024-05-09T09:23:10.332666+0000","last_unstale":"2024-05-09T09:25:06.160129+0000","last_undegraded":"2024-05-09T09:25:06.160129+0000","last_fullsized":"2024-05-09T09:25:06.160129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:28:50.306551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.158019+0000","last_change":"2024-05-09T09:23:14.367715+0000","last_active":"2024-05-09T09:25:06.158019+0000","last_peered":"2024-05-09T09:25:06.158019+0000","last_clean":"2024-05-09T09:25:06.158019+0000","last_became_active":"2024-05-09T09:23:14.367307+0000","last_became_peered":"2024-05-09T09:23:14.367307+0000","last_unstale":"2024-05-09T09:25:06.158019+0000","last_undegraded":"2024-05-09T09:25:06.158019+0000","last_fullsized":"2024-05-09T09:25:06.158019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:09.075572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164381+0000","last_change":"2024-05-09T09:23:09.306302+0000","last_active":"2024-05-09T09:25:06.164381+0000","last_peered":"2024-05-09T09:25:06.164381+0000","last_clean":"2024-05-09T09:25:06.164381+0000","last_became_active":"2024-05-09T09:23:09.306074+0000","last_became_peered":"2024-05-09T09:23:09.306074+0000","last_unstale":"2024-05-09T09:25:06.164381+0000","last_undegraded":"2024-05-09T09:25:06.164381+0000","last_fullsized":"2024-05-09T09:25:06.164381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:18.434899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156080+0000","last_change":"2024-05-09T09:23:10.331201+0000","last_active":"2024-05-09T09:25:06.156080+0000","last_peered":"2024-05-09T09:25:06.156080+0000","last_clean":"2024-05-09T09:25:06.156080+0000","last_became_active":"2024-05-09T09:23:10.329830+0000","last_became_peered":"2024-05-09T09:23:10.329830+0000","last_unstale":"2024-05-09T09:25:06.156080+0000","last_undegraded":"2024-05-09T09:25:06.156080+0000","last_fullsized":"2024-05-09T09:25:06.156080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:40.199622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160009+0000","last_change":"2024-05-09T09:23:14.359095+0000","last_active":"2024-05-09T09:25:06.160009+0000","last_peered":"2024-05-09T09:25:06.160009+0000","last_clean":"2024-05-09T09:25:06.160009+0000","last_became_active":"2024-05-09T09:23:14.358717+0000","last_became_peered":"2024-05-09T09:23:14.358717+0000","last_unstale":"2024-05-09T09:25:06.160009+0000","last_undegraded":"2024-05-09T09:25:06.160009+0000","last_fullsized":"2024-05-09T09:25:06.160009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:16:09.874767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157568+0000","last_change":"2024-05-09T09:23:09.305370+0000","last_active":"2024-05-09T09:25:06.157568+0000","last_peered":"2024-05-09T09:25:06.157568+0000","last_clean":"2024-05-09T09:25:06.157568+0000","last_became_active":"2024-05-09T09:23:09.305054+0000","last_became_peered":"2024-05-09T09:23:09.305054+0000","last_unstale":"2024-05-09T09:25:06.157568+0000","last_undegraded":"2024-05-09T09:25:06.157568+0000","last_fullsized":"2024-05-09T09:25:06.157568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:28.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161625+0000","last_change":"2024-05-09T09:23:10.332541+0000","last_active":"2024-05-09T09:25:06.161625+0000","last_peered":"2024-05-09T09:25:06.161625+0000","last_clean":"2024-05-09T09:25:06.161625+0000","last_became_active":"2024-05-09T09:23:10.332155+0000","last_became_peered":"2024-05-09T09:23:10.332155+0000","last_unstale":"2024-05-09T09:25:06.161625+0000","last_undegraded":"2024-05-09T09:25:06.161625+0000","last_fullsized":"2024-05-09T09:25:06.161625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:39.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159430+0000","last_change":"2024-05-09T09:23:14.374499+0000","last_active":"2024-05-09T09:25:06.159430+0000","last_peered":"2024-05-09T09:25:06.159430+0000","last_clean":"2024-05-09T09:25:06.159430+0000","last_became_active":"2024-05-09T09:23:14.370688+0000","last_became_peered":"2024-05-09T09:23:14.370688+0000","last_unstale":"2024-05-09T09:25:06.159430+0000","last_undegraded":"2024-05-09T09:25:06.159430+0000","last_fullsized":"2024-05-09T09:25:06.159430+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:53.367237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'8","reported_seq":165,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:32.763278+0000","last_change":"2024-05-09T09:23:09.297776+0000","last_active":"2024-05-09T09:25:32.763278+0000","last_peered":"2024-05-09T09:25:32.763278+0000","last_clean":"2024-05-09T09:25:32.763278+0000","last_became_active":"2024-05-09T09:23:09.297407+0000","last_became_peered":"2024-05-09T09:23:09.297407+0000","last_unstale":"2024-05-09T09:25:32.763278+0000","last_undegraded":"2024-05-09T09:25:32.763278+0000","last_fullsized":"2024-05-09T09:25:32.763278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:21.762941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161199+0000","last_change":"2024-05-09T09:23:10.339127+0000","last_active":"2024-05-09T09:25:06.161199+0000","last_peered":"2024-05-09T09:25:06.161199+0000","last_clean":"2024-05-09T09:25:06.161199+0000","last_became_active":"2024-05-09T09:23:10.338790+0000","last_became_peered":"2024-05-09T09:23:10.338790+0000","last_unstale":"2024-05-09T09:25:06.161199+0000","last_undegraded":"2024-05-09T09:25:06.161199+0000","last_fullsized":"2024-05-09T09:25:06.161199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:36.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156425+0000","last_change":"2024-05-09T09:23:14.373873+0000","last_active":"2024-05-09T09:25:06.156425+0000","last_peered":"2024-05-09T09:25:06.156425+0000","last_clean":"2024-05-09T09:25:06.156425+0000","last_became_active":"2024-05-09T09:23:14.371453+0000","last_became_peered":"2024-05-09T09:23:14.371453+0000","last_unstale":"2024-05-09T09:25:06.156425+0000","last_undegraded":"2024-05-09T09:25:06.156425+0000","last_fullsized":"2024-05-09T09:25:06.156425+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:26.885938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159355+0000","last_change":"2024-05-09T09:23:09.299373+0000","last_active":"2024-05-09T09:25:06.159355+0000","last_peered":"2024-05-09T09:25:06.159355+0000","last_clean":"2024-05-09T09:25:06.159355+0000","last_became_active":"2024-05-09T09:23:09.298926+0000","last_became_peered":"2024-05-09T09:23:09.298926+0000","last_unstale":"2024-05-09T09:25:06.159355+0000","last_undegraded":"2024-05-09T09:25:06.159355+0000","last_fullsized":"2024-05-09T09:25:06.159355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:35.629929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157309+0000","last_change":"2024-05-09T09:23:10.343911+0000","last_active":"2024-05-09T09:25:06.157309+0000","last_peered":"2024-05-09T09:25:06.157309+0000","last_clean":"2024-05-09T09:25:06.157309+0000","last_became_active":"2024-05-09T09:23:10.343568+0000","last_became_peered":"2024-05-09T09:23:10.343568+0000","last_unstale":"2024-05-09T09:25:06.157309+0000","last_undegraded":"2024-05-09T09:25:06.157309+0000","last_fullsized":"2024-05-09T09:25:06.157309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:07.311554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157224+0000","last_change":"2024-05-09T09:23:14.368544+0000","last_active":"2024-05-09T09:25:06.157224+0000","last_peered":"2024-05-09T09:25:06.157224+0000","last_clean":"2024-05-09T09:25:06.157224+0000","last_became_active":"2024-05-09T09:23:14.368140+0000","last_became_peered":"2024-05-09T09:23:14.368140+0000","last_unstale":"2024-05-09T09:25:06.157224+0000","last_undegraded":"2024-05-09T09:25:06.157224+0000","last_fullsized":"2024-05-09T09:25:06.157224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.091768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:27.142002+0000","last_change":"2024-05-09T09:23:09.297612+0000","last_active":"2024-05-09T09:25:27.142002+0000","last_peered":"2024-05-09T09:25:27.142002+0000","last_clean":"2024-05-09T09:25:27.142002+0000","last_became_active":"2024-05-09T09:23:09.297216+0000","last_became_peered":"2024-05-09T09:23:09.297216+0000","last_unstale":"2024-05-09T09:25:27.142002+0000","last_undegraded":"2024-05-09T09:25:27.142002+0000","last_fullsized":"2024-05-09T09:25:27.142002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:09:45.108082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164210+0000","last_change":"2024-05-09T09:25:06.163625+0000","last_active":"2024-05-09T09:25:06.164210+0000","last_peered":"2024-05-09T09:25:06.164210+0000","last_clean":"2024-05-09T09:25:06.164210+0000","last_became_active":"2024-05-09T09:25:06.163198+0000","last_became_peered":"2024-05-09T09:25:06.163198+0000","last_unstale":"2024-05-09T09:25:06.164210+0000","last_undegraded":"2024-05-09T09:25:06.164210+0000","last_fullsized":"2024-05-09T09:25:06.164210+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:35.223030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160161+0000","last_change":"2024-05-09T09:23:14.366119+0000","last_active":"2024-05-09T09:25:06.160161+0000","last_peered":"2024-05-09T09:25:06.160161+0000","last_clean":"2024-05-09T09:25:06.160161+0000","last_became_active":"2024-05-09T09:23:14.365659+0000","last_became_peered":"2024-05-09T09:23:14.365659+0000","last_unstale":"2024-05-09T09:25:06.160161+0000","last_undegraded":"2024-05-09T09:25:06.160161+0000","last_fullsized":"2024-05-09T09:25:06.160161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:23.312632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159327+0000","last_change":"2024-05-09T09:24:05.740866+0000","last_active":"2024-05-09T09:25:06.159327+0000","last_peered":"2024-05-09T09:25:06.159327+0000","last_clean":"2024-05-09T09:25:06.159327+0000","last_became_active":"2024-05-09T09:24:05.740418+0000","last_became_peered":"2024-05-09T09:24:05.740418+0000","last_unstale":"2024-05-09T09:25:06.159327+0000","last_undegraded":"2024-05-09T09:25:06.159327+0000","last_fullsized":"2024-05-09T09:25:06.159327+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:45.946040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.257359+0000","last_change":"2024-05-09T09:23:10.339788+0000","last_active":"2024-05-09T09:25:06.257359+0000","last_peered":"2024-05-09T09:25:06.257359+0000","last_clean":"2024-05-09T09:25:06.257359+0000","last_became_active":"2024-05-09T09:23:10.339430+0000","last_became_peered":"2024-05-09T09:23:10.339430+0000","last_unstale":"2024-05-09T09:25:06.257359+0000","last_undegraded":"2024-05-09T09:25:06.257359+0000","last_fullsized":"2024-05-09T09:25:06.257359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:22.467287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156237+0000","last_change":"2024-05-09T09:23:14.368710+0000","last_active":"2024-05-09T09:25:06.156237+0000","last_peered":"2024-05-09T09:25:06.156237+0000","last_clean":"2024-05-09T09:25:06.156237+0000","last_became_active":"2024-05-09T09:23:14.368288+0000","last_became_peered":"2024-05-09T09:23:14.368288+0000","last_unstale":"2024-05-09T09:25:06.156237+0000","last_undegraded":"2024-05-09T09:25:06.156237+0000","last_fullsized":"2024-05-09T09:25:06.156237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:40.941273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:34.765316+0000","last_change":"2024-05-09T09:23:09.301976+0000","last_active":"2024-05-09T09:25:34.765316+0000","last_peered":"2024-05-09T09:25:34.765316+0000","last_clean":"2024-05-09T09:25:34.765316+0000","last_became_active":"2024-05-09T09:23:09.301494+0000","last_became_peered":"2024-05-09T09:23:09.301494+0000","last_unstale":"2024-05-09T09:25:34.765316+0000","last_undegraded":"2024-05-09T09:25:34.765316+0000","last_fullsized":"2024-05-09T09:25:34.765316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:54.448246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256796+0000","last_change":"2024-05-09T09:23:10.331691+0000","last_active":"2024-05-09T09:25:06.256796+0000","last_peered":"2024-05-09T09:25:06.256796+0000","last_clean":"2024-05-09T09:25:06.256796+0000","last_became_active":"2024-05-09T09:23:10.331332+0000","last_became_peered":"2024-05-09T09:23:10.331332+0000","last_unstale":"2024-05-09T09:25:06.256796+0000","last_undegraded":"2024-05-09T09:25:06.256796+0000","last_fullsized":"2024-05-09T09:25:06.256796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:51.643695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156538+0000","last_change":"2024-05-09T09:23:14.367539+0000","last_active":"2024-05-09T09:25:06.156538+0000","last_peered":"2024-05-09T09:25:06.156538+0000","last_clean":"2024-05-09T09:25:06.156538+0000","last_became_active":"2024-05-09T09:23:14.366806+0000","last_became_peered":"2024-05-09T09:23:14.366806+0000","last_unstale":"2024-05-09T09:25:06.156538+0000","last_undegraded":"2024-05-09T09:25:06.156538+0000","last_fullsized":"2024-05-09T09:25:06.156538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:58.679095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157306+0000","last_change":"2024-05-09T09:23:10.340849+0000","last_active":"2024-05-09T09:25:06.157306+0000","last_peered":"2024-05-09T09:25:06.157306+0000","last_clean":"2024-05-09T09:25:06.157306+0000","last_became_active":"2024-05-09T09:23:10.340430+0000","last_became_peered":"2024-05-09T09:23:10.340430+0000","last_unstale":"2024-05-09T09:25:06.157306+0000","last_undegraded":"2024-05-09T09:25:06.157306+0000","last_fullsized":"2024-05-09T09:25:06.157306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:33:47.827659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'7","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:34.765356+0000","last_change":"2024-05-09T09:23:09.308544+0000","last_active":"2024-05-09T09:25:34.765356+0000","last_peered":"2024-05-09T09:25:34.765356+0000","last_clean":"2024-05-09T09:25:34.765356+0000","last_became_active":"2024-05-09T09:23:09.308195+0000","last_became_peered":"2024-05-09T09:23:09.308195+0000","last_unstale":"2024-05-09T09:25:34.765356+0000","last_undegraded":"2024-05-09T09:25:34.765356+0000","last_fullsized":"2024-05-09T09:25:34.765356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:17:34.884385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161350+0000","last_change":"2024-05-09T09:24:05.755846+0000","last_active":"2024-05-09T09:25:06.161350+0000","last_peered":"2024-05-09T09:25:06.161350+0000","last_clean":"2024-05-09T09:25:06.161350+0000","last_became_active":"2024-05-09T09:24:05.747246+0000","last_became_peered":"2024-05-09T09:24:05.747246+0000","last_unstale":"2024-05-09T09:25:06.161350+0000","last_undegraded":"2024-05-09T09:25:06.161350+0000","last_fullsized":"2024-05-09T09:25:06.161350+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:23:40.544770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161873+0000","last_change":"2024-05-09T09:23:10.341320+0000","last_active":"2024-05-09T09:25:06.161873+0000","last_peered":"2024-05-09T09:25:06.161873+0000","last_clean":"2024-05-09T09:25:06.161873+0000","last_became_active":"2024-05-09T09:23:10.340868+0000","last_became_peered":"2024-05-09T09:23:10.340868+0000","last_unstale":"2024-05-09T09:25:06.161873+0000","last_undegraded":"2024-05-09T09:25:06.161873+0000","last_fullsized":"2024-05-09T09:25:06.161873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:55.846642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157331+0000","last_change":"2024-05-09T09:23:09.299574+0000","last_active":"2024-05-09T09:25:06.157331+0000","last_peered":"2024-05-09T09:25:06.157331+0000","last_clean":"2024-05-09T09:25:06.157331+0000","last_became_active":"2024-05-09T09:23:09.298878+0000","last_became_peered":"2024-05-09T09:23:09.298878+0000","last_unstale":"2024-05-09T09:25:06.157331+0000","last_undegraded":"2024-05-09T09:25:06.157331+0000","last_fullsized":"2024-05-09T09:25:06.157331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.275907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":352,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160988+0000","last_change":"2024-05-09T09:21:57.576132+0000","last_active":"2024-05-09T09:25:06.160988+0000","last_peered":"2024-05-09T09:25:06.160988+0000","last_clean":"2024-05-09T09:25:06.160988+0000","last_became_active":"2024-05-09T09:21:57.575602+0000","last_became_peered":"2024-05-09T09:21:57.575602+0000","last_unstale":"2024-05-09T09:25:06.160988+0000","last_undegraded":"2024-05-09T09:25:06.160988+0000","last_fullsized":"2024-05-09T09:25:06.160988+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:18:31.335063+0000","last_clean_scrub_stamp":"2024-05-09T09:18:31.335063+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:07.639787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157785+0000","last_change":"2024-05-09T09:23:14.371184+0000","last_active":"2024-05-09T09:25:06.157785+0000","last_peered":"2024-05-09T09:25:06.157785+0000","last_clean":"2024-05-09T09:25:06.157785+0000","last_became_active":"2024-05-09T09:23:14.370780+0000","last_became_peered":"2024-05-09T09:23:14.370780+0000","last_unstale":"2024-05-09T09:25:06.157785+0000","last_undegraded":"2024-05-09T09:25:06.157785+0000","last_fullsized":"2024-05-09T09:25:06.157785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:39.361947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.155976+0000","last_change":"2024-05-09T09:23:10.330915+0000","last_active":"2024-05-09T09:25:06.155976+0000","last_peered":"2024-05-09T09:25:06.155976+0000","last_clean":"2024-05-09T09:25:06.155976+0000","last_became_active":"2024-05-09T09:23:10.330458+0000","last_became_peered":"2024-05-09T09:23:10.330458+0000","last_unstale":"2024-05-09T09:25:06.155976+0000","last_undegraded":"2024-05-09T09:25:06.155976+0000","last_fullsized":"2024-05-09T09:25:06.155976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:10.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161228+0000","last_change":"2024-05-09T09:23:09.303895+0000","last_active":"2024-05-09T09:25:06.161228+0000","last_peered":"2024-05-09T09:25:06.161228+0000","last_clean":"2024-05-09T09:25:06.161228+0000","last_became_active":"2024-05-09T09:23:09.303555+0000","last_became_peered":"2024-05-09T09:23:09.303555+0000","last_unstale":"2024-05-09T09:25:06.161228+0000","last_undegraded":"2024-05-09T09:25:06.161228+0000","last_fullsized":"2024-05-09T09:25:06.161228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:04.350161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159708+0000","last_change":"2024-05-09T09:23:14.359390+0000","last_active":"2024-05-09T09:25:06.159708+0000","last_peered":"2024-05-09T09:25:06.159708+0000","last_clean":"2024-05-09T09:25:06.159708+0000","last_became_active":"2024-05-09T09:23:14.359000+0000","last_became_peered":"2024-05-09T09:23:14.359000+0000","last_unstale":"2024-05-09T09:25:06.159708+0000","last_undegraded":"2024-05-09T09:25:06.159708+0000","last_fullsized":"2024-05-09T09:25:06.159708+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:25.077835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160314+0000","last_change":"2024-05-09T09:23:09.304535+0000","last_active":"2024-05-09T09:25:06.160314+0000","last_peered":"2024-05-09T09:25:06.160314+0000","last_clean":"2024-05-09T09:25:06.160314+0000","last_became_active":"2024-05-09T09:23:09.304166+0000","last_became_peered":"2024-05-09T09:23:09.304166+0000","last_unstale":"2024-05-09T09:25:06.160314+0000","last_undegraded":"2024-05-09T09:25:06.160314+0000","last_fullsized":"2024-05-09T09:25:06.160314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:03:20.281620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156507+0000","last_change":"2024-05-09T09:23:10.335657+0000","last_active":"2024-05-09T09:25:06.156507+0000","last_peered":"2024-05-09T09:25:06.156507+0000","last_clean":"2024-05-09T09:25:06.156507+0000","last_became_active":"2024-05-09T09:23:10.335231+0000","last_became_peered":"2024-05-09T09:23:10.335231+0000","last_unstale":"2024-05-09T09:25:06.156507+0000","last_undegraded":"2024-05-09T09:25:06.156507+0000","last_fullsized":"2024-05-09T09:25:06.156507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:07.990165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161193+0000","last_change":"2024-05-09T09:23:14.366988+0000","last_active":"2024-05-09T09:25:06.161193+0000","last_peered":"2024-05-09T09:25:06.161193+0000","last_clean":"2024-05-09T09:25:06.161193+0000","last_became_active":"2024-05-09T09:23:14.366562+0000","last_became_peered":"2024-05-09T09:23:14.366562+0000","last_unstale":"2024-05-09T09:25:06.161193+0000","last_undegraded":"2024-05-09T09:25:06.161193+0000","last_fullsized":"2024-05-09T09:25:06.161193+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:42.670036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'15","reported_seq":62,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:18.524078+0000","last_change":"2024-05-09T09:24:05.752748+0000","last_active":"2024-05-09T09:25:18.524078+0000","last_peered":"2024-05-09T09:25:18.524078+0000","last_clean":"2024-05-09T09:25:18.524078+0000","last_became_active":"2024-05-09T09:24:05.743638+0000","last_became_peered":"2024-05-09T09:24:05.743638+0000","last_unstale":"2024-05-09T09:25:18.524078+0000","last_undegraded":"2024-05-09T09:25:18.524078+0000","last_fullsized":"2024-05-09T09:25:18.524078+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:01.834934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161808+0000","last_change":"2024-05-09T09:23:10.345840+0000","last_active":"2024-05-09T09:25:06.161808+0000","last_peered":"2024-05-09T09:25:06.161808+0000","last_clean":"2024-05-09T09:25:06.161808+0000","last_became_active":"2024-05-09T09:23:10.345487+0000","last_became_peered":"2024-05-09T09:23:10.345487+0000","last_unstale":"2024-05-09T09:25:06.161808+0000","last_undegraded":"2024-05-09T09:25:06.161808+0000","last_fullsized":"2024-05-09T09:25:06.161808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:47.723429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162727+0000","last_change":"2024-05-09T09:23:14.365485+0000","last_active":"2024-05-09T09:25:06.162727+0000","last_peered":"2024-05-09T09:25:06.162727+0000","last_clean":"2024-05-09T09:25:06.162727+0000","last_became_active":"2024-05-09T09:23:14.365018+0000","last_became_peered":"2024-05-09T09:23:14.365018+0000","last_unstale":"2024-05-09T09:25:06.162727+0000","last_undegraded":"2024-05-09T09:25:06.162727+0000","last_fullsized":"2024-05-09T09:25:06.162727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:54:04.712920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157667+0000","last_change":"2024-05-09T09:23:09.302365+0000","last_active":"2024-05-09T09:25:06.157667+0000","last_peered":"2024-05-09T09:25:06.157667+0000","last_clean":"2024-05-09T09:25:06.157667+0000","last_became_active":"2024-05-09T09:23:09.302008+0000","last_became_peered":"2024-05-09T09:23:09.302008+0000","last_unstale":"2024-05-09T09:25:06.157667+0000","last_undegraded":"2024-05-09T09:25:06.157667+0000","last_fullsized":"2024-05-09T09:25:06.157667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:48.829940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165695+0000","last_change":"2024-05-09T09:23:10.339170+0000","last_active":"2024-05-09T09:25:06.165695+0000","last_peered":"2024-05-09T09:25:06.165695+0000","last_clean":"2024-05-09T09:25:06.165695+0000","last_became_active":"2024-05-09T09:23:10.338829+0000","last_became_peered":"2024-05-09T09:23:10.338829+0000","last_unstale":"2024-05-09T09:25:06.165695+0000","last_undegraded":"2024-05-09T09:25:06.165695+0000","last_fullsized":"2024-05-09T09:25:06.165695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:04.510495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157578+0000","last_change":"2024-05-09T09:24:05.747227+0000","last_active":"2024-05-09T09:25:06.157578+0000","last_peered":"2024-05-09T09:25:06.157578+0000","last_clean":"2024-05-09T09:25:06.157578+0000","last_became_active":"2024-05-09T09:24:05.746338+0000","last_became_peered":"2024-05-09T09:24:05.746338+0000","last_unstale":"2024-05-09T09:25:06.157578+0000","last_undegraded":"2024-05-09T09:25:06.157578+0000","last_fullsized":"2024-05-09T09:25:06.157578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:20.899664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256404+0000","last_change":"2024-05-09T09:23:09.297418+0000","last_active":"2024-05-09T09:25:06.256404+0000","last_peered":"2024-05-09T09:25:06.256404+0000","last_clean":"2024-05-09T09:25:06.256404+0000","last_became_active":"2024-05-09T09:23:09.296963+0000","last_became_peered":"2024-05-09T09:23:09.296963+0000","last_unstale":"2024-05-09T09:25:06.256404+0000","last_undegraded":"2024-05-09T09:25:06.256404+0000","last_fullsized":"2024-05-09T09:25:06.256404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:54.552857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159481+0000","last_change":"2024-05-09T09:23:10.325882+0000","last_active":"2024-05-09T09:25:06.159481+0000","last_peered":"2024-05-09T09:25:06.159481+0000","last_clean":"2024-05-09T09:25:06.159481+0000","last_became_active":"2024-05-09T09:23:10.325494+0000","last_became_peered":"2024-05-09T09:23:10.325494+0000","last_unstale":"2024-05-09T09:25:06.159481+0000","last_undegraded":"2024-05-09T09:25:06.159481+0000","last_fullsized":"2024-05-09T09:25:06.159481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:27.191310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159617+0000","last_change":"2024-05-09T09:23:14.375572+0000","last_active":"2024-05-09T09:25:06.159617+0000","last_peered":"2024-05-09T09:25:06.159617+0000","last_clean":"2024-05-09T09:25:06.159617+0000","last_became_active":"2024-05-09T09:23:14.374069+0000","last_became_peered":"2024-05-09T09:23:14.374069+0000","last_unstale":"2024-05-09T09:25:06.159617+0000","last_undegraded":"2024-05-09T09:25:06.159617+0000","last_fullsized":"2024-05-09T09:25:06.159617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:44.045012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161469+0000","last_change":"2024-05-09T09:23:09.304137+0000","last_active":"2024-05-09T09:25:06.161469+0000","last_peered":"2024-05-09T09:25:06.161469+0000","last_clean":"2024-05-09T09:25:06.161469+0000","last_became_active":"2024-05-09T09:23:09.303762+0000","last_became_peered":"2024-05-09T09:23:09.303762+0000","last_unstale":"2024-05-09T09:25:06.161469+0000","last_undegraded":"2024-05-09T09:25:06.161469+0000","last_fullsized":"2024-05-09T09:25:06.161469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:56.811194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159736+0000","last_change":"2024-05-09T09:23:10.331998+0000","last_active":"2024-05-09T09:25:06.159736+0000","last_peered":"2024-05-09T09:25:06.159736+0000","last_clean":"2024-05-09T09:25:06.159736+0000","last_became_active":"2024-05-09T09:23:10.331642+0000","last_became_peered":"2024-05-09T09:23:10.331642+0000","last_unstale":"2024-05-09T09:25:06.159736+0000","last_undegraded":"2024-05-09T09:25:06.159736+0000","last_fullsized":"2024-05-09T09:25:06.159736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:41.569300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157348+0000","last_change":"2024-05-09T09:23:14.377444+0000","last_active":"2024-05-09T09:25:06.157348+0000","last_peered":"2024-05-09T09:25:06.157348+0000","last_clean":"2024-05-09T09:25:06.157348+0000","last_became_active":"2024-05-09T09:23:14.376189+0000","last_became_peered":"2024-05-09T09:23:14.376189+0000","last_unstale":"2024-05-09T09:25:06.157348+0000","last_undegraded":"2024-05-09T09:25:06.157348+0000","last_fullsized":"2024-05-09T09:25:06.157348+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:06.973535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:19.360581+0000","last_change":"2024-05-09T09:23:09.304101+0000","last_active":"2024-05-09T09:25:19.360581+0000","last_peered":"2024-05-09T09:25:19.360581+0000","last_clean":"2024-05-09T09:25:19.360581+0000","last_became_active":"2024-05-09T09:23:09.303743+0000","last_became_peered":"2024-05-09T09:23:09.303743+0000","last_unstale":"2024-05-09T09:25:19.360581+0000","last_undegraded":"2024-05-09T09:25:19.360581+0000","last_fullsized":"2024-05-09T09:25:19.360581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:39.339815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162614+0000","last_change":"2024-05-09T09:23:10.345938+0000","last_active":"2024-05-09T09:25:06.162614+0000","last_peered":"2024-05-09T09:25:06.162614+0000","last_clean":"2024-05-09T09:25:06.162614+0000","last_became_active":"2024-05-09T09:23:10.345576+0000","last_became_peered":"2024-05-09T09:23:10.345576+0000","last_unstale":"2024-05-09T09:25:06.162614+0000","last_undegraded":"2024-05-09T09:25:06.162614+0000","last_fullsized":"2024-05-09T09:25:06.162614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:08.571900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157437+0000","last_change":"2024-05-09T09:23:14.371549+0000","last_active":"2024-05-09T09:25:06.157437+0000","last_peered":"2024-05-09T09:25:06.157437+0000","last_clean":"2024-05-09T09:25:06.157437+0000","last_became_active":"2024-05-09T09:23:14.369931+0000","last_became_peered":"2024-05-09T09:23:14.369931+0000","last_unstale":"2024-05-09T09:25:06.157437+0000","last_undegraded":"2024-05-09T09:25:06.157437+0000","last_fullsized":"2024-05-09T09:25:06.157437+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:20.777172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159654+0000","last_change":"2024-05-09T09:23:09.302909+0000","last_active":"2024-05-09T09:25:06.159654+0000","last_peered":"2024-05-09T09:25:06.159654+0000","last_clean":"2024-05-09T09:25:06.159654+0000","last_became_active":"2024-05-09T09:23:09.302570+0000","last_became_peered":"2024-05-09T09:23:09.302570+0000","last_unstale":"2024-05-09T09:25:06.159654+0000","last_undegraded":"2024-05-09T09:25:06.159654+0000","last_fullsized":"2024-05-09T09:25:06.159654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:23.856755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161777+0000","last_change":"2024-05-09T09:23:10.342676+0000","last_active":"2024-05-09T09:25:06.161777+0000","last_peered":"2024-05-09T09:25:06.161777+0000","last_clean":"2024-05-09T09:25:06.161777+0000","last_became_active":"2024-05-09T09:23:10.341906+0000","last_became_peered":"2024-05-09T09:23:10.341906+0000","last_unstale":"2024-05-09T09:25:06.161777+0000","last_undegraded":"2024-05-09T09:25:06.161777+0000","last_fullsized":"2024-05-09T09:25:06.161777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:59:11.137648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.256737+0000","last_change":"2024-05-09T09:23:14.363779+0000","last_active":"2024-05-09T09:25:06.256737+0000","last_peered":"2024-05-09T09:25:06.256737+0000","last_clean":"2024-05-09T09:25:06.256737+0000","last_became_active":"2024-05-09T09:23:14.363307+0000","last_became_peered":"2024-05-09T09:23:14.363307+0000","last_unstale":"2024-05-09T09:25:06.256737+0000","last_undegraded":"2024-05-09T09:25:06.256737+0000","last_fullsized":"2024-05-09T09:25:06.256737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:46.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"93'6","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:32.763660+0000","last_change":"2024-05-09T09:23:09.298094+0000","last_active":"2024-05-09T09:25:32.763660+0000","last_peered":"2024-05-09T09:25:32.763660+0000","last_clean":"2024-05-09T09:25:32.763660+0000","last_became_active":"2024-05-09T09:23:09.297777+0000","last_became_peered":"2024-05-09T09:23:09.297777+0000","last_unstale":"2024-05-09T09:25:32.763660+0000","last_undegraded":"2024-05-09T09:25:32.763660+0000","last_fullsized":"2024-05-09T09:25:32.763660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:25.099193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161899+0000","last_change":"2024-05-09T09:23:10.340120+0000","last_active":"2024-05-09T09:25:06.161899+0000","last_peered":"2024-05-09T09:25:06.161899+0000","last_clean":"2024-05-09T09:25:06.161899+0000","last_became_active":"2024-05-09T09:23:10.339690+0000","last_became_peered":"2024-05-09T09:23:10.339690+0000","last_unstale":"2024-05-09T09:25:06.161899+0000","last_undegraded":"2024-05-09T09:25:06.161899+0000","last_fullsized":"2024-05-09T09:25:06.161899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:36.783411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165251+0000","last_change":"2024-05-09T09:24:05.744892+0000","last_active":"2024-05-09T09:25:06.165251+0000","last_peered":"2024-05-09T09:25:06.165251+0000","last_clean":"2024-05-09T09:25:06.165251+0000","last_became_active":"2024-05-09T09:24:05.744437+0000","last_became_peered":"2024-05-09T09:24:05.744437+0000","last_unstale":"2024-05-09T09:25:06.165251+0000","last_undegraded":"2024-05-09T09:25:06.165251+0000","last_fullsized":"2024-05-09T09:25:06.165251+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:38:33.709821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157886+0000","last_change":"2024-05-09T09:23:09.307592+0000","last_active":"2024-05-09T09:25:06.157886+0000","last_peered":"2024-05-09T09:25:06.157886+0000","last_clean":"2024-05-09T09:25:06.157886+0000","last_became_active":"2024-05-09T09:23:09.307341+0000","last_became_peered":"2024-05-09T09:23:09.307341+0000","last_unstale":"2024-05-09T09:25:06.157886+0000","last_undegraded":"2024-05-09T09:25:06.157886+0000","last_fullsized":"2024-05-09T09:25:06.157886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:21.790980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161686+0000","last_change":"2024-05-09T09:23:10.349217+0000","last_active":"2024-05-09T09:25:06.161686+0000","last_peered":"2024-05-09T09:25:06.161686+0000","last_clean":"2024-05-09T09:25:06.161686+0000","last_became_active":"2024-05-09T09:23:10.348861+0000","last_became_peered":"2024-05-09T09:23:10.348861+0000","last_unstale":"2024-05-09T09:25:06.161686+0000","last_undegraded":"2024-05-09T09:25:06.161686+0000","last_fullsized":"2024-05-09T09:25:06.161686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:51.621045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157038+0000","last_change":"2024-05-09T09:23:14.363773+0000","last_active":"2024-05-09T09:25:06.157038+0000","last_peered":"2024-05-09T09:25:06.157038+0000","last_clean":"2024-05-09T09:25:06.157038+0000","last_became_active":"2024-05-09T09:23:14.363393+0000","last_became_peered":"2024-05-09T09:23:14.363393+0000","last_unstale":"2024-05-09T09:25:06.157038+0000","last_undegraded":"2024-05-09T09:25:06.157038+0000","last_fullsized":"2024-05-09T09:25:06.157038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:13.332343+0000","last_clean_scrub_stamp":"2024-05-09T09:23:13.332343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:57.322985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156417+0000","last_change":"2024-05-09T09:23:09.308723+0000","last_active":"2024-05-09T09:25:06.156417+0000","last_peered":"2024-05-09T09:25:06.156417+0000","last_clean":"2024-05-09T09:25:06.156417+0000","last_became_active":"2024-05-09T09:23:09.308267+0000","last_became_peered":"2024-05-09T09:23:09.308267+0000","last_unstale":"2024-05-09T09:25:06.156417+0000","last_undegraded":"2024-05-09T09:25:06.156417+0000","last_fullsized":"2024-05-09T09:25:06.156417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:08.273491+0000","last_clean_scrub_stamp":"2024-05-09T09:23:08.273491+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:09:25.036421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.162140+0000","last_change":"2024-05-09T09:23:10.342246+0000","last_active":"2024-05-09T09:25:06.162140+0000","last_peered":"2024-05-09T09:25:06.162140+0000","last_clean":"2024-05-09T09:25:06.162140+0000","last_became_active":"2024-05-09T09:23:10.341927+0000","last_became_peered":"2024-05-09T09:23:10.341927+0000","last_unstale":"2024-05-09T09:25:06.162140+0000","last_undegraded":"2024-05-09T09:25:06.162140+0000","last_fullsized":"2024-05-09T09:25:06.162140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:20:01.182226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.159866+0000","last_change":"2024-05-09T09:23:10.332768+0000","last_active":"2024-05-09T09:25:06.159866+0000","last_peered":"2024-05-09T09:25:06.159866+0000","last_clean":"2024-05-09T09:25:06.159866+0000","last_became_active":"2024-05-09T09:23:10.332441+0000","last_became_peered":"2024-05-09T09:23:10.332441+0000","last_unstale":"2024-05-09T09:25:06.159866+0000","last_undegraded":"2024-05-09T09:25:06.159866+0000","last_fullsized":"2024-05-09T09:25:06.159866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:02:53.377058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164248+0000","last_change":"2024-05-09T09:23:10.337944+0000","last_active":"2024-05-09T09:25:06.164248+0000","last_peered":"2024-05-09T09:25:06.164248+0000","last_clean":"2024-05-09T09:25:06.164248+0000","last_became_active":"2024-05-09T09:23:10.337593+0000","last_became_peered":"2024-05-09T09:23:10.337593+0000","last_unstale":"2024-05-09T09:25:06.164248+0000","last_undegraded":"2024-05-09T09:25:06.164248+0000","last_fullsized":"2024-05-09T09:25:06.164248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:20.381543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.167531+0000","last_change":"2024-05-09T09:25:06.167114+0000","last_active":"2024-05-09T09:25:06.167531+0000","last_peered":"2024-05-09T09:25:06.167531+0000","last_clean":"2024-05-09T09:25:06.167531+0000","last_became_active":"2024-05-09T09:25:06.166712+0000","last_became_peered":"2024-05-09T09:25:06.166712+0000","last_unstale":"2024-05-09T09:25:06.167531+0000","last_undegraded":"2024-05-09T09:25:06.167531+0000","last_fullsized":"2024-05-09T09:25:06.167531+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:00.253528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.165435+0000","last_change":"2024-05-09T09:23:10.340243+0000","last_active":"2024-05-09T09:25:06.165435+0000","last_peered":"2024-05-09T09:25:06.165435+0000","last_clean":"2024-05-09T09:25:06.165435+0000","last_became_active":"2024-05-09T09:23:10.339919+0000","last_became_peered":"2024-05-09T09:23:10.339919+0000","last_unstale":"2024-05-09T09:25:06.165435+0000","last_undegraded":"2024-05-09T09:25:06.165435+0000","last_fullsized":"2024-05-09T09:25:06.165435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:21.460928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156424+0000","last_change":"2024-05-09T09:23:10.333136+0000","last_active":"2024-05-09T09:25:06.156424+0000","last_peered":"2024-05-09T09:25:06.156424+0000","last_clean":"2024-05-09T09:25:06.156424+0000","last_became_active":"2024-05-09T09:23:10.332799+0000","last_became_peered":"2024-05-09T09:23:10.332799+0000","last_unstale":"2024-05-09T09:25:06.156424+0000","last_undegraded":"2024-05-09T09:25:06.156424+0000","last_fullsized":"2024-05-09T09:25:06.156424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:50.682747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160593+0000","last_change":"2024-05-09T09:24:05.745143+0000","last_active":"2024-05-09T09:25:06.160593+0000","last_peered":"2024-05-09T09:25:06.160593+0000","last_clean":"2024-05-09T09:25:06.160593+0000","last_became_active":"2024-05-09T09:24:05.744117+0000","last_became_peered":"2024-05-09T09:24:05.744117+0000","last_unstale":"2024-05-09T09:25:06.160593+0000","last_undegraded":"2024-05-09T09:25:06.160593+0000","last_fullsized":"2024-05-09T09:25:06.160593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:25:09.853820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164776+0000","last_change":"2024-05-09T09:23:10.341776+0000","last_active":"2024-05-09T09:25:06.164776+0000","last_peered":"2024-05-09T09:25:06.164776+0000","last_clean":"2024-05-09T09:25:06.164776+0000","last_became_active":"2024-05-09T09:23:10.341436+0000","last_became_peered":"2024-05-09T09:23:10.341436+0000","last_unstale":"2024-05-09T09:25:06.164776+0000","last_undegraded":"2024-05-09T09:25:06.164776+0000","last_fullsized":"2024-05-09T09:25:06.164776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:42.178856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161898+0000","last_change":"2024-05-09T09:24:05.745324+0000","last_active":"2024-05-09T09:25:06.161898+0000","last_peered":"2024-05-09T09:25:06.161898+0000","last_clean":"2024-05-09T09:25:06.161898+0000","last_became_active":"2024-05-09T09:24:05.744899+0000","last_became_peered":"2024-05-09T09:24:05.744899+0000","last_unstale":"2024-05-09T09:25:06.161898+0000","last_undegraded":"2024-05-09T09:25:06.161898+0000","last_fullsized":"2024-05-09T09:25:06.161898+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:59.372367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.157167+0000","last_change":"2024-05-09T09:23:10.337117+0000","last_active":"2024-05-09T09:25:06.157167+0000","last_peered":"2024-05-09T09:25:06.157167+0000","last_clean":"2024-05-09T09:25:06.157167+0000","last_became_active":"2024-05-09T09:23:10.336672+0000","last_became_peered":"2024-05-09T09:23:10.336672+0000","last_unstale":"2024-05-09T09:25:06.157167+0000","last_undegraded":"2024-05-09T09:25:06.157167+0000","last_fullsized":"2024-05-09T09:25:06.157167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:59:20.723742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.161586+0000","last_change":"2024-05-09T09:23:10.337733+0000","last_active":"2024-05-09T09:25:06.161586+0000","last_peered":"2024-05-09T09:25:06.161586+0000","last_clean":"2024-05-09T09:25:06.161586+0000","last_became_active":"2024-05-09T09:23:10.337343+0000","last_became_peered":"2024-05-09T09:23:10.337343+0000","last_unstale":"2024-05-09T09:25:06.161586+0000","last_undegraded":"2024-05-09T09:25:06.161586+0000","last_fullsized":"2024-05-09T09:25:06.161586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:53.447911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.164449+0000","last_change":"2024-05-09T09:25:06.163856+0000","last_active":"2024-05-09T09:25:06.164449+0000","last_peered":"2024-05-09T09:25:06.164449+0000","last_clean":"2024-05-09T09:25:06.164449+0000","last_became_active":"2024-05-09T09:25:06.163367+0000","last_became_peered":"2024-05-09T09:25:06.163367+0000","last_unstale":"2024-05-09T09:25:06.164449+0000","last_undegraded":"2024-05-09T09:25:06.164449+0000","last_fullsized":"2024-05-09T09:25:06.164449+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:59.920586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,3],"acting":[9,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.156375+0000","last_change":"2024-05-09T09:23:10.332271+0000","last_active":"2024-05-09T09:25:06.156375+0000","last_peered":"2024-05-09T09:25:06.156375+0000","last_clean":"2024-05-09T09:25:06.156375+0000","last_became_active":"2024-05-09T09:23:10.331941+0000","last_became_peered":"2024-05-09T09:23:10.331941+0000","last_unstale":"2024-05-09T09:25:06.156375+0000","last_undegraded":"2024-05-09T09:25:06.156375+0000","last_fullsized":"2024-05-09T09:25:06.156375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:27.707554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-09T09:25:06.160211+0000","last_change":"2024-05-09T09:23:10.328965+0000","last_active":"2024-05-09T09:25:06.160211+0000","last_peered":"2024-05-09T09:25:06.160211+0000","last_clean":"2024-05-09T09:25:06.160211+0000","last_became_active":"2024-05-09T09:23:10.328618+0000","last_became_peered":"2024-05-09T09:23:10.328618+0000","last_unstale":"2024-05-09T09:25:06.160211+0000","last_undegraded":"2024-05-09T09:25:06.160211+0000","last_fullsized":"2024-05-09T09:25:06.160211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:23:09.278973+0000","last_clean_scrub_stamp":"2024-05-09T09:23:09.278973+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:16:02.003378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73024,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1255,"num_read_kb":707,"num_write":265,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":223386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":520504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28740,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716412,"statfs":{"total":95995035648,"available":95965605888,"internally_reserved":0,"allocated":1765376,"data_stored":1176046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":519941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":520319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1683456,"data_stored":1118436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":515938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":585380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":517394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40624,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704528,"statfs":{"total":95995035648,"available":95953436672,"internally_reserved":0,"allocated":1155072,"data_stored":581779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":515153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":520544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1112500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:25:37.236 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:25:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.377+0000 7ff54b910640 1 -- 172.21.15.12:0/301369209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff544100790 msgr2=0x7ff544100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.377+0000 7ff54b910640 1 --2- 172.21.15.12:0/301369209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff544100790 0x7ff544100bb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff540009900 tx=0x7ff54002a410 comp rx=0 tx=0).stop 2024-05-09T09:25:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.377+0000 7ff54b910640 1 -- 172.21.15.12:0/301369209 shutdown_connections 2024-05-09T09:25:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.377+0000 7ff54b910640 1 --2- 172.21.15.12:0/301369209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544102340 0x7ff544108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.377+0000 7ff54b910640 1 --2- 172.21.15.12:0/301369209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff544101980 0x7ff544101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.377+0000 7ff54b910640 1 --2- 172.21.15.12:0/301369209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff544100790 0x7ff544100bb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.377+0000 7ff54b910640 1 -- 172.21.15.12:0/301369209 >> 172.21.15.12:0/301369209 conn(0x7ff5440fbd20 msgr2=0x7ff5440fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.377+0000 7ff54b910640 1 -- 172.21.15.12:0/301369209 shutdown_connections 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.377+0000 7ff54b910640 1 -- 172.21.15.12:0/301369209 wait complete. 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff54b910640 1 Processor -- start 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff54b910640 1 -- start start 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff54b910640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff544100790 0x7ff54406aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff54b910640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544101980 0x7ff54419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff54b910640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff544102340 0x7ff54419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff54b910640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff54410c450 con 0x7ff544102340 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff54b910640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff54410c2d0 con 0x7ff544101980 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff54b910640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff54410c5d0 con 0x7ff544100790 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff549ead640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff544102340 0x7ff54419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff549ead640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff544102340 0x7ff54419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34026/0 (socket says 172.21.15.12:34026) 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff549ead640 1 -- 172.21.15.12:0/252637777 learned_addr learned my addr 172.21.15.12:0/252637777 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff5496ac640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff544100790 0x7ff54406aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff548eab640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544101980 0x7ff54419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.378+0000 7ff549ead640 1 -- 172.21.15.12:0/252637777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff544100790 msgr2=0x7ff54406aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.379+0000 7ff549ead640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff544100790 0x7ff54406aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.379+0000 7ff549ead640 1 -- 172.21.15.12:0/252637777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544101980 msgr2=0x7ff54419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.379+0000 7ff549ead640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544101980 0x7ff54419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.379+0000 7ff549ead640 1 -- 172.21.15.12:0/252637777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54419f8a0 con 0x7ff544102340 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.379+0000 7ff5496ac640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff544100790 0x7ff54406aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.379+0000 7ff548eab640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544101980 0x7ff54419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.379+0000 7ff549ead640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff544102340 0x7ff54419ef90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00eaa0 tx=0x7ff53c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.379+0000 7ff53a7fc640 1 -- 172.21.15.12:0/252637777 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00cdb0 con 0x7ff544102340 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.379+0000 7ff53a7fc640 1 -- 172.21.15.12:0/252637777 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff53c005020 con 0x7ff544102340 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.379+0000 7ff54b910640 1 -- 172.21.15.12:0/252637777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54419fb30 con 0x7ff544102340 2024-05-09T09:25:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.379+0000 7ff53a7fc640 1 -- 172.21.15.12:0/252637777 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0116b0 con 0x7ff544102340 2024-05-09T09:25:37.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.379+0000 7ff54b910640 1 -- 172.21.15.12:0/252637777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5441a00b0 con 0x7ff544102340 2024-05-09T09:25:37.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.380+0000 7ff54b910640 1 -- 172.21.15.12:0/252637777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff544107b80 con 0x7ff544102340 2024-05-09T09:25:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.381+0000 7ff53a7fc640 1 -- 172.21.15.12:0/252637777 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff53c0051d0 con 0x7ff544102340 2024-05-09T09:25:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.382+0000 7ff53a7fc640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff5300a2b90 0x7ff5300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.382+0000 7ff53a7fc640 1 -- 172.21.15.12:0/252637777 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7ff53c0cf730 con 0x7ff544102340 2024-05-09T09:25:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.382+0000 7ff5496ac640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff5300a2b90 0x7ff5300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.382+0000 7ff5496ac640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff5300a2b90 0x7ff5300a5050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff540006010 tx=0x7ff540000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:37.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.386+0000 7ff53a7fc640 1 -- 172.21.15.12:0/252637777 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff53c016030 con 0x7ff544102340 2024-05-09T09:25:37.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.575+0000 7ff54b910640 1 -- 172.21.15.12:0/252637777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5440624d0 con 0x7ff544102340 2024-05-09T09:25:37.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.576+0000 7ff53a7fc640 1 -- 172.21.15.12:0/252637777 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24762 (secure 0 0 0) 0x7ff53c09c2c0 con 0x7ff544102340 2024-05-09T09:25:37.577 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:37.577 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":93,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:25:06.152764+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:25:37.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.579+0000 7ff54b910640 1 -- 172.21.15.12:0/252637777 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff5300a2b90 msgr2=0x7ff5300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.579+0000 7ff54b910640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff5300a2b90 0x7ff5300a5050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff540006010 tx=0x7ff540000bc0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.579+0000 7ff54b910640 1 -- 172.21.15.12:0/252637777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff544102340 msgr2=0x7ff54419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.579+0000 7ff54b910640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff544102340 0x7ff54419ef90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00eaa0 tx=0x7ff53c00ef60 comp rx=0 tx=0).stop 2024-05-09T09:25:37.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.579+0000 7ff54b910640 1 -- 172.21.15.12:0/252637777 shutdown_connections 2024-05-09T09:25:37.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.579+0000 7ff54b910640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff5300a2b90 0x7ff5300a5050 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.579+0000 7ff54b910640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff544102340 0x7ff54419ef90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.579+0000 7ff54b910640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544101980 0x7ff54419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.579+0000 7ff54b910640 1 --2- 172.21.15.12:0/252637777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff544100790 0x7ff54406aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.579+0000 7ff54b910640 1 -- 172.21.15.12:0/252637777 >> 172.21.15.12:0/252637777 conn(0x7ff5440fbd20 msgr2=0x7ff544104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:37.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.579+0000 7ff54b910640 1 -- 172.21.15.12:0/252637777 shutdown_connections 2024-05-09T09:25:37.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.579+0000 7ff54b910640 1 -- 172.21.15.12:0/252637777 wait complete. 2024-05-09T09:25:37.590 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:23:08.167107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:23:09.065670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:23:12.773640+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:25:37.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:25:37.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.701+0000 7f248141c640 1 -- 172.21.15.12:0/2842521704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f247c102340 msgr2=0x7f247c108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.701+0000 7f248141c640 1 --2- 172.21.15.12:0/2842521704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f247c102340 0x7f247c108bd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2474009900 tx=0x7f247402a410 comp rx=0 tx=0).stop 2024-05-09T09:25:37.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.701+0000 7f248141c640 1 -- 172.21.15.12:0/2842521704 shutdown_connections 2024-05-09T09:25:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.701+0000 7f248141c640 1 --2- 172.21.15.12:0/2842521704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f247c102340 0x7f247c108bd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.701+0000 7f248141c640 1 --2- 172.21.15.12:0/2842521704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247c101980 0x7f247c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.701+0000 7f248141c640 1 --2- 172.21.15.12:0/2842521704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f247c100790 0x7f247c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.701+0000 7f248141c640 1 -- 172.21.15.12:0/2842521704 >> 172.21.15.12:0/2842521704 conn(0x7f247c0fbd20 msgr2=0x7f247c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.701+0000 7f248141c640 1 -- 172.21.15.12:0/2842521704 shutdown_connections 2024-05-09T09:25:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.701+0000 7f248141c640 1 -- 172.21.15.12:0/2842521704 wait complete. 2024-05-09T09:25:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.702+0000 7f248141c640 1 Processor -- start 2024-05-09T09:25:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.702+0000 7f248141c640 1 -- start start 2024-05-09T09:25:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.702+0000 7f248141c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f247c100790 0x7f247c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.702+0000 7f248141c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f247c101980 0x7f247c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.702+0000 7f248141c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247c102340 0x7f247c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.702+0000 7f248141c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f247c10c3f0 con 0x7f247c100790 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.702+0000 7f248141c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f247c10c270 con 0x7f247c102340 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.702+0000 7f248141c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f247c10c570 con 0x7f247c101980 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.702+0000 7f247affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f247c100790 0x7f247c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.702+0000 7f247affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f247c100790 0x7f247c197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34052/0 (socket says 172.21.15.12:34052) 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.702+0000 7f247affd640 1 -- 172.21.15.12:0/1424243459 learned_addr learned my addr 172.21.15.12:0/1424243459 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f247b7fe640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247c102340 0x7f247c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f247a7fc640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f247c101980 0x7f247c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f247affd640 1 -- 172.21.15.12:0/1424243459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f247c101980 msgr2=0x7f247c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f247affd640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f247c101980 0x7f247c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f247affd640 1 -- 172.21.15.12:0/1424243459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247c102340 msgr2=0x7f247c19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f247affd640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247c102340 0x7f247c19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f247affd640 1 -- 172.21.15.12:0/1424243459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c19f840 con 0x7f247c100790 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f247b7fe640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247c102340 0x7f247c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:37.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f247affd640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f247c100790 0x7f247c197c30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f246c00ccb0 tx=0x7f246c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:37.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f2473fff640 1 -- 172.21.15.12:0/1424243459 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c004b20 con 0x7f247c100790 2024-05-09T09:25:37.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f248141c640 1 -- 172.21.15.12:0/1424243459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247c19fb30 con 0x7f247c100790 2024-05-09T09:25:37.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f2473fff640 1 -- 172.21.15.12:0/1424243459 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f246c004cc0 con 0x7f247c100790 2024-05-09T09:25:37.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f2473fff640 1 -- 172.21.15.12:0/1424243459 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c005020 con 0x7f247c100790 2024-05-09T09:25:37.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.703+0000 7f248141c640 1 -- 172.21.15.12:0/1424243459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247c1a0080 con 0x7f247c100790 2024-05-09T09:25:37.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.704+0000 7f248141c640 1 -- 172.21.15.12:0/1424243459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247c1a0250 con 0x7f247c100790 2024-05-09T09:25:37.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.705+0000 7f2473fff640 1 -- 172.21.15.12:0/1424243459 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f246c003990 con 0x7f247c100790 2024-05-09T09:25:37.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.706+0000 7f2473fff640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f24680a2b90 0x7f24680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:37.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.706+0000 7f247a7fc640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f24680a2b90 0x7f24680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:37.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.706+0000 7f2473fff640 1 -- 172.21.15.12:0/1424243459 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f246c0cf6f0 con 0x7f247c100790 2024-05-09T09:25:37.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.706+0000 7f247a7fc640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f24680a2b90 0x7f24680a5050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2464005fd0 tx=0x7f246400b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:37.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.710+0000 7f2473fff640 1 -- 172.21.15.12:0/1424243459 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f246c014030 con 0x7f247c100790 2024-05-09T09:25:37.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.897+0000 7f248141c640 1 -- 172.21.15.12:0/1424243459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f247c1a0420 con 0x7f247c100790 2024-05-09T09:25:37.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.897+0000 7f2473fff640 1 -- 172.21.15.12:0/1424243459 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f246c09c280 con 0x7f247c100790 2024-05-09T09:25:37.897 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:25:37.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.899+0000 7f248141c640 1 -- 172.21.15.12:0/1424243459 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f24680a2b90 msgr2=0x7f24680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.899+0000 7f248141c640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f24680a2b90 0x7f24680a5050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2464005fd0 tx=0x7f246400b590 comp rx=0 tx=0).stop 2024-05-09T09:25:37.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.899+0000 7f248141c640 1 -- 172.21.15.12:0/1424243459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f247c100790 msgr2=0x7f247c197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:37.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.899+0000 7f248141c640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f247c100790 0x7f247c197c30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f246c00ccb0 tx=0x7f246c00a620 comp rx=0 tx=0).stop 2024-05-09T09:25:37.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.899+0000 7f248141c640 1 -- 172.21.15.12:0/1424243459 shutdown_connections 2024-05-09T09:25:37.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.899+0000 7f248141c640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f24680a2b90 0x7f24680a5050 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.899+0000 7f248141c640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247c102340 0x7f247c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.899+0000 7f248141c640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f247c101980 0x7f247c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.899+0000 7f248141c640 1 --2- 172.21.15.12:0/1424243459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f247c100790 0x7f247c197c30 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:37.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.899+0000 7f248141c640 1 -- 172.21.15.12:0/1424243459 >> 172.21.15.12:0/1424243459 conn(0x7f247c0fbd20 msgr2=0x7f247c0fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:37.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.899+0000 7f248141c640 1 -- 172.21.15.12:0/1424243459 shutdown_connections 2024-05-09T09:25:37.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:37.899+0000 7f248141c640 1 -- 172.21.15.12:0/1424243459 wait complete. 2024-05-09T09:25:37.910 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:25:38.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.019+0000 7f0d87342640 1 -- 172.21.15.12:0/2142270432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d80104ca0 msgr2=0x7f0d80107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.019+0000 7f0d87342640 1 --2- 172.21.15.12:0/2142270432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d80104ca0 0x7f0d80107090 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009900 tx=0x7f0d7402a410 comp rx=0 tx=0).stop 2024-05-09T09:25:38.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.019+0000 7f0d87342640 1 -- 172.21.15.12:0/2142270432 shutdown_connections 2024-05-09T09:25:38.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.019+0000 7f0d87342640 1 --2- 172.21.15.12:0/2142270432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d801075d0 0x7f0d801099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.019+0000 7f0d87342640 1 --2- 172.21.15.12:0/2142270432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d80104ca0 0x7f0d80107090 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.019+0000 7f0d87342640 1 --2- 172.21.15.12:0/2142270432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d80102370 0x7f0d80104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.019+0000 7f0d87342640 1 -- 172.21.15.12:0/2142270432 >> 172.21.15.12:0/2142270432 conn(0x7f0d800fbce0 msgr2=0x7f0d800fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:38.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.019+0000 7f0d87342640 1 -- 172.21.15.12:0/2142270432 shutdown_connections 2024-05-09T09:25:38.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.019+0000 7f0d87342640 1 -- 172.21.15.12:0/2142270432 wait complete. 2024-05-09T09:25:38.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.019+0000 7f0d87342640 1 Processor -- start 2024-05-09T09:25:38.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.019+0000 7f0d87342640 1 -- start start 2024-05-09T09:25:38.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d87342640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d80102370 0x7f0d80197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d87342640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d80104ca0 0x7f0d80198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d87342640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d801075d0 0x7f0d8019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d87342640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010d4d0 con 0x7f0d80104ca0 2024-05-09T09:25:38.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d87342640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010d350 con 0x7f0d80102370 2024-05-09T09:25:38.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d87342640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010d650 con 0x7f0d801075d0 2024-05-09T09:25:38.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d848dd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d80104ca0 0x7f0d80198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d848dd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d80104ca0 0x7f0d80198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34076/0 (socket says 172.21.15.12:34076) 2024-05-09T09:25:38.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d848dd640 1 -- 172.21.15.12:0/3101395453 learned_addr learned my addr 172.21.15.12:0/3101395453 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:38.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d858df640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d801075d0 0x7f0d8019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d850de640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d80102370 0x7f0d80197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d848dd640 1 -- 172.21.15.12:0/3101395453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d801075d0 msgr2=0x7f0d8019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d848dd640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d801075d0 0x7f0d8019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d848dd640 1 -- 172.21.15.12:0/3101395453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d80102370 msgr2=0x7f0d80197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d848dd640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d80102370 0x7f0d80197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d848dd640 1 -- 172.21.15.12:0/3101395453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8019f830 con 0x7f0d80104ca0 2024-05-09T09:25:38.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d848dd640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d80104ca0 0x7f0d80198160 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009230 tx=0x7f0d74031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:38.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d7e7fc640 1 -- 172.21.15.12:0/3101395453 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7403f030 con 0x7f0d80104ca0 2024-05-09T09:25:38.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d7e7fc640 1 -- 172.21.15.12:0/3101395453 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0d74003ac0 con 0x7f0d80104ca0 2024-05-09T09:25:38.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d87342640 1 -- 172.21.15.12:0/3101395453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8019fac0 con 0x7f0d80104ca0 2024-05-09T09:25:38.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d7e7fc640 1 -- 172.21.15.12:0/3101395453 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74035070 con 0x7f0d80104ca0 2024-05-09T09:25:38.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.020+0000 7f0d87342640 1 -- 172.21.15.12:0/3101395453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d801a0000 con 0x7f0d80104ca0 2024-05-09T09:25:38.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.022+0000 7f0d87342640 1 -- 172.21.15.12:0/3101395453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d800fd7d0 con 0x7f0d80104ca0 2024-05-09T09:25:38.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.023+0000 7f0d7e7fc640 1 -- 172.21.15.12:0/3101395453 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0d74042030 con 0x7f0d80104ca0 2024-05-09T09:25:38.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.023+0000 7f0d7e7fc640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0d680a2c10 0x7f0d680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.023+0000 7f0d850de640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0d680a2c10 0x7f0d680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.023+0000 7f0d7e7fc640 1 -- 172.21.15.12:0/3101395453 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f0d740efb50 con 0x7f0d80104ca0 2024-05-09T09:25:38.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.024+0000 7f0d850de640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0d680a2c10 0x7f0d680a50d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0d70005c40 tx=0x7f0d70005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.028+0000 7f0d7e7fc640 1 -- 172.21.15.12:0/3101395453 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0d740bc6e0 con 0x7f0d80104ca0 2024-05-09T09:25:38.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.213+0000 7f0d87342640 1 -- 172.21.15.12:0/3101395453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0d80062460 con 0x7f0d80104ca0 2024-05-09T09:25:38.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.214+0000 7f0d7e7fc640 1 -- 172.21.15.12:0/3101395453 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f0d740bbfe0 con 0x7f0d80104ca0 2024-05-09T09:25:38.214 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:25:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.216+0000 7f0d87342640 1 -- 172.21.15.12:0/3101395453 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0d680a2c10 msgr2=0x7f0d680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.216+0000 7f0d87342640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0d680a2c10 0x7f0d680a50d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0d70005c40 tx=0x7f0d70005bd0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.216+0000 7f0d87342640 1 -- 172.21.15.12:0/3101395453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d80104ca0 msgr2=0x7f0d80198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.216+0000 7f0d87342640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d80104ca0 0x7f0d80198160 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009230 tx=0x7f0d74031ac0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.216+0000 7f0d87342640 1 -- 172.21.15.12:0/3101395453 shutdown_connections 2024-05-09T09:25:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.216+0000 7f0d87342640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0d680a2c10 0x7f0d680a50d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.216+0000 7f0d87342640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d801075d0 0x7f0d8019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.216+0000 7f0d87342640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d80104ca0 0x7f0d80198160 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.216+0000 7f0d87342640 1 --2- 172.21.15.12:0/3101395453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d80102370 0x7f0d80197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.216+0000 7f0d87342640 1 -- 172.21.15.12:0/3101395453 >> 172.21.15.12:0/3101395453 conn(0x7f0d800fbce0 msgr2=0x7f0d80102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.216+0000 7f0d87342640 1 -- 172.21.15.12:0/3101395453 shutdown_connections 2024-05-09T09:25:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.216+0000 7f0d87342640 1 -- 172.21.15.12:0/3101395453 wait complete. 2024-05-09T09:25:38.227 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:25:38.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.335+0000 7f5e430a5640 1 -- 172.21.15.12:0/1290584172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e3c104ca0 msgr2=0x7f5e3c107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.335+0000 7f5e430a5640 1 --2- 172.21.15.12:0/1290584172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e3c104ca0 0x7f5e3c107090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009900 tx=0x7f5e2802a410 comp rx=0 tx=0).stop 2024-05-09T09:25:38.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.335+0000 7f5e430a5640 1 -- 172.21.15.12:0/1290584172 shutdown_connections 2024-05-09T09:25:38.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.335+0000 7f5e430a5640 1 --2- 172.21.15.12:0/1290584172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c1075d0 0x7f5e3c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.335+0000 7f5e430a5640 1 --2- 172.21.15.12:0/1290584172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e3c104ca0 0x7f5e3c107090 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.335+0000 7f5e430a5640 1 --2- 172.21.15.12:0/1290584172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3c102370 0x7f5e3c104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.335+0000 7f5e430a5640 1 -- 172.21.15.12:0/1290584172 >> 172.21.15.12:0/1290584172 conn(0x7f5e3c0fbd20 msgr2=0x7f5e3c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:38.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.335+0000 7f5e430a5640 1 -- 172.21.15.12:0/1290584172 shutdown_connections 2024-05-09T09:25:38.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.335+0000 7f5e430a5640 1 -- 172.21.15.12:0/1290584172 wait complete. 2024-05-09T09:25:38.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.336+0000 7f5e430a5640 1 Processor -- start 2024-05-09T09:25:38.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.336+0000 7f5e430a5640 1 -- start start 2024-05-09T09:25:38.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.336+0000 7f5e430a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c102370 0x7f5e3c197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.336+0000 7f5e430a5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e3c104ca0 0x7f5e3c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.336+0000 7f5e430a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3c1075d0 0x7f5e3c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.336+0000 7f5e430a5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c10d4f0 con 0x7f5e3c104ca0 2024-05-09T09:25:38.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.336+0000 7f5e430a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c10d370 con 0x7f5e3c1075d0 2024-05-09T09:25:38.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.336+0000 7f5e430a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c10d670 con 0x7f5e3c102370 2024-05-09T09:25:38.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.336+0000 7f5e3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e3c104ca0 0x7f5e3c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.336+0000 7f5e3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e3c104ca0 0x7f5e3c198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34092/0 (socket says 172.21.15.12:34092) 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.336+0000 7f5e3bfff640 1 -- 172.21.15.12:0/1749410897 learned_addr learned my addr 172.21.15.12:0/1749410897 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.336+0000 7f5e41642640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3c1075d0 0x7f5e3c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.336+0000 7f5e40e41640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c102370 0x7f5e3c197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.337+0000 7f5e3bfff640 1 -- 172.21.15.12:0/1749410897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c102370 msgr2=0x7f5e3c197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.337+0000 7f5e3bfff640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c102370 0x7f5e3c197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.337+0000 7f5e3bfff640 1 -- 172.21.15.12:0/1749410897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3c1075d0 msgr2=0x7f5e3c19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.337+0000 7f5e3bfff640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3c1075d0 0x7f5e3c19f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.337+0000 7f5e3bfff640 1 -- 172.21.15.12:0/1749410897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c19f7f0 con 0x7f5e3c104ca0 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.337+0000 7f5e40e41640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c102370 0x7f5e3c197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.337+0000 7f5e41642640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3c1075d0 0x7f5e3c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.337+0000 7f5e3bfff640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e3c104ca0 0x7f5e3c198110 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5e28005ab0 tx=0x7f5e280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.337+0000 7f5e39ffb640 1 -- 172.21.15.12:0/1749410897 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2803f030 con 0x7f5e3c104ca0 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.337+0000 7f5e39ffb640 1 -- 172.21.15.12:0/1749410897 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5e28003ac0 con 0x7f5e3c104ca0 2024-05-09T09:25:38.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.337+0000 7f5e39ffb640 1 -- 172.21.15.12:0/1749410897 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28035070 con 0x7f5e3c104ca0 2024-05-09T09:25:38.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.337+0000 7f5e430a5640 1 -- 172.21.15.12:0/1749410897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3c19fa80 con 0x7f5e3c104ca0 2024-05-09T09:25:38.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.337+0000 7f5e430a5640 1 -- 172.21.15.12:0/1749410897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e3c19ffc0 con 0x7f5e3c104ca0 2024-05-09T09:25:38.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.338+0000 7f5e430a5640 1 -- 172.21.15.12:0/1749410897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e3c0fd810 con 0x7f5e3c104ca0 2024-05-09T09:25:38.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.339+0000 7f5e39ffb640 1 -- 172.21.15.12:0/1749410897 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5e28042030 con 0x7f5e3c104ca0 2024-05-09T09:25:38.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.340+0000 7f5e39ffb640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5e2c0a2c10 0x7f5e2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.340+0000 7f5e39ffb640 1 -- 172.21.15.12:0/1749410897 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f5e280efa80 con 0x7f5e3c104ca0 2024-05-09T09:25:38.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.340+0000 7f5e40e41640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5e2c0a2c10 0x7f5e2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.341+0000 7f5e40e41640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5e2c0a2c10 0x7f5e2c0a50d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009b40 tx=0x7f5e30005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:38.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.345+0000 7f5e39ffb640 1 -- 172.21.15.12:0/1749410897 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5e280f1e20 con 0x7f5e3c104ca0 2024-05-09T09:25:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:38 smithi012 ceph-mon[30140]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:25:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:38 smithi012 ceph-mon[30140]: pgmap v369: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-09T09:25:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:38 smithi012 ceph-mon[30140]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:25:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:38 smithi012 ceph-mon[30140]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:25:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:38 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/252637777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:38.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:38 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1424243459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:38.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:38 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3101395453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:38.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:38 smithi170 ceph-mon[33632]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:25:38.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:38 smithi170 ceph-mon[33632]: pgmap v369: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-09T09:25:38.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:38 smithi170 ceph-mon[33632]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:25:38.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:38 smithi170 ceph-mon[33632]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:25:38.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:38 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/252637777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:38.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:38 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1424243459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:38.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:38 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3101395453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:38 smithi016 ceph-mon[33335]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:25:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:38 smithi016 ceph-mon[33335]: pgmap v369: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-09T09:25:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:38 smithi016 ceph-mon[33335]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:25:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:38 smithi016 ceph-mon[33335]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:25:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:38 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/252637777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:38 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1424243459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:38 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3101395453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:38.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.531+0000 7f5e430a5640 1 -- 172.21.15.12:0/1749410897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5e3c062430 con 0x7f5e3c104ca0 2024-05-09T09:25:38.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.532+0000 7f5e39ffb640 1 -- 172.21.15.12:0/1749410897 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f5e280bc5b0 con 0x7f5e3c104ca0 2024-05-09T09:25:38.532 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T09:25:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.534+0000 7f5e430a5640 1 -- 172.21.15.12:0/1749410897 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5e2c0a2c10 msgr2=0x7f5e2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.534+0000 7f5e430a5640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5e2c0a2c10 0x7f5e2c0a50d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009b40 tx=0x7f5e30005e50 comp rx=0 tx=0).stop 2024-05-09T09:25:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.534+0000 7f5e430a5640 1 -- 172.21.15.12:0/1749410897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e3c104ca0 msgr2=0x7f5e3c198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.534+0000 7f5e430a5640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e3c104ca0 0x7f5e3c198110 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5e28005ab0 tx=0x7f5e280042f0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.534+0000 7f5e430a5640 1 -- 172.21.15.12:0/1749410897 shutdown_connections 2024-05-09T09:25:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.534+0000 7f5e430a5640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5e2c0a2c10 0x7f5e2c0a50d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.534+0000 7f5e430a5640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3c1075d0 0x7f5e3c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.534+0000 7f5e430a5640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e3c104ca0 0x7f5e3c198110 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.534+0000 7f5e430a5640 1 --2- 172.21.15.12:0/1749410897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c102370 0x7f5e3c197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.534+0000 7f5e430a5640 1 -- 172.21.15.12:0/1749410897 >> 172.21.15.12:0/1749410897 conn(0x7f5e3c0fbd20 msgr2=0x7f5e3c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.534+0000 7f5e430a5640 1 -- 172.21.15.12:0/1749410897 shutdown_connections 2024-05-09T09:25:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.534+0000 7f5e430a5640 1 -- 172.21.15.12:0/1749410897 wait complete. 2024-05-09T09:25:38.545 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:25:38.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.654+0000 7fe255e4d640 1 -- 172.21.15.12:0/3517888812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250104c40 msgr2=0x7fe250107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.654+0000 7fe255e4d640 1 --2- 172.21.15.12:0/3517888812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250104c40 0x7fe250107030 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe238009900 tx=0x7fe23802a410 comp rx=0 tx=0).stop 2024-05-09T09:25:38.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.654+0000 7fe255e4d640 1 -- 172.21.15.12:0/3517888812 shutdown_connections 2024-05-09T09:25:38.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.654+0000 7fe255e4d640 1 --2- 172.21.15.12:0/3517888812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe250107570 0x7fe2501099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.654+0000 7fe255e4d640 1 --2- 172.21.15.12:0/3517888812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250104c40 0x7fe250107030 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.654+0000 7fe255e4d640 1 --2- 172.21.15.12:0/3517888812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe250102310 0x7fe250104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.654+0000 7fe255e4d640 1 -- 172.21.15.12:0/3517888812 >> 172.21.15.12:0/3517888812 conn(0x7fe2500fbd20 msgr2=0x7fe2500fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:38.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.654+0000 7fe255e4d640 1 -- 172.21.15.12:0/3517888812 shutdown_connections 2024-05-09T09:25:38.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.654+0000 7fe255e4d640 1 -- 172.21.15.12:0/3517888812 wait complete. 2024-05-09T09:25:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.655+0000 7fe255e4d640 1 Processor -- start 2024-05-09T09:25:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.655+0000 7fe255e4d640 1 -- start start 2024-05-09T09:25:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.655+0000 7fe255e4d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe250102310 0x7fe250197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.655+0000 7fe255e4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe250104c40 0x7fe250198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.655+0000 7fe255e4d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250107570 0x7fe25019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.655+0000 7fe255e4d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe25010d4d0 con 0x7fe250107570 2024-05-09T09:25:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.655+0000 7fe255e4d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe25010d350 con 0x7fe250104c40 2024-05-09T09:25:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.655+0000 7fe255e4d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe25010d650 con 0x7fe250102310 2024-05-09T09:25:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.655+0000 7fe24ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250107570 0x7fe25019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.655+0000 7fe24ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250107570 0x7fe25019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34118/0 (socket says 172.21.15.12:34118) 2024-05-09T09:25:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.655+0000 7fe24ffff640 1 -- 172.21.15.12:0/2084662681 learned_addr learned my addr 172.21.15.12:0/2084662681 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.656+0000 7fe24effd640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe250104c40 0x7fe250198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.656+0000 7fe24f7fe640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe250102310 0x7fe250197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.656+0000 7fe24ffff640 1 -- 172.21.15.12:0/2084662681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe250102310 msgr2=0x7fe250197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.656+0000 7fe24ffff640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe250102310 0x7fe250197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.656+0000 7fe24ffff640 1 -- 172.21.15.12:0/2084662681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe250104c40 msgr2=0x7fe250198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.656+0000 7fe24ffff640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe250104c40 0x7fe250198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.656+0000 7fe24ffff640 1 -- 172.21.15.12:0/2084662681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25019f800 con 0x7fe250107570 2024-05-09T09:25:38.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.656+0000 7fe24ffff640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250107570 0x7fe25019f190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe24800eaa0 tx=0x7fe24800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:38.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.656+0000 7fe24cff9640 1 -- 172.21.15.12:0/2084662681 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24800cdb0 con 0x7fe250107570 2024-05-09T09:25:38.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.656+0000 7fe255e4d640 1 -- 172.21.15.12:0/2084662681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25019faf0 con 0x7fe250107570 2024-05-09T09:25:38.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.656+0000 7fe24cff9640 1 -- 172.21.15.12:0/2084662681 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe248005020 con 0x7fe250107570 2024-05-09T09:25:38.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.656+0000 7fe255e4d640 1 -- 172.21.15.12:0/2084662681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2501a0010 con 0x7fe250107570 2024-05-09T09:25:38.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.656+0000 7fe24cff9640 1 -- 172.21.15.12:0/2084662681 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24801f060 con 0x7fe250107570 2024-05-09T09:25:38.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.657+0000 7fe255e4d640 1 -- 172.21.15.12:0/2084662681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2500fd810 con 0x7fe250107570 2024-05-09T09:25:38.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.659+0000 7fe24cff9640 1 -- 172.21.15.12:0/2084662681 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe2480051d0 con 0x7fe250107570 2024-05-09T09:25:38.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.660+0000 7fe24cff9640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe23c0a2b90 0x7fe23c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.660+0000 7fe24f7fe640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe23c0a2b90 0x7fe23c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.660+0000 7fe24cff9640 1 -- 172.21.15.12:0/2084662681 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fe2480cfab0 con 0x7fe250107570 2024-05-09T09:25:38.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.660+0000 7fe24f7fe640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe23c0a2b90 0x7fe23c0a5050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe240005fd0 tx=0x7fe240005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:38.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.664+0000 7fe24cff9640 1 -- 172.21.15.12:0/2084662681 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe248016030 con 0x7fe250107570 2024-05-09T09:25:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.849+0000 7fe255e4d640 1 -- 172.21.15.12:0/2084662681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe25002d1c0 con 0x7fe250107570 2024-05-09T09:25:38.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.849+0000 7fe24cff9640 1 -- 172.21.15.12:0/2084662681 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fe24809c5e0 con 0x7fe250107570 2024-05-09T09:25:38.849 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:25:38.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.851+0000 7fe255e4d640 1 -- 172.21.15.12:0/2084662681 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe23c0a2b90 msgr2=0x7fe23c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.851+0000 7fe255e4d640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe23c0a2b90 0x7fe23c0a5050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe240005fd0 tx=0x7fe240005850 comp rx=0 tx=0).stop 2024-05-09T09:25:38.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.851+0000 7fe255e4d640 1 -- 172.21.15.12:0/2084662681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250107570 msgr2=0x7fe25019f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.851+0000 7fe255e4d640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250107570 0x7fe25019f190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe24800eaa0 tx=0x7fe24800ef60 comp rx=0 tx=0).stop 2024-05-09T09:25:38.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.851+0000 7fe255e4d640 1 -- 172.21.15.12:0/2084662681 shutdown_connections 2024-05-09T09:25:38.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.851+0000 7fe255e4d640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe23c0a2b90 0x7fe23c0a5050 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.851+0000 7fe255e4d640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250107570 0x7fe25019f190 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.851+0000 7fe255e4d640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe250104c40 0x7fe250198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.851+0000 7fe255e4d640 1 --2- 172.21.15.12:0/2084662681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe250102310 0x7fe250197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.851+0000 7fe255e4d640 1 -- 172.21.15.12:0/2084662681 >> 172.21.15.12:0/2084662681 conn(0x7fe2500fbd20 msgr2=0x7fe250102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:38.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.852+0000 7fe255e4d640 1 -- 172.21.15.12:0/2084662681 shutdown_connections 2024-05-09T09:25:38.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.852+0000 7fe255e4d640 1 -- 172.21.15.12:0/2084662681 wait complete. 2024-05-09T09:25:38.863 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:25:38.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.968+0000 7fd5f4b89640 1 -- 172.21.15.12:0/910862249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5f0101440 msgr2=0x7fd5f01018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.968+0000 7fd5f4b89640 1 --2- 172.21.15.12:0/910862249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5f0101440 0x7fd5f01018c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009900 tx=0x7fd5e802a410 comp rx=0 tx=0).stop 2024-05-09T09:25:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.969+0000 7fd5f4b89640 1 -- 172.21.15.12:0/910862249 shutdown_connections 2024-05-09T09:25:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.969+0000 7fd5f4b89640 1 --2- 172.21.15.12:0/910862249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5f0101440 0x7fd5f01018c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.969+0000 7fd5f4b89640 1 --2- 172.21.15.12:0/910862249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f0100aa0 0x7fd5f0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.969+0000 7fd5f4b89640 1 --2- 172.21.15.12:0/910862249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5f0100140 0x7fd5f0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.969+0000 7fd5f4b89640 1 -- 172.21.15.12:0/910862249 >> 172.21.15.12:0/910862249 conn(0x7fd5f00fbce0 msgr2=0x7fd5f00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.969+0000 7fd5f4b89640 1 -- 172.21.15.12:0/910862249 shutdown_connections 2024-05-09T09:25:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.969+0000 7fd5f4b89640 1 -- 172.21.15.12:0/910862249 wait complete. 2024-05-09T09:25:38.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.969+0000 7fd5f4b89640 1 Processor -- start 2024-05-09T09:25:38.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.969+0000 7fd5f4b89640 1 -- start start 2024-05-09T09:25:38.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.969+0000 7fd5f4b89640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5f0100140 0x7fd5f0199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.969+0000 7fd5f4b89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f0100aa0 0x7fd5f019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5f4b89640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5f0101440 0x7fd5f01a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5f4b89640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f010f730 con 0x7fd5f0100140 2024-05-09T09:25:38.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5f4b89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f010f5b0 con 0x7fd5f0100aa0 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5f4b89640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f010f8b0 con 0x7fd5f0101440 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5ee59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5f0100140 0x7fd5f0199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5ee59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5f0100140 0x7fd5f0199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34144/0 (socket says 172.21.15.12:34144) 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5ee59c640 1 -- 172.21.15.12:0/806019904 learned_addr learned my addr 172.21.15.12:0/806019904 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5edd9b640 1 --2- 172.21.15.12:0/806019904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f0100aa0 0x7fd5f019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5ee59c640 1 -- 172.21.15.12:0/806019904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5f0101440 msgr2=0x7fd5f01a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5ee59c640 1 --2- 172.21.15.12:0/806019904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5f0101440 0x7fd5f01a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5ee59c640 1 -- 172.21.15.12:0/806019904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f0100aa0 msgr2=0x7fd5f019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5ee59c640 1 --2- 172.21.15.12:0/806019904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f0100aa0 0x7fd5f019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5ee59c640 1 -- 172.21.15.12:0/806019904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f01a18b0 con 0x7fd5f0100140 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5ee59c640 1 --2- 172.21.15.12:0/806019904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5f0100140 0x7fd5f0199c70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000c920 tx=0x7fd5e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5df7fe640 1 -- 172.21.15.12:0/806019904 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0007c70 con 0x7fd5f0100140 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5f4b89640 1 -- 172.21.15.12:0/806019904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f01a1ba0 con 0x7fd5f0100140 2024-05-09T09:25:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5df7fe640 1 -- 172.21.15.12:0/806019904 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd5e0007e10 con 0x7fd5f0100140 2024-05-09T09:25:38.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5df7fe640 1 -- 172.21.15.12:0/806019904 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0018800 con 0x7fd5f0100140 2024-05-09T09:25:38.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.970+0000 7fd5f4b89640 1 -- 172.21.15.12:0/806019904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f01a20f0 con 0x7fd5f0100140 2024-05-09T09:25:38.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.971+0000 7fd5f4b89640 1 -- 172.21.15.12:0/806019904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f010af60 con 0x7fd5f0100140 2024-05-09T09:25:38.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.973+0000 7fd5df7fe640 1 -- 172.21.15.12:0/806019904 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd5e0018aa0 con 0x7fd5f0100140 2024-05-09T09:25:38.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.973+0000 7fd5df7fe640 1 --2- 172.21.15.12:0/806019904 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd5d80a2b90 0x7fd5d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:38.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.973+0000 7fd5edd9b640 1 --2- 172.21.15.12:0/806019904 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd5d80a2b90 0x7fd5d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:38.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.973+0000 7fd5df7fe640 1 -- 172.21.15.12:0/806019904 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd5e00cf650 con 0x7fd5f0100140 2024-05-09T09:25:38.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.974+0000 7fd5edd9b640 1 --2- 172.21.15.12:0/806019904 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd5d80a2b90 0x7fd5d80a5050 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4005c40 tx=0x7fd5e4005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:38.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:38.978+0000 7fd5df7fe640 1 -- 172.21.15.12:0/806019904 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd5e0014030 con 0x7fd5f0100140 2024-05-09T09:25:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.200+0000 7fd5f4b89640 1 -- 172.21.15.12:0/806019904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5f0062430 con 0x7fd5f0100140 2024-05-09T09:25:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.201+0000 7fd5df7fe640 1 -- 172.21.15.12:0/806019904 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11931 (secure 0 0 0) 0x7fd5e001f300 con 0x7fd5f0100140 2024-05-09T09:25:39.201 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:39.201 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:25:31.411273+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[14628,24484,24515,24523,24502]},"id":1}]} 2024-05-09T09:25:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-05-09T09:25:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.203+0000 7fd5f4b89640 1 -- 172.21.15.12:0/806019904 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd5d80a2b90 msgr2=0x7fd5d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.203+0000 7fd5f4b89640 1 --2- 172.21.15.12:0/806019904 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd5d80a2b90 0x7fd5d80a5050 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4005c40 tx=0x7fd5e4005bd0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.203+0000 7fd5f4b89640 1 -- 172.21.15.12:0/806019904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5f0100140 msgr2=0x7fd5f0199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.203+0000 7fd5f4b89640 1 --2- 172.21.15.12:0/806019904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5f0100140 0x7fd5f0199c70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000c920 tx=0x7fd5e000cde0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.203+0000 7fd5f4b89640 1 -- 172.21.15.12:0/806019904 shutdown_connections 2024-05-09T09:25:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.203+0000 7fd5f4b89640 1 --2- 172.21.15.12:0/806019904 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd5d80a2b90 0x7fd5d80a5050 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.203+0000 7fd5f4b89640 1 --2- 172.21.15.12:0/806019904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5f0101440 0x7fd5f01a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.203+0000 7fd5f4b89640 1 --2- 172.21.15.12:0/806019904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f0100aa0 0x7fd5f019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.203+0000 7fd5f4b89640 1 --2- 172.21.15.12:0/806019904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5f0100140 0x7fd5f0199c70 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.203+0000 7fd5f4b89640 1 -- 172.21.15.12:0/806019904 >> 172.21.15.12:0/806019904 conn(0x7fd5f00fbce0 msgr2=0x7fd5f00fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.203+0000 7fd5f4b89640 1 -- 172.21.15.12:0/806019904 shutdown_connections 2024-05-09T09:25:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.203+0000 7fd5f4b89640 1 -- 172.21.15.12:0/806019904 wait complete. 2024-05-09T09:25:39.214 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:25:39.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.320+0000 7fc39405d640 1 -- 172.21.15.12:0/48315739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc38c102a70 msgr2=0x7fc38c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.320+0000 7fc39405d640 1 --2- 172.21.15.12:0/48315739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc38c102a70 0x7fc38c102ef0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc380009900 tx=0x7fc38002a410 comp rx=0 tx=0).stop 2024-05-09T09:25:39.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.320+0000 7fc39405d640 1 -- 172.21.15.12:0/48315739 shutdown_connections 2024-05-09T09:25:39.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.320+0000 7fc39405d640 1 --2- 172.21.15.12:0/48315739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c103430 0x7fc38c109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.320+0000 7fc39405d640 1 --2- 172.21.15.12:0/48315739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc38c102a70 0x7fc38c102ef0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.320+0000 7fc39405d640 1 --2- 172.21.15.12:0/48315739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc38c101880 0x7fc38c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.320+0000 7fc39405d640 1 -- 172.21.15.12:0/48315739 >> 172.21.15.12:0/48315739 conn(0x7fc38c0fce10 msgr2=0x7fc38c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.320+0000 7fc39405d640 1 -- 172.21.15.12:0/48315739 shutdown_connections 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.320+0000 7fc39405d640 1 -- 172.21.15.12:0/48315739 wait complete. 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.321+0000 7fc39405d640 1 Processor -- start 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.321+0000 7fc39405d640 1 -- start start 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.321+0000 7fc39405d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c101880 0x7fc38c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.321+0000 7fc39405d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc38c102a70 0x7fc38c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.321+0000 7fc39405d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc38c103430 0x7fc38c19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.321+0000 7fc39405d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c10d4e0 con 0x7fc38c103430 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.321+0000 7fc39405d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c10d360 con 0x7fc38c101880 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.321+0000 7fc39405d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c10d660 con 0x7fc38c102a70 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.321+0000 7fc3925fa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc38c103430 0x7fc38c19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.321+0000 7fc3925fa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc38c103430 0x7fc38c19f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34160/0 (socket says 172.21.15.12:34160) 2024-05-09T09:25:39.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.321+0000 7fc3925fa640 1 -- 172.21.15.12:0/3723174518 learned_addr learned my addr 172.21.15.12:0/3723174518 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.321+0000 7fc391df9640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c101880 0x7fc38c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.321+0000 7fc3915f8640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc38c102a70 0x7fc38c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.322+0000 7fc3925fa640 1 -- 172.21.15.12:0/3723174518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc38c102a70 msgr2=0x7fc38c198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.322+0000 7fc3925fa640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc38c102a70 0x7fc38c198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.322+0000 7fc3925fa640 1 -- 172.21.15.12:0/3723174518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c101880 msgr2=0x7fc38c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.322+0000 7fc3925fa640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c101880 0x7fc38c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.322+0000 7fc3925fa640 1 -- 172.21.15.12:0/3723174518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38c19f7e0 con 0x7fc38c103430 2024-05-09T09:25:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.322+0000 7fc3925fa640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc38c103430 0x7fc38c19f170 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc384009fd0 tx=0x7fc38400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.322+0000 7fc37effd640 1 -- 172.21.15.12:0/3723174518 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384017070 con 0x7fc38c103430 2024-05-09T09:25:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.322+0000 7fc37effd640 1 -- 172.21.15.12:0/3723174518 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc384003e40 con 0x7fc38c103430 2024-05-09T09:25:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.322+0000 7fc37effd640 1 -- 172.21.15.12:0/3723174518 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3840106c0 con 0x7fc38c103430 2024-05-09T09:25:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.322+0000 7fc39405d640 1 -- 172.21.15.12:0/3723174518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38c19fad0 con 0x7fc38c103430 2024-05-09T09:25:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.322+0000 7fc39405d640 1 -- 172.21.15.12:0/3723174518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc38c19ffa0 con 0x7fc38c103430 2024-05-09T09:25:39.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.323+0000 7fc39405d640 1 -- 172.21.15.12:0/3723174518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc38c068aa0 con 0x7fc38c103430 2024-05-09T09:25:39.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.324+0000 7fc37effd640 1 -- 172.21.15.12:0/3723174518 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc384003970 con 0x7fc38c103430 2024-05-09T09:25:39.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.325+0000 7fc37effd640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc3780a2b90 0x7fc3780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.325+0000 7fc391df9640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc3780a2b90 0x7fc3780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.325+0000 7fc37effd640 1 -- 172.21.15.12:0/3723174518 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fc3840cf150 con 0x7fc38c103430 2024-05-09T09:25:39.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.325+0000 7fc391df9640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc3780a2b90 0x7fc3780a5050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc388005fd0 tx=0x7fc38800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.329+0000 7fc37effd640 1 -- 172.21.15.12:0/3723174518 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc384003c60 con 0x7fc38c103430 2024-05-09T09:25:39.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:39 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1749410897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:39.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:39 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2084662681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:39.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:39 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/806019904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:39.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:39 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1749410897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:39.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:39 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2084662681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:39.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:39 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/806019904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.520+0000 7fc39405d640 1 -- 172.21.15.12:0/3723174518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc38c062430 con 0x7fc38c103430 2024-05-09T09:25:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.522+0000 7fc37effd640 1 -- 172.21.15.12:0/3723174518 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24762 (secure 0 0 0) 0x7fc38409b630 con 0x7fc38c103430 2024-05-09T09:25:39.522 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:25:39.522 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":93,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:25:06.152764+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:25:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.524+0000 7fc39405d640 1 -- 172.21.15.12:0/3723174518 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc3780a2b90 msgr2=0x7fc3780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.524+0000 7fc39405d640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc3780a2b90 0x7fc3780a5050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc388005fd0 tx=0x7fc38800b390 comp rx=0 tx=0).stop 2024-05-09T09:25:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.524+0000 7fc39405d640 1 -- 172.21.15.12:0/3723174518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc38c103430 msgr2=0x7fc38c19f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.524+0000 7fc39405d640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc38c103430 0x7fc38c19f170 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc384009fd0 tx=0x7fc38400b260 comp rx=0 tx=0).stop 2024-05-09T09:25:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.524+0000 7fc39405d640 1 -- 172.21.15.12:0/3723174518 shutdown_connections 2024-05-09T09:25:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.524+0000 7fc39405d640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc3780a2b90 0x7fc3780a5050 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.524+0000 7fc39405d640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc38c103430 0x7fc38c19f170 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.524+0000 7fc39405d640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc38c102a70 0x7fc38c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.524+0000 7fc39405d640 1 --2- 172.21.15.12:0/3723174518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c101880 0x7fc38c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.524+0000 7fc39405d640 1 -- 172.21.15.12:0/3723174518 >> 172.21.15.12:0/3723174518 conn(0x7fc38c0fce10 msgr2=0x7fc38c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.525+0000 7fc39405d640 1 -- 172.21.15.12:0/3723174518 shutdown_connections 2024-05-09T09:25:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.525+0000 7fc39405d640 1 -- 172.21.15.12:0/3723174518 wait complete. 2024-05-09T09:25:39.535 DEBUG:tasks.fwd_scrub:Yielding 2024-05-09T09:25:39.536 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-09T09:25:39.546 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-09T09:25:39.547 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-09T09:25:39.547 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-09T09:25:39.547 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:25:39.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:39.548 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:25:39.550 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T09:25:39.563 INFO:tasks.workunit:Pulling workunits from ref be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T09:25:39.564 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T09:25:39.564 INFO:tasks.workunit:timeout=3h 2024-05-09T09:25:39.564 INFO:tasks.workunit:cleanup=True 2024-05-09T09:25:39.564 DEBUG:teuthology.orchestra.run.smithi012:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T09:25:39.581 INFO:teuthology.orchestra.run.smithi012.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-09T09:25:39.581 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-09T09:25:39.581 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 72h/114d Inode: 1 Links: 2 2024-05-09T09:25:39.581 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:25:39.582 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-09T09:25:39.582 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-09 09:23:33.390452940 +0000 2024-05-09T09:25:39.582 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-09 09:23:33.390452940 +0000 2024-05-09T09:25:39.582 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-09 09:25:30.538993251 +0000 2024-05-09T09:25:39.582 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-05-09T09:25:39.582 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-09T09:25:39.582 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-09T09:25:39.656 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T09:25:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.660+0000 7f31cf5b0640 1 -- 172.21.15.12:0/1878714332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c8104ca0 msgr2=0x7f31c8107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.660+0000 7f31cf5b0640 1 --2- 172.21.15.12:0/1878714332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c8104ca0 0x7f31c8107090 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009900 tx=0x7f31b802a410 comp rx=0 tx=0).stop 2024-05-09T09:25:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.660+0000 7f31cf5b0640 1 -- 172.21.15.12:0/1878714332 shutdown_connections 2024-05-09T09:25:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.660+0000 7f31cf5b0640 1 --2- 172.21.15.12:0/1878714332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31c81075d0 0x7f31c81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.660+0000 7f31cf5b0640 1 --2- 172.21.15.12:0/1878714332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c8104ca0 0x7f31c8107090 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.660+0000 7f31cf5b0640 1 --2- 172.21.15.12:0/1878714332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31c8102370 0x7f31c8104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.660+0000 7f31cf5b0640 1 -- 172.21.15.12:0/1878714332 >> 172.21.15.12:0/1878714332 conn(0x7f31c80fbd00 msgr2=0x7f31c80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.660+0000 7f31cf5b0640 1 -- 172.21.15.12:0/1878714332 shutdown_connections 2024-05-09T09:25:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.660+0000 7f31cf5b0640 1 -- 172.21.15.12:0/1878714332 wait complete. 2024-05-09T09:25:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31cf5b0640 1 Processor -- start 2024-05-09T09:25:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31cf5b0640 1 -- start start 2024-05-09T09:25:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31cf5b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31c8102370 0x7f31c8101c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31cf5b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c8104ca0 0x7f31c8100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31cf5b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31c81075d0 0x7f31c8100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31cf5b0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f31c810d4f0 con 0x7f31c8104ca0 2024-05-09T09:25:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31cf5b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f31c810d370 con 0x7f31c8102370 2024-05-09T09:25:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31cf5b0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f31c810d670 con 0x7f31c81075d0 2024-05-09T09:25:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31ccb4b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c8104ca0 0x7f31c8100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31ccb4b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c8104ca0 0x7f31c8100320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34168/0 (socket says 172.21.15.12:34168) 2024-05-09T09:25:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31ccb4b640 1 -- 172.21.15.12:0/3360408424 learned_addr learned my addr 172.21.15.12:0/3360408424 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:39.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31cdb4d640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31c81075d0 0x7f31c8100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31cd34c640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31c8102370 0x7f31c8101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31ccb4b640 1 -- 172.21.15.12:0/3360408424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31c81075d0 msgr2=0x7f31c8100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31ccb4b640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31c81075d0 0x7f31c8100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31ccb4b640 1 -- 172.21.15.12:0/3360408424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31c8102370 msgr2=0x7f31c8101c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31ccb4b640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31c8102370 0x7f31c8101c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31ccb4b640 1 -- 172.21.15.12:0/3360408424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c8101120 con 0x7f31c8104ca0 2024-05-09T09:25:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.661+0000 7f31cd34c640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31c8102370 0x7f31c8101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.662+0000 7f31ccb4b640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c8104ca0 0x7f31c8100320 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009230 tx=0x7f31b8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.662+0000 7f31be7fc640 1 -- 172.21.15.12:0/3360408424 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b803f030 con 0x7f31c8104ca0 2024-05-09T09:25:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.662+0000 7f31be7fc640 1 -- 172.21.15.12:0/3360408424 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f31b8003ac0 con 0x7f31c8104ca0 2024-05-09T09:25:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.662+0000 7f31cf5b0640 1 -- 172.21.15.12:0/3360408424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c81a1c00 con 0x7f31c8104ca0 2024-05-09T09:25:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.662+0000 7f31be7fc640 1 -- 172.21.15.12:0/3360408424 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8035070 con 0x7f31c8104ca0 2024-05-09T09:25:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.662+0000 7f31cf5b0640 1 -- 172.21.15.12:0/3360408424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c81a20c0 con 0x7f31c8104ca0 2024-05-09T09:25:39.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.664+0000 7f31d239f740 1 Processor -- start 2024-05-09T09:25:39.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31d239f740 1 -- start start 2024-05-09T09:25:39.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31d239f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46deba4c0 0x55b46deba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31d239f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46debaeb0 0x55b46debf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31d239f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46dec24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31d239f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b46de1e990 con 0x55b46debaeb0 2024-05-09T09:25:39.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31d239f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b46ddc65b0 con 0x55b46dec0020 2024-05-09T09:25:39.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31d239f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b46de1a950 con 0x55b46deba4c0 2024-05-09T09:25:39.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31be7fc640 1 -- 172.21.15.12:0/3360408424 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f31b8042030 con 0x7f31c8104ca0 2024-05-09T09:25:39.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31cdb4d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46deba4c0 0x55b46deba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31cd34c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46debaeb0 0x55b46debf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31cdb4d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46deba4c0 0x55b46deba8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:53676/0 (socket says 172.21.15.12:53676) 2024-05-09T09:25:39.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31cdb4d640 1 -- 172.21.15.12:0/909457693 learned_addr learned my addr 172.21.15.12:0/909457693 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:39.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31ccb4b640 1 --2- 172.21.15.12:0/909457693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46dec24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31cdb4d640 1 -- 172.21.15.12:0/909457693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 msgr2=0x55b46dec24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31cdb4d640 1 --2- 172.21.15.12:0/909457693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46dec24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31cdb4d640 1 -- 172.21.15.12:0/909457693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46debaeb0 msgr2=0x55b46debf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31cdb4d640 1 --2- 172.21.15.12:0/909457693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46debaeb0 0x55b46debf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31cdb4d640 1 -- 172.21.15.12:0/909457693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b46dbfa7e0 con 0x55b46deba4c0 2024-05-09T09:25:39.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31cd34c640 1 --2- 172.21.15.12:0/909457693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46debaeb0 0x55b46debf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:39.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.665+0000 7f31ccb4b640 1 --2- 172.21.15.12:0/909457693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46dec24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:39.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.666+0000 7f31cdb4d640 1 --2- 172.21.15.12:0/909457693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46deba4c0 0x55b46deba8e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f31c0008da0 tx=0x7f31c000d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff89ca23608472c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.666+0000 7f31be7fc640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f31b40a2b90 0x7f31b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.666+0000 7f31b37fe640 1 -- 172.21.15.12:0/909457693 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c000f830 con 0x55b46deba4c0 2024-05-09T09:25:39.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.666+0000 7f31b37fe640 1 -- 172.21.15.12:0/909457693 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f31c000de20 con 0x55b46deba4c0 2024-05-09T09:25:39.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.666+0000 7f31b37fe640 1 -- 172.21.15.12:0/909457693 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c000e3b0 con 0x55b46deba4c0 2024-05-09T09:25:39.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.666+0000 7f31cd34c640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f31b40a2b90 0x7f31b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.666+0000 7f31be7fc640 1 -- 172.21.15.12:0/3360408424 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f31b80efbc0 con 0x7f31c8104ca0 2024-05-09T09:25:39.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 -- 172.21.15.12:0/909457693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46deba4c0 msgr2=0x55b46deba8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31cd34c640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f31b40a2b90 0x7f31b40a5050 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55b46debb2b0 tx=0x7f31c400b470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 --2- 172.21.15.12:0/909457693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46deba4c0 0x55b46deba8e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f31c0008da0 tx=0x7f31c000d710 comp rx=0 tx=0).stop 2024-05-09T09:25:39.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 -- 172.21.15.12:0/909457693 shutdown_connections 2024-05-09T09:25:39.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 --2- 172.21.15.12:0/909457693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46dec24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 --2- 172.21.15.12:0/909457693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46debaeb0 0x55b46debf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 --2- 172.21.15.12:0/909457693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46deba4c0 0x55b46deba8e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 -- 172.21.15.12:0/909457693 >> 172.21.15.12:0/909457693 conn(0x55b46de820a0 msgr2=0x55b46deb5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 -- 172.21.15.12:0/909457693 shutdown_connections 2024-05-09T09:25:39.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 -- 172.21.15.12:0/909457693 wait complete. 2024-05-09T09:25:39.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 -- start start 2024-05-09T09:25:39.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 Processor -- start 2024-05-09T09:25:39.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 0x55b46dec4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46debaeb0 0x55b46dec4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46decbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b46dec2fd0 con 0x55b46deba4c0 2024-05-09T09:25:39.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.667+0000 7f31d239f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b46ddd9b40 con 0x55b46dec0020 2024-05-09T09:25:39.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31d239f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b46dec3150 con 0x55b46debaeb0 2024-05-09T09:25:39.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31d239f740 10 client.0 _ll_register_callbacks cb 0x55b46ddce0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:39.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31cd34c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 0x55b46dec4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31cd34c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 0x55b46dec4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34180/0 (socket says 172.21.15.12:34180) 2024-05-09T09:25:39.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31cd34c640 1 -- 172.21.15.12:0/1210197455 learned_addr learned my addr 172.21.15.12:0/1210197455 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:39.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31ccb4b640 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46debaeb0 0x55b46dec4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31cdb4d640 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46decbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31cd34c640 1 -- 172.21.15.12:0/1210197455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46debaeb0 msgr2=0x55b46dec4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31cd34c640 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46debaeb0 0x55b46dec4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31cd34c640 1 -- 172.21.15.12:0/1210197455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 msgr2=0x55b46decbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31cd34c640 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46decbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31cd34c640 1 -- 172.21.15.12:0/1210197455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b46dc5af70 con 0x55b46deba4c0 2024-05-09T09:25:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31cdb4d640 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46decbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31cd34c640 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 0x55b46dec4980 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f31c401bce0 tx=0x7f31c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723404ef21917779 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31b0ff9640 1 -- 172.21.15.12:0/1210197455 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c400eab0 con 0x55b46deba4c0 2024-05-09T09:25:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31b0ff9640 1 -- 172.21.15.12:0/1210197455 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f31c400ec50 con 0x55b46deba4c0 2024-05-09T09:25:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.668+0000 7f31d239f740 1 -- 172.21.15.12:0/1210197455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c000de20 con 0x55b46deba4c0 2024-05-09T09:25:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.669+0000 7f31b0ff9640 1 -- 172.21.15.12:0/1210197455 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31c402f050 con 0x55b46deba4c0 2024-05-09T09:25:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.669+0000 7f31d239f740 10 client.15663 fetch_fsmap learned FSMap version 29 2024-05-09T09:25:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.669+0000 7f31d239f740 1 -- 172.21.15.12:0/1210197455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b46daf2170 con 0x55b46deba4c0 2024-05-09T09:25:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.669+0000 7f31b0ff9640 1 -- 172.21.15.12:0/1210197455 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f31c000de20 con 0x55b46deba4c0 2024-05-09T09:25:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.669+0000 7f31b0ff9640 1 -- 172.21.15.12:0/1210197455 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f31c401c530 con 0x55b46deba4c0 2024-05-09T09:25:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.669+0000 7f31d239f740 10 client.15663 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:25:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.669+0000 7f31d239f740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:25:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.669+0000 7f31d239f740 20 client.15663 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.669+0000 7f31d239f740 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55b46ded0450 0x55b46ded28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.669+0000 7f31d239f740 4 client.15663 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.669+0000 7f31d239f740 1 -- 172.21.15.12:0/1210197455 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b46de2c0c0 con 0x55b46ded0450 2024-05-09T09:25:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.669+0000 7f31ccb4b640 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55b46ded0450 0x55b46ded28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.670+0000 7f31ccb4b640 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55b46ded0450 0x55b46ded28a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=346eef0b6c64b4e8 server_cookie=baee6d078ab9279a in_seq=0 out_seq=0 2024-05-09T09:25:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.670+0000 7f31b0ff9640 10 client.15663 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31b0ff9640 1 -- 172.21.15.12:0/1210197455 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b46de2c0c0 con 0x55b46ded0450 2024-05-09T09:25:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31b0ff9640 10 client.15663 handle_command_reply: tid=0 2024-05-09T09:25:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31d239f740 1 client.15663 shutdown 2024-05-09T09:25:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31d239f740 1 -- 172.21.15.12:0/1210197455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 msgr2=0x55b46dec4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31d239f740 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 0x55b46dec4980 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f31c401bce0 tx=0x7f31c40046c0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31d239f740 1 -- 172.21.15.12:0/1210197455 shutdown_connections 2024-05-09T09:25:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31d239f740 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55b46ded0450 0x55b46ded28a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31b0ff9640 0 client.15663 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31d239f740 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46decbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31d239f740 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46debaeb0 0x55b46dec4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31d239f740 1 --2- 172.21.15.12:0/1210197455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 0x55b46dec4980 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31d239f740 1 -- 172.21.15.12:0/1210197455 >> 172.21.15.12:0/1210197455 conn(0x55b46de820a0 msgr2=0x55b46deb5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31d239f740 1 -- 172.21.15.12:0/1210197455 shutdown_connections 2024-05-09T09:25:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31d239f740 1 -- 172.21.15.12:0/1210197455 wait complete. 2024-05-09T09:25:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.671+0000 7f31d239f740 20 client.15663 trim_cache size 0 max 16384 2024-05-09T09:25:39.680 INFO:tasks.workunit.client.0.smithi012.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T09:25:39.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.689+0000 7f31d239f740 1 Processor -- start 2024-05-09T09:25:39.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.689+0000 7f31d239f740 1 -- start start 2024-05-09T09:25:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31d239f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 0x55b46deb9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31d239f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46debaeb0 0x55b46decbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31d239f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46dec0020 0x55b46dec4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31d239f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f31c81ac7b0 con 0x55b46deba4c0 2024-05-09T09:25:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31d239f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f31c81ae2f0 con 0x55b46debaeb0 2024-05-09T09:25:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31d239f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f31c81ace80 con 0x55b46dec0020 2024-05-09T09:25:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31cd34c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 0x55b46deb9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31cd34c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 0x55b46deb9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34190/0 (socket says 172.21.15.12:34190) 2024-05-09T09:25:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31cd34c640 1 -- 172.21.15.12:0/1627907509 learned_addr learned my addr 172.21.15.12:0/1627907509 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31cdb4d640 1 --2- 172.21.15.12:0/1627907509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46dec0020 0x55b46dec4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31ccb4b640 1 --2- 172.21.15.12:0/1627907509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46debaeb0 0x55b46decbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31cd34c640 1 -- 172.21.15.12:0/1627907509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46dec0020 msgr2=0x55b46dec4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31cd34c640 1 --2- 172.21.15.12:0/1627907509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46dec0020 0x55b46dec4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31cd34c640 1 -- 172.21.15.12:0/1627907509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46debaeb0 msgr2=0x55b46decbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31cd34c640 1 --2- 172.21.15.12:0/1627907509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46debaeb0 0x55b46decbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31cd34c640 1 -- 172.21.15.12:0/1627907509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b46debf600 con 0x55b46deba4c0 2024-05-09T09:25:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.690+0000 7f31cd34c640 1 --2- 172.21.15.12:0/1627907509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 0x55b46deb9620 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f31c401b730 tx=0x7f31c400b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2584026365620365 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.691+0000 7f31b2ffd640 1 -- 172.21.15.12:0/1627907509 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c401cb70 con 0x55b46deba4c0 2024-05-09T09:25:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.691+0000 7f31b2ffd640 1 -- 172.21.15.12:0/1627907509 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f31c4005510 con 0x55b46deba4c0 2024-05-09T09:25:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.691+0000 7f31b2ffd640 1 -- 172.21.15.12:0/1627907509 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c400e9e0 con 0x55b46deba4c0 2024-05-09T09:25:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.691+0000 7f31d239f740 1 -- 172.21.15.12:0/1627907509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 msgr2=0x55b46deb9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.691+0000 7f31d239f740 1 --2- 172.21.15.12:0/1627907509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 0x55b46deb9620 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f31c401b730 tx=0x7f31c400b710 comp rx=0 tx=0).stop 2024-05-09T09:25:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.691+0000 7f31d239f740 1 -- 172.21.15.12:0/1627907509 shutdown_connections 2024-05-09T09:25:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.691+0000 7f31d239f740 1 --2- 172.21.15.12:0/1627907509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46dec0020 0x55b46dec4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.691+0000 7f31d239f740 1 --2- 172.21.15.12:0/1627907509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46debaeb0 0x55b46decbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.691+0000 7f31d239f740 1 --2- 172.21.15.12:0/1627907509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46deba4c0 0x55b46deb9620 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.691+0000 7f31d239f740 1 -- 172.21.15.12:0/1627907509 >> 172.21.15.12:0/1627907509 conn(0x55b46de820a0 msgr2=0x55b46deb8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.691+0000 7f31d239f740 1 -- 172.21.15.12:0/1627907509 shutdown_connections 2024-05-09T09:25:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.691+0000 7f31d239f740 1 -- 172.21.15.12:0/1627907509 wait complete. 2024-05-09T09:25:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31d239f740 1 -- start start 2024-05-09T09:25:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31d239f740 1 Processor -- start 2024-05-09T09:25:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31d239f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46deba4c0 0x55b46dec9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31d239f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46debaeb0 0x55b46deca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31d239f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46decb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31d239f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f31c81adc20 con 0x55b46debaeb0 2024-05-09T09:25:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31d239f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f31c81ad550 con 0x55b46dec0020 2024-05-09T09:25:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31d239f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f31c81aa790 con 0x55b46deba4c0 2024-05-09T09:25:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31d239f740 10 client.0 _ll_register_callbacks cb 0x7f31c401b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31ccb4b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46debaeb0 0x55b46deca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31ccb4b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46debaeb0 0x55b46deca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34200/0 (socket says 172.21.15.12:34200) 2024-05-09T09:25:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31ccb4b640 1 -- 172.21.15.12:0/2760817295 learned_addr learned my addr 172.21.15.12:0/2760817295 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31cdb4d640 1 --2- 172.21.15.12:0/2760817295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46decb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.692+0000 7f31cd34c640 1 --2- 172.21.15.12:0/2760817295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46deba4c0 0x55b46dec9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31ccb4b640 1 -- 172.21.15.12:0/2760817295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46deba4c0 msgr2=0x55b46dec9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31ccb4b640 1 --2- 172.21.15.12:0/2760817295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46deba4c0 0x55b46dec9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31ccb4b640 1 -- 172.21.15.12:0/2760817295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 msgr2=0x55b46decb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31ccb4b640 1 --2- 172.21.15.12:0/2760817295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46decb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31ccb4b640 1 -- 172.21.15.12:0/2760817295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b46decb7a0 con 0x55b46debaeb0 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31ccb4b640 1 --2- 172.21.15.12:0/2760817295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46debaeb0 0x55b46deca1a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f31b80bbd50 tx=0x7f31b802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20179dbb5cb4fba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31b0ff9640 1 -- 172.21.15.12:0/2760817295 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b80d4070 con 0x55b46debaeb0 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31b0ff9640 1 -- 172.21.15.12:0/2760817295 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f31b80ed370 con 0x55b46debaeb0 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31d239f740 1 -- 172.21.15.12:0/2760817295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c4005510 con 0x55b46debaeb0 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31b0ff9640 1 -- 172.21.15.12:0/2760817295 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31b8049040 con 0x55b46debaeb0 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31d239f740 10 client.15669 fetch_fsmap learned FSMap version 29 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31d239f740 1 -- 172.21.15.12:0/2760817295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b46df37110 con 0x55b46debaeb0 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31b0ff9640 1 -- 172.21.15.12:0/2760817295 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f31c4005510 con 0x55b46debaeb0 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.693+0000 7f31b0ff9640 1 -- 172.21.15.12:0/2760817295 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f31b8003c70 con 0x55b46debaeb0 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.694+0000 7f31d239f740 10 client.15669 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:25:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.694+0000 7f31d239f740 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:25:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.694+0000 7f31d239f740 20 client.15669 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.694+0000 7f31d239f740 1 --2- 172.21.15.12:0/2760817295 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55b46ded0450 0x55b46df39a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.694+0000 7f31d239f740 4 client.15669 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:25:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.694+0000 7f31d239f740 1 -- 172.21.15.12:0/2760817295 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b46de26410 con 0x55b46ded0450 2024-05-09T09:25:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.694+0000 7f31cd34c640 1 --2- 172.21.15.12:0/2760817295 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55b46ded0450 0x55b46df39a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.694+0000 7f31b0ff9640 10 client.15669 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.694+0000 7f31cd34c640 1 --2- 172.21.15.12:0/2760817295 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55b46ded0450 0x55b46df39a70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31fbb5ebc0db10ca server_cookie=add15d84e162d6ee in_seq=0 out_seq=0 2024-05-09T09:25:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.695+0000 7f31b0ff9640 1 -- 172.21.15.12:0/2760817295 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b46de26410 con 0x55b46ded0450 2024-05-09T09:25:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.695+0000 7f31b0ff9640 10 client.15669 handle_command_reply: tid=0 2024-05-09T09:25:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.695+0000 7f31d239f740 1 client.15669 shutdown 2024-05-09T09:25:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.695+0000 7f31d239f740 1 -- 172.21.15.12:0/2760817295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46debaeb0 msgr2=0x55b46deca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.695+0000 7f31d239f740 1 --2- 172.21.15.12:0/2760817295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46debaeb0 0x55b46deca1a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f31b80bbd50 tx=0x7f31b802c000 comp rx=0 tx=0).stop 2024-05-09T09:25:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.695+0000 7f31d239f740 1 -- 172.21.15.12:0/2760817295 shutdown_connections 2024-05-09T09:25:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.695+0000 7f31d239f740 1 --2- 172.21.15.12:0/2760817295 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55b46ded0450 0x55b46df39a70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.695+0000 7f31d239f740 1 --2- 172.21.15.12:0/2760817295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46dec0020 0x55b46decb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.695+0000 7f31d239f740 1 --2- 172.21.15.12:0/2760817295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b46debaeb0 0x55b46deca1a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.695+0000 7f31b0ff9640 0 client.15669 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.695+0000 7f31d239f740 1 --2- 172.21.15.12:0/2760817295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b46deba4c0 0x55b46dec9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31d239f740 1 -- 172.21.15.12:0/2760817295 >> 172.21.15.12:0/2760817295 conn(0x55b46de820a0 msgr2=0x55b46dec4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31d239f740 1 -- 172.21.15.12:0/2760817295 shutdown_connections 2024-05-09T09:25:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31d239f740 1 -- 172.21.15.12:0/2760817295 wait complete. 2024-05-09T09:25:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31d239f740 20 client.15669 trim_cache size 0 max 16384 2024-05-09T09:25:39.701 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:25:39.701 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T09:25:39.702 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "f6a7c7cf-cf24-48cf-9ad5-7f80b6e856a6", 2024-05-09T09:25:39.702 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T09:25:39.702 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:25:39.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31cf5b0640 1 -- 172.21.15.12:0/3360408424 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f31b40a2b90 msgr2=0x7f31b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31cf5b0640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f31b40a2b90 0x7f31b40a5050 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55b46debb2b0 tx=0x7f31c400b470 comp rx=0 tx=0).stop 2024-05-09T09:25:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31cf5b0640 1 -- 172.21.15.12:0/3360408424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c8104ca0 msgr2=0x7f31c8100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31cf5b0640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c8104ca0 0x7f31c8100320 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009230 tx=0x7f31b8004430 comp rx=0 tx=0).stop 2024-05-09T09:25:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31cf5b0640 1 -- 172.21.15.12:0/3360408424 shutdown_connections 2024-05-09T09:25:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31cf5b0640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f31b40a2b90 0x7f31b40a5050 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31cf5b0640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31c81075d0 0x7f31c8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31cf5b0640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c8104ca0 0x7f31c8100320 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31cf5b0640 1 --2- 172.21.15.12:0/3360408424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31c8102370 0x7f31c8101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31cf5b0640 1 -- 172.21.15.12:0/3360408424 >> 172.21.15.12:0/3360408424 conn(0x7f31c80fbd00 msgr2=0x7f31c8102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31cf5b0640 1 -- 172.21.15.12:0/3360408424 shutdown_connections 2024-05-09T09:25:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.696+0000 7f31cf5b0640 1 -- 172.21.15.12:0/3360408424 wait complete. 2024-05-09T09:25:39.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:25:39.707 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:25:39.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:39 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1749410897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:39.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:39 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2084662681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:39.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:39 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/806019904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:39.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.855+0000 7fa851ffb640 1 -- 172.21.15.12:0/2368460868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c107570 msgr2=0x7fa84c1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.855+0000 7fa851ffb640 1 --2- 172.21.15.12:0/2368460868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c107570 0x7fa84c1099a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa844009ca0 tx=0x7fa84402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.855+0000 7fa851ffb640 1 -- 172.21.15.12:0/2368460868 shutdown_connections 2024-05-09T09:25:39.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.855+0000 7fa851ffb640 1 --2- 172.21.15.12:0/2368460868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c107570 0x7fa84c1099a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.855+0000 7fa851ffb640 1 --2- 172.21.15.12:0/2368460868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa84c104c40 0x7fa84c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.855+0000 7fa851ffb640 1 --2- 172.21.15.12:0/2368460868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa84c102310 0x7fa84c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.855+0000 7fa851ffb640 1 -- 172.21.15.12:0/2368460868 >> 172.21.15.12:0/2368460868 conn(0x7fa84c0fbce0 msgr2=0x7fa84c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.856+0000 7fa851ffb640 1 -- 172.21.15.12:0/2368460868 shutdown_connections 2024-05-09T09:25:39.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.856+0000 7fa851ffb640 1 -- 172.21.15.12:0/2368460868 wait complete. 2024-05-09T09:25:39.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.856+0000 7fa851ffb640 1 Processor -- start 2024-05-09T09:25:39.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.856+0000 7fa851ffb640 1 -- start start 2024-05-09T09:25:39.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.856+0000 7fa851ffb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c102310 0x7fa84c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.856+0000 7fa851ffb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa84c104c40 0x7fa84c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.856+0000 7fa851ffb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa84c107570 0x7fa84c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.856+0000 7fa851ffb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c10d540 con 0x7fa84c102310 2024-05-09T09:25:39.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.856+0000 7fa851ffb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c10d3c0 con 0x7fa84c107570 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.856+0000 7fa851ffb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c10d6c0 con 0x7fa84c104c40 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c102310 0x7fa84c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c102310 0x7fa84c197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34220/0 (socket says 172.21.15.12:34220) 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa84b7fe640 1 -- 172.21.15.12:0/9990302 learned_addr learned my addr 172.21.15.12:0/9990302 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa84affd640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa84c104c40 0x7fa84c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa84bfff640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa84c107570 0x7fa84c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa84b7fe640 1 -- 172.21.15.12:0/9990302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa84c104c40 msgr2=0x7fa84c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa84c104c40 0x7fa84c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa84b7fe640 1 -- 172.21.15.12:0/9990302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa84c107570 msgr2=0x7fa84c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa84c107570 0x7fa84c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa84b7fe640 1 -- 172.21.15.12:0/9990302 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84c19f7b0 con 0x7fa84c102310 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa84bfff640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa84c107570 0x7fa84c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c102310 0x7fa84c197b90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c890 tx=0x7fa83c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa848ff9640 1 -- 172.21.15.12:0/9990302 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c004220 con 0x7fa84c102310 2024-05-09T09:25:39.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa848ff9640 1 -- 172.21.15.12:0/9990302 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa83c0043c0 con 0x7fa84c102310 2024-05-09T09:25:39.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa851ffb640 1 -- 172.21.15.12:0/9990302 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84c19faa0 con 0x7fa84c102310 2024-05-09T09:25:39.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa848ff9640 1 -- 172.21.15.12:0/9990302 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c004a00 con 0x7fa84c102310 2024-05-09T09:25:39.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.857+0000 7fa851ffb640 1 -- 172.21.15.12:0/9990302 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa84c1a0040 con 0x7fa84c102310 2024-05-09T09:25:39.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.859+0000 7fa848ff9640 1 -- 172.21.15.12:0/9990302 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa83c004ba0 con 0x7fa84c102310 2024-05-09T09:25:39.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa854de0740 1 Processor -- start 2024-05-09T09:25:39.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa854de0740 1 -- start start 2024-05-09T09:25:39.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa854de0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243c340 0x5648c243c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa854de0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243cd30 0x5648c2441830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa854de0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c2441ea0 0x5648c2444330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa854de0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5648c239cd00 con 0x5648c243cd30 2024-05-09T09:25:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa854de0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5648c23444c0 con 0x5648c2441ea0 2024-05-09T09:25:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa854de0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5648c2398cc0 con 0x5648c243c340 2024-05-09T09:25:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243cd30 0x5648c2441830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa848ff9640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa8340a2c10 0x7fa8340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243cd30 0x5648c2441830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34222/0 (socket says 172.21.15.12:34222) 2024-05-09T09:25:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa84b7fe640 1 -- 172.21.15.12:0/2339061088 learned_addr learned my addr 172.21.15.12:0/2339061088 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa84bfff640 1 --2- 172.21.15.12:0/2339061088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243c340 0x5648c243c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa84affd640 1 --2- 172.21.15.12:0/2339061088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c2441ea0 0x5648c2444330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa84bfff640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa8340a2c10 0x7fa8340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.860+0000 7fa848ff9640 1 -- 172.21.15.12:0/9990302 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fa83c0cf150 con 0x7fa84c102310 2024-05-09T09:25:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.861+0000 7fa84b7fe640 1 -- 172.21.15.12:0/2339061088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243c340 msgr2=0x5648c243c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.861+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/2339061088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243c340 0x5648c243c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.861+0000 7fa84b7fe640 1 -- 172.21.15.12:0/2339061088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c2441ea0 msgr2=0x5648c2444330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.861+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/2339061088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c2441ea0 0x5648c2444330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.861+0000 7fa84b7fe640 1 -- 172.21.15.12:0/2339061088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648c217a7d0 con 0x5648c243cd30 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.861+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/2339061088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243cd30 0x5648c2441830 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0a3cc0 tx=0x7fa83c00a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45b43aeb6079850 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.861+0000 7fa84bfff640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa8340a2c10 0x7fa8340a50d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa84402f540 tx=0x7fa8440095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.861+0000 7fa831ffb640 1 -- 172.21.15.12:0/2339061088 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c01d030 con 0x5648c243cd30 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.861+0000 7fa831ffb640 1 -- 172.21.15.12:0/2339061088 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa83c09bc80 con 0x5648c243cd30 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.861+0000 7fa831ffb640 1 -- 172.21.15.12:0/2339061088 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c005140 con 0x5648c243cd30 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 -- 172.21.15.12:0/2339061088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243cd30 msgr2=0x5648c2441830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 --2- 172.21.15.12:0/2339061088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243cd30 0x5648c2441830 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0a3cc0 tx=0x7fa83c00a5f0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 -- 172.21.15.12:0/2339061088 shutdown_connections 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 --2- 172.21.15.12:0/2339061088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c2441ea0 0x5648c2444330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 --2- 172.21.15.12:0/2339061088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243cd30 0x5648c2441830 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 --2- 172.21.15.12:0/2339061088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243c340 0x5648c243c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 -- 172.21.15.12:0/2339061088 >> 172.21.15.12:0/2339061088 conn(0x5648c24335a0 msgr2=0x5648c24339a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 -- 172.21.15.12:0/2339061088 shutdown_connections 2024-05-09T09:25:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 -- 172.21.15.12:0/2339061088 wait complete. 2024-05-09T09:25:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 -- start start 2024-05-09T09:25:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 Processor -- start 2024-05-09T09:25:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243c340 0x5648c2446780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243cd30 0x5648c2446cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c2441ea0 0x5648c244dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5648c2444e50 con 0x5648c2441ea0 2024-05-09T09:25:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5648c2357b90 con 0x5648c243c340 2024-05-09T09:25:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5648c2444fd0 con 0x5648c243cd30 2024-05-09T09:25:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.862+0000 7fa854de0740 10 client.0 _ll_register_callbacks cb 0x5648c2393da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa84bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c2441ea0 0x5648c244dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa84bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c2441ea0 0x5648c244dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34224/0 (socket says 172.21.15.12:34224) 2024-05-09T09:25:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa84bfff640 1 -- 172.21.15.12:0/3790929271 learned_addr learned my addr 172.21.15.12:0/3790929271 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243c340 0x5648c2446780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa84affd640 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243cd30 0x5648c2446cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa84bfff640 1 -- 172.21.15.12:0/3790929271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243cd30 msgr2=0x5648c2446cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa84bfff640 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243cd30 0x5648c2446cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa84bfff640 1 -- 172.21.15.12:0/3790929271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243c340 msgr2=0x5648c2446780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa84bfff640 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243c340 0x5648c2446780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa84bfff640 1 -- 172.21.15.12:0/3790929271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648c20b03f0 con 0x5648c2441ea0 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243c340 0x5648c2446780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa84bfff640 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c2441ea0 0x5648c244dd00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa844035000 tx=0x7fa84402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85eecfc5eb880cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa82affd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa82affd640 1 -- 172.21.15.12:0/3790929271 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844051030 con 0x5648c2441ea0 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa854de0740 1 -- 172.21.15.12:0/3790929271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa83c09bc80 con 0x5648c2441ea0 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa82affd640 1 -- 172.21.15.12:0/3790929271 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fa844047070 con 0x5648c2441ea0 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.863+0000 7fa82affd640 1 -- 172.21.15.12:0/3790929271 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa84405e050 con 0x5648c2441ea0 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.864+0000 7fa854de0740 10 client.15681 fetch_fsmap learned FSMap version 29 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.864+0000 7fa854de0740 1 -- 172.21.15.12:0/3790929271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648c20700f0 con 0x5648c2441ea0 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.864+0000 7fa82affd640 1 -- 172.21.15.12:0/3790929271 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa83c09bc80 con 0x5648c2441ea0 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.864+0000 7fa82affd640 1 -- 172.21.15.12:0/3790929271 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa84404a050 con 0x5648c2441ea0 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.864+0000 7fa854de0740 10 client.15681 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.864+0000 7fa854de0740 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.864+0000 7fa854de0740 20 client.15681 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.864+0000 7fa854de0740 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5648c24525a0 0x5648c2454a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.864+0000 7fa854de0740 4 client.15681 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.864+0000 7fa854de0740 1 -- 172.21.15.12:0/3790929271 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648c23aa430 con 0x5648c24525a0 2024-05-09T09:25:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.864+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5648c24525a0 0x5648c2454a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.865+0000 7fa82affd640 10 client.15681 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.865+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5648c24525a0 0x5648c2454a10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d72c5e0066f0dad9 server_cookie=c195c74e1a4bdd9b in_seq=0 out_seq=0 2024-05-09T09:25:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.865+0000 7fa82affd640 1 -- 172.21.15.12:0/3790929271 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648c23aa430 con 0x5648c24525a0 2024-05-09T09:25:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.865+0000 7fa82affd640 10 client.15681 handle_command_reply: tid=0 2024-05-09T09:25:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.865+0000 7fa854de0740 1 client.15681 shutdown 2024-05-09T09:25:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.866+0000 7fa854de0740 1 -- 172.21.15.12:0/3790929271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c2441ea0 msgr2=0x5648c244dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.866+0000 7fa854de0740 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c2441ea0 0x5648c244dd00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa844035000 tx=0x7fa84402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.866+0000 7fa854de0740 1 -- 172.21.15.12:0/3790929271 shutdown_connections 2024-05-09T09:25:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.866+0000 7fa854de0740 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5648c24525a0 0x5648c2454a10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.866+0000 7fa854de0740 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c2441ea0 0x5648c244dd00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.866+0000 7fa854de0740 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243cd30 0x5648c2446cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.866+0000 7fa854de0740 1 --2- 172.21.15.12:0/3790929271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243c340 0x5648c2446780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.866+0000 7fa82affd640 0 client.15681 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.866+0000 7fa854de0740 1 -- 172.21.15.12:0/3790929271 >> 172.21.15.12:0/3790929271 conn(0x5648c24335a0 msgr2=0x5648c24327e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.866+0000 7fa854de0740 1 -- 172.21.15.12:0/3790929271 shutdown_connections 2024-05-09T09:25:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.866+0000 7fa854de0740 1 -- 172.21.15.12:0/3790929271 wait complete. 2024-05-09T09:25:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.866+0000 7fa854de0740 20 client.15681 trim_cache size 0 max 16384 2024-05-09T09:25:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.883+0000 7fa854de0740 1 Processor -- start 2024-05-09T09:25:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.883+0000 7fa854de0740 1 -- start start 2024-05-09T09:25:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.883+0000 7fa854de0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243db10 0x5648c2432660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa854de0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c24335a0 0x5648c244dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa854de0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243c340 0x5648c24592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa854de0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1a84a0 con 0x5648c243c340 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa854de0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1a9fe0 con 0x5648c243db10 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa854de0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1a8b70 con 0x5648c24335a0 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa84bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243c340 0x5648c24592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa84bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243c340 0x5648c24592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34232/0 (socket says 172.21.15.12:34232) 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa84bfff640 1 -- 172.21.15.12:0/4173205358 learned_addr learned my addr 172.21.15.12:0/4173205358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/4173205358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243db10 0x5648c2432660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa84affd640 1 --2- 172.21.15.12:0/4173205358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c24335a0 0x5648c244dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa84bfff640 1 -- 172.21.15.12:0/4173205358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c24335a0 msgr2=0x5648c244dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa84bfff640 1 --2- 172.21.15.12:0/4173205358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c24335a0 0x5648c244dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa84bfff640 1 -- 172.21.15.12:0/4173205358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243db10 msgr2=0x5648c2432660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa84bfff640 1 --2- 172.21.15.12:0/4173205358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243db10 0x5648c2432660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa84bfff640 1 -- 172.21.15.12:0/4173205358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648c24333d0 con 0x5648c243c340 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa84affd640 1 --2- 172.21.15.12:0/4173205358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c24335a0 0x5648c244dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/4173205358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243db10 0x5648c2432660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa84bfff640 1 --2- 172.21.15.12:0/4173205358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243c340 0x5648c24592d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa844009bf0 tx=0x7fa84402afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45d38cc370584c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa8317fa640 1 -- 172.21.15.12:0/4173205358 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844051070 con 0x5648c243c340 2024-05-09T09:25:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa8317fa640 1 -- 172.21.15.12:0/4173205358 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa844036040 con 0x5648c243c340 2024-05-09T09:25:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.884+0000 7fa8317fa640 1 -- 172.21.15.12:0/4173205358 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84402f820 con 0x5648c243c340 2024-05-09T09:25:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.885+0000 7fa854de0740 1 -- 172.21.15.12:0/4173205358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243c340 msgr2=0x5648c24592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.885+0000 7fa854de0740 1 --2- 172.21.15.12:0/4173205358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243c340 0x5648c24592d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa844009bf0 tx=0x7fa84402afc0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.885+0000 7fa854de0740 1 -- 172.21.15.12:0/4173205358 shutdown_connections 2024-05-09T09:25:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.885+0000 7fa854de0740 1 --2- 172.21.15.12:0/4173205358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c243c340 0x5648c24592d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.885+0000 7fa854de0740 1 --2- 172.21.15.12:0/4173205358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c24335a0 0x5648c244dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.885+0000 7fa854de0740 1 --2- 172.21.15.12:0/4173205358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243db10 0x5648c2432660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.885+0000 7fa854de0740 1 -- 172.21.15.12:0/4173205358 >> 172.21.15.12:0/4173205358 conn(0x5648c242e9e0 msgr2=0x5648c2431f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.885+0000 7fa854de0740 1 -- 172.21.15.12:0/4173205358 shutdown_connections 2024-05-09T09:25:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.885+0000 7fa854de0740 1 -- 172.21.15.12:0/4173205358 wait complete. 2024-05-09T09:25:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.885+0000 7fa854de0740 1 -- start start 2024-05-09T09:25:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa854de0740 1 Processor -- start 2024-05-09T09:25:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa854de0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c24335a0 0x5648c244ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa854de0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243c340 0x5648c244bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa854de0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243db10 0x5648c244ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa854de0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1a9910 con 0x5648c24335a0 2024-05-09T09:25:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa854de0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1a9240 con 0x5648c243db10 2024-05-09T09:25:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa854de0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1a6480 con 0x5648c243c340 2024-05-09T09:25:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa854de0740 10 client.0 _ll_register_callbacks cb 0x7fa844032fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c24335a0 0x5648c244ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c24335a0 0x5648c244ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34240/0 (socket says 172.21.15.12:34240) 2024-05-09T09:25:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa84b7fe640 1 -- 172.21.15.12:0/1016156507 learned_addr learned my addr 172.21.15.12:0/1016156507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa84b7fe640 1 -- 172.21.15.12:0/1016156507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243c340 msgr2=0x5648c244bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/1016156507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243c340 0x5648c244bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa84b7fe640 1 -- 172.21.15.12:0/1016156507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243db10 msgr2=0x5648c244ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/1016156507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243db10 0x5648c244ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa84b7fe640 1 -- 172.21.15.12:0/1016156507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648c244d4a0 con 0x5648c24335a0 2024-05-09T09:25:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa84b7fe640 1 --2- 172.21.15.12:0/1016156507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c24335a0 0x5648c244ba00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa83c097f50 tx=0x7fa83c0cf3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff85db4d5c124f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa82affd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa82affd640 1 -- 172.21.15.12:0/1016156507 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c01d040 con 0x5648c24335a0 2024-05-09T09:25:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa82affd640 1 -- 172.21.15.12:0/1016156507 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fa83c004e90 con 0x5648c24335a0 2024-05-09T09:25:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.886+0000 7fa854de0740 1 -- 172.21.15.12:0/1016156507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa844036040 con 0x5648c24335a0 2024-05-09T09:25:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.887+0000 7fa82affd640 1 -- 172.21.15.12:0/1016156507 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa83c09bc80 con 0x5648c24335a0 2024-05-09T09:25:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.887+0000 7fa854de0740 10 client.15687 fetch_fsmap learned FSMap version 29 2024-05-09T09:25:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.887+0000 7fa854de0740 1 -- 172.21.15.12:0/1016156507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648c24b9070 con 0x5648c24335a0 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.887+0000 7fa82affd640 1 -- 172.21.15.12:0/1016156507 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa844036040 con 0x5648c24335a0 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.887+0000 7fa82affd640 1 -- 172.21.15.12:0/1016156507 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa83c0ccce0 con 0x5648c24335a0 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.887+0000 7fa854de0740 10 client.15687 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.887+0000 7fa854de0740 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.887+0000 7fa854de0740 20 client.15687 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.887+0000 7fa854de0740 1 --2- 172.21.15.12:0/1016156507 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5648c243cd30 0x5648c24bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.887+0000 7fa854de0740 4 client.15687 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.887+0000 7fa854de0740 1 -- 172.21.15.12:0/1016156507 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648c23a4780 con 0x5648c243cd30 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.887+0000 7fa84affd640 1 --2- 172.21.15.12:0/1016156507 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5648c243cd30 0x5648c24bb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa82affd640 10 client.15687 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa84affd640 1 --2- 172.21.15.12:0/1016156507 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5648c243cd30 0x5648c24bb5f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ea064de4dfc5b11 server_cookie=687cd86396d32aaf in_seq=0 out_seq=0 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa82affd640 1 -- 172.21.15.12:0/1016156507 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5648c23a4780 con 0x5648c243cd30 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa82affd640 10 client.15687 handle_command_reply: tid=0 2024-05-09T09:25:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa854de0740 1 client.15687 shutdown 2024-05-09T09:25:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa854de0740 1 -- 172.21.15.12:0/1016156507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c24335a0 msgr2=0x5648c244ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa854de0740 1 --2- 172.21.15.12:0/1016156507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c24335a0 0x5648c244ba00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa83c097f50 tx=0x7fa83c0cf3d0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa854de0740 1 -- 172.21.15.12:0/1016156507 shutdown_connections 2024-05-09T09:25:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa854de0740 1 --2- 172.21.15.12:0/1016156507 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5648c243cd30 0x5648c24bb5f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa854de0740 1 --2- 172.21.15.12:0/1016156507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648c243db10 0x5648c244ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa82affd640 0 client.15687 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa854de0740 1 --2- 172.21.15.12:0/1016156507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648c243c340 0x5648c244bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa854de0740 1 --2- 172.21.15.12:0/1016156507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648c24335a0 0x5648c244ba00 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa854de0740 1 -- 172.21.15.12:0/1016156507 >> 172.21.15.12:0/1016156507 conn(0x5648c242e9e0 msgr2=0x5648c24327e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa854de0740 1 -- 172.21.15.12:0/1016156507 shutdown_connections 2024-05-09T09:25:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa854de0740 1 -- 172.21.15.12:0/1016156507 wait complete. 2024-05-09T09:25:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.888+0000 7fa854de0740 20 client.15687 trim_cache size 0 max 16384 2024-05-09T09:25:39.897 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:25:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:25:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T09:25:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "f6a7c7cf-cf24-48cf-9ad5-7f80b6e856a6": { 2024-05-09T09:25:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T09:25:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "f6a7c7cf-cf24-48cf-9ad5-7f80b6e856a6", 2024-05-09T09:25:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T09:25:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:25:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:25:39.897 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:25:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.889+0000 7fa851ffb640 1 -- 172.21.15.12:0/9990302 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa8340a2c10 msgr2=0x7fa8340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.889+0000 7fa851ffb640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa8340a2c10 0x7fa8340a50d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa84402f540 tx=0x7fa8440095c0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.889+0000 7fa851ffb640 1 -- 172.21.15.12:0/9990302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c102310 msgr2=0x7fa84c197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.889+0000 7fa851ffb640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c102310 0x7fa84c197b90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c890 tx=0x7fa83c00cd50 comp rx=0 tx=0).stop 2024-05-09T09:25:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.889+0000 7fa851ffb640 1 -- 172.21.15.12:0/9990302 shutdown_connections 2024-05-09T09:25:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.889+0000 7fa851ffb640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa8340a2c10 0x7fa8340a50d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.889+0000 7fa851ffb640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa84c107570 0x7fa84c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.889+0000 7fa851ffb640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa84c104c40 0x7fa84c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.889+0000 7fa851ffb640 1 --2- 172.21.15.12:0/9990302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c102310 0x7fa84c197b90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.889+0000 7fa851ffb640 1 -- 172.21.15.12:0/9990302 >> 172.21.15.12:0/9990302 conn(0x7fa84c0fbce0 msgr2=0x7fa84c102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.889+0000 7fa851ffb640 1 -- 172.21.15.12:0/9990302 shutdown_connections 2024-05-09T09:25:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:25:39.889+0000 7fa851ffb640 1 -- 172.21.15.12:0/9990302 wait complete. 2024-05-09T09:25:39.900 INFO:tasks.cephfs.filesystem:scrub status for tag:f6a7c7cf-cf24-48cf-9ad5-7f80b6e856a6 - {'path': '/', 'tag': 'f6a7c7cf-cf24-48cf-9ad5-7f80b6e856a6', 'options': 'recursive,force'} 2024-05-09T09:25:40.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:40 smithi016 ceph-mon[33335]: pgmap v370: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T09:25:40.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:40 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3723174518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:40.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:40 smithi012 ceph-mon[30140]: pgmap v370: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T09:25:40.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:40 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3723174518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:40.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:40 smithi170 ceph-mon[33632]: pgmap v370: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T09:25:40.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:40 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3723174518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:41.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:41 smithi016 ceph-mon[33335]: scrub queued for path: / 2024-05-09T09:25:41.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:41 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:41.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:41 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:25:41.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:41 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:41.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:41 smithi012 ceph-mon[30140]: scrub queued for path: / 2024-05-09T09:25:41.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:41 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:41.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:41 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:25:41.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:41 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:41.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:41 smithi170 ceph-mon[33632]: scrub queued for path: / 2024-05-09T09:25:41.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:41 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:41.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:41 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:25:41.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:41 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:42.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:42 smithi012 ceph-mon[30140]: pgmap v371: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-09T09:25:42.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:42 smithi170 ceph-mon[33632]: pgmap v371: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-09T09:25:42.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:42 smithi016 ceph-mon[33335]: pgmap v371: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-09T09:25:44.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:44 smithi012 ceph-mon[30140]: pgmap v372: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T09:25:44.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:44 smithi170 ceph-mon[33632]: pgmap v372: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T09:25:44.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:44 smithi016 ceph-mon[33335]: pgmap v372: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T09:25:45.560 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:45 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:25:45.576 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:45 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:25:45.631 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:45 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:25:46.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:46 smithi012 ceph-mon[30140]: pgmap v373: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T09:25:46.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:46 smithi170 ceph-mon[33632]: pgmap v373: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T09:25:46.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:46 smithi016 ceph-mon[33335]: pgmap v373: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T09:25:47.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:25:47.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:25:47.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:25:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:25:47.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:47.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:25:47.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:25:47.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:25:48.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:48 smithi012 ceph-mon[30140]: pgmap v374: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T09:25:48.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:48 smithi170 ceph-mon[33632]: pgmap v374: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T09:25:48.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:48 smithi016 ceph-mon[33335]: pgmap v374: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T09:25:50.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:50 smithi012 ceph-mon[30140]: pgmap v375: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-09T09:25:50.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:50 smithi170 ceph-mon[33632]: pgmap v375: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-09T09:25:50.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:50 smithi016 ceph-mon[33335]: pgmap v375: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-09T09:25:51.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:51 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:25:51.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:51 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:25:51.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:51 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:25:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.696+0000 7f2750561640 1 -- 172.21.15.12:0/2128373839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2748102340 msgr2=0x7f2748108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.696+0000 7f2750561640 1 --2- 172.21.15.12:0/2128373839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2748102340 0x7f2748108bd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2740009ca0 tx=0x7f274002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.697+0000 7f2750561640 1 -- 172.21.15.12:0/2128373839 shutdown_connections 2024-05-09T09:25:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.697+0000 7f2750561640 1 --2- 172.21.15.12:0/2128373839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2748102340 0x7f2748108bd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.697+0000 7f2750561640 1 --2- 172.21.15.12:0/2128373839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2748101980 0x7f2748101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.697+0000 7f2750561640 1 --2- 172.21.15.12:0/2128373839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2748100790 0x7f2748100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.697+0000 7f2750561640 1 -- 172.21.15.12:0/2128373839 >> 172.21.15.12:0/2128373839 conn(0x7f27480fbd00 msgr2=0x7f27480fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.697+0000 7f2750561640 1 -- 172.21.15.12:0/2128373839 shutdown_connections 2024-05-09T09:25:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.697+0000 7f2750561640 1 -- 172.21.15.12:0/2128373839 wait complete. 2024-05-09T09:25:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.697+0000 7f2750561640 1 Processor -- start 2024-05-09T09:25:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.697+0000 7f2750561640 1 -- start start 2024-05-09T09:25:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.697+0000 7f2750561640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2748100790 0x7f2748197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.697+0000 7f2750561640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2748101980 0x7f2748197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f2750561640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2748102340 0x7f274819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f2750561640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f274810d4d0 con 0x7f2748100790 2024-05-09T09:25:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f2750561640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f274810d350 con 0x7f2748101980 2024-05-09T09:25:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f2750561640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f274810d650 con 0x7f2748102340 2024-05-09T09:25:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f274e2fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2748100790 0x7f2748197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f274e2fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2748100790 0x7f2748197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44608/0 (socket says 172.21.15.12:44608) 2024-05-09T09:25:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f274e2fd640 1 -- 172.21.15.12:0/171449780 learned_addr learned my addr 172.21.15.12:0/171449780 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f274dafc640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2748101980 0x7f2748197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f274eafe640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2748102340 0x7f274819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f274e2fd640 1 -- 172.21.15.12:0/171449780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2748102340 msgr2=0x7f274819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f274e2fd640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2748102340 0x7f274819f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f274e2fd640 1 -- 172.21.15.12:0/171449780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2748101980 msgr2=0x7f2748197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f274e2fd640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2748101980 0x7f2748197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f274e2fd640 1 -- 172.21.15.12:0/171449780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274819f6d0 con 0x7f2748100790 2024-05-09T09:25:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f274e2fd640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2748100790 0x7f2748197ab0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f274400d950 tx=0x7f274400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f273f7fe640 1 -- 172.21.15.12:0/171449780 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744017020 con 0x7f2748100790 2024-05-09T09:25:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f273f7fe640 1 -- 172.21.15.12:0/171449780 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2744009c30 con 0x7f2748100790 2024-05-09T09:25:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f2750561640 1 -- 172.21.15.12:0/171449780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274819f9c0 con 0x7f2748100790 2024-05-09T09:25:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f273f7fe640 1 -- 172.21.15.12:0/171449780 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274400f740 con 0x7f2748100790 2024-05-09T09:25:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.698+0000 7f2750561640 1 -- 172.21.15.12:0/171449780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274819ff10 con 0x7f2748100790 2024-05-09T09:25:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.701+0000 7f273f7fe640 1 -- 172.21.15.12:0/171449780 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f274401a020 con 0x7f2748100790 2024-05-09T09:25:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.701+0000 7f275326b740 1 Processor -- start 2024-05-09T09:25:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.701+0000 7f275326b740 1 -- start start 2024-05-09T09:25:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f275326b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 0x55c2aa539670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f275326b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539c40 0x55c2aa53e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f275326b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 0x55c2aa541240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f275326b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c2aa49d340 con 0x55c2aa539250 2024-05-09T09:25:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f275326b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c2aa445290 con 0x55c2aa539c40 2024-05-09T09:25:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f275326b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c2aa499300 con 0x55c2aa53edb0 2024-05-09T09:25:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f273f7fe640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f27340a2c30 0x7f27340a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f274eafe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 0x55c2aa539670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f274e2fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539c40 0x55c2aa53e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f274eafe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 0x55c2aa539670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44616/0 (socket says 172.21.15.12:44616) 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f274eafe640 1 -- 172.21.15.12:0/274475 learned_addr learned my addr 172.21.15.12:0/274475 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f274dafc640 1 --2- 172.21.15.12:0/274475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 0x55c2aa541240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f274eafe640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f27340a2c30 0x7f27340a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f273f7fe640 1 -- 172.21.15.12:0/171449780 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f27440cee90 con 0x7f2748100790 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f274eafe640 1 -- 172.21.15.12:0/274475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 msgr2=0x55c2aa541240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f274eafe640 1 --2- 172.21.15.12:0/274475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 0x55c2aa541240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f274eafe640 1 -- 172.21.15.12:0/274475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539c40 msgr2=0x55c2aa53e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f274eafe640 1 --2- 172.21.15.12:0/274475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539c40 0x55c2aa53e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f274eafe640 1 -- 172.21.15.12:0/274475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2aa4698e0 con 0x55c2aa539250 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.702+0000 7f274e2fd640 1 --2- 172.21.15.12:0/274475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539c40 0x55c2aa53e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.703+0000 7f274eafe640 1 --2- 172.21.15.12:0/274475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 0x55c2aa539670 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2740005e30 tx=0x7f2740003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97b6bafc5cafc53 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.703+0000 7f274eafe640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f27340a2c30 0x7f27340a50f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2740031be0 tx=0x7f2740047000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.703+0000 7f273cbfe640 1 -- 172.21.15.12:0/274475 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740031570 con 0x55c2aa539250 2024-05-09T09:25:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.703+0000 7f273cbfe640 1 -- 172.21.15.12:0/274475 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2740036070 con 0x55c2aa539250 2024-05-09T09:25:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.703+0000 7f273cbfe640 1 -- 172.21.15.12:0/274475 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274002af40 con 0x55c2aa539250 2024-05-09T09:25:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.704+0000 7f275326b740 1 -- 172.21.15.12:0/274475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 msgr2=0x55c2aa539670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.704+0000 7f275326b740 1 --2- 172.21.15.12:0/274475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 0x55c2aa539670 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2740005e30 tx=0x7f2740003a70 comp rx=0 tx=0).stop 2024-05-09T09:25:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.704+0000 7f275326b740 1 -- 172.21.15.12:0/274475 shutdown_connections 2024-05-09T09:25:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.704+0000 7f275326b740 1 --2- 172.21.15.12:0/274475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 0x55c2aa541240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.704+0000 7f275326b740 1 --2- 172.21.15.12:0/274475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539c40 0x55c2aa53e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.704+0000 7f275326b740 1 --2- 172.21.15.12:0/274475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 0x55c2aa539670 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.704+0000 7f275326b740 1 -- 172.21.15.12:0/274475 >> 172.21.15.12:0/274475 conn(0x55c2aa5348c0 msgr2=0x55c2aa534cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.704+0000 7f275326b740 1 -- 172.21.15.12:0/274475 shutdown_connections 2024-05-09T09:25:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.704+0000 7f275326b740 1 -- 172.21.15.12:0/274475 wait complete. 2024-05-09T09:25:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.704+0000 7f275326b740 1 -- start start 2024-05-09T09:25:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.704+0000 7f275326b740 1 Processor -- start 2024-05-09T09:25:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f275326b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 0x55c2aa543440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f275326b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539c40 0x55c2aa543980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f275326b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 0x55c2aa54a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f275326b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c2aa541ca0 con 0x55c2aa539250 2024-05-09T09:25:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f275326b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c2aa458790 con 0x55c2aa539c40 2024-05-09T09:25:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f275326b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c2aa541e20 con 0x55c2aa53edb0 2024-05-09T09:25:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f275326b740 10 client.0 _ll_register_callbacks cb 0x55c2aa493a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f274e2fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 0x55c2aa543440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f274e2fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 0x55c2aa543440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44626/0 (socket says 172.21.15.12:44626) 2024-05-09T09:25:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f274e2fd640 1 -- 172.21.15.12:0/413036206 learned_addr learned my addr 172.21.15.12:0/413036206 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f274eafe640 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 0x55c2aa54a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f274dafc640 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539c40 0x55c2aa543980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f274e2fd640 1 -- 172.21.15.12:0/413036206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 msgr2=0x55c2aa54a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f274e2fd640 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 0x55c2aa54a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f274e2fd640 1 -- 172.21.15.12:0/413036206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539c40 msgr2=0x55c2aa543980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f274e2fd640 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539c40 0x55c2aa543980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f274e2fd640 1 -- 172.21.15.12:0/413036206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2aa1b1210 con 0x55c2aa539250 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f274dafc640 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539c40 0x55c2aa543980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f274e2fd640 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 0x55c2aa543440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f27440cc930 tx=0x7f274406b1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44d4348a7c65297 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f2731ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f2731ffb640 1 -- 172.21.15.12:0/413036206 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744066400 con 0x55c2aa539250 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f2731ffb640 1 -- 172.21.15.12:0/413036206 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f2744097990 con 0x55c2aa539250 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.705+0000 7f275326b740 1 -- 172.21.15.12:0/413036206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2740036070 con 0x55c2aa539250 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.706+0000 7f2731ffb640 1 -- 172.21.15.12:0/413036206 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f274406a0d0 con 0x55c2aa539250 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.706+0000 7f275326b740 10 client.15699 fetch_fsmap learned FSMap version 29 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.706+0000 7f275326b740 1 -- 172.21.15.12:0/413036206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2aa27d620 con 0x55c2aa539250 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.706+0000 7f2731ffb640 1 -- 172.21.15.12:0/413036206 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2740036070 con 0x55c2aa539250 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.706+0000 7f2731ffb640 1 -- 172.21.15.12:0/413036206 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f27440a3b00 con 0x55c2aa539250 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.706+0000 7f275326b740 10 client.15699 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:25:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.706+0000 7f275326b740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.706+0000 7f275326b740 20 client.15699 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.706+0000 7f275326b740 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c2aa54ef30 0x55c2aa551340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.706+0000 7f275326b740 4 client.15699 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.706+0000 7f275326b740 1 -- 172.21.15.12:0/413036206 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2aa4aaa70 con 0x55c2aa54ef30 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.706+0000 7f274dafc640 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c2aa54ef30 0x55c2aa551340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.707+0000 7f274dafc640 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c2aa54ef30 0x55c2aa551340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4fd48fd2d943cbe server_cookie=947b659610acd8b in_seq=0 out_seq=0 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.707+0000 7f2731ffb640 10 client.15699 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f2731ffb640 1 -- 172.21.15.12:0/413036206 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2aa4aaa70 con 0x55c2aa54ef30 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f2731ffb640 10 client.15699 handle_command_reply: tid=0 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f275326b740 1 client.15699 shutdown 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f275326b740 1 -- 172.21.15.12:0/413036206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 msgr2=0x55c2aa543440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f275326b740 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 0x55c2aa543440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f27440cc930 tx=0x7f274406b1d0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f275326b740 1 -- 172.21.15.12:0/413036206 shutdown_connections 2024-05-09T09:25:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f275326b740 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c2aa54ef30 0x55c2aa551340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f275326b740 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 0x55c2aa54a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f275326b740 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539c40 0x55c2aa543980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f275326b740 1 --2- 172.21.15.12:0/413036206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539250 0x55c2aa543440 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f2731ffb640 0 client.15699 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f275326b740 1 -- 172.21.15.12:0/413036206 >> 172.21.15.12:0/413036206 conn(0x55c2aa5348c0 msgr2=0x55c2aa533a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f275326b740 1 -- 172.21.15.12:0/413036206 shutdown_connections 2024-05-09T09:25:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f275326b740 1 -- 172.21.15.12:0/413036206 wait complete. 2024-05-09T09:25:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.708+0000 7f275326b740 20 client.15699 trim_cache size 0 max 16384 2024-05-09T09:25:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f275326b740 1 Processor -- start 2024-05-09T09:25:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f275326b740 1 -- start start 2024-05-09T09:25:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f275326b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539250 0x55c2aa537f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f275326b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539c40 0x55c2aa556c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f275326b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 0x55c2aa557160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f275326b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f27481aa580 con 0x55c2aa539c40 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f275326b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27481ac0c0 con 0x55c2aa539250 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f275326b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f27481aac50 con 0x55c2aa53edb0 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f274dafc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539c40 0x55c2aa556c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f274dafc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539c40 0x55c2aa556c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44634/0 (socket says 172.21.15.12:44634) 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f274dafc640 1 -- 172.21.15.12:0/3773641589 learned_addr learned my addr 172.21.15.12:0/3773641589 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f274e2fd640 1 --2- 172.21.15.12:0/3773641589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539250 0x55c2aa537f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f274dafc640 1 -- 172.21.15.12:0/3773641589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 msgr2=0x55c2aa557160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f274dafc640 1 --2- 172.21.15.12:0/3773641589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 0x55c2aa557160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f274dafc640 1 -- 172.21.15.12:0/3773641589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539250 msgr2=0x55c2aa537f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.724+0000 7f274dafc640 1 --2- 172.21.15.12:0/3773641589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539250 0x55c2aa537f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.725+0000 7f274dafc640 1 -- 172.21.15.12:0/3773641589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2aa551990 con 0x55c2aa539c40 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.725+0000 7f274e2fd640 1 --2- 172.21.15.12:0/3773641589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539250 0x55c2aa537f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.725+0000 7f274dafc640 1 --2- 172.21.15.12:0/3773641589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539c40 0x55c2aa556c20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55c2aa551bc0 tx=0x7f2738009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befad871914cda server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.725+0000 7f2733fff640 1 -- 172.21.15.12:0/3773641589 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738014e50 con 0x55c2aa539c40 2024-05-09T09:25:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.725+0000 7f2733fff640 1 -- 172.21.15.12:0/3773641589 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f273801b820 con 0x55c2aa539c40 2024-05-09T09:25:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.725+0000 7f2733fff640 1 -- 172.21.15.12:0/3773641589 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738005580 con 0x55c2aa539c40 2024-05-09T09:25:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.725+0000 7f275326b740 1 -- 172.21.15.12:0/3773641589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539c40 msgr2=0x55c2aa556c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.725+0000 7f275326b740 1 --2- 172.21.15.12:0/3773641589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539c40 0x55c2aa556c20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55c2aa551bc0 tx=0x7f2738009350 comp rx=0 tx=0).stop 2024-05-09T09:25:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.725+0000 7f275326b740 1 -- 172.21.15.12:0/3773641589 shutdown_connections 2024-05-09T09:25:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.725+0000 7f275326b740 1 --2- 172.21.15.12:0/3773641589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa53edb0 0x55c2aa557160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.725+0000 7f275326b740 1 --2- 172.21.15.12:0/3773641589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa539c40 0x55c2aa556c20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.725+0000 7f275326b740 1 --2- 172.21.15.12:0/3773641589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539250 0x55c2aa537f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.725+0000 7f275326b740 1 -- 172.21.15.12:0/3773641589 >> 172.21.15.12:0/3773641589 conn(0x55c2aa5348c0 msgr2=0x55c2aa537870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f275326b740 1 -- 172.21.15.12:0/3773641589 shutdown_connections 2024-05-09T09:25:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f275326b740 1 -- 172.21.15.12:0/3773641589 wait complete. 2024-05-09T09:25:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f275326b740 1 -- start start 2024-05-09T09:25:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f275326b740 1 Processor -- start 2024-05-09T09:25:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f275326b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539250 0x55c2aa5b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f275326b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa539c40 0x55c2aa5b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f275326b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa53edb0 0x55c2aa5b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f275326b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f27481ab9f0 con 0x55c2aa53edb0 2024-05-09T09:25:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f275326b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27481ab320 con 0x55c2aa539250 2024-05-09T09:25:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f275326b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f27481a8560 con 0x55c2aa539c40 2024-05-09T09:25:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f275326b740 10 client.0 _ll_register_callbacks cb 0x7f27440976b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f274eafe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa53edb0 0x55c2aa5b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f274eafe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa53edb0 0x55c2aa5b81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44644/0 (socket says 172.21.15.12:44644) 2024-05-09T09:25:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f274eafe640 1 -- 172.21.15.12:0/2943861472 learned_addr learned my addr 172.21.15.12:0/2943861472 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:25:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f274e2fd640 1 --2- 172.21.15.12:0/2943861472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539250 0x55c2aa5b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.726+0000 7f274dafc640 1 --2- 172.21.15.12:0/2943861472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa539c40 0x55c2aa5b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f274eafe640 1 -- 172.21.15.12:0/2943861472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa539c40 msgr2=0x55c2aa5b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f274eafe640 1 --2- 172.21.15.12:0/2943861472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa539c40 0x55c2aa5b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f274eafe640 1 -- 172.21.15.12:0/2943861472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539250 msgr2=0x55c2aa5b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f274eafe640 1 --2- 172.21.15.12:0/2943861472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539250 0x55c2aa5b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f274eafe640 1 -- 172.21.15.12:0/2943861472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2aa5b87b0 con 0x55c2aa53edb0 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f274eafe640 1 --2- 172.21.15.12:0/2943861472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa53edb0 0x55c2aa5b81c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2740009ca0 tx=0x7f274002fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c93e96a73356470 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f2731ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f2731ffb640 1 -- 172.21.15.12:0/2943861472 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740057020 con 0x55c2aa53edb0 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f2731ffb640 1 -- 172.21.15.12:0/2943861472 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f27400305c0 con 0x55c2aa53edb0 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f275326b740 1 -- 172.21.15.12:0/2943861472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f273801b820 con 0x55c2aa53edb0 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f2731ffb640 1 -- 172.21.15.12:0/2943861472 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2740031570 con 0x55c2aa53edb0 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f275326b740 10 client.15705 fetch_fsmap learned FSMap version 29 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f275326b740 1 -- 172.21.15.12:0/2943861472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2aa5b9aa0 con 0x55c2aa53edb0 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.727+0000 7f2731ffb640 1 -- 172.21.15.12:0/2943861472 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f273801b820 con 0x55c2aa53edb0 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.728+0000 7f2731ffb640 1 -- 172.21.15.12:0/2943861472 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2740048070 con 0x55c2aa53edb0 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.728+0000 7f275326b740 10 client.15705 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.728+0000 7f275326b740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.728+0000 7f275326b740 20 client.15705 populate_metadata read hostname 'smithi012' 2024-05-09T09:25:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.728+0000 7f275326b740 1 --2- 172.21.15.12:0/2943861472 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c2aa5ba170 0x55c2aa5bc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.728+0000 7f275326b740 4 client.15705 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:25:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.728+0000 7f275326b740 1 -- 172.21.15.12:0/2943861472 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2aa4a4dc0 con 0x55c2aa5ba170 2024-05-09T09:25:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.728+0000 7f274e2fd640 1 --2- 172.21.15.12:0/2943861472 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c2aa5ba170 0x55c2aa5bc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.728+0000 7f2731ffb640 10 client.15705 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.728+0000 7f274e2fd640 1 --2- 172.21.15.12:0/2943861472 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c2aa5ba170 0x55c2aa5bc560 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f147453d3938c0 server_cookie=2da7241b39caf2ee in_seq=0 out_seq=0 2024-05-09T09:25:52.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:52 smithi012 ceph-mon[30140]: pgmap v376: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T09:25:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.741+0000 7f2731ffb640 1 -- 172.21.15.12:0/2943861472 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2aa4a4dc0 con 0x55c2aa5ba170 2024-05-09T09:25:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.741+0000 7f2731ffb640 10 client.15705 handle_command_reply: tid=0 2024-05-09T09:25:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.741+0000 7f275326b740 1 client.15705 shutdown 2024-05-09T09:25:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.741+0000 7f275326b740 1 -- 172.21.15.12:0/2943861472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa53edb0 msgr2=0x55c2aa5b81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.741+0000 7f275326b740 1 --2- 172.21.15.12:0/2943861472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa53edb0 0x55c2aa5b81c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2740009ca0 tx=0x7f274002fcc0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.741+0000 7f275326b740 1 -- 172.21.15.12:0/2943861472 shutdown_connections 2024-05-09T09:25:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.741+0000 7f275326b740 1 --2- 172.21.15.12:0/2943861472 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c2aa5ba170 0x55c2aa5bc560 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.741+0000 7f275326b740 1 --2- 172.21.15.12:0/2943861472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2aa53edb0 0x55c2aa5b81c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.741+0000 7f275326b740 1 --2- 172.21.15.12:0/2943861472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c2aa539c40 0x55c2aa5b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.741+0000 7f275326b740 1 --2- 172.21.15.12:0/2943861472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2aa539250 0x55c2aa5b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.741+0000 7f2731ffb640 0 client.15705 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:25:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.741+0000 7f275326b740 1 -- 172.21.15.12:0/2943861472 >> 172.21.15.12:0/2943861472 conn(0x55c2aa5348c0 msgr2=0x55c2aa543050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f275326b740 1 -- 172.21.15.12:0/2943861472 shutdown_connections 2024-05-09T09:25:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f275326b740 1 -- 172.21.15.12:0/2943861472 wait complete. 2024-05-09T09:25:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f275326b740 20 client.15705 trim_cache size 0 max 16384 2024-05-09T09:25:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:25:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:25:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:25:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:25:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f2750561640 1 -- 172.21.15.12:0/171449780 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f27340a2c30 msgr2=0x7f27340a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f2750561640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f27340a2c30 0x7f27340a50f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2740031be0 tx=0x7f2740047000 comp rx=0 tx=0).stop 2024-05-09T09:25:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f2750561640 1 -- 172.21.15.12:0/171449780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2748100790 msgr2=0x7f2748197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f2750561640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2748100790 0x7f2748197ab0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f274400d950 tx=0x7f274400de10 comp rx=0 tx=0).stop 2024-05-09T09:25:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f2750561640 1 -- 172.21.15.12:0/171449780 shutdown_connections 2024-05-09T09:25:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f2750561640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f27340a2c30 0x7f27340a50f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f2750561640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2748102340 0x7f274819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f2750561640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2748101980 0x7f2748197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f2750561640 1 --2- 172.21.15.12:0/171449780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2748100790 0x7f2748197ab0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f2750561640 1 -- 172.21.15.12:0/171449780 >> 172.21.15.12:0/171449780 conn(0x7f27480fbd00 msgr2=0x7f2748104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f2750561640 1 -- 172.21.15.12:0/171449780 shutdown_connections 2024-05-09T09:25:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:25:52.742+0000 7f2750561640 1 -- 172.21.15.12:0/171449780 wait complete. 2024-05-09T09:25:52.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:52 smithi170 ceph-mon[33632]: pgmap v376: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T09:25:52.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:52 smithi016 ceph-mon[33335]: pgmap v376: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T09:25:54.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:54 smithi012 ceph-mon[30140]: pgmap v377: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-09T09:25:54.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:54 smithi170 ceph-mon[33632]: pgmap v377: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-09T09:25:54.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:54 smithi016 ceph-mon[33335]: pgmap v377: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-09T09:25:56.739 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:56 smithi012 ceph-mon[30140]: pgmap v378: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-05-09T09:25:56.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:56 smithi170 ceph-mon[33632]: pgmap v378: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-05-09T09:25:56.758 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:56 smithi016 ceph-mon[33335]: pgmap v378: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-05-09T09:25:58.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:25:58 smithi012 ceph-mon[30140]: pgmap v379: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-05-09T09:25:58.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:25:58 smithi170 ceph-mon[33632]: pgmap v379: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-05-09T09:25:58.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:25:58 smithi016 ceph-mon[33335]: pgmap v379: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-05-09T09:26:00.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:00 smithi012 ceph-mon[30140]: pgmap v380: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-05-09T09:26:00.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:00 smithi170 ceph-mon[33632]: pgmap v380: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-05-09T09:26:00.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:00 smithi016 ceph-mon[33335]: pgmap v380: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-05-09T09:26:02.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:02 smithi012 ceph-mon[30140]: pgmap v381: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-05-09T09:26:02.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:02 smithi170 ceph-mon[33632]: pgmap v381: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-05-09T09:26:02.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:02 smithi016 ceph-mon[33335]: pgmap v381: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-05-09T09:26:04.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:04 smithi012 ceph-mon[30140]: pgmap v382: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-05-09T09:26:04.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:04 smithi170 ceph-mon[33632]: pgmap v382: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-05-09T09:26:04.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:04 smithi016 ceph-mon[33335]: pgmap v382: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-05-09T09:26:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.844+0000 7fe4283b0640 1 -- 172.21.15.12:0/2560553831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4201019b0 msgr2=0x7fe420101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.844+0000 7fe4283b0640 1 --2- 172.21.15.12:0/2560553831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4201019b0 0x7fe420101e30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe410009900 tx=0x7fe41002a410 comp rx=0 tx=0).stop 2024-05-09T09:26:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.844+0000 7fe4283b0640 1 -- 172.21.15.12:0/2560553831 shutdown_connections 2024-05-09T09:26:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.844+0000 7fe4283b0640 1 --2- 172.21.15.12:0/2560553831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420102370 0x7fe420108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.844+0000 7fe4283b0640 1 --2- 172.21.15.12:0/2560553831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4201019b0 0x7fe420101e30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.844+0000 7fe4283b0640 1 --2- 172.21.15.12:0/2560553831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4201007c0 0x7fe420100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.844+0000 7fe4283b0640 1 -- 172.21.15.12:0/2560553831 >> 172.21.15.12:0/2560553831 conn(0x7fe4200fbd30 msgr2=0x7fe4200fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.844+0000 7fe4283b0640 1 -- 172.21.15.12:0/2560553831 shutdown_connections 2024-05-09T09:26:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.845+0000 7fe4283b0640 1 -- 172.21.15.12:0/2560553831 wait complete. 2024-05-09T09:26:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.845+0000 7fe4283b0640 1 Processor -- start 2024-05-09T09:26:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.845+0000 7fe4283b0640 1 -- start start 2024-05-09T09:26:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.845+0000 7fe4283b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4201007c0 0x7fe420104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.845+0000 7fe4283b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4201019b0 0x7fe420102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.845+0000 7fe4283b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420102370 0x7fe4201032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.845+0000 7fe4283b0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe42010c420 con 0x7fe4201007c0 2024-05-09T09:26:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.845+0000 7fe4283b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe42010c2a0 con 0x7fe420102370 2024-05-09T09:26:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.845+0000 7fe4283b0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe42010c5a0 con 0x7fe4201019b0 2024-05-09T09:26:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.845+0000 7fe42614c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4201007c0 0x7fe420104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe42614c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4201007c0 0x7fe420104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53410/0 (socket says 172.21.15.12:53410) 2024-05-09T09:26:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe42614c640 1 -- 172.21.15.12:0/2440404344 learned_addr learned my addr 172.21.15.12:0/2440404344 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe42694d640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420102370 0x7fe4201032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe42594b640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4201019b0 0x7fe420102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe42614c640 1 -- 172.21.15.12:0/2440404344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4201019b0 msgr2=0x7fe420102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe42614c640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4201019b0 0x7fe420102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe42614c640 1 -- 172.21.15.12:0/2440404344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420102370 msgr2=0x7fe4201032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe42614c640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420102370 0x7fe4201032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe42614c640 1 -- 172.21.15.12:0/2440404344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe420103b90 con 0x7fe4201007c0 2024-05-09T09:26:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe42614c640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4201007c0 0x7fe420104700 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00eb70 tx=0x7fe41c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe4177fe640 1 -- 172.21.15.12:0/2440404344 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c003b70 con 0x7fe4201007c0 2024-05-09T09:26:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe4177fe640 1 -- 172.21.15.12:0/2440404344 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe41c00ee70 con 0x7fe4201007c0 2024-05-09T09:26:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe4177fe640 1 -- 172.21.15.12:0/2440404344 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c0107b0 con 0x7fe4201007c0 2024-05-09T09:26:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe4283b0640 1 -- 172.21.15.12:0/2440404344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4201a1c60 con 0x7fe4201007c0 2024-05-09T09:26:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.846+0000 7fe4283b0640 1 -- 172.21.15.12:0/2440404344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4201a2130 con 0x7fe4201007c0 2024-05-09T09:26:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.849+0000 7fe4177fe640 1 -- 172.21.15.12:0/2440404344 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe41c003f30 con 0x7fe4201007c0 2024-05-09T09:26:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.849+0000 7fe42b19f740 1 Processor -- start 2024-05-09T09:26:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.849+0000 7fe42b19f740 1 -- start start 2024-05-09T09:26:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42b19f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a25a60 0x562424a25e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42b19f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a26450 0x562424a2af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42b19f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a2da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42b19f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562424981340 con 0x562424a2b5c0 2024-05-09T09:26:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42b19f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562424929290 con 0x562424a25a60 2024-05-09T09:26:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42b19f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56242497d300 con 0x562424a26450 2024-05-09T09:26:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42594b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a2da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42594b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a2da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53426/0 (socket says 172.21.15.12:53426) 2024-05-09T09:26:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42594b640 1 -- 172.21.15.12:0/2318688466 learned_addr learned my addr 172.21.15.12:0/2318688466 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42694d640 1 --2- 172.21.15.12:0/2318688466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a25a60 0x562424a25e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42614c640 1 --2- 172.21.15.12:0/2318688466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a26450 0x562424a2af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe4177fe640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe40c0a2c30 0x7fe40c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42594b640 1 -- 172.21.15.12:0/2318688466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a26450 msgr2=0x562424a2af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42694d640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe40c0a2c30 0x7fe40c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42594b640 1 --2- 172.21.15.12:0/2318688466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a26450 0x562424a2af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42594b640 1 -- 172.21.15.12:0/2318688466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a25a60 msgr2=0x562424a25e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42594b640 1 --2- 172.21.15.12:0/2318688466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a25a60 0x562424a25e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42594b640 1 -- 172.21.15.12:0/2318688466 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242494d8e0 con 0x562424a2b5c0 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe42614c640 1 --2- 172.21.15.12:0/2318688466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a26450 0x562424a2af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.850+0000 7fe4177fe640 1 -- 172.21.15.12:0/2440404344 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fe41c0cf9e0 con 0x7fe4201007c0 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.851+0000 7fe42594b640 1 --2- 172.21.15.12:0/2318688466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a2da50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe410009900 tx=0x7fe410005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6710312714012c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.851+0000 7fe42694d640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe40c0a2c30 0x7fe40c0a50f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x562424a26cb0 tx=0x7fe418004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.851+0000 7fe414bfe640 1 -- 172.21.15.12:0/2318688466 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410040070 con 0x562424a2b5c0 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.851+0000 7fe414bfe640 1 -- 172.21.15.12:0/2318688466 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe410004020 con 0x562424a2b5c0 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.851+0000 7fe414bfe640 1 -- 172.21.15.12:0/2318688466 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410035040 con 0x562424a2b5c0 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.851+0000 7fe42b19f740 1 -- 172.21.15.12:0/2318688466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 msgr2=0x562424a2da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.851+0000 7fe42b19f740 1 --2- 172.21.15.12:0/2318688466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a2da50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe410009900 tx=0x7fe410005dd0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 -- 172.21.15.12:0/2318688466 shutdown_connections 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 --2- 172.21.15.12:0/2318688466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a2da50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 --2- 172.21.15.12:0/2318688466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a26450 0x562424a2af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 --2- 172.21.15.12:0/2318688466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a25a60 0x562424a25e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 -- 172.21.15.12:0/2318688466 >> 172.21.15.12:0/2318688466 conn(0x562424a188c0 msgr2=0x562424a18cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 -- 172.21.15.12:0/2318688466 shutdown_connections 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 -- 172.21.15.12:0/2318688466 wait complete. 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 -- start start 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 Processor -- start 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a25a60 0x562424a2fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a26450 0x562424a30150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a2b5c0 0x562424a37190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562424a2e510 con 0x562424a26450 2024-05-09T09:26:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56242493c790 con 0x562424a25a60 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562424a2e690 con 0x562424a2b5c0 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42b19f740 10 client.0 _ll_register_callbacks cb 0x562424977a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42594b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a26450 0x562424a30150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42594b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a26450 0x562424a30150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53438/0 (socket says 172.21.15.12:53438) 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.852+0000 7fe42594b640 1 -- 172.21.15.12:0/4006740899 learned_addr learned my addr 172.21.15.12:0/4006740899 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe42614c640 1 --2- 172.21.15.12:0/4006740899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a25a60 0x562424a2fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe42594b640 1 -- 172.21.15.12:0/4006740899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a2b5c0 msgr2=0x562424a37190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe42594b640 1 --2- 172.21.15.12:0/4006740899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a2b5c0 0x562424a37190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe42594b640 1 -- 172.21.15.12:0/4006740899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a25a60 msgr2=0x562424a2fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe42594b640 1 --2- 172.21.15.12:0/4006740899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a25a60 0x562424a2fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe42594b640 1 -- 172.21.15.12:0/4006740899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562424695210 con 0x562424a26450 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe42594b640 1 --2- 172.21.15.12:0/4006740899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a26450 0x562424a30150 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe410000c00 tx=0x7fe4100056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa80eb5141e3faf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe409ffb640 1 -- 172.21.15.12:0/4006740899 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41004c020 con 0x562424a26450 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe409ffb640 1 -- 172.21.15.12:0/4006740899 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fe410031810 con 0x562424a26450 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe42b19f740 1 -- 172.21.15.12:0/4006740899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe410004020 con 0x562424a26450 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe409ffb640 1 -- 172.21.15.12:0/4006740899 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe410057050 con 0x562424a26450 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe42b19f740 10 client.15717 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe42b19f740 1 -- 172.21.15.12:0/4006740899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562424761620 con 0x562424a26450 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.853+0000 7fe409ffb640 1 -- 172.21.15.12:0/4006740899 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe410004020 con 0x562424a26450 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.854+0000 7fe409ffb640 1 -- 172.21.15.12:0/4006740899 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe4100047f0 con 0x562424a26450 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.854+0000 7fe42b19f740 10 client.15717 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.854+0000 7fe42b19f740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.854+0000 7fe42b19f740 20 client.15717 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.854+0000 7fe42b19f740 1 --2- 172.21.15.12:0/4006740899 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x562424a3b2f0 0x562424a3d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.854+0000 7fe42b19f740 4 client.15717 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.854+0000 7fe42b19f740 1 -- 172.21.15.12:0/4006740899 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56242498ea70 con 0x562424a3b2f0 2024-05-09T09:26:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.854+0000 7fe42614c640 1 --2- 172.21.15.12:0/4006740899 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x562424a3b2f0 0x562424a3d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.854+0000 7fe42614c640 1 --2- 172.21.15.12:0/4006740899 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x562424a3b2f0 0x562424a3d6e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a2882ca4ac72e1 server_cookie=7173496949e5f19d in_seq=0 out_seq=0 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.854+0000 7fe409ffb640 10 client.15717 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe409ffb640 1 -- 172.21.15.12:0/4006740899 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56242498ea70 con 0x562424a3b2f0 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe409ffb640 10 client.15717 handle_command_reply: tid=0 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe42b19f740 1 client.15717 shutdown 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe42b19f740 1 -- 172.21.15.12:0/4006740899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a26450 msgr2=0x562424a30150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe42b19f740 1 --2- 172.21.15.12:0/4006740899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a26450 0x562424a30150 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe410000c00 tx=0x7fe4100056c0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe42b19f740 1 -- 172.21.15.12:0/4006740899 shutdown_connections 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe42b19f740 1 --2- 172.21.15.12:0/4006740899 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x562424a3b2f0 0x562424a3d6e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe42b19f740 1 --2- 172.21.15.12:0/4006740899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a2b5c0 0x562424a37190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe42b19f740 1 --2- 172.21.15.12:0/4006740899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a26450 0x562424a30150 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe42b19f740 1 --2- 172.21.15.12:0/4006740899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a25a60 0x562424a2fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe409ffb640 0 client.15717 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe42b19f740 1 -- 172.21.15.12:0/4006740899 >> 172.21.15.12:0/4006740899 conn(0x562424a188c0 msgr2=0x562424a17a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe42b19f740 1 -- 172.21.15.12:0/4006740899 shutdown_connections 2024-05-09T09:26:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.855+0000 7fe42b19f740 1 -- 172.21.15.12:0/4006740899 wait complete. 2024-05-09T09:26:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.856+0000 7fe42b19f740 20 client.15717 trim_cache size 0 max 16384 2024-05-09T09:26:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42b19f740 1 Processor -- start 2024-05-09T09:26:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42b19f740 1 -- start start 2024-05-09T09:26:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42b19f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a25a60 0x562424a13620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42b19f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a26450 0x562424a37150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42b19f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a18ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42b19f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe4201a82a0 con 0x562424a2b5c0 2024-05-09T09:26:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42b19f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4201a9de0 con 0x562424a26450 2024-05-09T09:26:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42b19f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4201a8970 con 0x562424a25a60 2024-05-09T09:26:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42694d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a18ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42694d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a18ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53446/0 (socket says 172.21.15.12:53446) 2024-05-09T09:26:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42694d640 1 -- 172.21.15.12:0/2551811069 learned_addr learned my addr 172.21.15.12:0/2551811069 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42594b640 1 --2- 172.21.15.12:0/2551811069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a26450 0x562424a37150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42614c640 1 --2- 172.21.15.12:0/2551811069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a25a60 0x562424a13620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42694d640 1 -- 172.21.15.12:0/2551811069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a25a60 msgr2=0x562424a13620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42694d640 1 --2- 172.21.15.12:0/2551811069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a25a60 0x562424a13620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42694d640 1 -- 172.21.15.12:0/2551811069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a26450 msgr2=0x562424a37150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.871+0000 7fe42694d640 1 --2- 172.21.15.12:0/2551811069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a26450 0x562424a37150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.872+0000 7fe42694d640 1 -- 172.21.15.12:0/2551811069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562424a16f00 con 0x562424a2b5c0 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.872+0000 7fe42694d640 1 --2- 172.21.15.12:0/2551811069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a18ca0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe4180104b0 tx=0x7fe418005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf1e02fb9a67dea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.872+0000 7fe40bfff640 1 -- 172.21.15.12:0/2551811069 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418010aa0 con 0x562424a2b5c0 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.872+0000 7fe40bfff640 1 -- 172.21.15.12:0/2551811069 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe418010c40 con 0x562424a2b5c0 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.872+0000 7fe40bfff640 1 -- 172.21.15.12:0/2551811069 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418020700 con 0x562424a2b5c0 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.872+0000 7fe42b19f740 1 -- 172.21.15.12:0/2551811069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 msgr2=0x562424a18ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.872+0000 7fe42b19f740 1 --2- 172.21.15.12:0/2551811069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a18ca0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe4180104b0 tx=0x7fe418005850 comp rx=0 tx=0).stop 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.872+0000 7fe42b19f740 1 -- 172.21.15.12:0/2551811069 shutdown_connections 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.872+0000 7fe42b19f740 1 --2- 172.21.15.12:0/2551811069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a18ca0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.872+0000 7fe42b19f740 1 --2- 172.21.15.12:0/2551811069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a26450 0x562424a37150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.872+0000 7fe42b19f740 1 --2- 172.21.15.12:0/2551811069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a25a60 0x562424a13620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.872+0000 7fe42b19f740 1 -- 172.21.15.12:0/2551811069 >> 172.21.15.12:0/2551811069 conn(0x562424a188c0 msgr2=0x562424a2ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42b19f740 1 -- 172.21.15.12:0/2551811069 shutdown_connections 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42b19f740 1 -- 172.21.15.12:0/2551811069 wait complete. 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42b19f740 1 -- start start 2024-05-09T09:26:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42b19f740 1 Processor -- start 2024-05-09T09:26:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42b19f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a25a60 0x562424a34910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42b19f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a26450 0x562424a34e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42b19f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a35df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42b19f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe4201a9710 con 0x562424a2b5c0 2024-05-09T09:26:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42b19f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4201a9040 con 0x562424a26450 2024-05-09T09:26:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42b19f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4201a62e0 con 0x562424a25a60 2024-05-09T09:26:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42b19f740 10 client.0 _ll_register_callbacks cb 0x7fe418009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42694d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a35df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42694d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a35df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53462/0 (socket says 172.21.15.12:53462) 2024-05-09T09:26:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42694d640 1 -- 172.21.15.12:0/4139821469 learned_addr learned my addr 172.21.15.12:0/4139821469 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42594b640 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a26450 0x562424a34e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.873+0000 7fe42614c640 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a25a60 0x562424a34910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe42694d640 1 -- 172.21.15.12:0/4139821469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a25a60 msgr2=0x562424a34910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe42694d640 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a25a60 0x562424a34910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe42694d640 1 -- 172.21.15.12:0/4139821469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a26450 msgr2=0x562424a34e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe42694d640 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a26450 0x562424a34e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe42694d640 1 -- 172.21.15.12:0/4139821469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562424a363e0 con 0x562424a2b5c0 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe42614c640 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a25a60 0x562424a34910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe42694d640 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a35df0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe4180058c0 tx=0x7fe41801ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c6b2fb61072dc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe409ffb640 1 -- 172.21.15.12:0/4139821469 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4180120d0 con 0x562424a2b5c0 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe409ffb640 1 -- 172.21.15.12:0/4139821469 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fe418005020 con 0x562424a2b5c0 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe42b19f740 1 -- 172.21.15.12:0/4139821469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe418010c40 con 0x562424a2b5c0 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe409ffb640 1 -- 172.21.15.12:0/4139821469 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe41803c050 con 0x562424a2b5c0 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe42b19f740 10 client.15723 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe42b19f740 1 -- 172.21.15.12:0/4139821469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562424aa1860 con 0x562424a2b5c0 2024-05-09T09:26:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe409ffb640 1 -- 172.21.15.12:0/4139821469 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe418010c40 con 0x562424a2b5c0 2024-05-09T09:26:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.874+0000 7fe409ffb640 1 -- 172.21.15.12:0/4139821469 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe418020480 con 0x562424a2b5c0 2024-05-09T09:26:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.875+0000 7fe42b19f740 10 client.15723 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.875+0000 7fe42b19f740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.875+0000 7fe42b19f740 20 client.15723 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.875+0000 7fe42b19f740 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x562424aa2230 0x562424aa4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.875+0000 7fe42b19f740 4 client.15723 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:26:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.875+0000 7fe42b19f740 1 -- 172.21.15.12:0/4139821469 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562424988dc0 con 0x562424aa2230 2024-05-09T09:26:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.875+0000 7fe42614c640 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x562424aa2230 0x562424aa4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.875+0000 7fe42614c640 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x562424aa2230 0x562424aa4650 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12899c40c6bcc275 server_cookie=b1fb0f9e6b936312 in_seq=0 out_seq=0 2024-05-09T09:26:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.875+0000 7fe409ffb640 10 client.15723 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe409ffb640 1 -- 172.21.15.12:0/4139821469 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562424988dc0 con 0x562424aa2230 2024-05-09T09:26:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe409ffb640 10 client.15723 handle_command_reply: tid=0 2024-05-09T09:26:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe42b19f740 1 client.15723 shutdown 2024-05-09T09:26:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe42b19f740 1 -- 172.21.15.12:0/4139821469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 msgr2=0x562424a35df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe42b19f740 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a35df0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe4180058c0 tx=0x7fe41801ea60 comp rx=0 tx=0).stop 2024-05-09T09:26:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe42b19f740 1 -- 172.21.15.12:0/4139821469 shutdown_connections 2024-05-09T09:26:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe42b19f740 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x562424aa2230 0x562424aa4650 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe42b19f740 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562424a2b5c0 0x562424a35df0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe42b19f740 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562424a26450 0x562424a34e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe409ffb640 0 client.15723 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe42b19f740 1 --2- 172.21.15.12:0/4139821469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562424a25a60 0x562424a34910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe42b19f740 1 -- 172.21.15.12:0/4139821469 >> 172.21.15.12:0/4139821469 conn(0x562424a188c0 msgr2=0x562424a2fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe42b19f740 1 -- 172.21.15.12:0/4139821469 shutdown_connections 2024-05-09T09:26:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe42b19f740 1 -- 172.21.15.12:0/4139821469 wait complete. 2024-05-09T09:26:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.879+0000 7fe42b19f740 20 client.15723 trim_cache size 0 max 16384 2024-05-09T09:26:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:26:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:26:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:26:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:26:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.880+0000 7fe4283b0640 1 -- 172.21.15.12:0/2440404344 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe40c0a2c30 msgr2=0x7fe40c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.880+0000 7fe4283b0640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe40c0a2c30 0x7fe40c0a50f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x562424a26cb0 tx=0x7fe418004730 comp rx=0 tx=0).stop 2024-05-09T09:26:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.880+0000 7fe4283b0640 1 -- 172.21.15.12:0/2440404344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4201007c0 msgr2=0x7fe420104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.880+0000 7fe4283b0640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4201007c0 0x7fe420104700 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00eb70 tx=0x7fe41c00c6a0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.880+0000 7fe4283b0640 1 -- 172.21.15.12:0/2440404344 shutdown_connections 2024-05-09T09:26:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.880+0000 7fe4283b0640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe40c0a2c30 0x7fe40c0a50f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.880+0000 7fe4283b0640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420102370 0x7fe4201032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.880+0000 7fe4283b0640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4201019b0 0x7fe420102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.880+0000 7fe4283b0640 1 --2- 172.21.15.12:0/2440404344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4201007c0 0x7fe420104700 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.880+0000 7fe4283b0640 1 -- 172.21.15.12:0/2440404344 >> 172.21.15.12:0/2440404344 conn(0x7fe4200fbd30 msgr2=0x7fe4200fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.880+0000 7fe4283b0640 1 -- 172.21.15.12:0/2440404344 shutdown_connections 2024-05-09T09:26:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:05.880+0000 7fe4283b0640 1 -- 172.21.15.12:0/2440404344 wait complete. 2024-05-09T09:26:06.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:06 smithi012 ceph-mon[30140]: pgmap v383: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-05-09T09:26:06.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:06 smithi170 ceph-mon[33632]: pgmap v383: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-05-09T09:26:06.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:06 smithi016 ceph-mon[33335]: pgmap v383: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-05-09T09:26:08.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:08 smithi012 ceph-mon[30140]: pgmap v384: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:26:08.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:08 smithi170 ceph-mon[33632]: pgmap v384: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:26:08.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:08 smithi016 ceph-mon[33335]: pgmap v384: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:26:09.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:26:09.901 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:26:10.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.013+0000 7f5f83aa5640 1 -- 172.21.15.12:0/418873473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f7c100770 msgr2=0x7f5f7c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.013+0000 7f5f83aa5640 1 --2- 172.21.15.12:0/418873473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f7c100770 0x7f5f7c100b90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009900 tx=0x7f5f7802a410 comp rx=0 tx=0).stop 2024-05-09T09:26:10.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.013+0000 7f5f83aa5640 1 -- 172.21.15.12:0/418873473 shutdown_connections 2024-05-09T09:26:10.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.013+0000 7f5f83aa5640 1 --2- 172.21.15.12:0/418873473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f7c102320 0x7f5f7c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.013+0000 7f5f83aa5640 1 --2- 172.21.15.12:0/418873473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f7c101960 0x7f5f7c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.013+0000 7f5f83aa5640 1 --2- 172.21.15.12:0/418873473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f7c100770 0x7f5f7c100b90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.013+0000 7f5f83aa5640 1 -- 172.21.15.12:0/418873473 >> 172.21.15.12:0/418873473 conn(0x7f5f7c0fbce0 msgr2=0x7f5f7c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.013+0000 7f5f83aa5640 1 -- 172.21.15.12:0/418873473 shutdown_connections 2024-05-09T09:26:10.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.013+0000 7f5f83aa5640 1 -- 172.21.15.12:0/418873473 wait complete. 2024-05-09T09:26:10.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.013+0000 7f5f83aa5640 1 Processor -- start 2024-05-09T09:26:10.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.013+0000 7f5f83aa5640 1 -- start start 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f83aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f7c100770 0x7f5f7c10b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f83aa5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f7c101960 0x7f5f7c109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f83aa5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f7c102320 0x7f5f7c109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f83aa5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c10d4c0 con 0x7f5f7c102320 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f83aa5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c10d340 con 0x7f5f7c100770 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f83aa5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c10d640 con 0x7f5f7c101960 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f82042640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f7c102320 0x7f5f7c109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f82042640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f7c102320 0x7f5f7c109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53476/0 (socket says 172.21.15.12:53476) 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f82042640 1 -- 172.21.15.12:0/3703699876 learned_addr learned my addr 172.21.15.12:0/3703699876 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f81841640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f7c100770 0x7f5f7c10b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f81040640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f7c101960 0x7f5f7c109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f82042640 1 -- 172.21.15.12:0/3703699876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f7c101960 msgr2=0x7f5f7c109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f82042640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f7c101960 0x7f5f7c109730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f82042640 1 -- 172.21.15.12:0/3703699876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f7c100770 msgr2=0x7f5f7c10b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f82042640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f7c100770 0x7f5f7c10b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f82042640 1 -- 172.21.15.12:0/3703699876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7c10a500 con 0x7f5f7c102320 2024-05-09T09:26:10.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f82042640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f7c102320 0x7f5f7c109c70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400cce0 tx=0x7f5f7400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.014+0000 7f5f72ffd640 1 -- 172.21.15.12:0/3703699876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7400f450 con 0x7f5f7c102320 2024-05-09T09:26:10.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.015+0000 7f5f72ffd640 1 -- 172.21.15.12:0/3703699876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f7400fac0 con 0x7f5f7c102320 2024-05-09T09:26:10.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.015+0000 7f5f83aa5640 1 -- 172.21.15.12:0/3703699876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7c1a1cb0 con 0x7f5f7c102320 2024-05-09T09:26:10.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.015+0000 7f5f72ffd640 1 -- 172.21.15.12:0/3703699876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74018be0 con 0x7f5f7c102320 2024-05-09T09:26:10.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.015+0000 7f5f83aa5640 1 -- 172.21.15.12:0/3703699876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f7c1a21b0 con 0x7f5f7c102320 2024-05-09T09:26:10.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.017+0000 7f5f72ffd640 1 -- 172.21.15.12:0/3703699876 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f7400fc70 con 0x7f5f7c102320 2024-05-09T09:26:10.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.017+0000 7f5f867b5740 1 Processor -- start 2024-05-09T09:26:10.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f867b5740 1 -- start start 2024-05-09T09:26:10.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f867b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af069df30 0x559af069e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f867b5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06a3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f867b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af06a3a90 0x559af06a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f867b5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559af0602d00 con 0x559af069e920 2024-05-09T09:26:10.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f867b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559af05aa4c0 con 0x559af06a3a90 2024-05-09T09:26:10.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f867b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559af05fecc0 con 0x559af069df30 2024-05-09T09:26:10.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f81841640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f81841640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06a3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53480/0 (socket says 172.21.15.12:53480) 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f81841640 1 -- 172.21.15.12:0/3668376042 learned_addr learned my addr 172.21.15.12:0/3668376042 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f81040640 1 --2- 172.21.15.12:0/3668376042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af069df30 0x559af069e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f81040640 1 --2- 172.21.15.12:0/3668376042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af06a3a90 0x559af06a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f81841640 1 -- 172.21.15.12:0/3668376042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af069df30 msgr2=0x559af069e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f81841640 1 --2- 172.21.15.12:0/3668376042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af069df30 0x559af069e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f81841640 1 -- 172.21.15.12:0/3668376042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af06a3a90 msgr2=0x559af06a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f81841640 1 --2- 172.21.15.12:0/3668376042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af06a3a90 0x559af06a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f81841640 1 -- 172.21.15.12:0/3668376042 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af03e07d0 con 0x559af069e920 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.018+0000 7f5f81040640 1 --2- 172.21.15.12:0/3668376042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af06a3a90 0x559af06a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.019+0000 7f5f81841640 1 --2- 172.21.15.12:0/3668376042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06a3420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c1017c0 tx=0x7f5f78031cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00b2564f75ef47f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.019+0000 7f5f72ffd640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5f680a2b90 0x7f5f680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.019+0000 7f5f67fff640 1 -- 172.21.15.12:0/3668376042 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f78040070 con 0x559af069e920 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.019+0000 7f5f81040640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5f680a2b90 0x7f5f680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.019+0000 7f5f67fff640 1 -- 172.21.15.12:0/3668376042 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f78031810 con 0x559af069e920 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.019+0000 7f5f67fff640 1 -- 172.21.15.12:0/3668376042 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f78035040 con 0x559af069e920 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.019+0000 7f5f72ffd640 1 -- 172.21.15.12:0/3703699876 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f5f740cfae0 con 0x7f5f7c102320 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.020+0000 7f5f867b5740 1 -- 172.21.15.12:0/3668376042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 msgr2=0x559af06a3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.020+0000 7f5f81040640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5f680a2b90 0x7f5f680a5050 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x559af069f180 tx=0x7f5f6c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.020+0000 7f5f867b5740 1 --2- 172.21.15.12:0/3668376042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06a3420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c1017c0 tx=0x7f5f78031cb0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.020+0000 7f5f867b5740 1 -- 172.21.15.12:0/3668376042 shutdown_connections 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.020+0000 7f5f867b5740 1 --2- 172.21.15.12:0/3668376042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af06a3a90 0x559af06a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.020+0000 7f5f867b5740 1 --2- 172.21.15.12:0/3668376042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06a3420 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.020+0000 7f5f867b5740 1 --2- 172.21.15.12:0/3668376042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af069df30 0x559af069e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.020+0000 7f5f867b5740 1 -- 172.21.15.12:0/3668376042 >> 172.21.15.12:0/3668376042 conn(0x559af06995a0 msgr2=0x559af06999a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.020+0000 7f5f867b5740 1 -- 172.21.15.12:0/3668376042 shutdown_connections 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.020+0000 7f5f867b5740 1 -- 172.21.15.12:0/3668376042 wait complete. 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.020+0000 7f5f867b5740 1 -- start start 2024-05-09T09:26:10.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.020+0000 7f5f867b5740 1 Processor -- start 2024-05-09T09:26:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f867b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af06a8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f867b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af069e920 0x559af06a8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f867b5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af06a3a90 0x559af06af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f867b5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559af06a69e0 con 0x559af06a3a90 2024-05-09T09:26:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f867b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559af05bdb90 con 0x559af069df30 2024-05-09T09:26:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f867b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559af06a6b60 con 0x559af069e920 2024-05-09T09:26:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f867b5740 10 client.0 _ll_register_callbacks cb 0x559af05f9da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f82042640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af06a3a90 0x559af06af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f82042640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af06a3a90 0x559af06af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53494/0 (socket says 172.21.15.12:53494) 2024-05-09T09:26:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f82042640 1 -- 172.21.15.12:0/1795923828 learned_addr learned my addr 172.21.15.12:0/1795923828 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f81040640 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af069e920 0x559af06a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f81841640 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af06a8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f82042640 1 -- 172.21.15.12:0/1795923828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af069e920 msgr2=0x559af06a8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f82042640 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af069e920 0x559af06a8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f82042640 1 -- 172.21.15.12:0/1795923828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 msgr2=0x559af06a8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f82042640 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af06a8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f82042640 1 -- 172.21.15.12:0/1795923828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af03163f0 con 0x559af06a3a90 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f81841640 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af06a8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f82042640 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af06a3a90 0x559af06af880 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5f74098940 tx=0x7f5f74005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a13762d5f2b6f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.021+0000 7f5f657fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f657fa640 1 -- 172.21.15.12:0/1795923828 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f740b6060 con 0x559af06a3a90 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f657fa640 1 -- 172.21.15.12:0/1795923828 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f5f740a4950 con 0x559af06a3a90 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f867b5740 1 -- 172.21.15.12:0/1795923828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f78031810 con 0x559af06a3a90 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f657fa640 1 -- 172.21.15.12:0/1795923828 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5f740a0320 con 0x559af06a3a90 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f867b5740 10 client.15735 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f867b5740 1 -- 172.21.15.12:0/1795923828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559af02d60f0 con 0x559af06a3a90 2024-05-09T09:26:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f657fa640 1 -- 172.21.15.12:0/1795923828 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f78031810 con 0x559af06a3a90 2024-05-09T09:26:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f657fa640 1 -- 172.21.15.12:0/1795923828 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5f7400f680 con 0x559af06a3a90 2024-05-09T09:26:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f867b5740 10 client.15735 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f867b5740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f867b5740 20 client.15735 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f867b5740 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559af06b41d0 0x559af06b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f867b5740 4 client.15735 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.022+0000 7f5f867b5740 1 -- 172.21.15.12:0/1795923828 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559af0610430 con 0x559af06b41d0 2024-05-09T09:26:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.023+0000 7f5f81841640 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559af06b41d0 0x559af06b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.023+0000 7f5f657fa640 10 client.15735 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.023+0000 7f5f81841640 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559af06b41d0 0x559af06b6660 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=860eb728c7fd0042 server_cookie=2ff69fbf9109e05a in_seq=0 out_seq=0 2024-05-09T09:26:10.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f657fa640 1 -- 172.21.15.12:0/1795923828 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559af0610430 con 0x559af06b41d0 2024-05-09T09:26:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f657fa640 10 client.15735 handle_command_reply: tid=0 2024-05-09T09:26:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f867b5740 1 client.15735 shutdown 2024-05-09T09:26:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f867b5740 1 -- 172.21.15.12:0/1795923828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af06a3a90 msgr2=0x559af06af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f867b5740 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af06a3a90 0x559af06af880 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5f74098940 tx=0x7f5f74005b10 comp rx=0 tx=0).stop 2024-05-09T09:26:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f867b5740 1 -- 172.21.15.12:0/1795923828 shutdown_connections 2024-05-09T09:26:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f867b5740 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559af06b41d0 0x559af06b6660 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f867b5740 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af06a3a90 0x559af06af880 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f867b5740 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af069e920 0x559af06a8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f657fa640 0 client.15735 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f867b5740 1 --2- 172.21.15.12:0/1795923828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af06a8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f867b5740 1 -- 172.21.15.12:0/1795923828 >> 172.21.15.12:0/1795923828 conn(0x559af06995a0 msgr2=0x559af06987e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f867b5740 1 -- 172.21.15.12:0/1795923828 shutdown_connections 2024-05-09T09:26:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f867b5740 1 -- 172.21.15.12:0/1795923828 wait complete. 2024-05-09T09:26:10.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.024+0000 7f5f867b5740 20 client.15735 trim_cache size 0 max 16384 2024-05-09T09:26:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f867b5740 1 Processor -- start 2024-05-09T09:26:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f867b5740 1 -- start start 2024-05-09T09:26:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f867b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af069f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f867b5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06af840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f867b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af06a3a90 0x559af069fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f867b5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c1ac700 con 0x559af069e920 2024-05-09T09:26:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f867b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c1ae240 con 0x559af069df30 2024-05-09T09:26:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f867b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c1acdd0 con 0x559af06a3a90 2024-05-09T09:26:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f81040640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f81040640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06af840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53508/0 (socket says 172.21.15.12:53508) 2024-05-09T09:26:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f81040640 1 -- 172.21.15.12:0/4065679481 learned_addr learned my addr 172.21.15.12:0/4065679481 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f82042640 1 --2- 172.21.15.12:0/4065679481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af06a3a90 0x559af069fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f81841640 1 --2- 172.21.15.12:0/4065679481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af069f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f81040640 1 -- 172.21.15.12:0/4065679481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af06a3a90 msgr2=0x559af069fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f81040640 1 --2- 172.21.15.12:0/4065679481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af06a3a90 0x559af069fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f81040640 1 -- 172.21.15.12:0/4065679481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 msgr2=0x559af069f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f81040640 1 --2- 172.21.15.12:0/4065679481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af069f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f81040640 1 -- 172.21.15.12:0/4065679481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af069f1b0 con 0x559af069e920 2024-05-09T09:26:10.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.057+0000 7f5f81841640 1 --2- 172.21.15.12:0/4065679481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af069f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:10.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.058+0000 7f5f81040640 1 --2- 172.21.15.12:0/4065679481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06af840 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c020490 tx=0x7f5f6c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0b4eeb6eaa636 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.058+0000 7f5f677fe640 1 -- 172.21.15.12:0/4065679481 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c01e700 con 0x559af069e920 2024-05-09T09:26:10.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.058+0000 7f5f677fe640 1 -- 172.21.15.12:0/4065679481 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f6c01ed70 con 0x559af069e920 2024-05-09T09:26:10.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.058+0000 7f5f677fe640 1 -- 172.21.15.12:0/4065679481 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c01ce20 con 0x559af069e920 2024-05-09T09:26:10.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.059+0000 7f5f867b5740 1 -- 172.21.15.12:0/4065679481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 msgr2=0x559af06af840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.059+0000 7f5f867b5740 1 --2- 172.21.15.12:0/4065679481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06af840 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c020490 tx=0x7f5f6c014040 comp rx=0 tx=0).stop 2024-05-09T09:26:10.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.059+0000 7f5f867b5740 1 -- 172.21.15.12:0/4065679481 shutdown_connections 2024-05-09T09:26:10.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.059+0000 7f5f867b5740 1 --2- 172.21.15.12:0/4065679481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af06a3a90 0x559af069fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.059+0000 7f5f867b5740 1 --2- 172.21.15.12:0/4065679481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06af840 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.059+0000 7f5f867b5740 1 --2- 172.21.15.12:0/4065679481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af069f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.059+0000 7f5f867b5740 1 -- 172.21.15.12:0/4065679481 >> 172.21.15.12:0/4065679481 conn(0x559af06995a0 msgr2=0x559af0699980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.059+0000 7f5f867b5740 1 -- 172.21.15.12:0/4065679481 shutdown_connections 2024-05-09T09:26:10.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.059+0000 7f5f867b5740 1 -- 172.21.15.12:0/4065679481 wait complete. 2024-05-09T09:26:10.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.059+0000 7f5f867b5740 1 -- start start 2024-05-09T09:26:10.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.059+0000 7f5f867b5740 1 Processor -- start 2024-05-09T09:26:10.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f867b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af06ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f867b5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06ad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f867b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af06a3a90 0x559af06ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f867b5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c1adb70 con 0x559af069e920 2024-05-09T09:26:10.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f867b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c1ad4a0 con 0x559af069df30 2024-05-09T09:26:10.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f867b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c1aa6e0 con 0x559af06a3a90 2024-05-09T09:26:10.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f867b5740 10 client.0 _ll_register_callbacks cb 0x7f5f740077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:10.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f81040640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f81040640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06ad650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53518/0 (socket says 172.21.15.12:53518) 2024-05-09T09:26:10.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f81040640 1 -- 172.21.15.12:0/247984132 learned_addr learned my addr 172.21.15.12:0/247984132 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f81841640 1 --2- 172.21.15.12:0/247984132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af06ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f82042640 1 --2- 172.21.15.12:0/247984132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af06a3a90 0x559af06ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f81040640 1 -- 172.21.15.12:0/247984132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af06a3a90 msgr2=0x559af06ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f81040640 1 --2- 172.21.15.12:0/247984132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af06a3a90 0x559af06ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f81040640 1 -- 172.21.15.12:0/247984132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 msgr2=0x559af06ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f81040640 1 --2- 172.21.15.12:0/247984132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af06ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.060+0000 7f5f81040640 1 -- 172.21.15.12:0/247984132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af06aebe0 con 0x559af069e920 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f81040640 1 --2- 172.21.15.12:0/247984132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06ad650 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c01c930 tx=0x7f5f6c01c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59512a43da1865df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f657fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f657fa640 1 -- 172.21.15.12:0/247984132 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c037060 con 0x559af069e920 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f657fa640 1 -- 172.21.15.12:0/247984132 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f5f6c01cba0 con 0x559af069e920 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f867b5740 1 -- 172.21.15.12:0/247984132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f6c01ed70 con 0x559af069e920 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f657fa640 1 -- 172.21.15.12:0/247984132 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5f6c03e050 con 0x559af069e920 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f867b5740 10 client.15741 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f867b5740 1 -- 172.21.15.12:0/247984132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559af071a860 con 0x559af069e920 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f657fa640 1 -- 172.21.15.12:0/247984132 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f6c01ed70 con 0x559af069e920 2024-05-09T09:26:10.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f657fa640 1 -- 172.21.15.12:0/247984132 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5f6c027070 con 0x559af069e920 2024-05-09T09:26:10.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f867b5740 10 client.15741 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:10.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f867b5740 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:10.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f867b5740 20 client.15741 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:10.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f867b5740 1 --2- 172.21.15.12:0/247984132 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559af06b41d0 0x559af071d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f867b5740 4 client.15741 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:26:10.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.061+0000 7f5f867b5740 1 -- 172.21.15.12:0/247984132 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559af060a780 con 0x559af06b41d0 2024-05-09T09:26:10.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.062+0000 7f5f81841640 1 --2- 172.21.15.12:0/247984132 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559af06b41d0 0x559af071d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.062+0000 7f5f81841640 1 --2- 172.21.15.12:0/247984132 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559af06b41d0 0x559af071d240 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4680d7e84fefc05 server_cookie=a23c0b551f2a792c in_seq=0 out_seq=0 2024-05-09T09:26:10.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.062+0000 7f5f657fa640 10 client.15741 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:10.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.062+0000 7f5f657fa640 1 -- 172.21.15.12:0/247984132 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559af060a780 con 0x559af06b41d0 2024-05-09T09:26:10.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.062+0000 7f5f657fa640 10 client.15741 handle_command_reply: tid=0 2024-05-09T09:26:10.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.062+0000 7f5f867b5740 1 client.15741 shutdown 2024-05-09T09:26:10.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.062+0000 7f5f867b5740 1 -- 172.21.15.12:0/247984132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 msgr2=0x559af06ad650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.062+0000 7f5f867b5740 1 --2- 172.21.15.12:0/247984132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06ad650 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c01c930 tx=0x7f5f6c01c6b0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f867b5740 1 -- 172.21.15.12:0/247984132 shutdown_connections 2024-05-09T09:26:10.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f867b5740 1 --2- 172.21.15.12:0/247984132 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559af06b41d0 0x559af071d240 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f867b5740 1 --2- 172.21.15.12:0/247984132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559af06a3a90 0x559af06ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f867b5740 1 --2- 172.21.15.12:0/247984132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af069e920 0x559af06ad650 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f657fa640 0 client.15741 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:10.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f867b5740 1 --2- 172.21.15.12:0/247984132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af069df30 0x559af06ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f867b5740 1 -- 172.21.15.12:0/247984132 >> 172.21.15.12:0/247984132 conn(0x559af06995a0 msgr2=0x559af069fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f867b5740 1 -- 172.21.15.12:0/247984132 shutdown_connections 2024-05-09T09:26:10.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f867b5740 1 -- 172.21.15.12:0/247984132 wait complete. 2024-05-09T09:26:10.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f867b5740 20 client.15741 trim_cache size 0 max 16384 2024-05-09T09:26:10.069 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:26:10.069 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T09:26:10.069 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T09:26:10.069 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:26:10.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f83aa5640 1 -- 172.21.15.12:0/3703699876 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5f680a2b90 msgr2=0x7f5f680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f83aa5640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5f680a2b90 0x7f5f680a5050 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x559af069f180 tx=0x7f5f6c00b390 comp rx=0 tx=0).stop 2024-05-09T09:26:10.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f83aa5640 1 -- 172.21.15.12:0/3703699876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f7c102320 msgr2=0x7f5f7c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f83aa5640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f7c102320 0x7f5f7c109c70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400cce0 tx=0x7f5f7400a620 comp rx=0 tx=0).stop 2024-05-09T09:26:10.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f83aa5640 1 -- 172.21.15.12:0/3703699876 shutdown_connections 2024-05-09T09:26:10.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f83aa5640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5f680a2b90 0x7f5f680a5050 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f83aa5640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f7c102320 0x7f5f7c109c70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f83aa5640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f7c101960 0x7f5f7c109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f83aa5640 1 --2- 172.21.15.12:0/3703699876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f7c100770 0x7f5f7c10b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.063+0000 7f5f83aa5640 1 -- 172.21.15.12:0/3703699876 >> 172.21.15.12:0/3703699876 conn(0x7f5f7c0fbce0 msgr2=0x7f5f7c104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.064+0000 7f5f83aa5640 1 -- 172.21.15.12:0/3703699876 shutdown_connections 2024-05-09T09:26:10.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.064+0000 7f5f83aa5640 1 -- 172.21.15.12:0/3703699876 wait complete. 2024-05-09T09:26:10.074 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:26:10.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:26:10.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.184+0000 7f587dc09640 1 -- 172.21.15.12:0/1611286998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5878101960 msgr2=0x7f5878101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.184+0000 7f587dc09640 1 --2- 172.21.15.12:0/1611286998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5878101960 0x7f5878101de0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5860009900 tx=0x7f586002a410 comp rx=0 tx=0).stop 2024-05-09T09:26:10.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.184+0000 7f587dc09640 1 -- 172.21.15.12:0/1611286998 shutdown_connections 2024-05-09T09:26:10.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.184+0000 7f587dc09640 1 --2- 172.21.15.12:0/1611286998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5878102320 0x7f5878108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.184+0000 7f587dc09640 1 --2- 172.21.15.12:0/1611286998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5878101960 0x7f5878101de0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.184+0000 7f587dc09640 1 --2- 172.21.15.12:0/1611286998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5878100770 0x7f5878100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.184+0000 7f587dc09640 1 -- 172.21.15.12:0/1611286998 >> 172.21.15.12:0/1611286998 conn(0x7f58780fbce0 msgr2=0x7f58780fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.184+0000 7f587dc09640 1 -- 172.21.15.12:0/1611286998 shutdown_connections 2024-05-09T09:26:10.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.184+0000 7f587dc09640 1 -- 172.21.15.12:0/1611286998 wait complete. 2024-05-09T09:26:10.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.184+0000 7f587dc09640 1 Processor -- start 2024-05-09T09:26:10.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.184+0000 7f587dc09640 1 -- start start 2024-05-09T09:26:10.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f587dc09640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5878100770 0x7f5878197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f587dc09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5878101960 0x7f58781980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f587dc09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5878102320 0x7f587819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f587dc09640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f587810c3d0 con 0x7f5878102320 2024-05-09T09:26:10.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f587dc09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f587810c250 con 0x7f5878101960 2024-05-09T09:26:10.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f587dc09640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f587810c550 con 0x7f5878100770 2024-05-09T09:26:10.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f5877fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5878102320 0x7f587819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f5877fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5878102320 0x7f587819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53524/0 (socket says 172.21.15.12:53524) 2024-05-09T09:26:10.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f5877fff640 1 -- 172.21.15.12:0/3734100244 learned_addr learned my addr 172.21.15.12:0/3734100244 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f58777fe640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5878100770 0x7f5878197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f586ffff640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5878101960 0x7f58781980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f5877fff640 1 -- 172.21.15.12:0/3734100244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5878100770 msgr2=0x7f5878197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f5877fff640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5878100770 0x7f5878197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f5877fff640 1 -- 172.21.15.12:0/3734100244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5878101960 msgr2=0x7f58781980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f5877fff640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5878101960 0x7f58781980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f5877fff640 1 -- 172.21.15.12:0/3734100244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587819f780 con 0x7f5878102320 2024-05-09T09:26:10.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.185+0000 7f5877fff640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5878102320 0x7f587819f110 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f587000eb10 tx=0x7f587000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.186+0000 7f58757fa640 1 -- 172.21.15.12:0/3734100244 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870004b20 con 0x7f5878102320 2024-05-09T09:26:10.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.186+0000 7f587dc09640 1 -- 172.21.15.12:0/3734100244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587819fa70 con 0x7f5878102320 2024-05-09T09:26:10.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.186+0000 7f58757fa640 1 -- 172.21.15.12:0/3734100244 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5870004cc0 con 0x7f5878102320 2024-05-09T09:26:10.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.186+0000 7f58757fa640 1 -- 172.21.15.12:0/3734100244 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870010700 con 0x7f5878102320 2024-05-09T09:26:10.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.186+0000 7f587dc09640 1 -- 172.21.15.12:0/3734100244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587819ffc0 con 0x7f5878102320 2024-05-09T09:26:10.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.187+0000 7f587dc09640 1 -- 172.21.15.12:0/3734100244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58781a02b0 con 0x7f5878102320 2024-05-09T09:26:10.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.188+0000 7f58757fa640 1 -- 172.21.15.12:0/3734100244 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5870004e70 con 0x7f5878102320 2024-05-09T09:26:10.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.189+0000 7f58757fa640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f58640a2b90 0x7f58640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.189+0000 7f58777fe640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f58640a2b90 0x7f58640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.189+0000 7f58757fa640 1 -- 172.21.15.12:0/3734100244 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f58700cfad0 con 0x7f5878102320 2024-05-09T09:26:10.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.189+0000 7f58777fe640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f58640a2b90 0x7f58640a5050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f58781017c0 tx=0x7f586800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.194+0000 7f58757fa640 1 -- 172.21.15.12:0/3734100244 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f587009c600 con 0x7f5878102320 2024-05-09T09:26:10.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.429+0000 7f587dc09640 1 -- 172.21.15.12:0/3734100244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58781a0480 con 0x7f5878102320 2024-05-09T09:26:10.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.430+0000 7f58757fa640 1 -- 172.21.15.12:0/3734100244 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11931 (secure 0 0 0) 0x7f587009bf00 con 0x7f5878102320 2024-05-09T09:26:10.429 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:26:10.430 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:25:31.411273+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[14628,24484,24515,24523,24502]},"id":1}]} 2024-05-09T09:26:10.430 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-05-09T09:26:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.432+0000 7f587dc09640 1 -- 172.21.15.12:0/3734100244 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f58640a2b90 msgr2=0x7f58640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.432+0000 7f587dc09640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f58640a2b90 0x7f58640a5050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f58781017c0 tx=0x7f586800b390 comp rx=0 tx=0).stop 2024-05-09T09:26:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.432+0000 7f587dc09640 1 -- 172.21.15.12:0/3734100244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5878102320 msgr2=0x7f587819f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.432+0000 7f587dc09640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5878102320 0x7f587819f110 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f587000eb10 tx=0x7f587000c2d0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.432+0000 7f587dc09640 1 -- 172.21.15.12:0/3734100244 shutdown_connections 2024-05-09T09:26:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.432+0000 7f587dc09640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f58640a2b90 0x7f58640a5050 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.432+0000 7f587dc09640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5878102320 0x7f587819f110 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.432+0000 7f587dc09640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5878101960 0x7f58781980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.432+0000 7f587dc09640 1 --2- 172.21.15.12:0/3734100244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5878100770 0x7f5878197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.432+0000 7f587dc09640 1 -- 172.21.15.12:0/3734100244 >> 172.21.15.12:0/3734100244 conn(0x7f58780fbce0 msgr2=0x7f58780fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.432+0000 7f587dc09640 1 -- 172.21.15.12:0/3734100244 shutdown_connections 2024-05-09T09:26:10.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.432+0000 7f587dc09640 1 -- 172.21.15.12:0/3734100244 wait complete. 2024-05-09T09:26:10.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-09T09:26:10.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.555+0000 7f39a75fb640 1 -- 172.21.15.12:0/540035223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0101960 msgr2=0x7f39a0101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.555+0000 7f39a75fb640 1 --2- 172.21.15.12:0/540035223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0101960 0x7f39a0101de0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3990009900 tx=0x7f399002a410 comp rx=0 tx=0).stop 2024-05-09T09:26:10.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.555+0000 7f39a75fb640 1 -- 172.21.15.12:0/540035223 shutdown_connections 2024-05-09T09:26:10.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.555+0000 7f39a75fb640 1 --2- 172.21.15.12:0/540035223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39a0102320 0x7f39a0108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.555+0000 7f39a75fb640 1 --2- 172.21.15.12:0/540035223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0101960 0x7f39a0101de0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.555+0000 7f39a75fb640 1 --2- 172.21.15.12:0/540035223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a0100770 0x7f39a0100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.555+0000 7f39a75fb640 1 -- 172.21.15.12:0/540035223 >> 172.21.15.12:0/540035223 conn(0x7f39a00fbce0 msgr2=0x7f39a00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.555+0000 7f39a75fb640 1 -- 172.21.15.12:0/540035223 shutdown_connections 2024-05-09T09:26:10.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.555+0000 7f39a75fb640 1 -- 172.21.15.12:0/540035223 wait complete. 2024-05-09T09:26:10.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a75fb640 1 Processor -- start 2024-05-09T09:26:10.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a75fb640 1 -- start start 2024-05-09T09:26:10.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a75fb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0100770 0x7f39a0104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a75fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39a0101960 0x7f39a0102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a75fb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a0102320 0x7f39a0103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a75fb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f39a010c420 con 0x7f39a0100770 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a75fb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39a010c2a0 con 0x7f39a0101960 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a75fb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39a010c5a0 con 0x7f39a0102320 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a5397640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0100770 0x7f39a0104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a5397640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0100770 0x7f39a0104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53538/0 (socket says 172.21.15.12:53538) 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a5397640 1 -- 172.21.15.12:0/4018229074 learned_addr learned my addr 172.21.15.12:0/4018229074 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a4b96640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39a0101960 0x7f39a0102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a5397640 1 -- 172.21.15.12:0/4018229074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a0102320 msgr2=0x7f39a0103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.556+0000 7f39a5b98640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a0102320 0x7f39a0103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.557+0000 7f39a5397640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a0102320 0x7f39a0103280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.557+0000 7f39a5397640 1 -- 172.21.15.12:0/4018229074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39a0101960 msgr2=0x7f39a0102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.557+0000 7f39a5397640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39a0101960 0x7f39a0102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.557+0000 7f39a5397640 1 -- 172.21.15.12:0/4018229074 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a0103b10 con 0x7f39a0100770 2024-05-09T09:26:10.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.557+0000 7f39a5397640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0100770 0x7f39a0104700 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f399c00c970 tx=0x7f399c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.557+0000 7f39967fc640 1 -- 172.21.15.12:0/4018229074 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c004220 con 0x7f39a0100770 2024-05-09T09:26:10.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.557+0000 7f39967fc640 1 -- 172.21.15.12:0/4018229074 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f399c0043c0 con 0x7f39a0100770 2024-05-09T09:26:10.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.557+0000 7f39a75fb640 1 -- 172.21.15.12:0/4018229074 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a01a1cc0 con 0x7f39a0100770 2024-05-09T09:26:10.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.557+0000 7f39967fc640 1 -- 172.21.15.12:0/4018229074 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c00f450 con 0x7f39a0100770 2024-05-09T09:26:10.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.557+0000 7f39a75fb640 1 -- 172.21.15.12:0/4018229074 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a01a2190 con 0x7f39a0100770 2024-05-09T09:26:10.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39967fc640 1 -- 172.21.15.12:0/4018229074 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f399c0079b0 con 0x7f39a0100770 2024-05-09T09:26:10.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39aa3e0740 1 Processor -- start 2024-05-09T09:26:10.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39aa3e0740 1 -- start start 2024-05-09T09:26:10.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39aa3e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee78360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39aa3e0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 0x5565dee7d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39aa3e0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee7ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39aa3e0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5565deddcc40 con 0x5565dee7daa0 2024-05-09T09:26:10.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39aa3e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5565ded84580 con 0x5565dee77f40 2024-05-09T09:26:10.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39aa3e0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5565dedd8c00 con 0x5565dee78930 2024-05-09T09:26:10.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39a4b96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee7ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39a4b96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee7ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53550/0 (socket says 172.21.15.12:53550) 2024-05-09T09:26:10.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39a4b96640 1 -- 172.21.15.12:0/3444198015 learned_addr learned my addr 172.21.15.12:0/3444198015 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39a5b98640 1 --2- 172.21.15.12:0/3444198015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee78360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39967fc640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f398c0a2b90 0x7f398c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.560+0000 7f39a5397640 1 --2- 172.21.15.12:0/3444198015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 0x5565dee7d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.561+0000 7f39a5b98640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f398c0a2b90 0x7f398c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.561+0000 7f39a5b98640 1 -- 172.21.15.12:0/3444198015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 msgr2=0x5565dee7d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.561+0000 7f39a5b98640 1 --2- 172.21.15.12:0/3444198015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 0x5565dee7d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.561+0000 7f39967fc640 1 -- 172.21.15.12:0/4018229074 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f399c0cf1f0 con 0x7f39a0100770 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.561+0000 7f39a5b98640 1 -- 172.21.15.12:0/3444198015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 msgr2=0x5565dee7ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.561+0000 7f39a5b98640 1 --2- 172.21.15.12:0/3444198015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee7ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.561+0000 7f39a5b98640 1 -- 172.21.15.12:0/3444198015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565deaf03f0 con 0x5565dee77f40 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.561+0000 7f39a4b96640 1 --2- 172.21.15.12:0/3444198015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee7ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.561+0000 7f39a5b98640 1 --2- 172.21.15.12:0/3444198015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee78360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f39980137f0 tx=0x7f3998013cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2c52849509eaa6f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.561+0000 7f39a5b98640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f398c0a2b90 0x7f398c0a5050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3998004000 tx=0x7f399801c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.561+0000 7f398b7fe640 1 -- 172.21.15.12:0/3444198015 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998022070 con 0x5565dee77f40 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.561+0000 7f398b7fe640 1 -- 172.21.15.12:0/3444198015 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3998003d40 con 0x5565dee77f40 2024-05-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.561+0000 7f398b7fe640 1 -- 172.21.15.12:0/3444198015 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399801d410 con 0x5565dee77f40 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.562+0000 7f39aa3e0740 1 -- 172.21.15.12:0/3444198015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 msgr2=0x5565dee78360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.562+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/3444198015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee78360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f39980137f0 tx=0x7f3998013cb0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.562+0000 7f39aa3e0740 1 -- 172.21.15.12:0/3444198015 shutdown_connections 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.562+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/3444198015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee7ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.562+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/3444198015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 0x5565dee7d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.562+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/3444198015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee78360 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.562+0000 7f39aa3e0740 1 -- 172.21.15.12:0/3444198015 >> 172.21.15.12:0/3444198015 conn(0x5565dee735b0 msgr2=0x5565dee739b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.562+0000 7f39aa3e0740 1 -- 172.21.15.12:0/3444198015 shutdown_connections 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.562+0000 7f39aa3e0740 1 -- 172.21.15.12:0/3444198015 wait complete. 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39aa3e0740 1 -- start start 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39aa3e0740 1 Processor -- start 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39aa3e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee82310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39aa3e0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 0x5565dee82850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39aa3e0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee89890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39aa3e0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5565dee809f0 con 0x5565dee7daa0 2024-05-09T09:26:10.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39aa3e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5565ded97c40 con 0x5565dee77f40 2024-05-09T09:26:10.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39aa3e0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5565dee80b70 con 0x5565dee78930 2024-05-09T09:26:10.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39aa3e0740 10 client.0 _ll_register_callbacks cb 0x5565deaefbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:10.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39a5b98640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee89890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39a5b98640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee89890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53562/0 (socket says 172.21.15.12:53562) 2024-05-09T09:26:10.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39a5b98640 1 -- 172.21.15.12:0/1680891954 learned_addr learned my addr 172.21.15.12:0/1680891954 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39a5397640 1 --2- 172.21.15.12:0/1680891954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee82310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.563+0000 7f39a4b96640 1 --2- 172.21.15.12:0/1680891954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 0x5565dee82850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f39a5b98640 1 -- 172.21.15.12:0/1680891954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 msgr2=0x5565dee82850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f39a5b98640 1 --2- 172.21.15.12:0/1680891954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 0x5565dee82850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f39a5b98640 1 -- 172.21.15.12:0/1680891954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 msgr2=0x5565dee82310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f39a5b98640 1 --2- 172.21.15.12:0/1680891954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee82310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f39a5b98640 1 -- 172.21.15.12:0/1680891954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565debba7d0 con 0x5565dee7daa0 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f39a5b98640 1 --2- 172.21.15.12:0/1680891954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee89890 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3998006010 tx=0x7f39980096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a56dca2700547eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f3988ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f3988ff9640 1 -- 172.21.15.12:0/1680891954 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998022040 con 0x5565dee7daa0 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f3988ff9640 1 -- 172.21.15.12:0/1680891954 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f3998004680 con 0x5565dee7daa0 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f39aa3e0740 1 -- 172.21.15.12:0/1680891954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3998003d40 con 0x5565dee7daa0 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f3988ff9640 1 -- 172.21.15.12:0/1680891954 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3998004030 con 0x5565dee7daa0 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f39aa3e0740 10 client.15759 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f39aa3e0740 1 -- 172.21.15.12:0/1680891954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565dee8cc70 con 0x5565dee7daa0 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.564+0000 7f3988ff9640 1 -- 172.21.15.12:0/1680891954 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3998003d40 con 0x5565dee7daa0 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.565+0000 7f3988ff9640 1 -- 172.21.15.12:0/1680891954 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3998028040 con 0x5565dee7daa0 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.565+0000 7f39aa3e0740 10 client.15759 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.565+0000 7f39aa3e0740 10 client.15759 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.565+0000 7f39aa3e0740 20 client.15759 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.565+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/1680891954 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5565dee8e0d0 0x5565dee904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.565+0000 7f39aa3e0740 4 client.15759 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.565+0000 7f39aa3e0740 1 -- 172.21.15.12:0/1680891954 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565dedea370 con 0x5565dee8e0d0 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.565+0000 7f39a5397640 1 --2- 172.21.15.12:0/1680891954 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5565dee8e0d0 0x5565dee904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.565+0000 7f39a5397640 1 --2- 172.21.15.12:0/1680891954 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5565dee8e0d0 0x5565dee904c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8ba18f55090e4857 server_cookie=d397e953e652549 in_seq=0 out_seq=0 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.565+0000 7f3988ff9640 10 client.15759 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f3988ff9640 1 -- 172.21.15.12:0/1680891954 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565dedea370 con 0x5565dee8e0d0 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f3988ff9640 10 client.15759 handle_command_reply: tid=0 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f39aa3e0740 1 client.15759 shutdown 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f39aa3e0740 1 -- 172.21.15.12:0/1680891954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 msgr2=0x5565dee89890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/1680891954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee89890 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3998006010 tx=0x7f39980096a0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f39aa3e0740 1 -- 172.21.15.12:0/1680891954 shutdown_connections 2024-05-09T09:26:10.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/1680891954 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5565dee8e0d0 0x5565dee904c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/1680891954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee89890 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/1680891954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 0x5565dee82850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/1680891954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee82310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f39aa3e0740 1 -- 172.21.15.12:0/1680891954 >> 172.21.15.12:0/1680891954 conn(0x5565dee735b0 msgr2=0x5565dee727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f3988ff9640 0 client.15759 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:26:10.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f39aa3e0740 1 -- 172.21.15.12:0/1680891954 shutdown_connections 2024-05-09T09:26:10.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f39aa3e0740 1 -- 172.21.15.12:0/1680891954 wait complete. 2024-05-09T09:26:10.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.566+0000 7f39aa3e0740 20 client.15759 trim_cache size 0 max 16384 2024-05-09T09:26:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.580+0000 7f39aa3e0740 1 Processor -- start 2024-05-09T09:26:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.580+0000 7f39aa3e0740 1 -- start start 2024-05-09T09:26:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.580+0000 7f39aa3e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee79190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.580+0000 7f39aa3e0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 0x5565dee89850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.580+0000 7f39aa3e0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee79d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.580+0000 7f39aa3e0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01ac810 con 0x5565dee7daa0 2024-05-09T09:26:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39aa3e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01ae350 con 0x5565dee77f40 2024-05-09T09:26:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39aa3e0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01acee0 con 0x5565dee78930 2024-05-09T09:26:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39a5b98640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee79d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39a5b98640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee79d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53566/0 (socket says 172.21.15.12:53566) 2024-05-09T09:26:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39a5b98640 1 -- 172.21.15.12:0/2503000490 learned_addr learned my addr 172.21.15.12:0/2503000490 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39a4b96640 1 --2- 172.21.15.12:0/2503000490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 0x5565dee89850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39a5397640 1 --2- 172.21.15.12:0/2503000490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee79190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39a5b98640 1 -- 172.21.15.12:0/2503000490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 msgr2=0x5565dee89850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39a5b98640 1 --2- 172.21.15.12:0/2503000490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 0x5565dee89850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39a5b98640 1 -- 172.21.15.12:0/2503000490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 msgr2=0x5565dee79190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39a5b98640 1 --2- 172.21.15.12:0/2503000490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee79190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39a5b98640 1 -- 172.21.15.12:0/2503000490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565dee64d20 con 0x5565dee7daa0 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39a5397640 1 --2- 172.21.15.12:0/2503000490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee79190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f39a5b98640 1 --2- 172.21.15.12:0/2503000490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee79d50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f39980137f0 tx=0x7f3998005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9258a9232c302ec6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f398affd640 1 -- 172.21.15.12:0/2503000490 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998022040 con 0x5565dee7daa0 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f398affd640 1 -- 172.21.15.12:0/2503000490 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3998011110 con 0x5565dee7daa0 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.581+0000 7f398affd640 1 -- 172.21.15.12:0/2503000490 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998013240 con 0x5565dee7daa0 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.582+0000 7f39aa3e0740 1 -- 172.21.15.12:0/2503000490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 msgr2=0x5565dee79d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.582+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/2503000490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee79d50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f39980137f0 tx=0x7f3998005be0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.582+0000 7f39aa3e0740 1 -- 172.21.15.12:0/2503000490 shutdown_connections 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.582+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/2503000490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee7daa0 0x5565dee79d50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.582+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/2503000490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee78930 0x5565dee89850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.582+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/2503000490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee79190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.582+0000 7f39aa3e0740 1 -- 172.21.15.12:0/2503000490 >> 172.21.15.12:0/2503000490 conn(0x5565dee735b0 msgr2=0x5565dee82310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.582+0000 7f39aa3e0740 1 -- 172.21.15.12:0/2503000490 shutdown_connections 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.582+0000 7f39aa3e0740 1 -- 172.21.15.12:0/2503000490 wait complete. 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.582+0000 7f39aa3e0740 1 -- start start 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39aa3e0740 1 Processor -- start 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39aa3e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee86750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39aa3e0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee78930 0x5565dee86c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39aa3e0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee7daa0 0x5565deef25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39aa3e0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01adc80 con 0x5565dee78930 2024-05-09T09:26:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39aa3e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01ad5b0 con 0x5565dee77f40 2024-05-09T09:26:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39aa3e0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01aa7f0 con 0x5565dee7daa0 2024-05-09T09:26:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39aa3e0740 10 client.0 _ll_register_callbacks cb 0x7f3998020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39a4b96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee78930 0x5565dee86c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39a4b96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee78930 0x5565dee86c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53576/0 (socket says 172.21.15.12:53576) 2024-05-09T09:26:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39a4b96640 1 -- 172.21.15.12:0/4232547504 learned_addr learned my addr 172.21.15.12:0/4232547504 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39a5397640 1 --2- 172.21.15.12:0/4232547504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee86750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39a5b98640 1 --2- 172.21.15.12:0/4232547504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee7daa0 0x5565deef25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39a4b96640 1 -- 172.21.15.12:0/4232547504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee7daa0 msgr2=0x5565deef25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39a4b96640 1 --2- 172.21.15.12:0/4232547504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee7daa0 0x5565deef25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39a4b96640 1 -- 172.21.15.12:0/4232547504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 msgr2=0x5565dee86750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39a4b96640 1 --2- 172.21.15.12:0/4232547504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee86750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.583+0000 7f39a4b96640 1 -- 172.21.15.12:0/4232547504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565deef2b10 con 0x5565dee78930 2024-05-09T09:26:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.584+0000 7f39a4b96640 1 --2- 172.21.15.12:0/4232547504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee78930 0x5565dee86c90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3990009900 tx=0x7f3990005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9fb8dec1c64ef2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.584+0000 7f3988ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.584+0000 7f3988ff9640 1 -- 172.21.15.12:0/4232547504 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990035040 con 0x5565dee78930 2024-05-09T09:26:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.584+0000 7f39aa3e0740 1 -- 172.21.15.12:0/4232547504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3998011110 con 0x5565dee78930 2024-05-09T09:26:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.584+0000 7f3988ff9640 1 -- 172.21.15.12:0/4232547504 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f39900054f0 con 0x5565dee78930 2024-05-09T09:26:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.584+0000 7f3988ff9640 1 -- 172.21.15.12:0/4232547504 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3990004750 con 0x5565dee78930 2024-05-09T09:26:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.584+0000 7f39aa3e0740 10 client.15765 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.584+0000 7f39aa3e0740 1 -- 172.21.15.12:0/4232547504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565deef3df0 con 0x5565dee78930 2024-05-09T09:26:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.584+0000 7f3988ff9640 1 -- 172.21.15.12:0/4232547504 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3998011110 con 0x5565dee78930 2024-05-09T09:26:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.584+0000 7f3988ff9640 1 -- 172.21.15.12:0/4232547504 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3990031b70 con 0x5565dee78930 2024-05-09T09:26:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 10 client.15765 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 10 client.15765 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:26:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 20 client.15765 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/4232547504 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5565deef47d0 0x5565deef6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 4 client.15765 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 1 -- 172.21.15.12:0/4232547504 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565dede46c0 con 0x5565deef47d0 2024-05-09T09:26:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39a5397640 1 --2- 172.21.15.12:0/4232547504 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5565deef47d0 0x5565deef6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f3988ff9640 10 client.15765 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:26:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39a5397640 1 --2- 172.21.15.12:0/4232547504 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5565deef47d0 0x5565deef6bf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4c4b3c8d8dbd3e42 server_cookie=ada557b2cdaeb20d in_seq=0 out_seq=0 2024-05-09T09:26:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f3988ff9640 1 -- 172.21.15.12:0/4232547504 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565dede46c0 con 0x5565deef47d0 2024-05-09T09:26:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f3988ff9640 10 client.15765 handle_command_reply: tid=0 2024-05-09T09:26:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 1 client.15765 shutdown 2024-05-09T09:26:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 1 -- 172.21.15.12:0/4232547504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee78930 msgr2=0x5565dee86c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/4232547504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee78930 0x5565dee86c90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3990009900 tx=0x7f3990005270 comp rx=0 tx=0).stop 2024-05-09T09:26:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 1 -- 172.21.15.12:0/4232547504 shutdown_connections 2024-05-09T09:26:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/4232547504 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5565deef47d0 0x5565deef6bf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/4232547504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565dee7daa0 0x5565deef25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f3988ff9640 0 client.15765 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:26:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/4232547504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565dee78930 0x5565dee86c90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 1 --2- 172.21.15.12:0/4232547504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565dee77f40 0x5565dee86750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.585+0000 7f39aa3e0740 1 -- 172.21.15.12:0/4232547504 >> 172.21.15.12:0/4232547504 conn(0x5565dee735b0 msgr2=0x5565dee79d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39aa3e0740 1 -- 172.21.15.12:0/4232547504 shutdown_connections 2024-05-09T09:26:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39aa3e0740 1 -- 172.21.15.12:0/4232547504 wait complete. 2024-05-09T09:26:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39aa3e0740 20 client.15765 trim_cache size 0 max 16384 2024-05-09T09:26:10.592 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:10.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39a75fb640 1 -- 172.21.15.12:0/4018229074 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f398c0a2b90 msgr2=0x7f398c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39a75fb640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f398c0a2b90 0x7f398c0a5050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3998004000 tx=0x7f399801c000 comp rx=0 tx=0).stop 2024-05-09T09:26:10.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39a75fb640 1 -- 172.21.15.12:0/4018229074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0100770 msgr2=0x7f39a0104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39a75fb640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0100770 0x7f39a0104700 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f399c00c970 tx=0x7f399c00ce30 comp rx=0 tx=0).stop 2024-05-09T09:26:10.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39a75fb640 1 -- 172.21.15.12:0/4018229074 shutdown_connections 2024-05-09T09:26:10.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39a75fb640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f398c0a2b90 0x7f398c0a5050 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39a75fb640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a0102320 0x7f39a0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39a75fb640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39a0101960 0x7f39a0102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39a75fb640 1 --2- 172.21.15.12:0/4018229074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0100770 0x7f39a0104700 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39a75fb640 1 -- 172.21.15.12:0/4018229074 >> 172.21.15.12:0/4018229074 conn(0x7f39a00fbce0 msgr2=0x7f39a00fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39a75fb640 1 -- 172.21.15.12:0/4018229074 shutdown_connections 2024-05-09T09:26:10.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.586+0000 7f39a75fb640 1 -- 172.21.15.12:0/4018229074 wait complete. 2024-05-09T09:26:10.597 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-09T09:26:10.619 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:10 smithi012 ceph-mon[30140]: pgmap v385: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.706+0000 7f86d9464640 1 -- 172.21.15.12:0/837120102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4101960 msgr2=0x7f86d4101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.706+0000 7f86d9464640 1 --2- 172.21.15.12:0/837120102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4101960 0x7f86d4101de0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009900 tx=0x7f86bc02a410 comp rx=0 tx=0).stop 2024-05-09T09:26:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.706+0000 7f86d9464640 1 -- 172.21.15.12:0/837120102 shutdown_connections 2024-05-09T09:26:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.706+0000 7f86d9464640 1 --2- 172.21.15.12:0/837120102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d4102320 0x7f86d4108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.706+0000 7f86d9464640 1 --2- 172.21.15.12:0/837120102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4101960 0x7f86d4101de0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.706+0000 7f86d9464640 1 --2- 172.21.15.12:0/837120102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4100770 0x7f86d4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.706+0000 7f86d9464640 1 -- 172.21.15.12:0/837120102 >> 172.21.15.12:0/837120102 conn(0x7f86d40fbce0 msgr2=0x7f86d40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.706+0000 7f86d9464640 1 -- 172.21.15.12:0/837120102 shutdown_connections 2024-05-09T09:26:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.706+0000 7f86d9464640 1 -- 172.21.15.12:0/837120102 wait complete. 2024-05-09T09:26:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d9464640 1 Processor -- start 2024-05-09T09:26:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d9464640 1 -- start start 2024-05-09T09:26:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d9464640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4100770 0x7f86d4199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d9464640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d4101960 0x7f86d419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d9464640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4102320 0x7f86d41a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d9464640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f86d410c3d0 con 0x7f86d4102320 2024-05-09T09:26:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d9464640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86d410c250 con 0x7f86d4100770 2024-05-09T09:26:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d9464640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86d410c550 con 0x7f86d4101960 2024-05-09T09:26:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4102320 0x7f86d41a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4102320 0x7f86d41a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53596/0 (socket says 172.21.15.12:53596) 2024-05-09T09:26:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d37fe640 1 -- 172.21.15.12:0/1309154295 learned_addr learned my addr 172.21.15.12:0/1309154295 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4100770 0x7f86d4199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86cbfff640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d4101960 0x7f86d419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d37fe640 1 -- 172.21.15.12:0/1309154295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d4101960 msgr2=0x7f86d419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d37fe640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d4101960 0x7f86d419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d37fe640 1 -- 172.21.15.12:0/1309154295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4100770 msgr2=0x7f86d4199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d37fe640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4100770 0x7f86d4199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d37fe640 1 -- 172.21.15.12:0/1309154295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d41a1970 con 0x7f86d4102320 2024-05-09T09:26:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.707+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4100770 0x7f86d4199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:10.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.708+0000 7f86cbfff640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d4101960 0x7f86d419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:10.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.708+0000 7f86d37fe640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4102320 0x7f86d41a1300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00eaa0 tx=0x7f86cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.708+0000 7f86d0ff9640 1 -- 172.21.15.12:0/1309154295 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc00cdb0 con 0x7f86d4102320 2024-05-09T09:26:10.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.708+0000 7f86d9464640 1 -- 172.21.15.12:0/1309154295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d41a1c60 con 0x7f86d4102320 2024-05-09T09:26:10.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.708+0000 7f86d0ff9640 1 -- 172.21.15.12:0/1309154295 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f86cc005020 con 0x7f86d4102320 2024-05-09T09:26:10.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.708+0000 7f86d0ff9640 1 -- 172.21.15.12:0/1309154295 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0116b0 con 0x7f86d4102320 2024-05-09T09:26:10.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.708+0000 7f86d9464640 1 -- 172.21.15.12:0/1309154295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d41a21b0 con 0x7f86d4102320 2024-05-09T09:26:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.711+0000 7f86dc171740 1 Processor -- start 2024-05-09T09:26:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.711+0000 7f86d0ff9640 1 -- 172.21.15.12:0/1309154295 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f86cc0051d0 con 0x7f86d4102320 2024-05-09T09:26:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.711+0000 7f86dc171740 1 -- start start 2024-05-09T09:26:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.711+0000 7f86dc171740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b8fff40 0x55ce0b900360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.711+0000 7f86dc171740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b900930 0x55ce0b905430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.711+0000 7f86dc171740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b907f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86dc171740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ce0b864c40 con 0x55ce0b905aa0 2024-05-09T09:26:10.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86dc171740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce0b80c580 con 0x55ce0b8fff40 2024-05-09T09:26:10.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86dc171740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ce0b860c00 con 0x55ce0b900930 2024-05-09T09:26:10.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b907f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b907f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53604/0 (socket says 172.21.15.12:53604) 2024-05-09T09:26:10.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86cbfff640 1 -- 172.21.15.12:0/2586510144 learned_addr learned my addr 172.21.15.12:0/2586510144 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/2586510144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b900930 0x55ce0b905430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86cbfff640 1 --2- 172.21.15.12:0/2586510144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b8fff40 0x55ce0b900360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86d0ff9640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86c00a2b90 0x7f86c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86d2ffd640 1 -- 172.21.15.12:0/2586510144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b8fff40 msgr2=0x55ce0b900360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/2586510144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b8fff40 0x55ce0b900360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86d2ffd640 1 -- 172.21.15.12:0/2586510144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 msgr2=0x55ce0b907f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/2586510144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b907f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86d2ffd640 1 -- 172.21.15.12:0/2586510144 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce0b5783f0 con 0x55ce0b900930 2024-05-09T09:26:10.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86d37fe640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86c00a2b90 0x7f86c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86d0ff9640 1 -- 172.21.15.12:0/1309154295 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f86cc0cf6f0 con 0x7f86d4102320 2024-05-09T09:26:10.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.712+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/2586510144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b900930 0x55ce0b905430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f86c4008ba0 tx=0x7f86c400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd79df40c2ae723 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.713+0000 7f86d37fe640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86c00a2b90 0x7f86c00a5050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f86cc086740 tx=0x7f86cc0cb040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.713+0000 7f86c97fa640 1 -- 172.21.15.12:0/2586510144 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4014070 con 0x55ce0b900930 2024-05-09T09:26:10.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.713+0000 7f86c97fa640 1 -- 172.21.15.12:0/2586510144 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f86c40054f0 con 0x55ce0b900930 2024-05-09T09:26:10.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.713+0000 7f86c97fa640 1 -- 172.21.15.12:0/2586510144 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4004b60 con 0x55ce0b900930 2024-05-09T09:26:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 -- 172.21.15.12:0/2586510144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b900930 msgr2=0x55ce0b905430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 --2- 172.21.15.12:0/2586510144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b900930 0x55ce0b905430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f86c4008ba0 tx=0x7f86c400e720 comp rx=0 tx=0).stop 2024-05-09T09:26:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 -- 172.21.15.12:0/2586510144 shutdown_connections 2024-05-09T09:26:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 --2- 172.21.15.12:0/2586510144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b907f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 --2- 172.21.15.12:0/2586510144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b900930 0x55ce0b905430 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 --2- 172.21.15.12:0/2586510144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b8fff40 0x55ce0b900360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 -- 172.21.15.12:0/2586510144 >> 172.21.15.12:0/2586510144 conn(0x55ce0b8fb5b0 msgr2=0x55ce0b8fb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 -- 172.21.15.12:0/2586510144 shutdown_connections 2024-05-09T09:26:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 -- 172.21.15.12:0/2586510144 wait complete. 2024-05-09T09:26:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 -- start start 2024-05-09T09:26:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 Processor -- start 2024-05-09T09:26:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b8fff40 0x55ce0b90a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b900930 0x55ce0b90a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b9118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ce0b908a50 con 0x55ce0b905aa0 2024-05-09T09:26:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce0b81fc40 con 0x55ce0b900930 2024-05-09T09:26:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ce0b908bd0 con 0x55ce0b8fff40 2024-05-09T09:26:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.714+0000 7f86dc171740 10 client.0 _ll_register_callbacks cb 0x55ce0b577be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b9118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b9118e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53618/0 (socket says 172.21.15.12:53618) 2024-05-09T09:26:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86cbfff640 1 -- 172.21.15.12:0/2591220657 learned_addr learned my addr 172.21.15.12:0/2591220657 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b900930 0x55ce0b90a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86cbfff640 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b8fff40 0x55ce0b90a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86cbfff640 1 -- 172.21.15.12:0/2591220657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b8fff40 msgr2=0x55ce0b90a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86cbfff640 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b8fff40 0x55ce0b90a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86cbfff640 1 -- 172.21.15.12:0/2591220657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b900930 msgr2=0x55ce0b90a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86cbfff640 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b900930 0x55ce0b90a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86cbfff640 1 -- 172.21.15.12:0/2591220657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce0b6427d0 con 0x55ce0b905aa0 2024-05-09T09:26:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b900930 0x55ce0b90a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86cbfff640 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b9118e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f86bc033000 tx=0x7f86bc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99c9ff8ed260845 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86ba7fc640 1 -- 172.21.15.12:0/2591220657 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc035070 con 0x55ce0b905aa0 2024-05-09T09:26:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86dc171740 1 -- 172.21.15.12:0/2591220657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86c40054f0 con 0x55ce0b905aa0 2024-05-09T09:26:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86ba7fc640 1 -- 172.21.15.12:0/2591220657 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f86bc004460 con 0x55ce0b905aa0 2024-05-09T09:26:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86ba7fc640 1 -- 172.21.15.12:0/2591220657 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f86bc0046c0 con 0x55ce0b905aa0 2024-05-09T09:26:10.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86dc171740 10 client.15777 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:10.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.715+0000 7f86dc171740 1 -- 172.21.15.12:0/2591220657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ce0b914d10 con 0x55ce0b905aa0 2024-05-09T09:26:10.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.716+0000 7f86ba7fc640 1 -- 172.21.15.12:0/2591220657 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f86c40054f0 con 0x55ce0b905aa0 2024-05-09T09:26:10.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.716+0000 7f86ba7fc640 1 -- 172.21.15.12:0/2591220657 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f86bc03a040 con 0x55ce0b905aa0 2024-05-09T09:26:10.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.716+0000 7f86dc171740 10 client.15777 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:10.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.716+0000 7f86dc171740 10 client.15777 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:26:10.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.716+0000 7f86dc171740 20 client.15777 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:10.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.716+0000 7f86dc171740 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ce0b9163a0 0x55ce0b918820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.716+0000 7f86dc171740 4 client.15777 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:10.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.716+0000 7f86dc171740 1 -- 172.21.15.12:0/2591220657 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce0b872370 con 0x55ce0b9163a0 2024-05-09T09:26:10.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.716+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ce0b9163a0 0x55ce0b918820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.717+0000 7f86ba7fc640 10 client.15777 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:26:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.717+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ce0b9163a0 0x55ce0b918820 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9348e5ebe0bc9cd7 server_cookie=3ebac493349510c8 in_seq=0 out_seq=0 2024-05-09T09:26:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.717+0000 7f86ba7fc640 1 -- 172.21.15.12:0/2591220657 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce0b872370 con 0x55ce0b9163a0 2024-05-09T09:26:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.717+0000 7f86ba7fc640 10 client.15777 handle_command_reply: tid=0 2024-05-09T09:26:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.717+0000 7f86dc171740 1 client.15777 shutdown 2024-05-09T09:26:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.718+0000 7f86dc171740 1 -- 172.21.15.12:0/2591220657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 msgr2=0x55ce0b9118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.718+0000 7f86dc171740 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b9118e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f86bc033000 tx=0x7f86bc02a410 comp rx=0 tx=0).stop 2024-05-09T09:26:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.718+0000 7f86dc171740 1 -- 172.21.15.12:0/2591220657 shutdown_connections 2024-05-09T09:26:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.718+0000 7f86dc171740 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ce0b9163a0 0x55ce0b918820 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.718+0000 7f86dc171740 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b9118e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.718+0000 7f86dc171740 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b900930 0x55ce0b90a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.718+0000 7f86ba7fc640 0 client.15777 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:26:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.718+0000 7f86dc171740 1 --2- 172.21.15.12:0/2591220657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b8fff40 0x55ce0b90a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.718+0000 7f86dc171740 1 -- 172.21.15.12:0/2591220657 >> 172.21.15.12:0/2591220657 conn(0x55ce0b8fb5b0 msgr2=0x55ce0b8fa7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.718+0000 7f86dc171740 1 -- 172.21.15.12:0/2591220657 shutdown_connections 2024-05-09T09:26:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.718+0000 7f86dc171740 1 -- 172.21.15.12:0/2591220657 wait complete. 2024-05-09T09:26:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.718+0000 7f86dc171740 20 client.15777 trim_cache size 0 max 16384 2024-05-09T09:26:10.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:10 smithi170 ceph-mon[33632]: pgmap v385: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.745+0000 7f86dc171740 1 Processor -- start 2024-05-09T09:26:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.745+0000 7f86dc171740 1 -- start start 2024-05-09T09:26:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.745+0000 7f86dc171740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b8fff40 0x55ce0b8fa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.745+0000 7f86dc171740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b900930 0x55ce0b8fab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.745+0000 7f86dc171740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b911870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.745+0000 7f86dc171740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41ac6f0 con 0x55ce0b905aa0 2024-05-09T09:26:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.745+0000 7f86dc171740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41ae230 con 0x55ce0b8fff40 2024-05-09T09:26:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.745+0000 7f86dc171740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41acdc0 con 0x55ce0b900930 2024-05-09T09:26:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b911870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b911870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53632/0 (socket says 172.21.15.12:53632) 2024-05-09T09:26:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86cbfff640 1 -- 172.21.15.12:0/2408775353 learned_addr learned my addr 172.21.15.12:0/2408775353 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/2408775353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b900930 0x55ce0b8fab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86cbfff640 1 --2- 172.21.15.12:0/2408775353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b8fff40 0x55ce0b8fa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86cbfff640 1 -- 172.21.15.12:0/2408775353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b900930 msgr2=0x55ce0b8fab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86cbfff640 1 --2- 172.21.15.12:0/2408775353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b900930 0x55ce0b8fab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86cbfff640 1 -- 172.21.15.12:0/2408775353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b8fff40 msgr2=0x55ce0b8fa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86cbfff640 1 --2- 172.21.15.12:0/2408775353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b8fff40 0x55ce0b8fa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86cbfff640 1 -- 172.21.15.12:0/2408775353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce0b8ecd20 con 0x55ce0b905aa0 2024-05-09T09:26:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/2408775353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b900930 0x55ce0b8fab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86cbfff640 1 --2- 172.21.15.12:0/2408775353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b911870 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f86bc003920 tx=0x7f86bc031610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b4595c59de905e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86c8ff9640 1 -- 172.21.15.12:0/2408775353 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc035070 con 0x55ce0b905aa0 2024-05-09T09:26:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86c8ff9640 1 -- 172.21.15.12:0/2408775353 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f86bc0035f0 con 0x55ce0b905aa0 2024-05-09T09:26:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.746+0000 7f86c8ff9640 1 -- 172.21.15.12:0/2408775353 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc004860 con 0x55ce0b905aa0 2024-05-09T09:26:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.747+0000 7f86dc171740 1 -- 172.21.15.12:0/2408775353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 msgr2=0x55ce0b911870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.747+0000 7f86dc171740 1 --2- 172.21.15.12:0/2408775353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b911870 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f86bc003920 tx=0x7f86bc031610 comp rx=0 tx=0).stop 2024-05-09T09:26:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.747+0000 7f86dc171740 1 -- 172.21.15.12:0/2408775353 shutdown_connections 2024-05-09T09:26:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.747+0000 7f86dc171740 1 --2- 172.21.15.12:0/2408775353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b905aa0 0x55ce0b911870 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.747+0000 7f86dc171740 1 --2- 172.21.15.12:0/2408775353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b900930 0x55ce0b8fab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.747+0000 7f86dc171740 1 --2- 172.21.15.12:0/2408775353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b8fff40 0x55ce0b8fa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.747+0000 7f86dc171740 1 -- 172.21.15.12:0/2408775353 >> 172.21.15.12:0/2408775353 conn(0x55ce0b8fb5b0 msgr2=0x55ce0b8f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.747+0000 7f86dc171740 1 -- 172.21.15.12:0/2408775353 shutdown_connections 2024-05-09T09:26:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.747+0000 7f86dc171740 1 -- 172.21.15.12:0/2408775353 wait complete. 2024-05-09T09:26:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.748+0000 7f86dc171740 1 -- start start 2024-05-09T09:26:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.748+0000 7f86dc171740 1 Processor -- start 2024-05-09T09:26:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.748+0000 7f86dc171740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b8fff40 0x55ce0b90ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.748+0000 7f86dc171740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b900930 0x55ce0b90f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.748+0000 7f86dc171740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b905aa0 0x55ce0b9103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.748+0000 7f86dc171740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41adb60 con 0x55ce0b8fff40 2024-05-09T09:26:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.748+0000 7f86dc171740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41ad490 con 0x55ce0b900930 2024-05-09T09:26:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.748+0000 7f86dc171740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41aa6d0 con 0x55ce0b905aa0 2024-05-09T09:26:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.748+0000 7f86dc171740 10 client.0 _ll_register_callbacks cb 0x7f86bc005520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.748+0000 7f86cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b8fff40 0x55ce0b90ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.748+0000 7f86cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b8fff40 0x55ce0b90ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53638/0 (socket says 172.21.15.12:53638) 2024-05-09T09:26:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.748+0000 7f86cbfff640 1 -- 172.21.15.12:0/794942001 learned_addr learned my addr 172.21.15.12:0/794942001 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.748+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b900930 0x55ce0b90f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86cbfff640 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b905aa0 0x55ce0b9103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86cbfff640 1 -- 172.21.15.12:0/794942001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b905aa0 msgr2=0x55ce0b9103b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86cbfff640 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b905aa0 0x55ce0b9103b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86cbfff640 1 -- 172.21.15.12:0/794942001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b900930 msgr2=0x55ce0b90f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86cbfff640 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b900930 0x55ce0b90f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86cbfff640 1 -- 172.21.15.12:0/794942001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce0b9109a0 con 0x55ce0b8fff40 2024-05-09T09:26:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b900930 0x55ce0b90f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86cbfff640 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b8fff40 0x55ce0b90ee60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f86bc005460 tx=0x7f86bc032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a411893c666ee7e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86ba7fc640 1 -- 172.21.15.12:0/794942001 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc035040 con 0x55ce0b8fff40 2024-05-09T09:26:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86ba7fc640 1 -- 172.21.15.12:0/794942001 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f86bc004860 con 0x55ce0b8fff40 2024-05-09T09:26:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86dc171740 1 -- 172.21.15.12:0/794942001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86bc0035f0 con 0x55ce0b8fff40 2024-05-09T09:26:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86ba7fc640 1 -- 172.21.15.12:0/794942001 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f86bc0056a0 con 0x55ce0b8fff40 2024-05-09T09:26:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86dc171740 10 client.15783 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.749+0000 7f86dc171740 1 -- 172.21.15.12:0/794942001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ce0b911ca0 con 0x55ce0b8fff40 2024-05-09T09:26:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.750+0000 7f86ba7fc640 1 -- 172.21.15.12:0/794942001 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f86bc0035f0 con 0x55ce0b8fff40 2024-05-09T09:26:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.750+0000 7f86ba7fc640 1 -- 172.21.15.12:0/794942001 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f86bc03db80 con 0x55ce0b8fff40 2024-05-09T09:26:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.750+0000 7f86dc171740 10 client.15783 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.750+0000 7f86dc171740 10 client.15783 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:26:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.750+0000 7f86dc171740 20 client.15783 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.750+0000 7f86dc171740 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ce0b97d110 0x55ce0b97f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.750+0000 7f86dc171740 4 client.15783 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.750+0000 7f86dc171740 1 -- 172.21.15.12:0/794942001 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce0b86c6c0 con 0x55ce0b97d110 2024-05-09T09:26:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.750+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ce0b97d110 0x55ce0b97f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86ba7fc640 10 client.15783 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:26:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86d2ffd640 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ce0b97d110 0x55ce0b97f530 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5a9e6b8ec2d6a7e3 server_cookie=df1ff09dd7626d9 in_seq=0 out_seq=0 2024-05-09T09:26:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86ba7fc640 1 -- 172.21.15.12:0/794942001 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce0b86c6c0 con 0x55ce0b97d110 2024-05-09T09:26:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86ba7fc640 10 client.15783 handle_command_reply: tid=0 2024-05-09T09:26:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86dc171740 1 client.15783 shutdown 2024-05-09T09:26:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86dc171740 1 -- 172.21.15.12:0/794942001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b8fff40 msgr2=0x55ce0b90ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86dc171740 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b8fff40 0x55ce0b90ee60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f86bc005460 tx=0x7f86bc032d00 comp rx=0 tx=0).stop 2024-05-09T09:26:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86dc171740 1 -- 172.21.15.12:0/794942001 shutdown_connections 2024-05-09T09:26:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86dc171740 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ce0b97d110 0x55ce0b97f530 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86dc171740 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce0b905aa0 0x55ce0b9103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86dc171740 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce0b900930 0x55ce0b90f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86dc171740 1 --2- 172.21.15.12:0/794942001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce0b8fff40 0x55ce0b90ee60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86dc171740 1 -- 172.21.15.12:0/794942001 >> 172.21.15.12:0/794942001 conn(0x55ce0b8fb5b0 msgr2=0x55ce0b8fa7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86ba7fc640 0 client.15783 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:26:10.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86dc171740 1 -- 172.21.15.12:0/794942001 shutdown_connections 2024-05-09T09:26:10.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86dc171740 1 -- 172.21.15.12:0/794942001 wait complete. 2024-05-09T09:26:10.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.751+0000 7f86dc171740 20 client.15783 trim_cache size 0 max 16384 2024-05-09T09:26:10.759 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:10.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.752+0000 7f86d9464640 1 -- 172.21.15.12:0/1309154295 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86c00a2b90 msgr2=0x7f86c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.752+0000 7f86d9464640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86c00a2b90 0x7f86c00a5050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f86cc086740 tx=0x7f86cc0cb040 comp rx=0 tx=0).stop 2024-05-09T09:26:10.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.752+0000 7f86d9464640 1 -- 172.21.15.12:0/1309154295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4102320 msgr2=0x7f86d41a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.752+0000 7f86d9464640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4102320 0x7f86d41a1300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00eaa0 tx=0x7f86cc00ef60 comp rx=0 tx=0).stop 2024-05-09T09:26:10.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.752+0000 7f86d9464640 1 -- 172.21.15.12:0/1309154295 shutdown_connections 2024-05-09T09:26:10.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.752+0000 7f86d9464640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f86c00a2b90 0x7f86c00a5050 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.752+0000 7f86d9464640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4102320 0x7f86d41a1300 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.752+0000 7f86d9464640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d4101960 0x7f86d419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.752+0000 7f86d9464640 1 --2- 172.21.15.12:0/1309154295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4100770 0x7f86d4199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.752+0000 7f86d9464640 1 -- 172.21.15.12:0/1309154295 >> 172.21.15.12:0/1309154295 conn(0x7f86d40fbce0 msgr2=0x7f86d40fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.752+0000 7f86d9464640 1 -- 172.21.15.12:0/1309154295 shutdown_connections 2024-05-09T09:26:10.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.752+0000 7f86d9464640 1 -- 172.21.15.12:0/1309154295 wait complete. 2024-05-09T09:26:10.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:10 smithi016 ceph-mon[33335]: pgmap v385: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:10.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:26:10.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.916+0000 7f4e20072640 1 -- 172.21.15.12:0/1419898609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e181023b0 msgr2=0x7f4e181047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.916+0000 7f4e20072640 1 --2- 172.21.15.12:0/1419898609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e181023b0 0x7f4e181047a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009900 tx=0x7f4e1402a410 comp rx=0 tx=0).stop 2024-05-09T09:26:10.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.916+0000 7f4e20072640 1 -- 172.21.15.12:0/1419898609 shutdown_connections 2024-05-09T09:26:10.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.916+0000 7f4e20072640 1 --2- 172.21.15.12:0/1419898609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e18107610 0x7f4e18109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.916+0000 7f4e20072640 1 --2- 172.21.15.12:0/1419898609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e18104ce0 0x7f4e181070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.916+0000 7f4e20072640 1 --2- 172.21.15.12:0/1419898609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e181023b0 0x7f4e181047a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.916+0000 7f4e20072640 1 -- 172.21.15.12:0/1419898609 >> 172.21.15.12:0/1419898609 conn(0x7f4e180fbce0 msgr2=0x7f4e180fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.916+0000 7f4e20072640 1 -- 172.21.15.12:0/1419898609 shutdown_connections 2024-05-09T09:26:10.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.916+0000 7f4e20072640 1 -- 172.21.15.12:0/1419898609 wait complete. 2024-05-09T09:26:10.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e20072640 1 Processor -- start 2024-05-09T09:26:10.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e20072640 1 -- start start 2024-05-09T09:26:10.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e20072640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e181023b0 0x7f4e18197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e20072640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e18104ce0 0x7f4e18198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e20072640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e18107610 0x7f4e1819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e20072640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1810d500 con 0x7f4e18104ce0 2024-05-09T09:26:10.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e20072640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1810d380 con 0x7f4e181023b0 2024-05-09T09:26:10.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e20072640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1810d680 con 0x7f4e18107610 2024-05-09T09:26:10.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e1d60d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e18104ce0 0x7f4e18198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e1d60d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e18104ce0 0x7f4e18198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53654/0 (socket says 172.21.15.12:53654) 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e1d60d640 1 -- 172.21.15.12:0/269956768 learned_addr learned my addr 172.21.15.12:0/269956768 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e1de0e640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e181023b0 0x7f4e18197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e1d60d640 1 -- 172.21.15.12:0/269956768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e18107610 msgr2=0x7f4e1819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e1d60d640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e18107610 0x7f4e1819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e1d60d640 1 -- 172.21.15.12:0/269956768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e181023b0 msgr2=0x7f4e18197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e1d60d640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e181023b0 0x7f4e18197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e1d60d640 1 -- 172.21.15.12:0/269956768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1819f860 con 0x7f4e18104ce0 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.917+0000 7f4e1de0e640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e181023b0 0x7f4e18197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.918+0000 7f4e1d60d640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e18104ce0 0x7f4e18198190 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800d740 tx=0x7f4e0800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.918+0000 7f4e0effd640 1 -- 172.21.15.12:0/269956768 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08017070 con 0x7f4e18104ce0 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.918+0000 7f4e20072640 1 -- 172.21.15.12:0/269956768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1819fb50 con 0x7f4e18104ce0 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.918+0000 7f4e0effd640 1 -- 172.21.15.12:0/269956768 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4e0800de50 con 0x7f4e18104ce0 2024-05-09T09:26:10.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.918+0000 7f4e0effd640 1 -- 172.21.15.12:0/269956768 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08004e00 con 0x7f4e18104ce0 2024-05-09T09:26:10.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.918+0000 7f4e20072640 1 -- 172.21.15.12:0/269956768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e181a0050 con 0x7f4e18104ce0 2024-05-09T09:26:10.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.920+0000 7f4e0effd640 1 -- 172.21.15.12:0/269956768 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4e08004fa0 con 0x7f4e18104ce0 2024-05-09T09:26:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.920+0000 7f4e22d7f740 1 Processor -- start 2024-05-09T09:26:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e22d7f740 1 -- start start 2024-05-09T09:26:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e22d7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4350 0x5565db6c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e22d7f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4d40 0x5565db6c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e22d7f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db6cc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e22d7f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5565db624c40 con 0x5565db6c9eb0 2024-05-09T09:26:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e22d7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5565db5cc580 con 0x5565db6c4350 2024-05-09T09:26:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e22d7f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5565db620c00 con 0x5565db6c4d40 2024-05-09T09:26:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e1d60d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db6cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e1d60d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db6cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53670/0 (socket says 172.21.15.12:53670) 2024-05-09T09:26:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e1de0e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4d40 0x5565db6c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e1d60d640 1 -- 172.21.15.12:0/1447906021 learned_addr learned my addr 172.21.15.12:0/1447906021 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e1e60f640 1 --2- 172.21.15.12:0/1447906021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4350 0x5565db6c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e0effd640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4e040a2b90 0x7f4e040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e1e60f640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4e040a2b90 0x7f4e040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e1d60d640 1 -- 172.21.15.12:0/1447906021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4d40 msgr2=0x5565db6c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e1d60d640 1 --2- 172.21.15.12:0/1447906021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4d40 0x5565db6c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e1d60d640 1 -- 172.21.15.12:0/1447906021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4350 msgr2=0x5565db6c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.921+0000 7f4e1d60d640 1 --2- 172.21.15.12:0/1447906021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4350 0x5565db6c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.922+0000 7f4e0effd640 1 -- 172.21.15.12:0/269956768 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f4e080cef10 con 0x7f4e18104ce0 2024-05-09T09:26:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.922+0000 7f4e1d60d640 1 -- 172.21.15.12:0/1447906021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565db3383f0 con 0x5565db6c9eb0 2024-05-09T09:26:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.922+0000 7f4e1de0e640 1 --2- 172.21.15.12:0/1447906021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4d40 0x5565db6c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.922+0000 7f4e1e60f640 1 --2- 172.21.15.12:0/1447906021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4350 0x5565db6c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.922+0000 7f4e1d60d640 1 --2- 172.21.15.12:0/1447906021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db6cc340 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4e08097770 tx=0x7f4e08004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7f683da32c4d45 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.922+0000 7f4e03fff640 1 -- 172.21.15.12:0/1447906021 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08020020 con 0x5565db6c9eb0 2024-05-09T09:26:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.922+0000 7f4e03fff640 1 -- 172.21.15.12:0/1447906021 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4e08009d50 con 0x5565db6c9eb0 2024-05-09T09:26:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.922+0000 7f4e03fff640 1 -- 172.21.15.12:0/1447906021 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0809f750 con 0x5565db6c9eb0 2024-05-09T09:26:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.923+0000 7f4e22d7f740 1 -- 172.21.15.12:0/1447906021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 msgr2=0x5565db6cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.923+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/1447906021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db6cc340 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4e08097770 tx=0x7f4e08004840 comp rx=0 tx=0).stop 2024-05-09T09:26:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.923+0000 7f4e1e60f640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4e040a2b90 0x7f4e040a5050 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5565db6c55a0 tx=0x7f4e10004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.923+0000 7f4e22d7f740 1 -- 172.21.15.12:0/1447906021 shutdown_connections 2024-05-09T09:26:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.923+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/1447906021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db6cc340 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.923+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/1447906021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4d40 0x5565db6c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.923+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/1447906021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4350 0x5565db6c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.923+0000 7f4e22d7f740 1 -- 172.21.15.12:0/1447906021 >> 172.21.15.12:0/1447906021 conn(0x5565db6bb5b0 msgr2=0x5565db6bb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.923+0000 7f4e22d7f740 1 -- 172.21.15.12:0/1447906021 shutdown_connections 2024-05-09T09:26:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.923+0000 7f4e22d7f740 1 -- 172.21.15.12:0/1447906021 wait complete. 2024-05-09T09:26:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.923+0000 7f4e22d7f740 1 -- start start 2024-05-09T09:26:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.923+0000 7f4e22d7f740 1 Processor -- start 2024-05-09T09:26:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e22d7f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4350 0x5565db6ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e22d7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4d40 0x5565db6cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e22d7f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db6d5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e22d7f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5565db6ccda0 con 0x5565db6c9eb0 2024-05-09T09:26:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e22d7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5565db5dfc40 con 0x5565db6c4d40 2024-05-09T09:26:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e22d7f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5565db6ccf20 con 0x5565db6c4350 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e22d7f740 10 client.0 _ll_register_callbacks cb 0x5565db337be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e1e60f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db6d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e1e60f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db6d5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53682/0 (socket says 172.21.15.12:53682) 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e1e60f640 1 -- 172.21.15.12:0/2660769997 learned_addr learned my addr 172.21.15.12:0/2660769997 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e1d60d640 1 --2- 172.21.15.12:0/2660769997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4d40 0x5565db6cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e1de0e640 1 --2- 172.21.15.12:0/2660769997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4350 0x5565db6ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e1e60f640 1 -- 172.21.15.12:0/2660769997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4350 msgr2=0x5565db6ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e1e60f640 1 --2- 172.21.15.12:0/2660769997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4350 0x5565db6ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e1e60f640 1 -- 172.21.15.12:0/2660769997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4d40 msgr2=0x5565db6cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e1e60f640 1 --2- 172.21.15.12:0/2660769997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4d40 0x5565db6cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.924+0000 7f4e1e60f640 1 -- 172.21.15.12:0/2660769997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565db4027d0 con 0x5565db6c9eb0 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.925+0000 7f4e1e60f640 1 --2- 172.21.15.12:0/2660769997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db6d5bc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4e10012c70 tx=0x7f4e10009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7faeabf6ca7ee59d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.925+0000 7f4e017fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.925+0000 7f4e017fa640 1 -- 172.21.15.12:0/2660769997 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10012ca0 con 0x5565db6c9eb0 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.925+0000 7f4e017fa640 1 -- 172.21.15.12:0/2660769997 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f4e1001f420 con 0x5565db6c9eb0 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.925+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2660769997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5565db632370 con 0x5565db6c9eb0 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.925+0000 7f4e017fa640 1 -- 172.21.15.12:0/2660769997 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4e1002d050 con 0x5565db6c9eb0 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.925+0000 7f4e22d7f740 10 client.15795 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.925+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2660769997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565db6d9000 con 0x5565db6c9eb0 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.925+0000 7f4e017fa640 1 -- 172.21.15.12:0/2660769997 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x5565db632370 con 0x5565db6c9eb0 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.925+0000 7f4e017fa640 1 -- 172.21.15.12:0/2660769997 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4e1000f050 con 0x5565db6c9eb0 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.925+0000 7f4e22d7f740 10 client.15795 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.925+0000 7f4e22d7f740 10 client.15795 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.925+0000 7f4e22d7f740 20 client.15795 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.926+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/2660769997 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5565db6da770 0x5565db6dcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.926+0000 7f4e22d7f740 4 client.15795 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.926+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2660769997 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565db62c6c0 con 0x5565db6da770 2024-05-09T09:26:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.926+0000 7f4e1de0e640 1 --2- 172.21.15.12:0/2660769997 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5565db6da770 0x5565db6dcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.926+0000 7f4e1de0e640 1 --2- 172.21.15.12:0/2660769997 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5565db6da770 0x5565db6dcbd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1aad52a067983b2 server_cookie=f55834210e2fc256 in_seq=0 out_seq=0 2024-05-09T09:26:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.926+0000 7f4e017fa640 10 client.15795 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:26:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e017fa640 1 -- 172.21.15.12:0/2660769997 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565db62c6c0 con 0x5565db6da770 2024-05-09T09:26:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e017fa640 10 client.15795 handle_command_reply: tid=0 2024-05-09T09:26:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e22d7f740 1 client.15795 shutdown 2024-05-09T09:26:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2660769997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 msgr2=0x5565db6d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/2660769997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db6d5bc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4e10012c70 tx=0x7f4e10009440 comp rx=0 tx=0).stop 2024-05-09T09:26:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2660769997 shutdown_connections 2024-05-09T09:26:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/2660769997 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5565db6da770 0x5565db6dcbd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/2660769997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db6d5bc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/2660769997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4d40 0x5565db6cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e017fa640 0 client.15795 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:26:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/2660769997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4350 0x5565db6ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2660769997 >> 172.21.15.12:0/2660769997 conn(0x5565db6bb5b0 msgr2=0x5565db6ba7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2660769997 shutdown_connections 2024-05-09T09:26:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2660769997 wait complete. 2024-05-09T09:26:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.927+0000 7f4e22d7f740 20 client.15795 trim_cache size 0 max 16384 2024-05-09T09:26:10.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e22d7f740 1 Processor -- start 2024-05-09T09:26:10.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e22d7f740 1 -- start start 2024-05-09T09:26:10.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e22d7f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c4350 0x5565db6c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e22d7f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4d40 0x5565db6c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e22d7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c9eb0 0x5565db6d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e22d7f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4e181a8520 con 0x5565db6c4350 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e22d7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e181aa060 con 0x5565db6c9eb0 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e22d7f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4e181a8bf0 con 0x5565db6c4d40 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e1de0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c4350 0x5565db6c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e1de0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c4350 0x5565db6c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53696/0 (socket says 172.21.15.12:53696) 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e1de0e640 1 -- 172.21.15.12:0/1247312929 learned_addr learned my addr 172.21.15.12:0/1247312929 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e1d60d640 1 --2- 172.21.15.12:0/1247312929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4d40 0x5565db6c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e1e60f640 1 --2- 172.21.15.12:0/1247312929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c9eb0 0x5565db6d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e1de0e640 1 -- 172.21.15.12:0/1247312929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4d40 msgr2=0x5565db6c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e1de0e640 1 --2- 172.21.15.12:0/1247312929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4d40 0x5565db6c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e1de0e640 1 -- 172.21.15.12:0/1247312929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c9eb0 msgr2=0x5565db6d5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e1de0e640 1 --2- 172.21.15.12:0/1247312929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c9eb0 0x5565db6d5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e1de0e640 1 -- 172.21.15.12:0/1247312929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565db6c5120 con 0x5565db6c4350 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.941+0000 7f4e1e60f640 1 --2- 172.21.15.12:0/1247312929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c9eb0 0x5565db6d5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.942+0000 7f4e1de0e640 1 --2- 172.21.15.12:0/1247312929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c4350 0x5565db6c6170 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4e18199120 tx=0x7f4e140328f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58032bb8904b6db5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.942+0000 7f4e037fe640 1 -- 172.21.15.12:0/1247312929 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1404c070 con 0x5565db6c4350 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.942+0000 7f4e037fe640 1 -- 172.21.15.12:0/1247312929 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4e14032960 con 0x5565db6c4350 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.942+0000 7f4e037fe640 1 -- 172.21.15.12:0/1247312929 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14035040 con 0x5565db6c4350 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.942+0000 7f4e22d7f740 1 -- 172.21.15.12:0/1247312929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c4350 msgr2=0x5565db6c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.942+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/1247312929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c4350 0x5565db6c6170 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4e18199120 tx=0x7f4e140328f0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.942+0000 7f4e22d7f740 1 -- 172.21.15.12:0/1247312929 shutdown_connections 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.942+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/1247312929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c9eb0 0x5565db6d5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.942+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/1247312929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4d40 0x5565db6c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.942+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/1247312929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c4350 0x5565db6c6170 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.942+0000 7f4e22d7f740 1 -- 172.21.15.12:0/1247312929 >> 172.21.15.12:0/1247312929 conn(0x5565db6bb5b0 msgr2=0x5565db6ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e22d7f740 1 -- 172.21.15.12:0/1247312929 shutdown_connections 2024-05-09T09:26:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e22d7f740 1 -- 172.21.15.12:0/1247312929 wait complete. 2024-05-09T09:26:10.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e22d7f740 1 -- start start 2024-05-09T09:26:10.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e22d7f740 1 Processor -- start 2024-05-09T09:26:10.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e22d7f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4350 0x5565db6d2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e22d7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4d40 0x5565db6d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e22d7f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db73ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e22d7f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4e181a9990 con 0x5565db6c9eb0 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e22d7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e181a92c0 con 0x5565db6c4d40 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e22d7f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4e181a6500 con 0x5565db6c4350 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e22d7f740 10 client.0 _ll_register_callbacks cb 0x7f4e10009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e1e60f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db73ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e1e60f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db73ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53698/0 (socket says 172.21.15.12:53698) 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.943+0000 7f4e1e60f640 1 -- 172.21.15.12:0/2183134230 learned_addr learned my addr 172.21.15.12:0/2183134230 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e1de0e640 1 --2- 172.21.15.12:0/2183134230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4350 0x5565db6d2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e1d60d640 1 --2- 172.21.15.12:0/2183134230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4d40 0x5565db6d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e1e60f640 1 -- 172.21.15.12:0/2183134230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4350 msgr2=0x5565db6d2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e1e60f640 1 --2- 172.21.15.12:0/2183134230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4350 0x5565db6d2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e1e60f640 1 -- 172.21.15.12:0/2183134230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4d40 msgr2=0x5565db6d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:10.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e1e60f640 1 --2- 172.21.15.12:0/2183134230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4d40 0x5565db6d3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e1e60f640 1 -- 172.21.15.12:0/2183134230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565db73f2f0 con 0x5565db6c9eb0 2024-05-09T09:26:10.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e1e60f640 1 --2- 172.21.15.12:0/2183134230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db73ed00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4e10012040 tx=0x7f4e10005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb6ddf08071fdf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:10.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e017fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:10.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e017fa640 1 -- 172.21.15.12:0/2183134230 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1001eba0 con 0x5565db6c9eb0 2024-05-09T09:26:10.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e017fa640 1 -- 172.21.15.12:0/2183134230 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f4e10025030 con 0x5565db6c9eb0 2024-05-09T09:26:10.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2183134230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e14032960 con 0x5565db6c9eb0 2024-05-09T09:26:10.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e017fa640 1 -- 172.21.15.12:0/2183134230 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4e10042050 con 0x5565db6c9eb0 2024-05-09T09:26:10.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e22d7f740 10 client.15801 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:10.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.944+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2183134230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565db740590 con 0x5565db6c9eb0 2024-05-09T09:26:10.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.945+0000 7f4e017fa640 1 -- 172.21.15.12:0/2183134230 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4e14032960 con 0x5565db6c9eb0 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.945+0000 7f4e017fa640 1 -- 172.21.15.12:0/2183134230 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4e10030070 con 0x5565db6c9eb0 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.945+0000 7f4e22d7f740 10 client.15801 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.945+0000 7f4e22d7f740 10 client.15801 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.945+0000 7f4e22d7f740 20 client.15801 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.945+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/2183134230 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5565db740fe0 0x5565db7433d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.945+0000 7f4e22d7f740 4 client.15801 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.945+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2183134230 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565db5a9820 con 0x5565db740fe0 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.945+0000 7f4e1de0e640 1 --2- 172.21.15.12:0/2183134230 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5565db740fe0 0x5565db7433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.945+0000 7f4e1de0e640 1 --2- 172.21.15.12:0/2183134230 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5565db740fe0 0x5565db7433d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70e38b41920a8179 server_cookie=6bb5c926a87c8f5b in_seq=0 out_seq=0 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.945+0000 7f4e017fa640 10 client.15801 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e017fa640 1 -- 172.21.15.12:0/2183134230 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565db5a9820 con 0x5565db740fe0 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e017fa640 10 client.15801 handle_command_reply: tid=0 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e22d7f740 1 client.15801 shutdown 2024-05-09T09:26:10.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2183134230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 msgr2=0x5565db73ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/2183134230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db73ed00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4e10012040 tx=0x7f4e10005ce0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2183134230 shutdown_connections 2024-05-09T09:26:10.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/2183134230 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5565db740fe0 0x5565db7433d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/2183134230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565db6c9eb0 0x5565db73ed00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e017fa640 0 client.15801 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:26:10.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/2183134230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db6c4d40 0x5565db6d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e22d7f740 1 --2- 172.21.15.12:0/2183134230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565db6c4350 0x5565db6d2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2183134230 >> 172.21.15.12:0/2183134230 conn(0x5565db6bb5b0 msgr2=0x5565db6ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2183134230 shutdown_connections 2024-05-09T09:26:10.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e22d7f740 1 -- 172.21.15.12:0/2183134230 wait complete. 2024-05-09T09:26:10.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.946+0000 7f4e22d7f740 20 client.15801 trim_cache size 0 max 16384 2024-05-09T09:26:10.953 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:10.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.947+0000 7f4e20072640 1 -- 172.21.15.12:0/269956768 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4e040a2b90 msgr2=0x7f4e040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.947+0000 7f4e20072640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4e040a2b90 0x7f4e040a5050 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5565db6c55a0 tx=0x7f4e10004a40 comp rx=0 tx=0).stop 2024-05-09T09:26:10.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.947+0000 7f4e20072640 1 -- 172.21.15.12:0/269956768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e18104ce0 msgr2=0x7f4e18198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:10.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.947+0000 7f4e20072640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e18104ce0 0x7f4e18198190 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800d740 tx=0x7f4e0800dc00 comp rx=0 tx=0).stop 2024-05-09T09:26:10.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.947+0000 7f4e20072640 1 -- 172.21.15.12:0/269956768 shutdown_connections 2024-05-09T09:26:10.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.947+0000 7f4e20072640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4e040a2b90 0x7f4e040a5050 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.947+0000 7f4e20072640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e18107610 0x7f4e1819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.947+0000 7f4e20072640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e18104ce0 0x7f4e18198190 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.947+0000 7f4e20072640 1 --2- 172.21.15.12:0/269956768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e181023b0 0x7f4e18197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:10.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.947+0000 7f4e20072640 1 -- 172.21.15.12:0/269956768 >> 172.21.15.12:0/269956768 conn(0x7f4e180fbce0 msgr2=0x7f4e18102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:10.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.947+0000 7f4e20072640 1 -- 172.21.15.12:0/269956768 shutdown_connections 2024-05-09T09:26:10.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:10.947+0000 7f4e20072640 1 -- 172.21.15.12:0/269956768 wait complete. 2024-05-09T09:26:10.957 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:26:11.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.070+0000 7fd066eee640 1 -- 172.21.15.12:0/1389151264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd060100770 msgr2=0x7fd060100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.070+0000 7fd066eee640 1 --2- 172.21.15.12:0/1389151264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd060100770 0x7fd060100b90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd054009900 tx=0x7fd05402a410 comp rx=0 tx=0).stop 2024-05-09T09:26:11.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.070+0000 7fd066eee640 1 -- 172.21.15.12:0/1389151264 shutdown_connections 2024-05-09T09:26:11.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.070+0000 7fd066eee640 1 --2- 172.21.15.12:0/1389151264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd060102320 0x7fd060108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.070+0000 7fd066eee640 1 --2- 172.21.15.12:0/1389151264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060101960 0x7fd060101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.070+0000 7fd066eee640 1 --2- 172.21.15.12:0/1389151264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd060100770 0x7fd060100b90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.070+0000 7fd066eee640 1 -- 172.21.15.12:0/1389151264 >> 172.21.15.12:0/1389151264 conn(0x7fd0600fbce0 msgr2=0x7fd0600fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:11.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.070+0000 7fd066eee640 1 -- 172.21.15.12:0/1389151264 shutdown_connections 2024-05-09T09:26:11.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.070+0000 7fd066eee640 1 -- 172.21.15.12:0/1389151264 wait complete. 2024-05-09T09:26:11.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.071+0000 7fd066eee640 1 Processor -- start 2024-05-09T09:26:11.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.071+0000 7fd066eee640 1 -- start start 2024-05-09T09:26:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.071+0000 7fd066eee640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd060100770 0x7fd060069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.071+0000 7fd066eee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd060101960 0x7fd06019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd066eee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060102320 0x7fd06019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd066eee640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd06010c3d0 con 0x7fd060100770 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd066eee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd06010c250 con 0x7fd060102320 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd066eee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd06010c550 con 0x7fd060101960 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd064c8a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd060100770 0x7fd060069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd064c8a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd060100770 0x7fd060069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53718/0 (socket says 172.21.15.12:53718) 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd064c8a640 1 -- 172.21.15.12:0/2302745724 learned_addr learned my addr 172.21.15.12:0/2302745724 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd05ffff640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd060101960 0x7fd06019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd06548b640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060102320 0x7fd06019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd064c8a640 1 -- 172.21.15.12:0/2302745724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd060101960 msgr2=0x7fd06019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd064c8a640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd060101960 0x7fd06019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd064c8a640 1 -- 172.21.15.12:0/2302745724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060102320 msgr2=0x7fd06019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd064c8a640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060102320 0x7fd06019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd064c8a640 1 -- 172.21.15.12:0/2302745724 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06019f810 con 0x7fd060100770 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd064c8a640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd060100770 0x7fd060069590 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd054005ab0 tx=0x7fd0540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd05dffb640 1 -- 172.21.15.12:0/2302745724 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05403f030 con 0x7fd060100770 2024-05-09T09:26:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd05dffb640 1 -- 172.21.15.12:0/2302745724 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd054003ac0 con 0x7fd060100770 2024-05-09T09:26:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd066eee640 1 -- 172.21.15.12:0/2302745724 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06019faa0 con 0x7fd060100770 2024-05-09T09:26:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.072+0000 7fd05dffb640 1 -- 172.21.15.12:0/2302745724 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054035070 con 0x7fd060100770 2024-05-09T09:26:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.073+0000 7fd066eee640 1 -- 172.21.15.12:0/2302745724 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06019ff90 con 0x7fd060100770 2024-05-09T09:26:11.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.075+0000 7fd05dffb640 1 -- 172.21.15.12:0/2302745724 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd054042030 con 0x7fd060100770 2024-05-09T09:26:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.075+0000 7fd069bf8740 1 Processor -- start 2024-05-09T09:26:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd069bf8740 1 -- start start 2024-05-09T09:26:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd069bf8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041350 0x564587041770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd069bf8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 0x564587046840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd069bf8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 0x564587049340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd069bf8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564586fa1c40 con 0x564587041d40 2024-05-09T09:26:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd069bf8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564586f49580 con 0x564587046eb0 2024-05-09T09:26:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd069bf8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564586f9dc00 con 0x564587041350 2024-05-09T09:26:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd064c8a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 0x564587046840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd064c8a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 0x564587046840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53726/0 (socket says 172.21.15.12:53726) 2024-05-09T09:26:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd064c8a640 1 -- 172.21.15.12:0/819589198 learned_addr learned my addr 172.21.15.12:0/819589198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd06548b640 1 --2- 172.21.15.12:0/819589198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041350 0x564587041770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd05ffff640 1 --2- 172.21.15.12:0/819589198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 0x564587049340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd05dffb640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd0500a2b90 0x7fd0500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd064c8a640 1 -- 172.21.15.12:0/819589198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041350 msgr2=0x564587041770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd064c8a640 1 --2- 172.21.15.12:0/819589198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041350 0x564587041770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd064c8a640 1 -- 172.21.15.12:0/819589198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 msgr2=0x564587049340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd064c8a640 1 --2- 172.21.15.12:0/819589198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 0x564587049340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd064c8a640 1 -- 172.21.15.12:0/819589198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564586cb53f0 con 0x564587041d40 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.076+0000 7fd06548b640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd0500a2b90 0x7fd0500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.077+0000 7fd05dffb640 1 -- 172.21.15.12:0/2302745724 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd0540ef9a0 con 0x7fd060100770 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.077+0000 7fd064c8a640 1 --2- 172.21.15.12:0/819589198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 0x564587046840 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd0540d3d20 tx=0x7fd0540b7bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac753f5ad9529f65 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.077+0000 7fd04affd640 1 -- 172.21.15.12:0/819589198 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054049070 con 0x564587041d40 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.077+0000 7fd04affd640 1 -- 172.21.15.12:0/819589198 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd0540047a0 con 0x564587041d40 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.077+0000 7fd04affd640 1 -- 172.21.15.12:0/819589198 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0540bb9c0 con 0x564587041d40 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.077+0000 7fd06548b640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd0500a2b90 0x7fd0500a5050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x5645870425a0 tx=0x7fd058004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 -- 172.21.15.12:0/819589198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 msgr2=0x564587046840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 --2- 172.21.15.12:0/819589198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 0x564587046840 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd0540d3d20 tx=0x7fd0540b7bd0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 -- 172.21.15.12:0/819589198 shutdown_connections 2024-05-09T09:26:11.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 --2- 172.21.15.12:0/819589198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 0x564587049340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 --2- 172.21.15.12:0/819589198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 0x564587046840 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 --2- 172.21.15.12:0/819589198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041350 0x564587041770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 -- 172.21.15.12:0/819589198 >> 172.21.15.12:0/819589198 conn(0x5645870385b0 msgr2=0x5645870389b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:11.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 -- 172.21.15.12:0/819589198 shutdown_connections 2024-05-09T09:26:11.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 -- 172.21.15.12:0/819589198 wait complete. 2024-05-09T09:26:11.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 -- start start 2024-05-09T09:26:11.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 Processor -- start 2024-05-09T09:26:11.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 0x56458704b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041d40 0x56458704b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.078+0000 7fd069bf8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 0x564587052960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd069bf8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564587049c20 con 0x564587041350 2024-05-09T09:26:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd069bf8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564586f5cc40 con 0x564587046eb0 2024-05-09T09:26:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd069bf8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564587049da0 con 0x564587041d40 2024-05-09T09:26:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd064c8a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 0x56458704b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd064c8a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 0x56458704b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53732/0 (socket says 172.21.15.12:53732) 2024-05-09T09:26:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd064c8a640 1 -- 172.21.15.12:0/1574366707 learned_addr learned my addr 172.21.15.12:0/1574366707 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd069bf8740 10 client.0 _ll_register_callbacks cb 0x564586cb4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd05ffff640 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041d40 0x56458704b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd06548b640 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 0x564587052960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd064c8a640 1 -- 172.21.15.12:0/1574366707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041d40 msgr2=0x56458704b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd064c8a640 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041d40 0x56458704b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd064c8a640 1 -- 172.21.15.12:0/1574366707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 msgr2=0x564587052960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd064c8a640 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 0x564587052960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd064c8a640 1 -- 172.21.15.12:0/1574366707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564586d7f7d0 con 0x564587041350 2024-05-09T09:26:11.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd06548b640 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 0x564587052960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:11.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd064c8a640 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 0x56458704b3e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd0540c4510 tx=0x7fd05402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998d4a542fe322fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:11.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd03ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:11.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd03ffff640 1 -- 172.21.15.12:0/1574366707 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0540d9060 con 0x564587041350 2024-05-09T09:26:11.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd03ffff640 1 -- 172.21.15.12:0/1574366707 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd054003c70 con 0x564587041350 2024-05-09T09:26:11.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd069bf8740 1 -- 172.21.15.12:0/1574366707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0540047a0 con 0x564587041350 2024-05-09T09:26:11.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd03ffff640 1 -- 172.21.15.12:0/1574366707 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd054049030 con 0x564587041350 2024-05-09T09:26:11.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd069bf8740 10 client.15813 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:11.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.079+0000 7fd069bf8740 1 -- 172.21.15.12:0/1574366707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564587049c20 con 0x564587041350 2024-05-09T09:26:11.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.080+0000 7fd03ffff640 1 -- 172.21.15.12:0/1574366707 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd0540047a0 con 0x564587041350 2024-05-09T09:26:11.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.080+0000 7fd03ffff640 1 -- 172.21.15.12:0/1574366707 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd0540d1070 con 0x564587041350 2024-05-09T09:26:11.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.080+0000 7fd069bf8740 10 client.15813 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:11.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.080+0000 7fd069bf8740 10 client.15813 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:26:11.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.080+0000 7fd069bf8740 20 client.15813 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:11.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.080+0000 7fd069bf8740 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x564587056fd0 0x564587059410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.080+0000 7fd069bf8740 4 client.15813 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:11.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.080+0000 7fd069bf8740 1 -- 172.21.15.12:0/1574366707 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564586faf370 con 0x564587056fd0 2024-05-09T09:26:11.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.080+0000 7fd05ffff640 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x564587056fd0 0x564587059410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.081+0000 7fd03ffff640 10 client.15813 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.081+0000 7fd05ffff640 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x564587056fd0 0x564587059410 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0790635112c403d server_cookie=d85873c9ccd544cc in_seq=0 out_seq=0 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.081+0000 7fd03ffff640 1 -- 172.21.15.12:0/1574366707 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564586faf370 con 0x564587056fd0 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.081+0000 7fd03ffff640 10 client.15813 handle_command_reply: tid=0 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.081+0000 7fd069bf8740 1 client.15813 shutdown 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.082+0000 7fd069bf8740 1 -- 172.21.15.12:0/1574366707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 msgr2=0x56458704b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.082+0000 7fd069bf8740 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 0x56458704b3e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd0540c4510 tx=0x7fd05402a410 comp rx=0 tx=0).stop 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.082+0000 7fd069bf8740 1 -- 172.21.15.12:0/1574366707 shutdown_connections 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.082+0000 7fd069bf8740 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x564587056fd0 0x564587059410 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.082+0000 7fd069bf8740 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 0x564587052960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.082+0000 7fd069bf8740 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041d40 0x56458704b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.082+0000 7fd069bf8740 1 --2- 172.21.15.12:0/1574366707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 0x56458704b3e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.082+0000 7fd069bf8740 1 -- 172.21.15.12:0/1574366707 >> 172.21.15.12:0/1574366707 conn(0x5645870385b0 msgr2=0x5645870377b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.082+0000 7fd03ffff640 0 client.15813 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:26:11.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.082+0000 7fd069bf8740 1 -- 172.21.15.12:0/1574366707 shutdown_connections 2024-05-09T09:26:11.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.082+0000 7fd069bf8740 1 -- 172.21.15.12:0/1574366707 wait complete. 2024-05-09T09:26:11.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.082+0000 7fd069bf8740 20 client.15813 trim_cache size 0 max 16384 2024-05-09T09:26:11.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.095+0000 7fd069bf8740 1 Processor -- start 2024-05-09T09:26:11.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.095+0000 7fd069bf8740 1 -- start start 2024-05-09T09:26:11.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.095+0000 7fd069bf8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587041350 0x56458704b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.095+0000 7fd069bf8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 0x5645870528f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.095+0000 7fd069bf8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587046eb0 0x56458703bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.095+0000 7fd069bf8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601a8460 con 0x564587041d40 2024-05-09T09:26:11.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.095+0000 7fd069bf8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601a9fa0 con 0x564587041350 2024-05-09T09:26:11.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.095+0000 7fd069bf8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601a8b30 con 0x564587046eb0 2024-05-09T09:26:11.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd05ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 0x5645870528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd05ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 0x5645870528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53746/0 (socket says 172.21.15.12:53746) 2024-05-09T09:26:11.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd05ffff640 1 -- 172.21.15.12:0/826046412 learned_addr learned my addr 172.21.15.12:0/826046412 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:11.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd06548b640 1 --2- 172.21.15.12:0/826046412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587046eb0 0x56458703bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd064c8a640 1 --2- 172.21.15.12:0/826046412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587041350 0x56458704b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd05ffff640 1 -- 172.21.15.12:0/826046412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587046eb0 msgr2=0x56458703bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd05ffff640 1 --2- 172.21.15.12:0/826046412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587046eb0 0x56458703bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd05ffff640 1 -- 172.21.15.12:0/826046412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587041350 msgr2=0x56458704b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd05ffff640 1 --2- 172.21.15.12:0/826046412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587041350 0x56458704b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd05ffff640 1 -- 172.21.15.12:0/826046412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564587039020 con 0x564587041d40 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd064c8a640 1 --2- 172.21.15.12:0/826046412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587041350 0x56458704b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd05ffff640 1 --2- 172.21.15.12:0/826046412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 0x5645870528f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x564587039520 tx=0x7fd04c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155b09534216c432 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd04a7fc640 1 -- 172.21.15.12:0/826046412 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c01f630 con 0x564587041d40 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd04a7fc640 1 -- 172.21.15.12:0/826046412 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd04c01f7d0 con 0x564587041d40 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.096+0000 7fd04a7fc640 1 -- 172.21.15.12:0/826046412 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c01ee60 con 0x564587041d40 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.097+0000 7fd069bf8740 1 -- 172.21.15.12:0/826046412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 msgr2=0x5645870528f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.097+0000 7fd069bf8740 1 --2- 172.21.15.12:0/826046412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 0x5645870528f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x564587039520 tx=0x7fd04c005f90 comp rx=0 tx=0).stop 2024-05-09T09:26:11.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.097+0000 7fd069bf8740 1 -- 172.21.15.12:0/826046412 shutdown_connections 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.097+0000 7fd069bf8740 1 --2- 172.21.15.12:0/826046412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587046eb0 0x56458703bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.097+0000 7fd069bf8740 1 --2- 172.21.15.12:0/826046412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041d40 0x5645870528f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.097+0000 7fd069bf8740 1 --2- 172.21.15.12:0/826046412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587041350 0x56458704b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.097+0000 7fd069bf8740 1 -- 172.21.15.12:0/826046412 >> 172.21.15.12:0/826046412 conn(0x5645870385b0 msgr2=0x56458703b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.097+0000 7fd069bf8740 1 -- 172.21.15.12:0/826046412 shutdown_connections 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.097+0000 7fd069bf8740 1 -- 172.21.15.12:0/826046412 wait complete. 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.097+0000 7fd069bf8740 1 -- start start 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd069bf8740 1 Processor -- start 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd069bf8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 0x56458704fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd069bf8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041d40 0x564587050190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd069bf8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 0x5645870511a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd069bf8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601a98d0 con 0x564587041350 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd069bf8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601a9200 con 0x564587046eb0 2024-05-09T09:26:11.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd069bf8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601a6440 con 0x564587041d40 2024-05-09T09:26:11.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd069bf8740 10 client.0 _ll_register_callbacks cb 0x7fd058009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:11.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd064c8a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 0x56458704fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd064c8a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 0x56458704fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53754/0 (socket says 172.21.15.12:53754) 2024-05-09T09:26:11.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd064c8a640 1 -- 172.21.15.12:0/1522215122 learned_addr learned my addr 172.21.15.12:0/1522215122 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:11.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd05ffff640 1 --2- 172.21.15.12:0/1522215122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041d40 0x564587050190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd06548b640 1 --2- 172.21.15.12:0/1522215122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 0x5645870511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd064c8a640 1 -- 172.21.15.12:0/1522215122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041d40 msgr2=0x564587050190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.098+0000 7fd064c8a640 1 --2- 172.21.15.12:0/1522215122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041d40 0x564587050190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd064c8a640 1 -- 172.21.15.12:0/1522215122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 msgr2=0x5645870511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd064c8a640 1 --2- 172.21.15.12:0/1522215122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 0x5645870511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd064c8a640 1 -- 172.21.15.12:0/1522215122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564587051790 con 0x564587041350 2024-05-09T09:26:11.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd064c8a640 1 --2- 172.21.15.12:0/1522215122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 0x56458704fc50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd0540b87e0 tx=0x7fd05403b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7602611554910e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:11.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd03ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd03ffff640 1 -- 172.21.15.12:0/1522215122 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0540d9070 con 0x564587041350 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd069bf8740 1 -- 172.21.15.12:0/1522215122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd04c01f7d0 con 0x564587041350 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd03ffff640 1 -- 172.21.15.12:0/1522215122 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd054003c70 con 0x564587041350 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd03ffff640 1 -- 172.21.15.12:0/1522215122 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0540044e0 con 0x564587041350 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd069bf8740 10 client.15819 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd069bf8740 1 -- 172.21.15.12:0/1522215122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564587051790 con 0x564587041350 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd03ffff640 1 -- 172.21.15.12:0/1522215122 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd04c01f7d0 con 0x564587041350 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd03ffff640 1 -- 172.21.15.12:0/1522215122 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd0540d1060 con 0x564587041350 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd069bf8740 10 client.15819 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd069bf8740 10 client.15819 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.099+0000 7fd069bf8740 20 client.15819 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.100+0000 7fd069bf8740 1 --2- 172.21.15.12:0/1522215122 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x5645870bd6f0 0x5645870bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.100+0000 7fd069bf8740 4 client.15819 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.100+0000 7fd069bf8740 1 -- 172.21.15.12:0/1522215122 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564586fa96c0 con 0x5645870bd6f0 2024-05-09T09:26:11.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.100+0000 7fd05ffff640 1 --2- 172.21.15.12:0/1522215122 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x5645870bd6f0 0x5645870bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.100+0000 7fd05ffff640 1 --2- 172.21.15.12:0/1522215122 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x5645870bd6f0 0x5645870bfae0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61b5e06c506bcfb0 server_cookie=5d3b07eba81f7473 in_seq=0 out_seq=0 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.100+0000 7fd03ffff640 10 client.15819 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.100+0000 7fd03ffff640 1 -- 172.21.15.12:0/1522215122 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564586fa96c0 con 0x5645870bd6f0 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.100+0000 7fd03ffff640 10 client.15819 handle_command_reply: tid=0 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.100+0000 7fd069bf8740 1 client.15819 shutdown 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.100+0000 7fd069bf8740 1 -- 172.21.15.12:0/1522215122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 msgr2=0x56458704fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.100+0000 7fd069bf8740 1 --2- 172.21.15.12:0/1522215122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 0x56458704fc50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd0540b87e0 tx=0x7fd05403b460 comp rx=0 tx=0).stop 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd069bf8740 1 -- 172.21.15.12:0/1522215122 shutdown_connections 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd069bf8740 1 --2- 172.21.15.12:0/1522215122 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x5645870bd6f0 0x5645870bfae0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd069bf8740 1 --2- 172.21.15.12:0/1522215122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564587046eb0 0x5645870511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd069bf8740 1 --2- 172.21.15.12:0/1522215122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564587041d40 0x564587050190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd03ffff640 0 client.15819 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd069bf8740 1 --2- 172.21.15.12:0/1522215122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564587041350 0x56458704fc50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd069bf8740 1 -- 172.21.15.12:0/1522215122 >> 172.21.15.12:0/1522215122 conn(0x5645870385b0 msgr2=0x56458703bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:11.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd069bf8740 1 -- 172.21.15.12:0/1522215122 shutdown_connections 2024-05-09T09:26:11.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd069bf8740 1 -- 172.21.15.12:0/1522215122 wait complete. 2024-05-09T09:26:11.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd069bf8740 20 client.15819 trim_cache size 0 max 16384 2024-05-09T09:26:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:11.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd066eee640 1 -- 172.21.15.12:0/2302745724 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd0500a2b90 msgr2=0x7fd0500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd066eee640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd0500a2b90 0x7fd0500a5050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x5645870425a0 tx=0x7fd058004a40 comp rx=0 tx=0).stop 2024-05-09T09:26:11.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd066eee640 1 -- 172.21.15.12:0/2302745724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd060100770 msgr2=0x7fd060069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd066eee640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd060100770 0x7fd060069590 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd054005ab0 tx=0x7fd0540042f0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd066eee640 1 -- 172.21.15.12:0/2302745724 shutdown_connections 2024-05-09T09:26:11.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd066eee640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd0500a2b90 0x7fd0500a5050 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd066eee640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060102320 0x7fd06019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd066eee640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd060101960 0x7fd06019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd066eee640 1 --2- 172.21.15.12:0/2302745724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd060100770 0x7fd060069590 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.101+0000 7fd066eee640 1 -- 172.21.15.12:0/2302745724 >> 172.21.15.12:0/2302745724 conn(0x7fd0600fbce0 msgr2=0x7fd060104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:11.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.102+0000 7fd066eee640 1 -- 172.21.15.12:0/2302745724 shutdown_connections 2024-05-09T09:26:11.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.102+0000 7fd066eee640 1 -- 172.21.15.12:0/2302745724 wait complete. 2024-05-09T09:26:11.113 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:26:11.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.222+0000 7fbd92a58640 1 -- 172.21.15.12:0/709183378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd8c102a70 msgr2=0x7fbd8c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.222+0000 7fbd92a58640 1 --2- 172.21.15.12:0/709183378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd8c102a70 0x7fbd8c102ef0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009900 tx=0x7fbd8002a410 comp rx=0 tx=0).stop 2024-05-09T09:26:11.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.222+0000 7fbd92a58640 1 -- 172.21.15.12:0/709183378 shutdown_connections 2024-05-09T09:26:11.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.222+0000 7fbd92a58640 1 --2- 172.21.15.12:0/709183378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8c103430 0x7fbd8c109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.222+0000 7fbd92a58640 1 --2- 172.21.15.12:0/709183378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd8c102a70 0x7fbd8c102ef0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.222+0000 7fbd92a58640 1 --2- 172.21.15.12:0/709183378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c101880 0x7fbd8c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.222+0000 7fbd92a58640 1 -- 172.21.15.12:0/709183378 >> 172.21.15.12:0/709183378 conn(0x7fbd8c0fce10 msgr2=0x7fbd8c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:11.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.222+0000 7fbd92a58640 1 -- 172.21.15.12:0/709183378 shutdown_connections 2024-05-09T09:26:11.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.222+0000 7fbd92a58640 1 -- 172.21.15.12:0/709183378 wait complete. 2024-05-09T09:26:11.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.223+0000 7fbd92a58640 1 Processor -- start 2024-05-09T09:26:11.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.223+0000 7fbd92a58640 1 -- start start 2024-05-09T09:26:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.223+0000 7fbd92a58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8c101880 0x7fbd8c0755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.223+0000 7fbd92a58640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd8c102a70 0x7fbd8c073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.223+0000 7fbd92a58640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c103430 0x7fbd8c074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.223+0000 7fbd92a58640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c10d4e0 con 0x7fbd8c102a70 2024-05-09T09:26:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.223+0000 7fbd92a58640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c10d360 con 0x7fbd8c101880 2024-05-09T09:26:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.223+0000 7fbd92a58640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c10d660 con 0x7fbd8c103430 2024-05-09T09:26:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.223+0000 7fbd8b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd8c102a70 0x7fbd8c073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.223+0000 7fbd8b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd8c102a70 0x7fbd8c073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53780/0 (socket says 172.21.15.12:53780) 2024-05-09T09:26:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.223+0000 7fbd8b7fe640 1 -- 172.21.15.12:0/2305209272 learned_addr learned my addr 172.21.15.12:0/2305209272 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.223+0000 7fbd8bfff640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8c101880 0x7fbd8c0755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.223+0000 7fbd90ff5640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c103430 0x7fbd8c074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.224+0000 7fbd8b7fe640 1 -- 172.21.15.12:0/2305209272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c103430 msgr2=0x7fbd8c074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.224+0000 7fbd8b7fe640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c103430 0x7fbd8c074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.224+0000 7fbd8b7fe640 1 -- 172.21.15.12:0/2305209272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8c101880 msgr2=0x7fbd8c0755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.224+0000 7fbd8b7fe640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8c101880 0x7fbd8c0755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.224+0000 7fbd8b7fe640 1 -- 172.21.15.12:0/2305209272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c0749f0 con 0x7fbd8c102a70 2024-05-09T09:26:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.224+0000 7fbd90ff5640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c103430 0x7fbd8c074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.224+0000 7fbd8b7fe640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd8c102a70 0x7fbd8c073c20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009230 tx=0x7fbd800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.224+0000 7fbd897fa640 1 -- 172.21.15.12:0/2305209272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8003f030 con 0x7fbd8c102a70 2024-05-09T09:26:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.224+0000 7fbd92a58640 1 -- 172.21.15.12:0/2305209272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8c1a1c40 con 0x7fbd8c102a70 2024-05-09T09:26:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.224+0000 7fbd897fa640 1 -- 172.21.15.12:0/2305209272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd80031980 con 0x7fbd8c102a70 2024-05-09T09:26:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.224+0000 7fbd897fa640 1 -- 172.21.15.12:0/2305209272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80035070 con 0x7fbd8c102a70 2024-05-09T09:26:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.224+0000 7fbd92a58640 1 -- 172.21.15.12:0/2305209272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd8c1a2190 con 0x7fbd8c102a70 2024-05-09T09:26:11.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd897fa640 1 -- 172.21.15.12:0/2305209272 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbd80042030 con 0x7fbd8c102a70 2024-05-09T09:26:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd95766740 1 Processor -- start 2024-05-09T09:26:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd95766740 1 -- start start 2024-05-09T09:26:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd95766740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c2f40 0x55ab242c3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd95766740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 0x55ab242c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd95766740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c8aa0 0x55ab242caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd95766740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ab24227c40 con 0x55ab242c3930 2024-05-09T09:26:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd95766740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ab241cf580 con 0x55ab242c8aa0 2024-05-09T09:26:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd95766740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ab24223c00 con 0x55ab242c2f40 2024-05-09T09:26:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 0x55ab242c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 0x55ab242c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53794/0 (socket says 172.21.15.12:53794) 2024-05-09T09:26:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd8bfff640 1 -- 172.21.15.12:0/3140414735 learned_addr learned my addr 172.21.15.12:0/3140414735 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd8b7fe640 1 --2- 172.21.15.12:0/3140414735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c8aa0 0x55ab242caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd897fa640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbd740a2b90 0x7fbd740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.227+0000 7fbd90ff5640 1 --2- 172.21.15.12:0/3140414735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c2f40 0x55ab242c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.228+0000 7fbd8bfff640 1 -- 172.21.15.12:0/3140414735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c2f40 msgr2=0x55ab242c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.228+0000 7fbd8bfff640 1 --2- 172.21.15.12:0/3140414735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c2f40 0x55ab242c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.228+0000 7fbd8bfff640 1 -- 172.21.15.12:0/3140414735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c8aa0 msgr2=0x55ab242caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.228+0000 7fbd90ff5640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbd740a2b90 0x7fbd740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.228+0000 7fbd8bfff640 1 --2- 172.21.15.12:0/3140414735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c8aa0 0x55ab242caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.228+0000 7fbd8bfff640 1 -- 172.21.15.12:0/3140414735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab23f3b3f0 con 0x55ab242c3930 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.228+0000 7fbd897fa640 1 -- 172.21.15.12:0/2305209272 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbd800ef970 con 0x7fbd8c102a70 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.228+0000 7fbd8bfff640 1 --2- 172.21.15.12:0/3140414735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 0x55ab242c8430 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009ca0 tx=0x7fbd7c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936262e4730de646 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.228+0000 7fbd727fc640 1 -- 172.21.15.12:0/3140414735 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c004220 con 0x55ab242c3930 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.228+0000 7fbd727fc640 1 -- 172.21.15.12:0/3140414735 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd7c0043c0 con 0x55ab242c3930 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.228+0000 7fbd727fc640 1 -- 172.21.15.12:0/3140414735 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c019850 con 0x55ab242c3930 2024-05-09T09:26:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.229+0000 7fbd95766740 1 -- 172.21.15.12:0/3140414735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 msgr2=0x55ab242c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.229+0000 7fbd90ff5640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbd740a2b90 0x7fbd740a5050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55ab242c4190 tx=0x7fbd84004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.229+0000 7fbd95766740 1 --2- 172.21.15.12:0/3140414735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 0x55ab242c8430 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009ca0 tx=0x7fbd7c011810 comp rx=0 tx=0).stop 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.229+0000 7fbd95766740 1 -- 172.21.15.12:0/3140414735 shutdown_connections 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.229+0000 7fbd95766740 1 --2- 172.21.15.12:0/3140414735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c8aa0 0x55ab242caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.229+0000 7fbd95766740 1 --2- 172.21.15.12:0/3140414735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 0x55ab242c8430 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.229+0000 7fbd95766740 1 --2- 172.21.15.12:0/3140414735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c2f40 0x55ab242c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.229+0000 7fbd95766740 1 -- 172.21.15.12:0/3140414735 >> 172.21.15.12:0/3140414735 conn(0x55ab242be5b0 msgr2=0x55ab242be9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.229+0000 7fbd95766740 1 -- 172.21.15.12:0/3140414735 shutdown_connections 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.229+0000 7fbd95766740 1 -- 172.21.15.12:0/3140414735 wait complete. 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.229+0000 7fbd95766740 1 -- start start 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.229+0000 7fbd95766740 1 Processor -- start 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd95766740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c2f40 0x55ab242cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd95766740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 0x55ab242cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd95766740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 0x55ab242d48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd95766740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ab242cba50 con 0x55ab242c3930 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd95766740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ab241e2c40 con 0x55ab242c2f40 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd95766740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ab242cbbd0 con 0x55ab242c8aa0 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd95766740 10 client.0 _ll_register_callbacks cb 0x55ab23f3abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd8b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 0x55ab242cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd8b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 0x55ab242cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53804/0 (socket says 172.21.15.12:53804) 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd8b7fe640 1 -- 172.21.15.12:0/1790958158 learned_addr learned my addr 172.21.15.12:0/1790958158 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd8bfff640 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c2f40 0x55ab242cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd90ff5640 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 0x55ab242d48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd8b7fe640 1 -- 172.21.15.12:0/1790958158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 msgr2=0x55ab242d48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd8b7fe640 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 0x55ab242d48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd8b7fe640 1 -- 172.21.15.12:0/1790958158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c2f40 msgr2=0x55ab242cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd8b7fe640 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c2f40 0x55ab242cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd8b7fe640 1 -- 172.21.15.12:0/1790958158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab240057d0 con 0x55ab242c3930 2024-05-09T09:26:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd90ff5640 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 0x55ab242d48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd8b7fe640 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 0x55ab242cd8a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbd800ed410 tx=0x7fbd800092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4027333e9a8ed26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.230+0000 7fbd6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd6b7fe640 1 -- 172.21.15.12:0/1790958158 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80049040 con 0x55ab242c3930 2024-05-09T09:26:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd6b7fe640 1 -- 172.21.15.12:0/1790958158 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbd80003cc0 con 0x55ab242c3930 2024-05-09T09:26:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd95766740 1 -- 172.21.15.12:0/1790958158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd7c0043c0 con 0x55ab242c3930 2024-05-09T09:26:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd6b7fe640 1 -- 172.21.15.12:0/1790958158 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd800bb760 con 0x55ab242c3930 2024-05-09T09:26:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd95766740 10 client.15831 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd95766740 1 -- 172.21.15.12:0/1790958158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ab242d7c50 con 0x55ab242c3930 2024-05-09T09:26:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd6b7fe640 1 -- 172.21.15.12:0/1790958158 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd7c0043c0 con 0x55ab242c3930 2024-05-09T09:26:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd6b7fe640 1 -- 172.21.15.12:0/1790958158 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbd800bc060 con 0x55ab242c3930 2024-05-09T09:26:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd95766740 10 client.15831 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd95766740 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd95766740 20 client.15831 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd95766740 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ab242d9330 0x55ab242db790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd95766740 4 client.15831 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.231+0000 7fbd95766740 1 -- 172.21.15.12:0/1790958158 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab24235370 con 0x55ab242d9330 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.232+0000 7fbd8bfff640 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ab242d9330 0x55ab242db790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.232+0000 7fbd8bfff640 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ab242d9330 0x55ab242db790 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ecc3f67ab7d4fc7 server_cookie=f5124bc53413c51c in_seq=0 out_seq=0 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.232+0000 7fbd6b7fe640 10 client.15831 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd6b7fe640 1 -- 172.21.15.12:0/1790958158 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab24235370 con 0x55ab242d9330 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd6b7fe640 10 client.15831 handle_command_reply: tid=0 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd95766740 1 client.15831 shutdown 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd95766740 1 -- 172.21.15.12:0/1790958158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 msgr2=0x55ab242cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd95766740 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 0x55ab242cd8a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbd800ed410 tx=0x7fbd800092e0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd95766740 1 -- 172.21.15.12:0/1790958158 shutdown_connections 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd95766740 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ab242d9330 0x55ab242db790 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd95766740 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 0x55ab242d48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd95766740 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c3930 0x55ab242cd8a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd6b7fe640 0 client.15831 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:11.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd95766740 1 --2- 172.21.15.12:0/1790958158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c2f40 0x55ab242cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd95766740 1 -- 172.21.15.12:0/1790958158 >> 172.21.15.12:0/1790958158 conn(0x55ab242be5b0 msgr2=0x55ab242bd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:11.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd95766740 1 -- 172.21.15.12:0/1790958158 shutdown_connections 2024-05-09T09:26:11.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd95766740 1 -- 172.21.15.12:0/1790958158 wait complete. 2024-05-09T09:26:11.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.233+0000 7fbd95766740 20 client.15831 trim_cache size 0 max 16384 2024-05-09T09:26:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.247+0000 7fbd95766740 1 Processor -- start 2024-05-09T09:26:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.247+0000 7fbd95766740 1 -- start start 2024-05-09T09:26:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.247+0000 7fbd95766740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 0x55ab242c1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.247+0000 7fbd95766740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c3930 0x55ab242d48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.247+0000 7fbd95766740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 0x55ab242c1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.247+0000 7fbd95766740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c1ac740 con 0x55ab242c2f40 2024-05-09T09:26:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.247+0000 7fbd95766740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c1ae280 con 0x55ab242c3930 2024-05-09T09:26:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.247+0000 7fbd95766740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c1ace10 con 0x55ab242c8aa0 2024-05-09T09:26:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 0x55ab242c1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 0x55ab242c1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53806/0 (socket says 172.21.15.12:53806) 2024-05-09T09:26:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd8bfff640 1 -- 172.21.15.12:0/3931559166 learned_addr learned my addr 172.21.15.12:0/3931559166 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd8b7fe640 1 --2- 172.21.15.12:0/3931559166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c3930 0x55ab242d48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd90ff5640 1 --2- 172.21.15.12:0/3931559166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 0x55ab242c1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd8bfff640 1 -- 172.21.15.12:0/3931559166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 msgr2=0x55ab242c1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd8bfff640 1 --2- 172.21.15.12:0/3931559166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 0x55ab242c1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd8bfff640 1 -- 172.21.15.12:0/3931559166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c3930 msgr2=0x55ab242d48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd8bfff640 1 --2- 172.21.15.12:0/3931559166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c3930 0x55ab242d48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd8bfff640 1 -- 172.21.15.12:0/3931559166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab242afd20 con 0x55ab242c2f40 2024-05-09T09:26:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd8b7fe640 1 --2- 172.21.15.12:0/3931559166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c3930 0x55ab242d48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd8bfff640 1 --2- 172.21.15.12:0/3931559166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 0x55ab242c1560 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0040c0 tx=0x7fbd7c003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d631152f08243da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd71ffb640 1 -- 172.21.15.12:0/3931559166 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c031070 con 0x55ab242c2f40 2024-05-09T09:26:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd71ffb640 1 -- 172.21.15.12:0/3931559166 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd7c02bb50 con 0x55ab242c2f40 2024-05-09T09:26:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.248+0000 7fbd71ffb640 1 -- 172.21.15.12:0/3931559166 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c02b410 con 0x55ab242c2f40 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.249+0000 7fbd95766740 1 -- 172.21.15.12:0/3931559166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 msgr2=0x55ab242c1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.249+0000 7fbd95766740 1 --2- 172.21.15.12:0/3931559166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 0x55ab242c1560 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0040c0 tx=0x7fbd7c003e20 comp rx=0 tx=0).stop 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.249+0000 7fbd95766740 1 -- 172.21.15.12:0/3931559166 shutdown_connections 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.249+0000 7fbd95766740 1 --2- 172.21.15.12:0/3931559166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 0x55ab242c1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.249+0000 7fbd95766740 1 --2- 172.21.15.12:0/3931559166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c3930 0x55ab242d48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.249+0000 7fbd95766740 1 --2- 172.21.15.12:0/3931559166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 0x55ab242c1560 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.249+0000 7fbd95766740 1 -- 172.21.15.12:0/3931559166 >> 172.21.15.12:0/3931559166 conn(0x55ab242be5b0 msgr2=0x55ab242c4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.249+0000 7fbd95766740 1 -- 172.21.15.12:0/3931559166 shutdown_connections 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.249+0000 7fbd95766740 1 -- 172.21.15.12:0/3931559166 wait complete. 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.249+0000 7fbd95766740 1 -- start start 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd95766740 1 Processor -- start 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd95766740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 0x55ab242d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd95766740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c3930 0x55ab242d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd95766740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 0x55ab242d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd95766740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c1adbb0 con 0x55ab242c2f40 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd95766740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c1ad4e0 con 0x55ab242c3930 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd95766740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c1aa720 con 0x55ab242c8aa0 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd95766740 10 client.0 _ll_register_callbacks cb 0x7fbd800f5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 0x55ab242d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 0x55ab242d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53816/0 (socket says 172.21.15.12:53816) 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd8bfff640 1 -- 172.21.15.12:0/139648434 learned_addr learned my addr 172.21.15.12:0/139648434 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd90ff5640 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 0x55ab242d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd8b7fe640 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c3930 0x55ab242d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd8bfff640 1 -- 172.21.15.12:0/139648434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 msgr2=0x55ab242d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd8bfff640 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 0x55ab242d3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd8bfff640 1 -- 172.21.15.12:0/139648434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c3930 msgr2=0x55ab242d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd8bfff640 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c3930 0x55ab242d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd8bfff640 1 -- 172.21.15.12:0/139648434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab242d4010 con 0x55ab242c2f40 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.250+0000 7fbd8b7fe640 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c3930 0x55ab242d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd8bfff640 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 0x55ab242d24d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c011d10 tx=0x7fbd7c011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9dbc8afb145552 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd6b7fe640 1 -- 172.21.15.12:0/139648434 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c031070 con 0x55ab242c2f40 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd6b7fe640 1 -- 172.21.15.12:0/139648434 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbd7c005460 con 0x55ab242c2f40 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd95766740 1 -- 172.21.15.12:0/139648434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd7c02bb50 con 0x55ab242c2f40 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd6b7fe640 1 -- 172.21.15.12:0/139648434 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd7c005660 con 0x55ab242c2f40 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd95766740 10 client.15837 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd95766740 1 -- 172.21.15.12:0/139648434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ab2433fdb0 con 0x55ab242c2f40 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd6b7fe640 1 -- 172.21.15.12:0/139648434 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd7c02bb50 con 0x55ab242c2f40 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd6b7fe640 1 -- 172.21.15.12:0/139648434 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbd7c036460 con 0x55ab242c2f40 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd95766740 10 client.15837 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd95766740 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd95766740 20 client.15837 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd95766740 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ab24340560 0x55ab24342980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd95766740 4 client.15837 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.251+0000 7fbd95766740 1 -- 172.21.15.12:0/139648434 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab2422f6c0 con 0x55ab24340560 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.252+0000 7fbd8b7fe640 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ab24340560 0x55ab24342980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.252+0000 7fbd6b7fe640 10 client.15837 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.252+0000 7fbd8b7fe640 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ab24340560 0x55ab24342980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc1d08c47be48892 server_cookie=eb907047013a58e in_seq=0 out_seq=0 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.252+0000 7fbd6b7fe640 1 -- 172.21.15.12:0/139648434 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab2422f6c0 con 0x55ab24340560 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.252+0000 7fbd6b7fe640 10 client.15837 handle_command_reply: tid=0 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.252+0000 7fbd95766740 1 client.15837 shutdown 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.252+0000 7fbd95766740 1 -- 172.21.15.12:0/139648434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 msgr2=0x55ab242d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.252+0000 7fbd95766740 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 0x55ab242d24d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c011d10 tx=0x7fbd7c011e50 comp rx=0 tx=0).stop 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd95766740 1 -- 172.21.15.12:0/139648434 shutdown_connections 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd95766740 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ab24340560 0x55ab24342980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd95766740 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab242c8aa0 0x55ab242d3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd95766740 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab242c3930 0x55ab242d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd6b7fe640 0 client.15837 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:11.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd95766740 1 --2- 172.21.15.12:0/139648434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab242c2f40 0x55ab242d24d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd95766740 1 -- 172.21.15.12:0/139648434 >> 172.21.15.12:0/139648434 conn(0x55ab242be5b0 msgr2=0x55ab242bcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:11.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd95766740 1 -- 172.21.15.12:0/139648434 shutdown_connections 2024-05-09T09:26:11.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd95766740 1 -- 172.21.15.12:0/139648434 wait complete. 2024-05-09T09:26:11.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd95766740 20 client.15837 trim_cache size 0 max 16384 2024-05-09T09:26:11.259 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:11.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd92a58640 1 -- 172.21.15.12:0/2305209272 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbd740a2b90 msgr2=0x7fbd740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd92a58640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbd740a2b90 0x7fbd740a5050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55ab242c4190 tx=0x7fbd84004ab0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd92a58640 1 -- 172.21.15.12:0/2305209272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd8c102a70 msgr2=0x7fbd8c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:11.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd92a58640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd8c102a70 0x7fbd8c073c20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009230 tx=0x7fbd800042f0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd92a58640 1 -- 172.21.15.12:0/2305209272 shutdown_connections 2024-05-09T09:26:11.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd92a58640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbd740a2b90 0x7fbd740a5050 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd92a58640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c103430 0x7fbd8c074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd92a58640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd8c102a70 0x7fbd8c073c20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd92a58640 1 --2- 172.21.15.12:0/2305209272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8c101880 0x7fbd8c0755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:11.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd92a58640 1 -- 172.21.15.12:0/2305209272 >> 172.21.15.12:0/2305209272 conn(0x7fbd8c0fce10 msgr2=0x7fbd8c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:11.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd92a58640 1 -- 172.21.15.12:0/2305209272 shutdown_connections 2024-05-09T09:26:11.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:11.253+0000 7fbd92a58640 1 -- 172.21.15.12:0/2305209272 wait complete. 2024-05-09T09:26:11.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:11 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3734100244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:11.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:11 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3734100244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:11.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:11 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3734100244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:12.264 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:26:12.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:26:12.265 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:26:12.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.382+0000 7f6d762a5640 1 -- 172.21.15.12:0/2328006287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d70102320 msgr2=0x7f6d70108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.382+0000 7f6d762a5640 1 --2- 172.21.15.12:0/2328006287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d70102320 0x7f6d70108bb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009ca0 tx=0x7f6d6802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.382+0000 7f6d762a5640 1 -- 172.21.15.12:0/2328006287 shutdown_connections 2024-05-09T09:26:12.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.382+0000 7f6d762a5640 1 --2- 172.21.15.12:0/2328006287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d70102320 0x7f6d70108bb0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.382+0000 7f6d762a5640 1 --2- 172.21.15.12:0/2328006287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d70101960 0x7f6d70101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.382+0000 7f6d762a5640 1 --2- 172.21.15.12:0/2328006287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d70100770 0x7f6d70100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.382+0000 7f6d762a5640 1 -- 172.21.15.12:0/2328006287 >> 172.21.15.12:0/2328006287 conn(0x7f6d700fbce0 msgr2=0x7f6d700fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:12.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.382+0000 7f6d762a5640 1 -- 172.21.15.12:0/2328006287 shutdown_connections 2024-05-09T09:26:12.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.382+0000 7f6d762a5640 1 -- 172.21.15.12:0/2328006287 wait complete. 2024-05-09T09:26:12.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.383+0000 7f6d762a5640 1 Processor -- start 2024-05-09T09:26:12.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.383+0000 7f6d762a5640 1 -- start start 2024-05-09T09:26:12.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d762a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d70100770 0x7f6d70196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d762a5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d70101960 0x7f6d70196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d762a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d70102320 0x7f6d7019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d762a5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7010c3d0 con 0x7f6d70101960 2024-05-09T09:26:12.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d762a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7010c250 con 0x7f6d70100770 2024-05-09T09:26:12.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d762a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7010c550 con 0x7f6d70102320 2024-05-09T09:26:12.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d6f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d70101960 0x7f6d70196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d6f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d70101960 0x7f6d70196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53702/0 (socket says 172.21.15.12:53702) 2024-05-09T09:26:12.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/9974584 learned_addr learned my addr 172.21.15.12:0/9974584 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:12.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d74842640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d70102320 0x7f6d7019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d6ffff640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d70100770 0x7f6d70196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/9974584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d70102320 msgr2=0x7f6d7019e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d6f7fe640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d70102320 0x7f6d7019e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/9974584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d70100770 msgr2=0x7f6d70196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d6f7fe640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d70100770 0x7f6d70196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/9974584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7019e690 con 0x7f6d70101960 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d6ffff640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d70100770 0x7f6d70196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d74842640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d70102320 0x7f6d7019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d6f7fe640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d70101960 0x7f6d70196fb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e9f0 tx=0x7f6d6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d6d7fa640 1 -- 172.21.15.12:0/9974584 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400cd80 con 0x7f6d70101960 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d6d7fa640 1 -- 172.21.15.12:0/9974584 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6d64010900 con 0x7f6d70101960 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.384+0000 7f6d762a5640 1 -- 172.21.15.12:0/9974584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7019e980 con 0x7f6d70101960 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.385+0000 7f6d6d7fa640 1 -- 172.21.15.12:0/9974584 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d640189e0 con 0x7f6d70101960 2024-05-09T09:26:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.385+0000 7f6d762a5640 1 -- 172.21.15.12:0/9974584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d7019eed0 con 0x7f6d70101960 2024-05-09T09:26:12.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.387+0000 7f6d6d7fa640 1 -- 172.21.15.12:0/9974584 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6d64010ab0 con 0x7f6d70101960 2024-05-09T09:26:12.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.387+0000 7f6d78fb4740 1 Processor -- start 2024-05-09T09:26:12.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.387+0000 7f6d78fb4740 1 -- start start 2024-05-09T09:26:12.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.387+0000 7f6d78fb4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc7caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.387+0000 7f6d78fb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc81bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.387+0000 7f6d78fb4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 0x55ec9dc846c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.387+0000 7f6d78fb4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ec9dbde990 con 0x55ec9dc82230 2024-05-09T09:26:12.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.387+0000 7f6d78fb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ec9db865b0 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.387+0000 7f6d78fb4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ec9dbda950 con 0x55ec9dc7c6d0 2024-05-09T09:26:12.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.387+0000 7f6d6f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 0x55ec9dc846c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d6f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 0x55ec9dc846c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53712/0 (socket says 172.21.15.12:53712) 2024-05-09T09:26:12.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/2512744434 learned_addr learned my addr 172.21.15.12:0/2512744434 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:12.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d6ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc81bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d74842640 1 --2- 172.21.15.12:0/2512744434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc7caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/2512744434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 msgr2=0x55ec9dc7caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d6f7fe640 1 --2- 172.21.15.12:0/2512744434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc7caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/2512744434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7d0c0 msgr2=0x55ec9dc81bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d6f7fe640 1 --2- 172.21.15.12:0/2512744434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc81bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/2512744434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec9d9ba7e0 con 0x55ec9dc82230 2024-05-09T09:26:12.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d6f7fe640 1 --2- 172.21.15.12:0/2512744434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 0x55ec9dc846c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6d640d5c10 tx=0x7f6d640d7690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce75a17f5fa8519f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:12.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d6d7fa640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6d580a2b90 0x7f6d580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d567fc640 1 -- 172.21.15.12:0/2512744434 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d640de070 con 0x55ec9dc82230 2024-05-09T09:26:12.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d74842640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6d580a2b90 0x7f6d580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d567fc640 1 -- 172.21.15.12:0/2512744434 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6d6409c450 con 0x55ec9dc82230 2024-05-09T09:26:12.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.388+0000 7f6d567fc640 1 -- 172.21.15.12:0/2512744434 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64091490 con 0x55ec9dc82230 2024-05-09T09:26:12.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.389+0000 7f6d6d7fa640 1 -- 172.21.15.12:0/9974584 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f6d640cf8c0 con 0x7f6d70101960 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.389+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2512744434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 msgr2=0x55ec9dc846c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.389+0000 7f6d74842640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6d580a2b90 0x7f6d580a5050 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6d68031be0 tx=0x7f6d680093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.389+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2512744434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 0x55ec9dc846c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6d640d5c10 tx=0x7f6d640d7690 comp rx=0 tx=0).stop 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.389+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2512744434 shutdown_connections 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.389+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2512744434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 0x55ec9dc846c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.389+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2512744434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc81bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.389+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2512744434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc7caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.389+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2512744434 >> 172.21.15.12:0/2512744434 conn(0x55ec9dc420a0 msgr2=0x55ec9dc75f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.389+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2512744434 shutdown_connections 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.389+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2512744434 wait complete. 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d78fb4740 1 -- start start 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d78fb4740 1 Processor -- start 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d78fb4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc86af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d78fb4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc87030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d78fb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc82230 0x55ec9dc8e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d78fb4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ec9dc851e0 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d78fb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ec9db99b40 con 0x55ec9dc82230 2024-05-09T09:26:12.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d78fb4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ec9dc85360 con 0x55ec9dc7c6d0 2024-05-09T09:26:12.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d78fb4740 10 client.0 _ll_register_callbacks cb 0x55ec9db8e0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:12.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d6f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc87030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d6f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc87030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53724/0 (socket says 172.21.15.12:53724) 2024-05-09T09:26:12.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/2341386785 learned_addr learned my addr 172.21.15.12:0/2341386785 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:12.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d74842640 1 --2- 172.21.15.12:0/2341386785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc82230 0x55ec9dc8e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.390+0000 7f6d6ffff640 1 --2- 172.21.15.12:0/2341386785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc86af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/2341386785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 msgr2=0x55ec9dc86af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d6f7fe640 1 --2- 172.21.15.12:0/2341386785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc86af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/2341386785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc82230 msgr2=0x55ec9dc8e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d6f7fe640 1 --2- 172.21.15.12:0/2341386785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc82230 0x55ec9dc8e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/2341386785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec9da1af70 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d6f7fe640 1 --2- 172.21.15.12:0/2341386785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc87030 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6d6409c600 tx=0x7f6d64091230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14390abf08fd705 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/2341386785 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64094fb0 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/2341386785 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f6d6409a050 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2341386785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6409c450 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/2341386785 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d640de020 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d78fb4740 10 client.15849 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:12.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2341386785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec9d8b2170 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.391+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/2341386785 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6d6409c450 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.392+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/2341386785 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6d6409bfb0 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.392+0000 7f6d78fb4740 10 client.15849 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:12.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.392+0000 7f6d78fb4740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:12.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.392+0000 7f6d78fb4740 20 client.15849 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:12.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.392+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2341386785 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ec9dc921d0 0x55ec9dc945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.392+0000 7f6d78fb4740 4 client.15849 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:12.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.392+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2341386785 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec9dbec0c0 con 0x55ec9dc921d0 2024-05-09T09:26:12.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.392+0000 7f6d6ffff640 1 --2- 172.21.15.12:0/2341386785 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ec9dc921d0 0x55ec9dc945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.392+0000 7f6d6ffff640 1 --2- 172.21.15.12:0/2341386785 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ec9dc921d0 0x55ec9dc945c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1142efca3e42107 server_cookie=fcd0233f96390e4 in_seq=0 out_seq=0 2024-05-09T09:26:12.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.392+0000 7f6d4f7fe640 10 client.15849 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:12.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.393+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/2341386785 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec9dbec0c0 con 0x55ec9dc921d0 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.393+0000 7f6d4f7fe640 10 client.15849 handle_command_reply: tid=0 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.393+0000 7f6d78fb4740 1 client.15849 shutdown 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.393+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2341386785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 msgr2=0x55ec9dc87030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.393+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2341386785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc87030 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6d6409c600 tx=0x7f6d64091230 comp rx=0 tx=0).stop 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.393+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2341386785 shutdown_connections 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.393+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2341386785 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ec9dc921d0 0x55ec9dc945c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.394+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2341386785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc82230 0x55ec9dc8e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.394+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2341386785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc87030 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.394+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2341386785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc86af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.394+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2341386785 >> 172.21.15.12:0/2341386785 conn(0x55ec9dc420a0 msgr2=0x55ec9dc75080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.394+0000 7f6d4f7fe640 0 client.15849 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.394+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2341386785 shutdown_connections 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.394+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2341386785 wait complete. 2024-05-09T09:26:12.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.394+0000 7f6d78fb4740 20 client.15849 trim_cache size 0 max 16384 2024-05-09T09:26:12.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.413+0000 7f6d78fb4740 1 Processor -- start 2024-05-09T09:26:12.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.413+0000 7f6d78fb4740 1 -- start start 2024-05-09T09:26:12.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.413+0000 7f6d78fb4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc78e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.413+0000 7f6d78fb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc8e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.413+0000 7f6d78fb4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 0x55ec9dc79370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.413+0000 7f6d78fb4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701a84b0 con 0x55ec9dc82230 2024-05-09T09:26:12.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.413+0000 7f6d78fb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701a9ff0 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.413+0000 7f6d78fb4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701a8b80 con 0x55ec9dc7c6d0 2024-05-09T09:26:12.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.413+0000 7f6d74842640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 0x55ec9dc79370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.413+0000 7f6d74842640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 0x55ec9dc79370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53730/0 (socket says 172.21.15.12:53730) 2024-05-09T09:26:12.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.413+0000 7f6d74842640 1 -- 172.21.15.12:0/2064176729 learned_addr learned my addr 172.21.15.12:0/2064176729 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:12.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.413+0000 7f6d6f7fe640 1 --2- 172.21.15.12:0/2064176729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc8e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.413+0000 7f6d6ffff640 1 --2- 172.21.15.12:0/2064176729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc78e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.414+0000 7f6d74842640 1 -- 172.21.15.12:0/2064176729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 msgr2=0x55ec9dc78e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.414+0000 7f6d74842640 1 --2- 172.21.15.12:0/2064176729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc78e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.414+0000 7f6d74842640 1 -- 172.21.15.12:0/2064176729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7d0c0 msgr2=0x55ec9dc8e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.414+0000 7f6d74842640 1 --2- 172.21.15.12:0/2064176729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc8e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.414+0000 7f6d74842640 1 -- 172.21.15.12:0/2064176729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec9dc74470 con 0x55ec9dc82230 2024-05-09T09:26:12.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.414+0000 7f6d74842640 1 --2- 172.21.15.12:0/2064176729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 0x55ec9dc79370 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6d68035000 tx=0x7f6d6803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223b051a98acaf92 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:12.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.414+0000 7f6d55ffb640 1 -- 172.21.15.12:0/2064176729 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68056070 con 0x55ec9dc82230 2024-05-09T09:26:12.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.414+0000 7f6d55ffb640 1 -- 172.21.15.12:0/2064176729 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6d68036040 con 0x55ec9dc82230 2024-05-09T09:26:12.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.414+0000 7f6d55ffb640 1 -- 172.21.15.12:0/2064176729 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6802fbe0 con 0x55ec9dc82230 2024-05-09T09:26:12.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.415+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2064176729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 msgr2=0x55ec9dc79370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.415+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2064176729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 0x55ec9dc79370 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6d68035000 tx=0x7f6d6803b000 comp rx=0 tx=0).stop 2024-05-09T09:26:12.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.415+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2064176729 shutdown_connections 2024-05-09T09:26:12.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.415+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2064176729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc82230 0x55ec9dc79370 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.415+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2064176729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc8e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.415+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/2064176729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc78e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.415+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2064176729 >> 172.21.15.12:0/2064176729 conn(0x55ec9dc420a0 msgr2=0x55ec9dc941e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.415+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2064176729 shutdown_connections 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.415+0000 7f6d78fb4740 1 -- 172.21.15.12:0/2064176729 wait complete. 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.415+0000 7f6d78fb4740 1 -- start start 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.415+0000 7f6d78fb4740 1 Processor -- start 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d78fb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc8b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d78fb4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc8bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d78fb4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc82230 0x55ec9dc8cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d78fb4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701a9920 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d78fb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701a9250 con 0x55ec9dc7c6d0 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d78fb4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701a6490 con 0x55ec9dc82230 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d78fb4740 10 client.0 _ll_register_callbacks cb 0x7f6d60009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d6f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc8bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d6f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc8bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53736/0 (socket says 172.21.15.12:53736) 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/3876337429 learned_addr learned my addr 172.21.15.12:0/3876337429 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d74842640 1 --2- 172.21.15.12:0/3876337429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc82230 0x55ec9dc8cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d6ffff640 1 --2- 172.21.15.12:0/3876337429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc8b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/3876337429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc82230 msgr2=0x55ec9dc8cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d6f7fe640 1 --2- 172.21.15.12:0/3876337429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc82230 0x55ec9dc8cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/3876337429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7c6d0 msgr2=0x55ec9dc8b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d6f7fe640 1 --2- 172.21.15.12:0/3876337429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc8b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d6f7fe640 1 -- 172.21.15.12:0/3876337429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec9dc8d0d0 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d6f7fe640 1 --2- 172.21.15.12:0/3876337429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc8bb40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400c6a0 tx=0x7f6d64018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585e3cf0c46e51e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/3876337429 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d640933e0 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/3876337429 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f6d640d5320 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.416+0000 7f6d78fb4740 1 -- 172.21.15.12:0/3876337429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d68036040 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.417+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/3876337429 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d64093760 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.417+0000 7f6d78fb4740 10 client.15855 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.417+0000 7f6d78fb4740 1 -- 172.21.15.12:0/3876337429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec9dcf8660 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.417+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/3876337429 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6d68036040 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.417+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/3876337429 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6d640cd040 con 0x55ec9dc7d0c0 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.417+0000 7f6d78fb4740 10 client.15855 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.417+0000 7f6d78fb4740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.417+0000 7f6d78fb4740 20 client.15855 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.417+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/3876337429 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ec9dcf8b90 0x55ec9dcfaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.417+0000 7f6d78fb4740 4 client.15855 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.417+0000 7f6d78fb4740 1 -- 172.21.15.12:0/3876337429 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ec9dbe6410 con 0x55ec9dcf8b90 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.417+0000 7f6d6ffff640 1 --2- 172.21.15.12:0/3876337429 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ec9dcf8b90 0x55ec9dcfaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.418+0000 7f6d6ffff640 1 --2- 172.21.15.12:0/3876337429 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ec9dcf8b90 0x55ec9dcfaff0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7e7713517eceb0 server_cookie=f926326f90dd746e in_seq=0 out_seq=0 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.418+0000 7f6d4f7fe640 10 client.15855 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.419+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/3876337429 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ec9dbe6410 con 0x55ec9dcf8b90 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.419+0000 7f6d4f7fe640 10 client.15855 handle_command_reply: tid=0 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.419+0000 7f6d78fb4740 1 client.15855 shutdown 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.419+0000 7f6d78fb4740 1 -- 172.21.15.12:0/3876337429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 msgr2=0x55ec9dc8bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.419+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/3876337429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc8bb40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400c6a0 tx=0x7f6d64018420 comp rx=0 tx=0).stop 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d78fb4740 1 -- 172.21.15.12:0/3876337429 shutdown_connections 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/3876337429 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ec9dcf8b90 0x55ec9dcfaff0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/3876337429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9dc82230 0x55ec9dc8cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/3876337429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9dc7d0c0 0x55ec9dc8bb40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d78fb4740 1 --2- 172.21.15.12:0/3876337429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec9dc7c6d0 0x55ec9dc8b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d78fb4740 1 -- 172.21.15.12:0/3876337429 >> 172.21.15.12:0/3876337429 conn(0x55ec9dc420a0 msgr2=0x55ec9dc747f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d4f7fe640 0 client.15855 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d78fb4740 1 -- 172.21.15.12:0/3876337429 shutdown_connections 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d78fb4740 1 -- 172.21.15.12:0/3876337429 wait complete. 2024-05-09T09:26:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d78fb4740 20 client.15855 trim_cache size 0 max 16384 2024-05-09T09:26:12.425 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:26:12.425 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T09:26:12.425 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "5bf48514-a6d6-4ce2-b649-69cd6e7e2759", 2024-05-09T09:26:12.425 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T09:26:12.425 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:26:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d762a5640 1 -- 172.21.15.12:0/9974584 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6d580a2b90 msgr2=0x7f6d580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d762a5640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6d580a2b90 0x7f6d580a5050 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6d68031be0 tx=0x7f6d680093f0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d762a5640 1 -- 172.21.15.12:0/9974584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d70101960 msgr2=0x7f6d70196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.420+0000 7f6d762a5640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d70101960 0x7f6d70196fb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e9f0 tx=0x7f6d6400eeb0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.421+0000 7f6d762a5640 1 -- 172.21.15.12:0/9974584 shutdown_connections 2024-05-09T09:26:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.421+0000 7f6d762a5640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6d580a2b90 0x7f6d580a5050 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.421+0000 7f6d762a5640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d70102320 0x7f6d7019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.421+0000 7f6d762a5640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d70101960 0x7f6d70196fb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.421+0000 7f6d762a5640 1 --2- 172.21.15.12:0/9974584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d70100770 0x7f6d70196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.421+0000 7f6d762a5640 1 -- 172.21.15.12:0/9974584 >> 172.21.15.12:0/9974584 conn(0x7f6d700fbce0 msgr2=0x7f6d70104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:12.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.421+0000 7f6d762a5640 1 -- 172.21.15.12:0/9974584 shutdown_connections 2024-05-09T09:26:12.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.421+0000 7f6d762a5640 1 -- 172.21.15.12:0/9974584 wait complete. 2024-05-09T09:26:12.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:26:12.435 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:26:12.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.546+0000 7f852fff4640 1 -- 172.21.15.12:0/2711127101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8528100770 msgr2=0x7f8528100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.546+0000 7f852fff4640 1 --2- 172.21.15.12:0/2711127101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8528100770 0x7f8528100b90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8524009900 tx=0x7f852402a410 comp rx=0 tx=0).stop 2024-05-09T09:26:12.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.546+0000 7f852fff4640 1 -- 172.21.15.12:0/2711127101 shutdown_connections 2024-05-09T09:26:12.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.546+0000 7f852fff4640 1 --2- 172.21.15.12:0/2711127101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8528102320 0x7f8528108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.546+0000 7f852fff4640 1 --2- 172.21.15.12:0/2711127101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8528101960 0x7f8528101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.546+0000 7f852fff4640 1 --2- 172.21.15.12:0/2711127101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8528100770 0x7f8528100b90 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.546+0000 7f852fff4640 1 -- 172.21.15.12:0/2711127101 >> 172.21.15.12:0/2711127101 conn(0x7f85280fbce0 msgr2=0x7f85280fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:12.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.546+0000 7f852fff4640 1 -- 172.21.15.12:0/2711127101 shutdown_connections 2024-05-09T09:26:12.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.546+0000 7f852fff4640 1 -- 172.21.15.12:0/2711127101 wait complete. 2024-05-09T09:26:12.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.546+0000 7f852fff4640 1 Processor -- start 2024-05-09T09:26:12.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852fff4640 1 -- start start 2024-05-09T09:26:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852fff4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8528100770 0x7f8528197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852fff4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8528101960 0x7f8528198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852fff4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8528102320 0x7f852819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852fff4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f852810c3d0 con 0x7f8528100770 2024-05-09T09:26:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852fff4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f852810c250 con 0x7f8528101960 2024-05-09T09:26:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852fff4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f852810c550 con 0x7f8528102320 2024-05-09T09:26:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852dd90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8528100770 0x7f8528197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852dd90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8528100770 0x7f8528197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53750/0 (socket says 172.21.15.12:53750) 2024-05-09T09:26:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852dd90640 1 -- 172.21.15.12:0/2853386351 learned_addr learned my addr 172.21.15.12:0/2853386351 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852e591640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8528102320 0x7f852819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852d58f640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8528101960 0x7f8528198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852dd90640 1 -- 172.21.15.12:0/2853386351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8528102320 msgr2=0x7f852819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852dd90640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8528102320 0x7f852819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852dd90640 1 -- 172.21.15.12:0/2853386351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8528101960 msgr2=0x7f8528198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852dd90640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8528101960 0x7f8528198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852dd90640 1 -- 172.21.15.12:0/2853386351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852819f830 con 0x7f8528100770 2024-05-09T09:26:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.547+0000 7f852d58f640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8528101960 0x7f8528198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.548+0000 7f852dd90640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8528100770 0x7f8528197c20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8524009230 tx=0x7f8524031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.548+0000 7f851effd640 1 -- 172.21.15.12:0/2853386351 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852403f030 con 0x7f8528100770 2024-05-09T09:26:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.548+0000 7f852fff4640 1 -- 172.21.15.12:0/2853386351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852819fac0 con 0x7f8528100770 2024-05-09T09:26:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.548+0000 7f851effd640 1 -- 172.21.15.12:0/2853386351 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8524003ac0 con 0x7f8528100770 2024-05-09T09:26:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.548+0000 7f851effd640 1 -- 172.21.15.12:0/2853386351 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524035070 con 0x7f8528100770 2024-05-09T09:26:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.548+0000 7f852fff4640 1 -- 172.21.15.12:0/2853386351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f852819ffb0 con 0x7f8528100770 2024-05-09T09:26:12.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.550+0000 7f851effd640 1 -- 172.21.15.12:0/2853386351 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8524042030 con 0x7f8528100770 2024-05-09T09:26:12.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.550+0000 7f8532dd6740 1 Processor -- start 2024-05-09T09:26:12.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f8532dd6740 1 -- start start 2024-05-09T09:26:12.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f8532dd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106cf30 0x56383106d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f8532dd6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106d920 0x563831072420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f8532dd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563831072a90 0x563831074f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f8532dd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563830fd1d00 con 0x56383106cf30 2024-05-09T09:26:12.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f8532dd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563830f794c0 con 0x563831072a90 2024-05-09T09:26:12.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f8532dd6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563830fcdcc0 con 0x56383106d920 2024-05-09T09:26:12.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f852e591640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106cf30 0x56383106d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f851effd640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85140a2b90 0x7f85140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f852e591640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106cf30 0x56383106d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53756/0 (socket says 172.21.15.12:53756) 2024-05-09T09:26:12.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f852e591640 1 -- 172.21.15.12:0/2225380709 learned_addr learned my addr 172.21.15.12:0/2225380709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:12.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f852d58f640 1 --2- 172.21.15.12:0/2225380709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563831072a90 0x563831074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f851effd640 1 -- 172.21.15.12:0/2853386351 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f85240ef7b0 con 0x7f8528100770 2024-05-09T09:26:12.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f852e591640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85140a2b90 0x7f85140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.551+0000 7f852dd90640 1 --2- 172.21.15.12:0/2225380709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106d920 0x563831072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.552+0000 7f852e591640 1 -- 172.21.15.12:0/2225380709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106d920 msgr2=0x563831072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.552+0000 7f852e591640 1 --2- 172.21.15.12:0/2225380709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106d920 0x563831072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.552+0000 7f852e591640 1 -- 172.21.15.12:0/2225380709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563831072a90 msgr2=0x563831074f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.552+0000 7f852e591640 1 --2- 172.21.15.12:0/2225380709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563831072a90 0x563831074f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.552+0000 7f852e591640 1 -- 172.21.15.12:0/2225380709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563830daf7d0 con 0x56383106cf30 2024-05-09T09:26:12.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.552+0000 7f852dd90640 1 --2- 172.21.15.12:0/2225380709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106d920 0x563831072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:12.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.552+0000 7f852e591640 1 --2- 172.21.15.12:0/2225380709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106cf30 0x56383106d350 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8520005f40 tx=0x7f8520014f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746f0448f74a57c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:12.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.552+0000 7f8513fff640 1 -- 172.21.15.12:0/2225380709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520012d90 con 0x56383106cf30 2024-05-09T09:26:12.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.552+0000 7f8513fff640 1 -- 172.21.15.12:0/2225380709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8520023070 con 0x56383106cf30 2024-05-09T09:26:12.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.552+0000 7f8513fff640 1 -- 172.21.15.12:0/2225380709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852000e360 con 0x56383106cf30 2024-05-09T09:26:12.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.552+0000 7f852e591640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85140a2b90 0x7f85140a5050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8520022000 tx=0x7f8520020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:12.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.553+0000 7f8532dd6740 1 -- 172.21.15.12:0/2225380709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106cf30 msgr2=0x56383106d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.553+0000 7f8532dd6740 1 --2- 172.21.15.12:0/2225380709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106cf30 0x56383106d350 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8520005f40 tx=0x7f8520014f70 comp rx=0 tx=0).stop 2024-05-09T09:26:12.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.553+0000 7f8532dd6740 1 -- 172.21.15.12:0/2225380709 shutdown_connections 2024-05-09T09:26:12.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.553+0000 7f8532dd6740 1 --2- 172.21.15.12:0/2225380709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563831072a90 0x563831074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.553+0000 7f8532dd6740 1 --2- 172.21.15.12:0/2225380709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106d920 0x563831072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.553+0000 7f8532dd6740 1 --2- 172.21.15.12:0/2225380709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106cf30 0x56383106d350 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.553+0000 7f8532dd6740 1 -- 172.21.15.12:0/2225380709 >> 172.21.15.12:0/2225380709 conn(0x5638310685a0 msgr2=0x5638310689a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:12.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.553+0000 7f8532dd6740 1 -- 172.21.15.12:0/2225380709 shutdown_connections 2024-05-09T09:26:12.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.553+0000 7f8532dd6740 1 -- 172.21.15.12:0/2225380709 wait complete. 2024-05-09T09:26:12.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.553+0000 7f8532dd6740 1 -- start start 2024-05-09T09:26:12.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.553+0000 7f8532dd6740 1 Processor -- start 2024-05-09T09:26:12.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f8532dd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56383106cf30 0x563831077380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f8532dd6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106d920 0x5638310778c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f8532dd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 0x56383107e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f8532dd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563831075a40 con 0x563831072a90 2024-05-09T09:26:12.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f8532dd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563830f8cb90 con 0x56383106cf30 2024-05-09T09:26:12.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f8532dd6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563831075bc0 con 0x56383106d920 2024-05-09T09:26:12.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f8532dd6740 10 client.0 _ll_register_callbacks cb 0x563830fc8da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:12.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f852e591640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 0x56383107e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f852e591640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 0x56383107e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53770/0 (socket says 172.21.15.12:53770) 2024-05-09T09:26:12.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f852e591640 1 -- 172.21.15.12:0/2928610462 learned_addr learned my addr 172.21.15.12:0/2928610462 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:12.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f852d58f640 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106d920 0x5638310778c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f852dd90640 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56383106cf30 0x563831077380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f852e591640 1 -- 172.21.15.12:0/2928610462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106d920 msgr2=0x5638310778c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f852e591640 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106d920 0x5638310778c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f852e591640 1 -- 172.21.15.12:0/2928610462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56383106cf30 msgr2=0x563831077380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f852e591640 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56383106cf30 0x563831077380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f852e591640 1 -- 172.21.15.12:0/2928610462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563830ce53f0 con 0x563831072a90 2024-05-09T09:26:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.554+0000 7f852dd90640 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56383106cf30 0x563831077380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.555+0000 7f852e591640 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 0x56383107e900 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8520005f10 tx=0x7f8520012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a72b1e8d65d9d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.555+0000 7f85117fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.555+0000 7f85117fa640 1 -- 172.21.15.12:0/2928610462 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852000e360 con 0x563831072a90 2024-05-09T09:26:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.555+0000 7f85117fa640 1 -- 172.21.15.12:0/2928610462 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f85200144d0 con 0x563831072a90 2024-05-09T09:26:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.555+0000 7f8532dd6740 1 -- 172.21.15.12:0/2928610462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8520023070 con 0x563831072a90 2024-05-09T09:26:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.555+0000 7f85117fa640 1 -- 172.21.15.12:0/2928610462 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f852003b050 con 0x563831072a90 2024-05-09T09:26:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.555+0000 7f8532dd6740 10 client.15867 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.555+0000 7f8532dd6740 1 -- 172.21.15.12:0/2928610462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563830ca50f0 con 0x563831072a90 2024-05-09T09:26:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.555+0000 7f85117fa640 1 -- 172.21.15.12:0/2928610462 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8520023070 con 0x563831072a90 2024-05-09T09:26:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.555+0000 7f85117fa640 1 -- 172.21.15.12:0/2928610462 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f852000f040 con 0x563831072a90 2024-05-09T09:26:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.556+0000 7f8532dd6740 10 client.15867 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.556+0000 7f8532dd6740 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.556+0000 7f8532dd6740 20 client.15867 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.556+0000 7f8532dd6740 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x563831082ee0 0x5638310852d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.556+0000 7f8532dd6740 4 client.15867 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.556+0000 7f8532dd6740 1 -- 172.21.15.12:0/2928610462 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563830fdf430 con 0x563831082ee0 2024-05-09T09:26:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.556+0000 7f852dd90640 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x563831082ee0 0x5638310852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.556+0000 7f85117fa640 10 client.15867 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.556+0000 7f852dd90640 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x563831082ee0 0x5638310852d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a10ba8298474f42 server_cookie=492c8c11728ec337 in_seq=0 out_seq=0 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f85117fa640 1 -- 172.21.15.12:0/2928610462 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563830fdf430 con 0x563831082ee0 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f85117fa640 10 client.15867 handle_command_reply: tid=0 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f8532dd6740 1 client.15867 shutdown 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f8532dd6740 1 -- 172.21.15.12:0/2928610462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 msgr2=0x56383107e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f8532dd6740 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 0x56383107e900 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8520005f10 tx=0x7f8520012910 comp rx=0 tx=0).stop 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f8532dd6740 1 -- 172.21.15.12:0/2928610462 shutdown_connections 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f8532dd6740 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x563831082ee0 0x5638310852d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f8532dd6740 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 0x56383107e900 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f8532dd6740 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106d920 0x5638310778c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f85117fa640 0 client.15867 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f8532dd6740 1 --2- 172.21.15.12:0/2928610462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56383106cf30 0x563831077380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f8532dd6740 1 -- 172.21.15.12:0/2928610462 >> 172.21.15.12:0/2928610462 conn(0x5638310685a0 msgr2=0x5638310677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f8532dd6740 1 -- 172.21.15.12:0/2928610462 shutdown_connections 2024-05-09T09:26:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f8532dd6740 1 -- 172.21.15.12:0/2928610462 wait complete. 2024-05-09T09:26:12.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.557+0000 7f8532dd6740 20 client.15867 trim_cache size 0 max 16384 2024-05-09T09:26:12.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.579+0000 7f8532dd6740 1 Processor -- start 2024-05-09T09:26:12.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.579+0000 7f8532dd6740 1 -- start start 2024-05-09T09:26:12.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.579+0000 7f8532dd6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106cf30 0x5638310852d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.579+0000 7f8532dd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106d920 0x56383107e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.579+0000 7f8532dd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563831072a90 0x563831067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.579+0000 7f8532dd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f85281aa610 con 0x56383106d920 2024-05-09T09:26:12.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.579+0000 7f8532dd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85281ac150 con 0x563831072a90 2024-05-09T09:26:12.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.579+0000 7f8532dd6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f85281aace0 con 0x56383106cf30 2024-05-09T09:26:12.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.579+0000 7f852d58f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106d920 0x56383107e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.579+0000 7f852d58f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106d920 0x56383107e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53782/0 (socket says 172.21.15.12:53782) 2024-05-09T09:26:12.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.579+0000 7f852d58f640 1 -- 172.21.15.12:0/1408730477 learned_addr learned my addr 172.21.15.12:0/1408730477 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.579+0000 7f852dd90640 1 --2- 172.21.15.12:0/1408730477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106cf30 0x5638310852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.580+0000 7f852d58f640 1 -- 172.21.15.12:0/1408730477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106cf30 msgr2=0x5638310852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.580+0000 7f852e591640 1 --2- 172.21.15.12:0/1408730477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563831072a90 0x563831067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.580+0000 7f852d58f640 1 --2- 172.21.15.12:0/1408730477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106cf30 0x5638310852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.580+0000 7f852d58f640 1 -- 172.21.15.12:0/1408730477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563831072a90 msgr2=0x563831067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.580+0000 7f852d58f640 1 --2- 172.21.15.12:0/1408730477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563831072a90 0x563831067660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.580+0000 7f852d58f640 1 -- 172.21.15.12:0/1408730477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56383106cb70 con 0x56383106d920 2024-05-09T09:26:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.580+0000 7f852e591640 1 --2- 172.21.15.12:0/1408730477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563831072a90 0x563831067660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.580+0000 7f852d58f640 1 --2- 172.21.15.12:0/1408730477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106d920 0x56383107e8c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8518009e00 tx=0x7f8518012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cb4450dab9fb1b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.580+0000 7f85137fe640 1 -- 172.21.15.12:0/1408730477 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518004020 con 0x56383106d920 2024-05-09T09:26:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.580+0000 7f85137fe640 1 -- 172.21.15.12:0/1408730477 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f85180041c0 con 0x56383106d920 2024-05-09T09:26:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.580+0000 7f85137fe640 1 -- 172.21.15.12:0/1408730477 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518011830 con 0x56383106d920 2024-05-09T09:26:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.581+0000 7f8532dd6740 1 -- 172.21.15.12:0/1408730477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106d920 msgr2=0x56383107e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.581+0000 7f8532dd6740 1 --2- 172.21.15.12:0/1408730477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106d920 0x56383107e8c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8518009e00 tx=0x7f8518012820 comp rx=0 tx=0).stop 2024-05-09T09:26:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.581+0000 7f8532dd6740 1 -- 172.21.15.12:0/1408730477 shutdown_connections 2024-05-09T09:26:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.581+0000 7f8532dd6740 1 --2- 172.21.15.12:0/1408730477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563831072a90 0x563831067660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.581+0000 7f8532dd6740 1 --2- 172.21.15.12:0/1408730477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56383106d920 0x56383107e8c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.581+0000 7f8532dd6740 1 --2- 172.21.15.12:0/1408730477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106cf30 0x5638310852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.581+0000 7f8532dd6740 1 -- 172.21.15.12:0/1408730477 >> 172.21.15.12:0/1408730477 conn(0x5638310685a0 msgr2=0x563831066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.581+0000 7f8532dd6740 1 -- 172.21.15.12:0/1408730477 shutdown_connections 2024-05-09T09:26:12.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.581+0000 7f8532dd6740 1 -- 172.21.15.12:0/1408730477 wait complete. 2024-05-09T09:26:12.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.581+0000 7f8532dd6740 1 -- start start 2024-05-09T09:26:12.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.581+0000 7f8532dd6740 1 Processor -- start 2024-05-09T09:26:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f8532dd6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106cf30 0x56383107c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f8532dd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56383106d920 0x56383107cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f8532dd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 0x56383107db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f8532dd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f85281aba80 con 0x563831072a90 2024-05-09T09:26:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f8532dd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85281ab3b0 con 0x56383106d920 2024-05-09T09:26:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f8532dd6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f85281a85f0 con 0x56383106cf30 2024-05-09T09:26:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f8532dd6740 10 client.0 _ll_register_callbacks cb 0x7f8518009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f852e591640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 0x56383107db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f852e591640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 0x56383107db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53786/0 (socket says 172.21.15.12:53786) 2024-05-09T09:26:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f852e591640 1 -- 172.21.15.12:0/4202303713 learned_addr learned my addr 172.21.15.12:0/4202303713 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f852dd90640 1 --2- 172.21.15.12:0/4202303713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106cf30 0x56383107c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f852d58f640 1 --2- 172.21.15.12:0/4202303713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56383106d920 0x56383107cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f852e591640 1 -- 172.21.15.12:0/4202303713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106cf30 msgr2=0x56383107c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f852e591640 1 --2- 172.21.15.12:0/4202303713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106cf30 0x56383107c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f852e591640 1 -- 172.21.15.12:0/4202303713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56383106d920 msgr2=0x56383107cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:12.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f852e591640 1 --2- 172.21.15.12:0/4202303713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56383106d920 0x56383107cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f852e591640 1 -- 172.21.15.12:0/4202303713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56383107e110 con 0x563831072a90 2024-05-09T09:26:12.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f852e591640 1 --2- 172.21.15.12:0/4202303713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 0x56383107db20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f85200142b0 tx=0x7f852001d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ea5fb504ba03ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:12.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f85117fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:12.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f85117fa640 1 -- 172.21.15.12:0/4202303713 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852002b020 con 0x563831072a90 2024-05-09T09:26:12.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f8532dd6740 1 -- 172.21.15.12:0/4202303713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85180041c0 con 0x563831072a90 2024-05-09T09:26:12.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.582+0000 7f85117fa640 1 -- 172.21.15.12:0/4202303713 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f8520038070 con 0x563831072a90 2024-05-09T09:26:12.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.583+0000 7f85117fa640 1 -- 172.21.15.12:0/4202303713 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85200144d0 con 0x563831072a90 2024-05-09T09:26:12.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.583+0000 7f8532dd6740 10 client.15873 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:12.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.583+0000 7f8532dd6740 1 -- 172.21.15.12:0/4202303713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5638310e9990 con 0x563831072a90 2024-05-09T09:26:12.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.583+0000 7f85117fa640 1 -- 172.21.15.12:0/4202303713 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f85180041c0 con 0x563831072a90 2024-05-09T09:26:12.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.583+0000 7f85117fa640 1 -- 172.21.15.12:0/4202303713 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f852000e6b0 con 0x563831072a90 2024-05-09T09:26:12.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.583+0000 7f8532dd6740 10 client.15873 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:12.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.583+0000 7f8532dd6740 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:12.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.583+0000 7f8532dd6740 20 client.15873 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:12.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.583+0000 7f8532dd6740 1 --2- 172.21.15.12:0/4202303713 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5638310ea3b0 0x5638310ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:12.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.583+0000 7f8532dd6740 4 client.15873 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:26:12.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.583+0000 7f8532dd6740 1 -- 172.21.15.12:0/4202303713 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563830fd9780 con 0x5638310ea3b0 2024-05-09T09:26:12.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.583+0000 7f852dd90640 1 --2- 172.21.15.12:0/4202303713 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5638310ea3b0 0x5638310ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:12.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f85117fa640 10 client.15873 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:12.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f852dd90640 1 --2- 172.21.15.12:0/4202303713 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5638310ea3b0 0x5638310ec7d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b0f3fb660323b server_cookie=5e1ac013e40728e8 in_seq=0 out_seq=0 2024-05-09T09:26:12.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f85117fa640 1 -- 172.21.15.12:0/4202303713 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563830fd9780 con 0x5638310ea3b0 2024-05-09T09:26:12.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f85117fa640 10 client.15873 handle_command_reply: tid=0 2024-05-09T09:26:12.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f8532dd6740 1 client.15873 shutdown 2024-05-09T09:26:12.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f8532dd6740 1 -- 172.21.15.12:0/4202303713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 msgr2=0x56383107db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f8532dd6740 1 --2- 172.21.15.12:0/4202303713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 0x56383107db20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f85200142b0 tx=0x7f852001d9c0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f8532dd6740 1 -- 172.21.15.12:0/4202303713 shutdown_connections 2024-05-09T09:26:12.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f8532dd6740 1 --2- 172.21.15.12:0/4202303713 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5638310ea3b0 0x5638310ec7d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f8532dd6740 1 --2- 172.21.15.12:0/4202303713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563831072a90 0x56383107db20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f8532dd6740 1 --2- 172.21.15.12:0/4202303713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56383106d920 0x56383107cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f85117fa640 0 client.15873 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:12.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f8532dd6740 1 --2- 172.21.15.12:0/4202303713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383106cf30 0x56383107c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f8532dd6740 1 -- 172.21.15.12:0/4202303713 >> 172.21.15.12:0/4202303713 conn(0x5638310685a0 msgr2=0x56383106bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:12.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f8532dd6740 1 -- 172.21.15.12:0/4202303713 shutdown_connections 2024-05-09T09:26:12.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.584+0000 7f8532dd6740 1 -- 172.21.15.12:0/4202303713 wait complete. 2024-05-09T09:26:12.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.585+0000 7f8532dd6740 20 client.15873 trim_cache size 0 max 16384 2024-05-09T09:26:12.599 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:26:12.599 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:26:12.599 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T09:26:12.599 INFO:teuthology.orchestra.run.smithi012.stdout: "5bf48514-a6d6-4ce2-b649-69cd6e7e2759": { 2024-05-09T09:26:12.599 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T09:26:12.599 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "5bf48514-a6d6-4ce2-b649-69cd6e7e2759", 2024-05-09T09:26:12.599 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T09:26:12.599 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:26:12.599 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:26:12.599 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:26:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.585+0000 7f852fff4640 1 -- 172.21.15.12:0/2853386351 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85140a2b90 msgr2=0x7f85140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.585+0000 7f852fff4640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85140a2b90 0x7f85140a5050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8520022000 tx=0x7f8520020000 comp rx=0 tx=0).stop 2024-05-09T09:26:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.585+0000 7f852fff4640 1 -- 172.21.15.12:0/2853386351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8528100770 msgr2=0x7f8528197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.585+0000 7f852fff4640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8528100770 0x7f8528197c20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8524009230 tx=0x7f8524031ac0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.585+0000 7f852fff4640 1 -- 172.21.15.12:0/2853386351 shutdown_connections 2024-05-09T09:26:12.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.585+0000 7f852fff4640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85140a2b90 0x7f85140a5050 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.585+0000 7f852fff4640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8528102320 0x7f852819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.585+0000 7f852fff4640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8528101960 0x7f8528198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.585+0000 7f852fff4640 1 --2- 172.21.15.12:0/2853386351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8528100770 0x7f8528197c20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:12.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.585+0000 7f852fff4640 1 -- 172.21.15.12:0/2853386351 >> 172.21.15.12:0/2853386351 conn(0x7f85280fbce0 msgr2=0x7f85280fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:12.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.585+0000 7f852fff4640 1 -- 172.21.15.12:0/2853386351 shutdown_connections 2024-05-09T09:26:12.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:12.585+0000 7f852fff4640 1 -- 172.21.15.12:0/2853386351 wait complete. 2024-05-09T09:26:12.601 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:12 smithi012 ceph-mon[30140]: pgmap v386: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:12.602 INFO:tasks.cephfs.filesystem:scrub status for tag:5bf48514-a6d6-4ce2-b649-69cd6e7e2759 - {'path': '/', 'tag': '5bf48514-a6d6-4ce2-b649-69cd6e7e2759', 'options': 'recursive,force'} 2024-05-09T09:26:12.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:12 smithi170 ceph-mon[33632]: pgmap v386: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:12.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:12 smithi016 ceph-mon[33335]: pgmap v386: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:13.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:13 smithi012 ceph-mon[30140]: scrub queued for path: / 2024-05-09T09:26:13.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:13 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:13.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:13 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:26:13.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:13 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:13.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:13 smithi012 ceph-mon[30140]: pgmap v387: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:13.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:13 smithi170 ceph-mon[33632]: scrub queued for path: / 2024-05-09T09:26:13.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:13 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:13.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:13 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:26:13.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:13 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:13.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:13 smithi170 ceph-mon[33632]: pgmap v387: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:13.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:13 smithi016 ceph-mon[33335]: scrub queued for path: / 2024-05-09T09:26:13.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:13 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:13.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:13 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:26:13.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:13 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:13.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:13 smithi016 ceph-mon[33335]: pgmap v387: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:15.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:15 smithi012 ceph-mon[30140]: pgmap v388: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T09:26:15.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:15 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:26:15.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:15 smithi170 ceph-mon[33632]: pgmap v388: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T09:26:15.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:15 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:26:16.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:15 smithi016 ceph-mon[33335]: pgmap v388: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T09:26:16.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:15 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:26:17.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:17 smithi012 ceph-mon[30140]: pgmap v389: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T09:26:17.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:17 smithi170 ceph-mon[33632]: pgmap v389: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T09:26:18.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:17 smithi016 ceph-mon[33335]: pgmap v389: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T09:26:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.971+0000 7f9b9357e640 1 -- 172.21.15.12:0/401226020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c100790 msgr2=0x7f9b8c100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.971+0000 7f9b9357e640 1 --2- 172.21.15.12:0/401226020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c100790 0x7f9b8c100bb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009900 tx=0x7f9b8802a410 comp rx=0 tx=0).stop 2024-05-09T09:26:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.971+0000 7f9b9357e640 1 -- 172.21.15.12:0/401226020 shutdown_connections 2024-05-09T09:26:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.971+0000 7f9b9357e640 1 --2- 172.21.15.12:0/401226020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b8c102340 0x7f9b8c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.971+0000 7f9b9357e640 1 --2- 172.21.15.12:0/401226020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c101980 0x7f9b8c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.971+0000 7f9b9357e640 1 --2- 172.21.15.12:0/401226020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c100790 0x7f9b8c100bb0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.971+0000 7f9b9357e640 1 -- 172.21.15.12:0/401226020 >> 172.21.15.12:0/401226020 conn(0x7f9b8c0fbd00 msgr2=0x7f9b8c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.971+0000 7f9b9357e640 1 -- 172.21.15.12:0/401226020 shutdown_connections 2024-05-09T09:26:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.971+0000 7f9b9357e640 1 -- 172.21.15.12:0/401226020 wait complete. 2024-05-09T09:26:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.972+0000 7f9b9357e640 1 Processor -- start 2024-05-09T09:26:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.972+0000 7f9b9357e640 1 -- start start 2024-05-09T09:26:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.972+0000 7f9b9357e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c100790 0x7f9b8c069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.972+0000 7f9b9357e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c101980 0x7f9b8c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.972+0000 7f9b9357e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b8c102340 0x7f9b8c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.972+0000 7f9b9357e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c10c440 con 0x7f9b8c101980 2024-05-09T09:26:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.972+0000 7f9b9357e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c10c2c0 con 0x7f9b8c100790 2024-05-09T09:26:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.972+0000 7f9b9357e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c10c5c0 con 0x7f9b8c102340 2024-05-09T09:26:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.972+0000 7f9b90b19640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c101980 0x7f9b8c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.972+0000 7f9b90b19640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c101980 0x7f9b8c19c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53810/0 (socket says 172.21.15.12:53810) 2024-05-09T09:26:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.972+0000 7f9b90b19640 1 -- 172.21.15.12:0/1152460713 learned_addr learned my addr 172.21.15.12:0/1152460713 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.972+0000 7f9b91b1b640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b8c102340 0x7f9b8c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.972+0000 7f9b9131a640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c100790 0x7f9b8c069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.973+0000 7f9b90b19640 1 -- 172.21.15.12:0/1152460713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b8c102340 msgr2=0x7f9b8c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.973+0000 7f9b90b19640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b8c102340 0x7f9b8c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.973+0000 7f9b90b19640 1 -- 172.21.15.12:0/1152460713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c100790 msgr2=0x7f9b8c069a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.973+0000 7f9b90b19640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c100790 0x7f9b8c069a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.973+0000 7f9b90b19640 1 -- 172.21.15.12:0/1152460713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c19f7d0 con 0x7f9b8c101980 2024-05-09T09:26:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.973+0000 7f9b9131a640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c100790 0x7f9b8c069a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.973+0000 7f9b90b19640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c101980 0x7f9b8c19c8a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000d950 tx=0x7f9b8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.973+0000 7f9b7e7fc640 1 -- 172.21.15.12:0/1152460713 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80017070 con 0x7f9b8c101980 2024-05-09T09:26:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.973+0000 7f9b7e7fc640 1 -- 172.21.15.12:0/1152460713 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9b80009c30 con 0x7f9b8c101980 2024-05-09T09:26:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.973+0000 7f9b9357e640 1 -- 172.21.15.12:0/1152460713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c19fa60 con 0x7f9b8c101980 2024-05-09T09:26:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.973+0000 7f9b7e7fc640 1 -- 172.21.15.12:0/1152460713 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8000f6b0 con 0x7f9b8c101980 2024-05-09T09:26:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.973+0000 7f9b9357e640 1 -- 172.21.15.12:0/1152460713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c19ffb0 con 0x7f9b8c101980 2024-05-09T09:26:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.975+0000 7f9b7e7fc640 1 -- 172.21.15.12:0/1152460713 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9b80010a50 con 0x7f9b8c101980 2024-05-09T09:26:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.976+0000 7f9b7e7fc640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9b780a2c30 0x7f9b780a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.976+0000 7f9b9131a640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9b780a2c30 0x7f9b780a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.976+0000 7f9b7e7fc640 1 -- 172.21.15.12:0/1152460713 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f9b800cf070 con 0x7f9b8c101980 2024-05-09T09:26:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b9636d740 1 Processor -- start 2024-05-09T09:26:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b9636d740 1 -- start start 2024-05-09T09:26:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b9131a640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9b780a2c30 0x7f9b780a50f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9b88006010 tx=0x7f9b88031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b9636d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a175670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b9636d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175c40 0x55716a17a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b9636d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a17adb0 0x55716a17d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b9636d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55716a0d9340 con 0x55716a17adb0 2024-05-09T09:26:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b9636d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55716a081290 con 0x55716a175c40 2024-05-09T09:26:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b9636d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55716a0d5300 con 0x55716a175250 2024-05-09T09:26:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b91b1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a17adb0 0x55716a17d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b9131a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a175670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b9131a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a175670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:55646/0 (socket says 172.21.15.12:55646) 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b9131a640 1 -- 172.21.15.12:0/2593367105 learned_addr learned my addr 172.21.15.12:0/2593367105 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.977+0000 7f9b90b19640 1 --2- 172.21.15.12:0/2593367105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175c40 0x55716a17a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.978+0000 7f9b9131a640 1 -- 172.21.15.12:0/2593367105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175c40 msgr2=0x55716a17a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.978+0000 7f9b9131a640 1 --2- 172.21.15.12:0/2593367105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175c40 0x55716a17a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.978+0000 7f9b9131a640 1 -- 172.21.15.12:0/2593367105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a17adb0 msgr2=0x55716a17d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.978+0000 7f9b9131a640 1 --2- 172.21.15.12:0/2593367105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a17adb0 0x55716a17d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.978+0000 7f9b9131a640 1 -- 172.21.15.12:0/2593367105 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55716a0a58e0 con 0x55716a175250 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.978+0000 7f9b91b1b640 1 --2- 172.21.15.12:0/2593367105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a17adb0 0x55716a17d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.978+0000 7f9b9131a640 1 --2- 172.21.15.12:0/2593367105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a175670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b88033000 tx=0x7f9b8803d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6700f8bb459148f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.978+0000 7f9b777fe640 1 -- 172.21.15.12:0/2593367105 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88005370 con 0x55716a175250 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.978+0000 7f9b777fe640 1 -- 172.21.15.12:0/2593367105 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9b8804d070 con 0x55716a175250 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.978+0000 7f9b777fe640 1 -- 172.21.15.12:0/2593367105 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88031940 con 0x55716a175250 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.979+0000 7f9b9636d740 1 -- 172.21.15.12:0/2593367105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 msgr2=0x55716a175670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.979+0000 7f9b9636d740 1 --2- 172.21.15.12:0/2593367105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a175670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b88033000 tx=0x7f9b8803d000 comp rx=0 tx=0).stop 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.979+0000 7f9b9636d740 1 -- 172.21.15.12:0/2593367105 shutdown_connections 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.979+0000 7f9b9636d740 1 --2- 172.21.15.12:0/2593367105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a17adb0 0x55716a17d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.979+0000 7f9b9636d740 1 --2- 172.21.15.12:0/2593367105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175c40 0x55716a17a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.979+0000 7f9b9636d740 1 --2- 172.21.15.12:0/2593367105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a175670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.979+0000 7f9b9636d740 1 -- 172.21.15.12:0/2593367105 >> 172.21.15.12:0/2593367105 conn(0x55716a1708c0 msgr2=0x55716a170cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.979+0000 7f9b9636d740 1 -- 172.21.15.12:0/2593367105 shutdown_connections 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.979+0000 7f9b9636d740 1 -- 172.21.15.12:0/2593367105 wait complete. 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.979+0000 7f9b9636d740 1 -- start start 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.979+0000 7f9b9636d740 1 Processor -- start 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b9636d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175250 0x55716a17f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b9636d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 0x55716a17f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b9636d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a17adb0 0x55716a1869b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b9636d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55716a17dd00 con 0x55716a175c40 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b9636d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55716a094790 con 0x55716a175250 2024-05-09T09:26:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b9636d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55716a17de80 con 0x55716a17adb0 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b9636d740 10 client.0 _ll_register_callbacks cb 0x55716a0cfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b90b19640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 0x55716a17f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b90b19640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 0x55716a17f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53828/0 (socket says 172.21.15.12:53828) 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b90b19640 1 -- 172.21.15.12:0/3552557109 learned_addr learned my addr 172.21.15.12:0/3552557109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b91b1b640 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a17adb0 0x55716a1869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b9131a640 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175250 0x55716a17f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b90b19640 1 -- 172.21.15.12:0/3552557109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a17adb0 msgr2=0x55716a1869b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b90b19640 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a17adb0 0x55716a1869b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b90b19640 1 -- 172.21.15.12:0/3552557109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175250 msgr2=0x55716a17f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b90b19640 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175250 0x55716a17f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b90b19640 1 -- 172.21.15.12:0/3552557109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557169ded210 con 0x55716a175c40 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b91b1b640 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a17adb0 0x55716a1869b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b90b19640 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 0x55716a17f970 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9b800ccb10 tx=0x7f9b80009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37de588aedbe7d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b74ff9640 1 -- 172.21.15.12:0/3552557109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800b4060 con 0x55716a175c40 2024-05-09T09:26:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b74ff9640 1 -- 172.21.15.12:0/3552557109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f9b800b4200 con 0x55716a175c40 2024-05-09T09:26:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.980+0000 7f9b9636d740 1 -- 172.21.15.12:0/3552557109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b8804d070 con 0x55716a175c40 2024-05-09T09:26:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.981+0000 7f9b74ff9640 1 -- 172.21.15.12:0/3552557109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9b80020040 con 0x55716a175c40 2024-05-09T09:26:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.981+0000 7f9b9636d740 10 client.15885 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.981+0000 7f9b9636d740 1 -- 172.21.15.12:0/3552557109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557169eb9620 con 0x55716a175c40 2024-05-09T09:26:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.981+0000 7f9b74ff9640 1 -- 172.21.15.12:0/3552557109 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9b8804d070 con 0x55716a175c40 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.981+0000 7f9b74ff9640 1 -- 172.21.15.12:0/3552557109 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9b800972e0 con 0x55716a175c40 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.981+0000 7f9b9636d740 10 client.15885 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.981+0000 7f9b9636d740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.981+0000 7f9b9636d740 20 client.15885 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.981+0000 7f9b9636d740 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55716a18af50 0x55716a18d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.981+0000 7f9b9636d740 4 client.15885 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.981+0000 7f9b9636d740 1 -- 172.21.15.12:0/3552557109 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55716a0e6a70 con 0x55716a18af50 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.981+0000 7f9b9131a640 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55716a18af50 0x55716a18d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.982+0000 7f9b9131a640 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55716a18af50 0x55716a18d360 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1926d8842644ce3 server_cookie=9490b4047df4babc in_seq=0 out_seq=0 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.982+0000 7f9b74ff9640 10 client.15885 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b74ff9640 1 -- 172.21.15.12:0/3552557109 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55716a0e6a70 con 0x55716a18af50 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b74ff9640 10 client.15885 handle_command_reply: tid=0 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b9636d740 1 client.15885 shutdown 2024-05-09T09:26:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b9636d740 1 -- 172.21.15.12:0/3552557109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 msgr2=0x55716a17f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b9636d740 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 0x55716a17f970 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9b800ccb10 tx=0x7f9b80009ef0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b9636d740 1 -- 172.21.15.12:0/3552557109 shutdown_connections 2024-05-09T09:26:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b9636d740 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55716a18af50 0x55716a18d360 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b9636d740 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a17adb0 0x55716a1869b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b9636d740 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 0x55716a17f970 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b74ff9640 0 client.15885 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b9636d740 1 --2- 172.21.15.12:0/3552557109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175250 0x55716a17f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b9636d740 1 -- 172.21.15.12:0/3552557109 >> 172.21.15.12:0/3552557109 conn(0x55716a1708c0 msgr2=0x55716a16fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b9636d740 1 -- 172.21.15.12:0/3552557109 shutdown_connections 2024-05-09T09:26:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b9636d740 1 -- 172.21.15.12:0/3552557109 wait complete. 2024-05-09T09:26:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.983+0000 7f9b9636d740 20 client.15885 trim_cache size 0 max 16384 2024-05-09T09:26:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b9636d740 1 Processor -- start 2024-05-09T09:26:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b9636d740 1 -- start start 2024-05-09T09:26:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b9636d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a170180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b9636d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 0x55716a186970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b9636d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a17adb0 0x55716a173f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b9636d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1aa550 con 0x55716a175c40 2024-05-09T09:26:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b9636d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1ac090 con 0x55716a17adb0 2024-05-09T09:26:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b9636d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1aac20 con 0x55716a175250 2024-05-09T09:26:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b90b19640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 0x55716a186970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b90b19640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 0x55716a186970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53844/0 (socket says 172.21.15.12:53844) 2024-05-09T09:26:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b90b19640 1 -- 172.21.15.12:0/1348879468 learned_addr learned my addr 172.21.15.12:0/1348879468 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b9131a640 1 --2- 172.21.15.12:0/1348879468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a170180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b91b1b640 1 --2- 172.21.15.12:0/1348879468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a17adb0 0x55716a173f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b90b19640 1 -- 172.21.15.12:0/1348879468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 msgr2=0x55716a170180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b90b19640 1 --2- 172.21.15.12:0/1348879468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a170180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b90b19640 1 -- 172.21.15.12:0/1348879468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a17adb0 msgr2=0x55716a173f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b90b19640 1 --2- 172.21.15.12:0/1348879468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a17adb0 0x55716a173f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b90b19640 1 -- 172.21.15.12:0/1348879468 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55716a16bc30 con 0x55716a175c40 2024-05-09T09:26:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b91b1b640 1 --2- 172.21.15.12:0/1348879468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a17adb0 0x55716a173f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:18.999+0000 7f9b90b19640 1 --2- 172.21.15.12:0/1348879468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 0x55716a186970 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9b8009b2f0 tx=0x7f9b80099a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ec464ee0492216 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.000+0000 7f9b76ffd640 1 -- 172.21.15.12:0/1348879468 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800b4040 con 0x55716a175c40 2024-05-09T09:26:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.000+0000 7f9b76ffd640 1 -- 172.21.15.12:0/1348879468 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9b800a4170 con 0x55716a175c40 2024-05-09T09:26:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.000+0000 7f9b76ffd640 1 -- 172.21.15.12:0/1348879468 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800b41e0 con 0x55716a175c40 2024-05-09T09:26:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.000+0000 7f9b9636d740 1 -- 172.21.15.12:0/1348879468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 msgr2=0x55716a186970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.000+0000 7f9b9636d740 1 --2- 172.21.15.12:0/1348879468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 0x55716a186970 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9b8009b2f0 tx=0x7f9b80099a10 comp rx=0 tx=0).stop 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.000+0000 7f9b9636d740 1 -- 172.21.15.12:0/1348879468 shutdown_connections 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.000+0000 7f9b9636d740 1 --2- 172.21.15.12:0/1348879468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a17adb0 0x55716a173f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.000+0000 7f9b9636d740 1 --2- 172.21.15.12:0/1348879468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a175c40 0x55716a186970 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.000+0000 7f9b9636d740 1 --2- 172.21.15.12:0/1348879468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a170180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.000+0000 7f9b9636d740 1 -- 172.21.15.12:0/1348879468 >> 172.21.15.12:0/1348879468 conn(0x55716a1708c0 msgr2=0x55716a173870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.000+0000 7f9b9636d740 1 -- 172.21.15.12:0/1348879468 shutdown_connections 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b9636d740 1 -- 172.21.15.12:0/1348879468 wait complete. 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b9636d740 1 -- start start 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b9636d740 1 Processor -- start 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b9636d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a183cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b9636d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175c40 0x55716a1841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b9636d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a17adb0 0x55716a185140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b9636d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1ab9c0 con 0x55716a17adb0 2024-05-09T09:26:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b9636d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1ab2f0 con 0x55716a175c40 2024-05-09T09:26:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b9636d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1a8530 con 0x55716a175250 2024-05-09T09:26:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b9636d740 10 client.0 _ll_register_callbacks cb 0x7f9b8009b320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b91b1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a17adb0 0x55716a185140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b9131a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a183cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b91b1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a17adb0 0x55716a185140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53848/0 (socket says 172.21.15.12:53848) 2024-05-09T09:26:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.001+0000 7f9b91b1b640 1 -- 172.21.15.12:0/1473872109 learned_addr learned my addr 172.21.15.12:0/1473872109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b90b19640 1 --2- 172.21.15.12:0/1473872109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175c40 0x55716a1841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b9131a640 1 -- 172.21.15.12:0/1473872109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175c40 msgr2=0x55716a1841f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b9131a640 1 --2- 172.21.15.12:0/1473872109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175c40 0x55716a1841f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b9131a640 1 -- 172.21.15.12:0/1473872109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a17adb0 msgr2=0x55716a185140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b9131a640 1 --2- 172.21.15.12:0/1473872109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a17adb0 0x55716a185140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b9131a640 1 -- 172.21.15.12:0/1473872109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55716a185730 con 0x55716a175250 2024-05-09T09:26:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b91b1b640 1 --2- 172.21.15.12:0/1473872109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a17adb0 0x55716a185140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b9131a640 1 --2- 172.21.15.12:0/1473872109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a183cb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b8802aa00 tx=0x7f9b880045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f15314ed45bd5ed4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:26:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b74ff9640 1 -- 172.21.15.12:0/1473872109 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88062060 con 0x55716a175250 2024-05-09T09:26:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b9636d740 1 -- 172.21.15.12:0/1473872109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b800a4170 con 0x55716a175250 2024-05-09T09:26:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b74ff9640 1 -- 172.21.15.12:0/1473872109 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f9b8805cdc0 con 0x55716a175250 2024-05-09T09:26:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b74ff9640 1 -- 172.21.15.12:0/1473872109 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9b88069050 con 0x55716a175250 2024-05-09T09:26:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b9636d740 10 client.25516 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.002+0000 7f9b9636d740 1 -- 172.21.15.12:0/1473872109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55716a186a20 con 0x55716a175250 2024-05-09T09:26:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.003+0000 7f9b74ff9640 1 -- 172.21.15.12:0/1473872109 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9b800a4170 con 0x55716a175250 2024-05-09T09:26:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.003+0000 7f9b74ff9640 1 -- 172.21.15.12:0/1473872109 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9b880049c0 con 0x55716a175250 2024-05-09T09:26:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.003+0000 7f9b9636d740 10 client.25516 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.003+0000 7f9b9636d740 10 client.25516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.003+0000 7f9b9636d740 20 client.25516 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.003+0000 7f9b9636d740 1 --2- 172.21.15.12:0/1473872109 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55716a1f1630 0x55716a1f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.003+0000 7f9b9636d740 4 client.25516 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:26:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.003+0000 7f9b9636d740 1 -- 172.21.15.12:0/1473872109 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55716a0e0dc0 con 0x55716a1f1630 2024-05-09T09:26:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.003+0000 7f9b90b19640 1 --2- 172.21.15.12:0/1473872109 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55716a1f1630 0x55716a1f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.003+0000 7f9b90b19640 1 --2- 172.21.15.12:0/1473872109 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55716a1f1630 0x55716a1f3a50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad8492daff7b437b server_cookie=6dbb816eab74456 in_seq=0 out_seq=0 2024-05-09T09:26:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.003+0000 7f9b74ff9640 10 client.25516 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b74ff9640 1 -- 172.21.15.12:0/1473872109 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55716a0e0dc0 con 0x55716a1f1630 2024-05-09T09:26:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b74ff9640 10 client.25516 handle_command_reply: tid=0 2024-05-09T09:26:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b9636d740 1 client.25516 shutdown 2024-05-09T09:26:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b9636d740 1 -- 172.21.15.12:0/1473872109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 msgr2=0x55716a183cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b9636d740 1 --2- 172.21.15.12:0/1473872109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a183cb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b8802aa00 tx=0x7f9b880045e0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b9636d740 1 -- 172.21.15.12:0/1473872109 shutdown_connections 2024-05-09T09:26:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b9636d740 1 --2- 172.21.15.12:0/1473872109 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55716a1f1630 0x55716a1f3a50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b9636d740 1 --2- 172.21.15.12:0/1473872109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55716a17adb0 0x55716a185140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b74ff9640 0 client.25516 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b9636d740 1 --2- 172.21.15.12:0/1473872109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55716a175c40 0x55716a1841f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b9636d740 1 --2- 172.21.15.12:0/1473872109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55716a175250 0x55716a183cb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b9636d740 1 -- 172.21.15.12:0/1473872109 >> 172.21.15.12:0/1473872109 conn(0x55716a1708c0 msgr2=0x55716a16f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b9636d740 1 -- 172.21.15.12:0/1473872109 shutdown_connections 2024-05-09T09:26:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b9636d740 1 -- 172.21.15.12:0/1473872109 wait complete. 2024-05-09T09:26:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.007+0000 7f9b9636d740 20 client.25516 trim_cache size 0 max 16384 2024-05-09T09:26:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:26:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:26:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:26:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:26:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.008+0000 7f9b9357e640 1 -- 172.21.15.12:0/1152460713 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9b780a2c30 msgr2=0x7f9b780a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.008+0000 7f9b9357e640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9b780a2c30 0x7f9b780a50f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9b88006010 tx=0x7f9b88031510 comp rx=0 tx=0).stop 2024-05-09T09:26:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.008+0000 7f9b9357e640 1 -- 172.21.15.12:0/1152460713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c101980 msgr2=0x7f9b8c19c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.008+0000 7f9b9357e640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c101980 0x7f9b8c19c8a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000d950 tx=0x7f9b8000de10 comp rx=0 tx=0).stop 2024-05-09T09:26:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.008+0000 7f9b9357e640 1 -- 172.21.15.12:0/1152460713 shutdown_connections 2024-05-09T09:26:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.008+0000 7f9b9357e640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f9b780a2c30 0x7f9b780a50f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.008+0000 7f9b9357e640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b8c102340 0x7f9b8c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.008+0000 7f9b9357e640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c101980 0x7f9b8c19c8a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.008+0000 7f9b9357e640 1 --2- 172.21.15.12:0/1152460713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c100790 0x7f9b8c069a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.008+0000 7f9b9357e640 1 -- 172.21.15.12:0/1152460713 >> 172.21.15.12:0/1152460713 conn(0x7f9b8c0fbd00 msgr2=0x7f9b8c104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.008+0000 7f9b9357e640 1 -- 172.21.15.12:0/1152460713 shutdown_connections 2024-05-09T09:26:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:19.008+0000 7f9b9357e640 1 -- 172.21.15.12:0/1152460713 wait complete. 2024-05-09T09:26:19.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:19 smithi012 ceph-mon[30140]: pgmap v390: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T09:26:19.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:19 smithi170 ceph-mon[33632]: pgmap v390: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T09:26:20.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:19 smithi016 ceph-mon[33335]: pgmap v390: 129 pgs: 129 active+clean; 664 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T09:26:21.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:21 smithi012 ceph-mon[30140]: pgmap v391: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-09T09:26:21.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:21 smithi170 ceph-mon[33632]: pgmap v391: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-09T09:26:22.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:21 smithi016 ceph-mon[33335]: pgmap v391: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-09T09:26:23.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:23 smithi012 ceph-mon[30140]: pgmap v392: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T09:26:23.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:23 smithi170 ceph-mon[33632]: pgmap v392: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T09:26:24.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:23 smithi016 ceph-mon[33335]: pgmap v392: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T09:26:25.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:25 smithi012 ceph-mon[30140]: pgmap v393: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T09:26:25.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:25 smithi170 ceph-mon[33632]: pgmap v393: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T09:26:26.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:25 smithi016 ceph-mon[33335]: pgmap v393: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T09:26:27.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:27 smithi012 ceph-mon[30140]: pgmap v394: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T09:26:27.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:27 smithi170 ceph-mon[33632]: pgmap v394: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T09:26:28.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:27 smithi016 ceph-mon[33335]: pgmap v394: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T09:26:29.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:29 smithi012 ceph-mon[30140]: pgmap v395: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:29.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:29 smithi170 ceph-mon[33632]: pgmap v395: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:30.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:29 smithi016 ceph-mon[33335]: pgmap v395: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:31.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:31 smithi012 ceph-mon[30140]: pgmap v396: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:31.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:31 smithi170 ceph-mon[33632]: pgmap v396: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:32.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:31 smithi016 ceph-mon[33335]: pgmap v396: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.145+0000 7f7b244ba640 1 -- 172.21.15.12:0/423238426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b1c100a00 msgr2=0x7f7b1c100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.145+0000 7f7b244ba640 1 --2- 172.21.15.12:0/423238426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b1c100a00 0x7f7b1c100e80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0099b0 tx=0x7f7b0c02c650 comp rx=0 tx=0).stop 2024-05-09T09:26:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.146+0000 7f7b244ba640 1 -- 172.21.15.12:0/423238426 shutdown_connections 2024-05-09T09:26:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.146+0000 7f7b244ba640 1 --2- 172.21.15.12:0/423238426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b1c1013c0 0x7f7b1c108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.146+0000 7f7b244ba640 1 --2- 172.21.15.12:0/423238426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b1c100a00 0x7f7b1c100e80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.146+0000 7f7b244ba640 1 --2- 172.21.15.12:0/423238426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c101d60 0x7f7b1c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.146+0000 7f7b244ba640 1 -- 172.21.15.12:0/423238426 >> 172.21.15.12:0/423238426 conn(0x7f7b1c0fb490 msgr2=0x7f7b1c0fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.146+0000 7f7b244ba640 1 -- 172.21.15.12:0/423238426 shutdown_connections 2024-05-09T09:26:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.146+0000 7f7b244ba640 1 -- 172.21.15.12:0/423238426 wait complete. 2024-05-09T09:26:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.147+0000 7f7b244ba640 1 Processor -- start 2024-05-09T09:26:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.147+0000 7f7b244ba640 1 -- start start 2024-05-09T09:26:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.147+0000 7f7b244ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c100a00 0x7f7b1c195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.147+0000 7f7b244ba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b1c1013c0 0x7f7b1c195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b244ba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b1c101d60 0x7f7b1c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b244ba640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c10b340 con 0x7f7b1c1013c0 2024-05-09T09:26:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b21a55640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b1c1013c0 0x7f7b1c195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b244ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c10b1c0 con 0x7f7b1c100a00 2024-05-09T09:26:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b21a55640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b1c1013c0 0x7f7b1c195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44868/0 (socket says 172.21.15.12:44868) 2024-05-09T09:26:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b21a55640 1 -- 172.21.15.12:0/1978176779 learned_addr learned my addr 172.21.15.12:0/1978176779 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b22a57640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b1c101d60 0x7f7b1c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b22256640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c100a00 0x7f7b1c195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b244ba640 1 -- 172.21.15.12:0/1978176779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c10b4c0 con 0x7f7b1c101d60 2024-05-09T09:26:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b22a57640 1 -- 172.21.15.12:0/1978176779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c100a00 msgr2=0x7f7b1c195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b22a57640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c100a00 0x7f7b1c195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b22a57640 1 -- 172.21.15.12:0/1978176779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b1c1013c0 msgr2=0x7f7b1c195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b22a57640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b1c1013c0 0x7f7b1c195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.148+0000 7f7b22a57640 1 -- 172.21.15.12:0/1978176779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1c19d520 con 0x7f7b1c101d60 2024-05-09T09:26:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.149+0000 7f7b22256640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c100a00 0x7f7b1c195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.149+0000 7f7b21a55640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b1c1013c0 0x7f7b1c195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.149+0000 7f7b22a57640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b1c101d60 0x7f7b1c19ceb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009fd0 tx=0x7f7b1400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.149+0000 7f7b137fe640 1 -- 172.21.15.12:0/1978176779 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1400ce20 con 0x7f7b1c101d60 2024-05-09T09:26:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.149+0000 7f7b244ba640 1 -- 172.21.15.12:0/1978176779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1c19d810 con 0x7f7b1c101d60 2024-05-09T09:26:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.149+0000 7f7b137fe640 1 -- 172.21.15.12:0/1978176779 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7b14005020 con 0x7f7b1c101d60 2024-05-09T09:26:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.149+0000 7f7b137fe640 1 -- 172.21.15.12:0/1978176779 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b140106b0 con 0x7f7b1c101d60 2024-05-09T09:26:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.150+0000 7f7b244ba640 1 -- 172.21.15.12:0/1978176779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b1c19dd00 con 0x7f7b1c101d60 2024-05-09T09:26:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.152+0000 7f7b137fe640 1 -- 172.21.15.12:0/1978176779 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7b140051d0 con 0x7f7b1c101d60 2024-05-09T09:26:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.154+0000 7f7b271c9740 1 Processor -- start 2024-05-09T09:26:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.154+0000 7f7b137fe640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7b080a2cb0 0x7f7b080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.154+0000 7f7b137fe640 1 -- 172.21.15.12:0/1978176779 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f7b140cf7e0 con 0x7f7b1c101d60 2024-05-09T09:26:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.154+0000 7f7b21a55640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7b080a2cb0 0x7f7b080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.155+0000 7f7b271c9740 1 -- start start 2024-05-09T09:26:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.155+0000 7f7b21a55640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7b080a2cb0 0x7f7b080a5170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0096c0 tx=0x7f7b0c033530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.155+0000 7f7b271c9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0da250 0x55ba8b0da670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.155+0000 7f7b271c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0dac40 0x55ba8b0df740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.155+0000 7f7b271c9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0e2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.155+0000 7f7b271c9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ba8b03e340 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.155+0000 7f7b271c9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ba8afe6290 con 0x55ba8b0da250 2024-05-09T09:26:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.155+0000 7f7b271c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ba8b03a300 con 0x55ba8b0dac40 2024-05-09T09:26:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.155+0000 7f7b22a57640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0e2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.155+0000 7f7b22a57640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0e2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44878/0 (socket says 172.21.15.12:44878) 2024-05-09T09:26:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.155+0000 7f7b22a57640 1 -- 172.21.15.12:0/2795907311 learned_addr learned my addr 172.21.15.12:0/2795907311 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.155+0000 7f7b21a55640 1 --2- 172.21.15.12:0/2795907311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0dac40 0x55ba8b0df740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.155+0000 7f7b22256640 1 --2- 172.21.15.12:0/2795907311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0da250 0x55ba8b0da670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.156+0000 7f7b22a57640 1 -- 172.21.15.12:0/2795907311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0dac40 msgr2=0x55ba8b0df740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.156+0000 7f7b22a57640 1 --2- 172.21.15.12:0/2795907311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0dac40 0x55ba8b0df740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.156+0000 7f7b22a57640 1 -- 172.21.15.12:0/2795907311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0da250 msgr2=0x55ba8b0da670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.156+0000 7f7b22a57640 1 --2- 172.21.15.12:0/2795907311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0da250 0x55ba8b0da670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.156+0000 7f7b22a57640 1 -- 172.21.15.12:0/2795907311 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba8b00a8e0 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.156+0000 7f7b22a57640 1 --2- 172.21.15.12:0/2795907311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0e2240 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7b140980d0 tx=0x7f7b14005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e661760df799087f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.156+0000 7f7b10bfe640 1 -- 172.21.15.12:0/2795907311 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b140a0020 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.156+0000 7f7b10bfe640 1 -- 172.21.15.12:0/2795907311 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7b140a48e0 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.156+0000 7f7b10bfe640 1 -- 172.21.15.12:0/2795907311 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1409ab30 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.157+0000 7f7b271c9740 1 -- 172.21.15.12:0/2795907311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 msgr2=0x55ba8b0e2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.157+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2795907311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0e2240 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7b140980d0 tx=0x7f7b14005670 comp rx=0 tx=0).stop 2024-05-09T09:26:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.157+0000 7f7b271c9740 1 -- 172.21.15.12:0/2795907311 shutdown_connections 2024-05-09T09:26:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.157+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2795907311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0e2240 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.157+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2795907311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0dac40 0x55ba8b0df740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.157+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2795907311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0da250 0x55ba8b0da670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.157+0000 7f7b271c9740 1 -- 172.21.15.12:0/2795907311 >> 172.21.15.12:0/2795907311 conn(0x55ba8b0d58c0 msgr2=0x55ba8b0d5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.158+0000 7f7b271c9740 1 -- 172.21.15.12:0/2795907311 shutdown_connections 2024-05-09T09:26:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.158+0000 7f7b271c9740 1 -- 172.21.15.12:0/2795907311 wait complete. 2024-05-09T09:26:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.158+0000 7f7b271c9740 1 -- start start 2024-05-09T09:26:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.158+0000 7f7b271c9740 1 Processor -- start 2024-05-09T09:26:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.158+0000 7f7b271c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 0x55ba8b0e4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.158+0000 7f7b271c9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dac40 0x55ba8b0e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.158+0000 7f7b271c9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0eb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.158+0000 7f7b271c9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ba8b0e2d00 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.158+0000 7f7b271c9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ba8aff9790 con 0x55ba8b0dac40 2024-05-09T09:26:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.158+0000 7f7b271c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ba8b0e2e80 con 0x55ba8b0da250 2024-05-09T09:26:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.158+0000 7f7b271c9740 10 client.0 _ll_register_callbacks cb 0x55ba8b034a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.158+0000 7f7b22a57640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0eb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b22a57640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0eb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44886/0 (socket says 172.21.15.12:44886) 2024-05-09T09:26:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b22a57640 1 -- 172.21.15.12:0/2719501821 learned_addr learned my addr 172.21.15.12:0/2719501821 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b21a55640 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dac40 0x55ba8b0e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b22256640 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 0x55ba8b0e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b22a57640 1 -- 172.21.15.12:0/2719501821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 msgr2=0x55ba8b0e4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b22a57640 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 0x55ba8b0e4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b22a57640 1 -- 172.21.15.12:0/2719501821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dac40 msgr2=0x55ba8b0e49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b22a57640 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dac40 0x55ba8b0e49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b22a57640 1 -- 172.21.15.12:0/2719501821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba8ad52210 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b21a55640 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dac40 0x55ba8b0e49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b22a57640 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0eb970 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7b1409bc00 tx=0x7f7b140a0140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3978261b186fe36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b05ffb640 1 -- 172.21.15.12:0/2719501821 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b140b9060 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b05ffb640 1 -- 172.21.15.12:0/2719501821 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f7b1409b2c0 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b271c9740 1 -- 172.21.15.12:0/2719501821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b140a48e0 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b05ffb640 1 -- 172.21.15.12:0/2719501821 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b1409ab30 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.159+0000 7f7b271c9740 10 client.15903 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.160+0000 7f7b271c9740 1 -- 172.21.15.12:0/2719501821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba8ae1e620 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.160+0000 7f7b05ffb640 1 -- 172.21.15.12:0/2719501821 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7b140a48e0 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.160+0000 7f7b05ffb640 1 -- 172.21.15.12:0/2719501821 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7b1401f020 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.160+0000 7f7b271c9740 10 client.15903 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.160+0000 7f7b271c9740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.160+0000 7f7b271c9740 20 client.15903 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.160+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ba8b0efad0 0x55ba8b0f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.160+0000 7f7b271c9740 4 client.15903 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.160+0000 7f7b271c9740 1 -- 172.21.15.12:0/2719501821 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba8b04ba70 con 0x55ba8b0efad0 2024-05-09T09:26:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.160+0000 7f7b22256640 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ba8b0efad0 0x55ba8b0f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.161+0000 7f7b22256640 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ba8b0efad0 0x55ba8b0f1ec0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d0dc733b324e0f server_cookie=5de9f75e5f3843bf in_seq=0 out_seq=0 2024-05-09T09:26:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.161+0000 7f7b05ffb640 10 client.15903 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b05ffb640 1 -- 172.21.15.12:0/2719501821 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba8b04ba70 con 0x55ba8b0efad0 2024-05-09T09:26:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b05ffb640 10 client.15903 handle_command_reply: tid=0 2024-05-09T09:26:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b271c9740 1 client.15903 shutdown 2024-05-09T09:26:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b271c9740 1 -- 172.21.15.12:0/2719501821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 msgr2=0x55ba8b0eb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0eb970 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7b1409bc00 tx=0x7f7b140a0140 comp rx=0 tx=0).stop 2024-05-09T09:26:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b271c9740 1 -- 172.21.15.12:0/2719501821 shutdown_connections 2024-05-09T09:26:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ba8b0efad0 0x55ba8b0f1ec0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0eb970 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dac40 0x55ba8b0e49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2719501821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 0x55ba8b0e4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b271c9740 1 -- 172.21.15.12:0/2719501821 >> 172.21.15.12:0/2719501821 conn(0x55ba8b0d58c0 msgr2=0x55ba8b0d4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b05ffb640 0 client.15903 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b271c9740 1 -- 172.21.15.12:0/2719501821 shutdown_connections 2024-05-09T09:26:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b271c9740 1 -- 172.21.15.12:0/2719501821 wait complete. 2024-05-09T09:26:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.162+0000 7f7b271c9740 20 client.15903 trim_cache size 0 max 16384 2024-05-09T09:26:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.192+0000 7f7b271c9740 1 Processor -- start 2024-05-09T09:26:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.192+0000 7f7b271c9740 1 -- start start 2024-05-09T09:26:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.192+0000 7f7b271c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 0x55ba8b0f7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.192+0000 7f7b271c9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dac40 0x55ba8b0f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.192+0000 7f7b271c9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0eb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.192+0000 7f7b271c9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1a83c0 con 0x55ba8b0dac40 2024-05-09T09:26:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.192+0000 7f7b271c9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1a9f00 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.192+0000 7f7b271c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1a8a90 con 0x55ba8b0da250 2024-05-09T09:26:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.193+0000 7f7b21a55640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dac40 0x55ba8b0f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.193+0000 7f7b21a55640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dac40 0x55ba8b0f7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44898/0 (socket says 172.21.15.12:44898) 2024-05-09T09:26:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.193+0000 7f7b21a55640 1 -- 172.21.15.12:0/2834732031 learned_addr learned my addr 172.21.15.12:0/2834732031 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.193+0000 7f7b22a57640 1 --2- 172.21.15.12:0/2834732031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.193+0000 7f7b22256640 1 --2- 172.21.15.12:0/2834732031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 0x55ba8b0f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.193+0000 7f7b21a55640 1 -- 172.21.15.12:0/2834732031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 msgr2=0x55ba8b0f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.193+0000 7f7b21a55640 1 --2- 172.21.15.12:0/2834732031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 0x55ba8b0f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.193+0000 7f7b21a55640 1 -- 172.21.15.12:0/2834732031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dfdb0 msgr2=0x55ba8b0eb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.193+0000 7f7b21a55640 1 --2- 172.21.15.12:0/2834732031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0eb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.193+0000 7f7b21a55640 1 -- 172.21.15.12:0/2834732031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba8b0db4d0 con 0x55ba8b0dac40 2024-05-09T09:26:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.193+0000 7f7b21a55640 1 --2- 172.21.15.12:0/2834732031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dac40 0x55ba8b0f7cc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c02cb70 tx=0x7f7b0c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbdc3ef80219335 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.194+0000 7f7b07fff640 1 -- 172.21.15.12:0/2834732031 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c037050 con 0x55ba8b0dac40 2024-05-09T09:26:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.194+0000 7f7b07fff640 1 -- 172.21.15.12:0/2834732031 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7b0c0047a0 con 0x55ba8b0dac40 2024-05-09T09:26:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.194+0000 7f7b07fff640 1 -- 172.21.15.12:0/2834732031 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c003b70 con 0x55ba8b0dac40 2024-05-09T09:26:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.195+0000 7f7b271c9740 1 -- 172.21.15.12:0/2834732031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dac40 msgr2=0x55ba8b0f7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.195+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2834732031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dac40 0x55ba8b0f7cc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c02cb70 tx=0x7f7b0c040000 comp rx=0 tx=0).stop 2024-05-09T09:26:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.195+0000 7f7b271c9740 1 -- 172.21.15.12:0/2834732031 shutdown_connections 2024-05-09T09:26:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.195+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2834732031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b0eb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.195+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2834732031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dac40 0x55ba8b0f7cc0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.195+0000 7f7b271c9740 1 --2- 172.21.15.12:0/2834732031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 0x55ba8b0f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.195+0000 7f7b271c9740 1 -- 172.21.15.12:0/2834732031 >> 172.21.15.12:0/2834732031 conn(0x55ba8b0d58c0 msgr2=0x55ba8b0d5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.195+0000 7f7b271c9740 1 -- 172.21.15.12:0/2834732031 shutdown_connections 2024-05-09T09:26:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.195+0000 7f7b271c9740 1 -- 172.21.15.12:0/2834732031 wait complete. 2024-05-09T09:26:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.195+0000 7f7b271c9740 1 -- start start 2024-05-09T09:26:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.195+0000 7f7b271c9740 1 Processor -- start 2024-05-09T09:26:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b271c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 0x55ba8b0e8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b271c9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dac40 0x55ba8b0e8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b271c9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b271c9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1a9830 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b271c9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1a9160 con 0x55ba8b0dac40 2024-05-09T09:26:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b271c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1a63a0 con 0x55ba8b0da250 2024-05-09T09:26:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b271c9740 10 client.0 _ll_register_callbacks cb 0x7f7b18009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b22a57640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b22256640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 0x55ba8b0e8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b22a57640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44914/0 (socket says 172.21.15.12:44914) 2024-05-09T09:26:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b22a57640 1 -- 172.21.15.12:0/3335196838 learned_addr learned my addr 172.21.15.12:0/3335196838 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b21a55640 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dac40 0x55ba8b0e8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b22a57640 1 -- 172.21.15.12:0/3335196838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 msgr2=0x55ba8b0e8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b22a57640 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 0x55ba8b0e8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b22a57640 1 -- 172.21.15.12:0/3335196838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dac40 msgr2=0x55ba8b0e8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b22a57640 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dac40 0x55ba8b0e8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b22a57640 1 -- 172.21.15.12:0/3335196838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba8b0ebc80 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b22256640 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 0x55ba8b0e8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.196+0000 7f7b21a55640 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dac40 0x55ba8b0e8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.197+0000 7f7b22a57640 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b153fe0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400c760 tx=0x7f7b140a00a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79f3dac109aa7da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.197+0000 7f7b05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.197+0000 7f7b05ffb640 1 -- 172.21.15.12:0/3335196838 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b140b9070 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.197+0000 7f7b05ffb640 1 -- 172.21.15.12:0/3335196838 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f7b1409b0c0 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.197+0000 7f7b271c9740 1 -- 172.21.15.12:0/3335196838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b0c0047a0 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.197+0000 7f7b05ffb640 1 -- 172.21.15.12:0/3335196838 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b14004e50 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.197+0000 7f7b271c9740 10 client.15909 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.197+0000 7f7b271c9740 1 -- 172.21.15.12:0/3335196838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba8b155590 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.197+0000 7f7b05ffb640 1 -- 172.21.15.12:0/3335196838 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7b0c0047a0 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.198+0000 7f7b05ffb640 1 -- 172.21.15.12:0/3335196838 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7b1407aae0 con 0x55ba8b0dfdb0 2024-05-09T09:26:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.198+0000 7f7b271c9740 10 client.15909 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.198+0000 7f7b271c9740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.198+0000 7f7b271c9740 20 client.15909 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.198+0000 7f7b271c9740 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ba8b155c50 0x55ba8b158070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.198+0000 7f7b271c9740 4 client.15909 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:26:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.198+0000 7f7b271c9740 1 -- 172.21.15.12:0/3335196838 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba8b045dc0 con 0x55ba8b155c50 2024-05-09T09:26:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.198+0000 7f7b22256640 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ba8b155c50 0x55ba8b158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.198+0000 7f7b05ffb640 10 client.15909 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.198+0000 7f7b22256640 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ba8b155c50 0x55ba8b158070 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffce2836702f5bc8 server_cookie=1a7943a5b4abdccb in_seq=0 out_seq=0 2024-05-09T09:26:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b05ffb640 1 -- 172.21.15.12:0/3335196838 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba8b045dc0 con 0x55ba8b155c50 2024-05-09T09:26:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b05ffb640 10 client.15909 handle_command_reply: tid=0 2024-05-09T09:26:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b271c9740 1 client.15909 shutdown 2024-05-09T09:26:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b271c9740 1 -- 172.21.15.12:0/3335196838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 msgr2=0x55ba8b153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b271c9740 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b153fe0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400c760 tx=0x7f7b140a00a0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b271c9740 1 -- 172.21.15.12:0/3335196838 shutdown_connections 2024-05-09T09:26:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b271c9740 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ba8b155c50 0x55ba8b158070 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b271c9740 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8b0dfdb0 0x55ba8b153fe0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b271c9740 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba8b0dac40 0x55ba8b0e8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b271c9740 1 --2- 172.21.15.12:0/3335196838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ba8b0da250 0x55ba8b0e8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b271c9740 1 -- 172.21.15.12:0/3335196838 >> 172.21.15.12:0/3335196838 conn(0x55ba8b0d58c0 msgr2=0x55ba8b0ec000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b05ffb640 0 client.15909 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b271c9740 1 -- 172.21.15.12:0/3335196838 shutdown_connections 2024-05-09T09:26:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b271c9740 1 -- 172.21.15.12:0/3335196838 wait complete. 2024-05-09T09:26:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.203+0000 7f7b271c9740 20 client.15909 trim_cache size 0 max 16384 2024-05-09T09:26:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:26:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:26:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:26:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:26:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.204+0000 7f7b244ba640 1 -- 172.21.15.12:0/1978176779 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7b080a2cb0 msgr2=0x7f7b080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.204+0000 7f7b244ba640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7b080a2cb0 0x7f7b080a5170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0096c0 tx=0x7f7b0c033530 comp rx=0 tx=0).stop 2024-05-09T09:26:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.204+0000 7f7b244ba640 1 -- 172.21.15.12:0/1978176779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b1c101d60 msgr2=0x7f7b1c19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.204+0000 7f7b244ba640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b1c101d60 0x7f7b1c19ceb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009fd0 tx=0x7f7b1400ef10 comp rx=0 tx=0).stop 2024-05-09T09:26:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.204+0000 7f7b244ba640 1 -- 172.21.15.12:0/1978176779 shutdown_connections 2024-05-09T09:26:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.204+0000 7f7b244ba640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7b080a2cb0 0x7f7b080a5170 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.204+0000 7f7b244ba640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b1c101d60 0x7f7b1c19ceb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.204+0000 7f7b244ba640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b1c1013c0 0x7f7b1c195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.204+0000 7f7b244ba640 1 --2- 172.21.15.12:0/1978176779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c100a00 0x7f7b1c195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.204+0000 7f7b244ba640 1 -- 172.21.15.12:0/1978176779 >> 172.21.15.12:0/1978176779 conn(0x7f7b1c0fb490 msgr2=0x7f7b1c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.204+0000 7f7b244ba640 1 -- 172.21.15.12:0/1978176779 shutdown_connections 2024-05-09T09:26:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:32.204+0000 7f7b244ba640 1 -- 172.21.15.12:0/1978176779 wait complete. 2024-05-09T09:26:33.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:33 smithi012 ceph-mon[30140]: pgmap v397: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T09:26:33.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:33 smithi170 ceph-mon[33632]: pgmap v397: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T09:26:34.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:33 smithi016 ceph-mon[33335]: pgmap v397: 129 pgs: 129 active+clean; 665 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T09:26:35.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:35 smithi012 ceph-mon[30140]: pgmap v398: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:35.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:35 smithi170 ceph-mon[33632]: pgmap v398: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:36.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:35 smithi016 ceph-mon[33335]: pgmap v398: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:37.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:37 smithi012 ceph-mon[30140]: pgmap v399: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:37.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:37 smithi170 ceph-mon[33632]: pgmap v399: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:38.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:37 smithi016 ceph-mon[33335]: pgmap v399: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:39.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:39 smithi012 ceph-mon[30140]: pgmap v400: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:39.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:39 smithi170 ceph-mon[33632]: pgmap v400: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:40.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:39 smithi016 ceph-mon[33335]: pgmap v400: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:41.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:41 smithi012 ceph-mon[30140]: pgmap v401: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:41.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:41 smithi170 ceph-mon[33632]: pgmap v401: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:42.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:41 smithi016 ceph-mon[33335]: pgmap v401: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:42.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:26:42.604 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:26:42.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.726+0000 7fbca5c12640 1 -- 172.21.15.12:0/1753661147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca0102320 msgr2=0x7fbca0108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:42.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.726+0000 7fbca5c12640 1 --2- 172.21.15.12:0/1753661147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca0102320 0x7fbca0108bb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009ca0 tx=0x7fbc9802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.726+0000 7fbca5c12640 1 -- 172.21.15.12:0/1753661147 shutdown_connections 2024-05-09T09:26:42.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.726+0000 7fbca5c12640 1 --2- 172.21.15.12:0/1753661147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca0102320 0x7fbca0108bb0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.726+0000 7fbca5c12640 1 --2- 172.21.15.12:0/1753661147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca0101960 0x7fbca0101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.726+0000 7fbca5c12640 1 --2- 172.21.15.12:0/1753661147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca0100770 0x7fbca0100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.726+0000 7fbca5c12640 1 -- 172.21.15.12:0/1753661147 >> 172.21.15.12:0/1753661147 conn(0x7fbca00fbce0 msgr2=0x7fbca00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:42.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.726+0000 7fbca5c12640 1 -- 172.21.15.12:0/1753661147 shutdown_connections 2024-05-09T09:26:42.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.726+0000 7fbca5c12640 1 -- 172.21.15.12:0/1753661147 wait complete. 2024-05-09T09:26:42.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.727+0000 7fbca5c12640 1 Processor -- start 2024-05-09T09:26:42.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.727+0000 7fbca5c12640 1 -- start start 2024-05-09T09:26:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.727+0000 7fbca5c12640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca0100770 0x7fbca0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.727+0000 7fbca5c12640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca0101960 0x7fbca0198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.727+0000 7fbca5c12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca0102320 0x7fbca019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.727+0000 7fbca5c12640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbca010c3d0 con 0x7fbca0100770 2024-05-09T09:26:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.727+0000 7fbca5c12640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbca010c250 con 0x7fbca0102320 2024-05-09T09:26:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.727+0000 7fbca5c12640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbca010c550 con 0x7fbca0101960 2024-05-09T09:26:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.727+0000 7fbc9f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca0100770 0x7fbca0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.727+0000 7fbc9f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca0100770 0x7fbca0197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34720/0 (socket says 172.21.15.12:34720) 2024-05-09T09:26:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.727+0000 7fbc9f7fe640 1 -- 172.21.15.12:0/3673505440 learned_addr learned my addr 172.21.15.12:0/3673505440 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.727+0000 7fbc9effd640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca0101960 0x7fbca0198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca0102320 0x7fbca019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbc9f7fe640 1 -- 172.21.15.12:0/3673505440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca0101960 msgr2=0x7fbca0198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbc9f7fe640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca0101960 0x7fbca0198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbc9f7fe640 1 -- 172.21.15.12:0/3673505440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca0102320 msgr2=0x7fbca019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbc9f7fe640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca0102320 0x7fbca019f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbc9f7fe640 1 -- 172.21.15.12:0/3673505440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca019f7d0 con 0x7fbca0100770 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbc9effd640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca0101960 0x7fbca0198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca0102320 0x7fbca019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbc9f7fe640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca0100770 0x7fbca0197c20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000e9c0 tx=0x7fbc9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbc9cff9640 1 -- 172.21.15.12:0/3673505440 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9000cd80 con 0x7fbca0100770 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbc9cff9640 1 -- 172.21.15.12:0/3673505440 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc900035f0 con 0x7fbca0100770 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbc9cff9640 1 -- 172.21.15.12:0/3673505440 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90010690 con 0x7fbca0100770 2024-05-09T09:26:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbca5c12640 1 -- 172.21.15.12:0/3673505440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca019fac0 con 0x7fbca0100770 2024-05-09T09:26:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.728+0000 7fbca5c12640 1 -- 172.21.15.12:0/3673505440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca019ffc0 con 0x7fbca0100770 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.731+0000 7fbca89f8740 1 Processor -- start 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.731+0000 7fbca89f8740 1 -- start start 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.731+0000 7fbca89f8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d691f30 0x55643d692350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.731+0000 7fbca89f8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d692920 0x55643d697420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.731+0000 7fbca89f8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 0x55643d699f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.731+0000 7fbca89f8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55643d5f6d00 con 0x55643d692920 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.731+0000 7fbca89f8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55643d59e4c0 con 0x55643d697a90 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.731+0000 7fbca89f8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55643d5f2cc0 con 0x55643d691f30 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.732+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d691f30 0x55643d692350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.732+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d691f30 0x55643d692350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:36772/0 (socket says 172.21.15.12:36772) 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.732+0000 7fbc9ffff640 1 -- 172.21.15.12:0/112431187 learned_addr learned my addr 172.21.15.12:0/112431187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.732+0000 7fbc9effd640 1 --2- 172.21.15.12:0/112431187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 0x55643d699f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.732+0000 7fbc9cff9640 1 -- 172.21.15.12:0/3673505440 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbc900037a0 con 0x7fbca0100770 2024-05-09T09:26:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.732+0000 7fbc9ffff640 1 -- 172.21.15.12:0/112431187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 msgr2=0x55643d699f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:42.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.732+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/112431187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 0x55643d699f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.732+0000 7fbc9ffff640 1 -- 172.21.15.12:0/112431187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d692920 msgr2=0x55643d697420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:26:42.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.732+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/112431187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d692920 0x55643d697420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.732+0000 7fbc9ffff640 1 -- 172.21.15.12:0/112431187 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55643d3d47d0 con 0x55643d691f30 2024-05-09T09:26:42.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.733+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/112431187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d691f30 0x55643d692350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc980093e0 tx=0x7fbc98005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc0ef3e303237d4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:42.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.733+0000 7fbc85ffb640 1 -- 172.21.15.12:0/112431187 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98031650 con 0x55643d691f30 2024-05-09T09:26:42.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.733+0000 7fbc85ffb640 1 -- 172.21.15.12:0/112431187 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc98036040 con 0x55643d691f30 2024-05-09T09:26:42.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.733+0000 7fbc85ffb640 1 -- 172.21.15.12:0/112431187 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98004660 con 0x55643d691f30 2024-05-09T09:26:42.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.733+0000 7fbc9cff9640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbc880a2c10 0x7fbc880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.733+0000 7fbc9f7fe640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbc880a2c10 0x7fbc880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.734+0000 7fbc9cff9640 1 -- 172.21.15.12:0/3673505440 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbc900cf820 con 0x7fbca0100770 2024-05-09T09:26:42.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.734+0000 7fbca89f8740 1 -- 172.21.15.12:0/112431187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d691f30 msgr2=0x55643d692350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:42.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.734+0000 7fbca89f8740 1 --2- 172.21.15.12:0/112431187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d691f30 0x55643d692350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc980093e0 tx=0x7fbc98005ef0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.734+0000 7fbca89f8740 1 -- 172.21.15.12:0/112431187 shutdown_connections 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.734+0000 7fbca89f8740 1 --2- 172.21.15.12:0/112431187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 0x55643d699f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.734+0000 7fbca89f8740 1 --2- 172.21.15.12:0/112431187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d692920 0x55643d697420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.734+0000 7fbca89f8740 1 --2- 172.21.15.12:0/112431187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d691f30 0x55643d692350 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.734+0000 7fbca89f8740 1 -- 172.21.15.12:0/112431187 >> 172.21.15.12:0/112431187 conn(0x55643d68d5a0 msgr2=0x55643d68d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.734+0000 7fbc9f7fe640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbc880a2c10 0x7fbc880a50d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbc900ccfa0 tx=0x7fbc90066d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.734+0000 7fbca89f8740 1 -- 172.21.15.12:0/112431187 shutdown_connections 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.734+0000 7fbca89f8740 1 -- 172.21.15.12:0/112431187 wait complete. 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.735+0000 7fbca89f8740 1 -- start start 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.735+0000 7fbca89f8740 1 Processor -- start 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.735+0000 7fbca89f8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d691f30 0x55643d69c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.735+0000 7fbca89f8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d692920 0x55643d69c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.735+0000 7fbca89f8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 0x55643d6a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.735+0000 7fbca89f8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55643d69aa80 con 0x55643d692920 2024-05-09T09:26:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.735+0000 7fbca89f8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55643d5b1b90 con 0x55643d697a90 2024-05-09T09:26:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.735+0000 7fbca89f8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55643d69ac00 con 0x55643d691f30 2024-05-09T09:26:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d691f30 0x55643d69c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbc9effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d692920 0x55643d69c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 0x55643d6a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbc9effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d692920 0x55643d69c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34724/0 (socket says 172.21.15.12:34724) 2024-05-09T09:26:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbc9effd640 1 -- 172.21.15.12:0/1550164966 learned_addr learned my addr 172.21.15.12:0/1550164966 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbca89f8740 10 client.0 _ll_register_callbacks cb 0x55643d5edda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbc9effd640 1 -- 172.21.15.12:0/1550164966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d691f30 msgr2=0x55643d69c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbc9effd640 1 --2- 172.21.15.12:0/1550164966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d691f30 0x55643d69c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbc9effd640 1 -- 172.21.15.12:0/1550164966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 msgr2=0x55643d6a38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbc9effd640 1 --2- 172.21.15.12:0/1550164966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 0x55643d6a38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbc9effd640 1 -- 172.21.15.12:0/1550164966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55643d30a3f0 con 0x55643d692920 2024-05-09T09:26:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/1550164966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 0x55643d6a38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbca89f8740 1 -- 172.21.15.12:0/1550164966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc98036040 con 0x55643d692920 2024-05-09T09:26:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbc9effd640 1 --2- 172.21.15.12:0/1550164966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d692920 0x55643d69c8a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009e80 tx=0x7fbc94011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77f2f8c401bbf97 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.736+0000 7fbc7effd640 10 client.15918 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.737+0000 7fbc7effd640 1 -- 172.21.15.12:0/1550164966 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94011e40 con 0x55643d692920 2024-05-09T09:26:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.737+0000 7fbc7effd640 1 -- 172.21.15.12:0/1550164966 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbc9400f460 con 0x55643d692920 2024-05-09T09:26:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.737+0000 7fbc7effd640 1 -- 172.21.15.12:0/1550164966 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc9400f660 con 0x55643d692920 2024-05-09T09:26:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.737+0000 7fbca89f8740 10 client.15918 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.737+0000 7fbca89f8740 1 -- 172.21.15.12:0/1550164966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55643d2ca0f0 con 0x55643d692920 2024-05-09T09:26:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.737+0000 7fbc7effd640 1 -- 172.21.15.12:0/1550164966 <== mon.0 v2:172.21.15.12:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbc9400f800 con 0x55643d692920 2024-05-09T09:26:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.737+0000 7fbca89f8740 10 client.15918 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.737+0000 7fbca89f8740 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.737+0000 7fbca89f8740 20 client.15918 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.737+0000 7fbca89f8740 1 --2- 172.21.15.12:0/1550164966 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55643d6a8090 0x55643d6aa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.737+0000 7fbca89f8740 4 client.15918 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.737+0000 7fbca89f8740 1 -- 172.21.15.12:0/1550164966 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55643d604430 con 0x55643d6a8090 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.738+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/1550164966 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55643d6a8090 0x55643d6aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.738+0000 7fbc7effd640 10 client.15918 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.738+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/1550164966 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55643d6a8090 0x55643d6aa480 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=173de586645885df server_cookie=8a44a5f998c8fffe in_seq=0 out_seq=0 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbc7effd640 1 -- 172.21.15.12:0/1550164966 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55643d604430 con 0x55643d6a8090 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbc7effd640 10 client.15918 handle_command_reply: tid=0 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbca89f8740 1 client.15918 shutdown 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbca89f8740 1 -- 172.21.15.12:0/1550164966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d692920 msgr2=0x55643d69c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbca89f8740 1 --2- 172.21.15.12:0/1550164966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d692920 0x55643d69c8a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009e80 tx=0x7fbc94011810 comp rx=0 tx=0).stop 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbca89f8740 1 -- 172.21.15.12:0/1550164966 shutdown_connections 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbca89f8740 1 --2- 172.21.15.12:0/1550164966 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55643d6a8090 0x55643d6aa480 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbca89f8740 1 --2- 172.21.15.12:0/1550164966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 0x55643d6a38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbca89f8740 1 --2- 172.21.15.12:0/1550164966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d692920 0x55643d69c8a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbc7effd640 0 client.15918 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:42.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbca89f8740 1 --2- 172.21.15.12:0/1550164966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d691f30 0x55643d69c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbca89f8740 1 -- 172.21.15.12:0/1550164966 >> 172.21.15.12:0/1550164966 conn(0x55643d68d5a0 msgr2=0x55643d68c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:42.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbca89f8740 1 -- 172.21.15.12:0/1550164966 shutdown_connections 2024-05-09T09:26:42.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.739+0000 7fbca89f8740 1 -- 172.21.15.12:0/1550164966 wait complete. 2024-05-09T09:26:42.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.740+0000 7fbca89f8740 20 client.15918 trim_cache size 0 max 16384 2024-05-09T09:26:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.757+0000 7fbca89f8740 1 Processor -- start 2024-05-09T09:26:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.757+0000 7fbca89f8740 1 -- start start 2024-05-09T09:26:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.757+0000 7fbca89f8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d691f30 0x55643d697310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.757+0000 7fbca89f8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d692920 0x55643d6a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.757+0000 7fbca89f8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d697a90 0x55643d690280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.757+0000 7fbca89f8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01a8450 con 0x55643d697a90 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.757+0000 7fbca89f8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01a9f90 con 0x55643d691f30 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.757+0000 7fbca89f8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01a8b20 con 0x55643d692920 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.758+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d697a90 0x55643d690280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.758+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d697a90 0x55643d690280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34726/0 (socket says 172.21.15.12:34726) 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.758+0000 7fbc9ffff640 1 -- 172.21.15.12:0/72648324 learned_addr learned my addr 172.21.15.12:0/72648324 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.758+0000 7fbc9effd640 1 --2- 172.21.15.12:0/72648324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d692920 0x55643d6a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.758+0000 7fbc9ffff640 1 -- 172.21.15.12:0/72648324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d692920 msgr2=0x55643d6a38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.758+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/72648324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d692920 0x55643d6a38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.758+0000 7fbc9ffff640 1 -- 172.21.15.12:0/72648324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d691f30 msgr2=0x55643d697310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.758+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/72648324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d691f30 0x55643d697310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.758+0000 7fbc9ffff640 1 -- 172.21.15.12:0/72648324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55643d6aa7f0 con 0x55643d697a90 2024-05-09T09:26:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.758+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/72648324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d697a90 0x55643d690280 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbc98031ab0 tx=0x7fbc98005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7805a7b296749474 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.759+0000 7fbc857fa640 1 -- 172.21.15.12:0/72648324 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98058070 con 0x55643d697a90 2024-05-09T09:26:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.759+0000 7fbc857fa640 1 -- 172.21.15.12:0/72648324 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc980040b0 con 0x55643d697a90 2024-05-09T09:26:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.759+0000 7fbc857fa640 1 -- 172.21.15.12:0/72648324 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9804a040 con 0x55643d697a90 2024-05-09T09:26:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.760+0000 7fbca89f8740 1 -- 172.21.15.12:0/72648324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d697a90 msgr2=0x55643d690280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.760+0000 7fbca89f8740 1 --2- 172.21.15.12:0/72648324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d697a90 0x55643d690280 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbc98031ab0 tx=0x7fbc98005ef0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.760+0000 7fbca89f8740 1 -- 172.21.15.12:0/72648324 shutdown_connections 2024-05-09T09:26:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.760+0000 7fbca89f8740 1 --2- 172.21.15.12:0/72648324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d697a90 0x55643d690280 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.760+0000 7fbca89f8740 1 --2- 172.21.15.12:0/72648324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d692920 0x55643d6a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.760+0000 7fbca89f8740 1 --2- 172.21.15.12:0/72648324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d691f30 0x55643d697310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.760+0000 7fbca89f8740 1 -- 172.21.15.12:0/72648324 >> 172.21.15.12:0/72648324 conn(0x55643d68d5a0 msgr2=0x55643d68d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.760+0000 7fbca89f8740 1 -- 172.21.15.12:0/72648324 shutdown_connections 2024-05-09T09:26:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.760+0000 7fbca89f8740 1 -- 172.21.15.12:0/72648324 wait complete. 2024-05-09T09:26:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.760+0000 7fbca89f8740 1 -- start start 2024-05-09T09:26:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.761+0000 7fbca89f8740 1 Processor -- start 2024-05-09T09:26:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.761+0000 7fbca89f8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d691f30 0x55643d6a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.761+0000 7fbca89f8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d692920 0x55643d6a1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.761+0000 7fbca89f8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 0x55643d6a2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.761+0000 7fbca89f8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01a98c0 con 0x55643d691f30 2024-05-09T09:26:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbca89f8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01a91f0 con 0x55643d697a90 2024-05-09T09:26:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbca89f8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01a6430 con 0x55643d692920 2024-05-09T09:26:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d691f30 0x55643d6a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d691f30 0x55643d6a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34734/0 (socket says 172.21.15.12:34734) 2024-05-09T09:26:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbc9ffff640 1 -- 172.21.15.12:0/4196462447 learned_addr learned my addr 172.21.15.12:0/4196462447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbc9effd640 1 --2- 172.21.15.12:0/4196462447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d692920 0x55643d6a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbca89f8740 10 client.0 _ll_register_callbacks cb 0x7fbc94016860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbc9effd640 1 -- 172.21.15.12:0/4196462447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 msgr2=0x55643d6a2460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:26:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbc9effd640 1 --2- 172.21.15.12:0/4196462447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 0x55643d6a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbc9effd640 1 -- 172.21.15.12:0/4196462447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d691f30 msgr2=0x55643d6a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbc9effd640 1 --2- 172.21.15.12:0/4196462447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d691f30 0x55643d6a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbc9effd640 1 -- 172.21.15.12:0/4196462447 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55643d6a2a50 con 0x55643d692920 2024-05-09T09:26:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/4196462447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d691f30 0x55643d6a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbc9effd640 1 --2- 172.21.15.12:0/4196462447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d692920 0x55643d6a1450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400a010 tx=0x7fbc940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f42c8aff6004b8f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.762+0000 7fbc7effd640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:26:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbc7effd640 1 -- 172.21.15.12:0/4196462447 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94005020 con 0x55643d692920 2024-05-09T09:26:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbca89f8740 1 -- 172.21.15.12:0/4196462447 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc980040b0 con 0x55643d692920 2024-05-09T09:26:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbc7effd640 1 -- 172.21.15.12:0/4196462447 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbc940051c0 con 0x55643d692920 2024-05-09T09:26:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbc7effd640 1 -- 172.21.15.12:0/4196462447 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc9403e050 con 0x55643d692920 2024-05-09T09:26:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbca89f8740 10 client.25552 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbca89f8740 1 -- 172.21.15.12:0/4196462447 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55643d70e5c0 con 0x55643d692920 2024-05-09T09:26:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbc7effd640 1 -- 172.21.15.12:0/4196462447 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc980040b0 con 0x55643d692920 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbc7effd640 1 -- 172.21.15.12:0/4196462447 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbc9402a470 con 0x55643d692920 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbca89f8740 10 client.25552 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbca89f8740 10 client.25552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbca89f8740 20 client.25552 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbca89f8740 1 --2- 172.21.15.12:0/4196462447 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55643d70ee40 0x55643d711280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbca89f8740 4 client.25552 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.763+0000 7fbca89f8740 1 -- 172.21.15.12:0/4196462447 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55643d5fe780 con 0x55643d70ee40 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.764+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/4196462447 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55643d70ee40 0x55643d711280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.764+0000 7fbc7effd640 10 client.25552 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.764+0000 7fbc9ffff640 1 --2- 172.21.15.12:0/4196462447 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55643d70ee40 0x55643d711280 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c0d179bcf17b87 server_cookie=840b94cb187f36ab in_seq=0 out_seq=0 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.764+0000 7fbc7effd640 1 -- 172.21.15.12:0/4196462447 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55643d5fe780 con 0x55643d70ee40 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.764+0000 7fbc7effd640 10 client.25552 handle_command_reply: tid=0 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.764+0000 7fbca89f8740 1 client.25552 shutdown 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.765+0000 7fbca89f8740 1 -- 172.21.15.12:0/4196462447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d692920 msgr2=0x55643d6a1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.765+0000 7fbca89f8740 1 --2- 172.21.15.12:0/4196462447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d692920 0x55643d6a1450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400a010 tx=0x7fbc940046c0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.765+0000 7fbca89f8740 1 -- 172.21.15.12:0/4196462447 shutdown_connections 2024-05-09T09:26:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.765+0000 7fbca89f8740 1 --2- 172.21.15.12:0/4196462447 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55643d70ee40 0x55643d711280 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.765+0000 7fbca89f8740 1 --2- 172.21.15.12:0/4196462447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55643d697a90 0x55643d6a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.765+0000 7fbc7effd640 0 client.25552 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.765+0000 7fbca89f8740 1 --2- 172.21.15.12:0/4196462447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643d692920 0x55643d6a1450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.765+0000 7fbca89f8740 1 --2- 172.21.15.12:0/4196462447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643d691f30 0x55643d6a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.765+0000 7fbca89f8740 1 -- 172.21.15.12:0/4196462447 >> 172.21.15.12:0/4196462447 conn(0x55643d68d5a0 msgr2=0x55643d693ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.765+0000 7fbca89f8740 1 -- 172.21.15.12:0/4196462447 shutdown_connections 2024-05-09T09:26:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.765+0000 7fbca89f8740 1 -- 172.21.15.12:0/4196462447 wait complete. 2024-05-09T09:26:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.765+0000 7fbca89f8740 20 client.25552 trim_cache size 0 max 16384 2024-05-09T09:26:42.770 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:26:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T09:26:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T09:26:42.770 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:26:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.766+0000 7fbca5c12640 1 -- 172.21.15.12:0/3673505440 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbc880a2c10 msgr2=0x7fbc880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.766+0000 7fbca5c12640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbc880a2c10 0x7fbc880a50d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbc900ccfa0 tx=0x7fbc90066d20 comp rx=0 tx=0).stop 2024-05-09T09:26:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.766+0000 7fbca5c12640 1 -- 172.21.15.12:0/3673505440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca0100770 msgr2=0x7fbca0197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.766+0000 7fbca5c12640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca0100770 0x7fbca0197c20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000e9c0 tx=0x7fbc9000ee80 comp rx=0 tx=0).stop 2024-05-09T09:26:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.766+0000 7fbca5c12640 1 -- 172.21.15.12:0/3673505440 shutdown_connections 2024-05-09T09:26:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.766+0000 7fbca5c12640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbc880a2c10 0x7fbc880a50d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.766+0000 7fbca5c12640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca0102320 0x7fbca019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.766+0000 7fbca5c12640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca0101960 0x7fbca0198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.766+0000 7fbca5c12640 1 --2- 172.21.15.12:0/3673505440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca0100770 0x7fbca0197c20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.766+0000 7fbca5c12640 1 -- 172.21.15.12:0/3673505440 >> 172.21.15.12:0/3673505440 conn(0x7fbca00fbce0 msgr2=0x7fbca00fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.766+0000 7fbca5c12640 1 -- 172.21.15.12:0/3673505440 shutdown_connections 2024-05-09T09:26:42.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.766+0000 7fbca5c12640 1 -- 172.21.15.12:0/3673505440 wait complete. 2024-05-09T09:26:42.780 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:26:42.781 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:26:42.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.888+0000 7fe7e135a640 1 -- 172.21.15.12:0/4084498842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7dc100770 msgr2=0x7fe7dc100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:42.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.888+0000 7fe7e135a640 1 --2- 172.21.15.12:0/4084498842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7dc100770 0x7fe7dc100b90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009ca0 tx=0x7fe7cc02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.888+0000 7fe7e135a640 1 -- 172.21.15.12:0/4084498842 shutdown_connections 2024-05-09T09:26:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.888+0000 7fe7e135a640 1 --2- 172.21.15.12:0/4084498842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7dc102320 0x7fe7dc108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.888+0000 7fe7e135a640 1 --2- 172.21.15.12:0/4084498842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc101960 0x7fe7dc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.888+0000 7fe7e135a640 1 --2- 172.21.15.12:0/4084498842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7dc100770 0x7fe7dc100b90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.888+0000 7fe7e135a640 1 -- 172.21.15.12:0/4084498842 >> 172.21.15.12:0/4084498842 conn(0x7fe7dc0fbce0 msgr2=0x7fe7dc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.888+0000 7fe7e135a640 1 -- 172.21.15.12:0/4084498842 shutdown_connections 2024-05-09T09:26:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.888+0000 7fe7e135a640 1 -- 172.21.15.12:0/4084498842 wait complete. 2024-05-09T09:26:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.889+0000 7fe7e135a640 1 Processor -- start 2024-05-09T09:26:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.889+0000 7fe7e135a640 1 -- start start 2024-05-09T09:26:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.889+0000 7fe7e135a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7dc101960 0x7fe7dc197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.889+0000 7fe7e135a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc102320 0x7fe7dc198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.889+0000 7fe7e135a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7dc19d370 0x7fe7dc19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.889+0000 7fe7e135a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc10c3d0 con 0x7fe7dc101960 2024-05-09T09:26:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.889+0000 7fe7e135a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc10c250 con 0x7fe7dc19d370 2024-05-09T09:26:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.889+0000 7fe7e135a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc10c550 con 0x7fe7dc102320 2024-05-09T09:26:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.889+0000 7fe7daffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7dc101960 0x7fe7dc197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.889+0000 7fe7db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7dc19d370 0x7fe7dc19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.889+0000 7fe7db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7dc19d370 0x7fe7dc19f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47168/0 (socket says 172.21.15.12:47168) 2024-05-09T09:26:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.889+0000 7fe7db7fe640 1 -- 172.21.15.12:0/387771729 learned_addr learned my addr 172.21.15.12:0/387771729 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.890+0000 7fe7da7fc640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc102320 0x7fe7dc198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.890+0000 7fe7daffd640 1 -- 172.21.15.12:0/387771729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc102320 msgr2=0x7fe7dc198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.890+0000 7fe7daffd640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc102320 0x7fe7dc198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.890+0000 7fe7daffd640 1 -- 172.21.15.12:0/387771729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7dc19d370 msgr2=0x7fe7dc19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.890+0000 7fe7daffd640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7dc19d370 0x7fe7dc19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.890+0000 7fe7daffd640 1 -- 172.21.15.12:0/387771729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc19fdd0 con 0x7fe7dc101960 2024-05-09T09:26:42.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.890+0000 7fe7da7fc640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc102320 0x7fe7dc198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:42.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.890+0000 7fe7daffd640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7dc101960 0x7fe7dc197e20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc1002b0 tx=0x7fe7cc031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:42.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.890+0000 7fe7c3fff640 1 -- 172.21.15.12:0/387771729 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc003bf0 con 0x7fe7dc101960 2024-05-09T09:26:42.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.890+0000 7fe7e135a640 1 -- 172.21.15.12:0/387771729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7dc1a0060 con 0x7fe7dc101960 2024-05-09T09:26:42.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.890+0000 7fe7c3fff640 1 -- 172.21.15.12:0/387771729 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe7cc003d90 con 0x7fe7dc101960 2024-05-09T09:26:42.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.890+0000 7fe7c3fff640 1 -- 172.21.15.12:0/387771729 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc02fca0 con 0x7fe7dc101960 2024-05-09T09:26:42.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.890+0000 7fe7e135a640 1 -- 172.21.15.12:0/387771729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7dc1a0580 con 0x7fe7dc101960 2024-05-09T09:26:42.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.892+0000 7fe7e135a640 1 -- 172.21.15.12:0/387771729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7dc068650 con 0x7fe7dc101960 2024-05-09T09:26:42.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.893+0000 7fe7c3fff640 1 -- 172.21.15.12:0/387771729 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe7cc004040 con 0x7fe7dc101960 2024-05-09T09:26:42.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.894+0000 7fe7c3fff640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe7c40a2b90 0x7fe7c40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:42.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.894+0000 7fe7c3fff640 1 -- 172.21.15.12:0/387771729 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fe7cc0f06e0 con 0x7fe7dc101960 2024-05-09T09:26:42.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.895+0000 7fe7da7fc640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe7c40a2b90 0x7fe7c40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:42.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.895+0000 7fe7da7fc640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe7c40a2b90 0x7fe7c40a5050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc1992e0 tx=0x7fe7d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:42.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:42.899+0000 7fe7c3fff640 1 -- 172.21.15.12:0/387771729 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe7cc0bd300 con 0x7fe7dc101960 2024-05-09T09:26:43.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.145+0000 7fe7e135a640 1 -- 172.21.15.12:0/387771729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7dc1a0a60 con 0x7fe7dc101960 2024-05-09T09:26:43.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.146+0000 7fe7c3fff640 1 -- 172.21.15.12:0/387771729 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11931 (secure 0 0 0) 0x7fe7cc0bd300 con 0x7fe7dc101960 2024-05-09T09:26:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:26:43.146 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:25:31.411273+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[14628,24484,24515,24523,24502]},"id":1}]} 2024-05-09T09:26:43.147 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-05-09T09:26:43.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.149+0000 7fe7e135a640 1 -- 172.21.15.12:0/387771729 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe7c40a2b90 msgr2=0x7fe7c40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.149+0000 7fe7e135a640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe7c40a2b90 0x7fe7c40a5050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc1992e0 tx=0x7fe7d0005f90 comp rx=0 tx=0).stop 2024-05-09T09:26:43.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.149+0000 7fe7e135a640 1 -- 172.21.15.12:0/387771729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7dc101960 msgr2=0x7fe7dc197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.149+0000 7fe7e135a640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7dc101960 0x7fe7dc197e20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc1002b0 tx=0x7fe7cc031840 comp rx=0 tx=0).stop 2024-05-09T09:26:43.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.149+0000 7fe7e135a640 1 -- 172.21.15.12:0/387771729 shutdown_connections 2024-05-09T09:26:43.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.149+0000 7fe7e135a640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe7c40a2b90 0x7fe7c40a5050 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.149+0000 7fe7e135a640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7dc19d370 0x7fe7dc19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.149+0000 7fe7e135a640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc102320 0x7fe7dc198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.149+0000 7fe7e135a640 1 --2- 172.21.15.12:0/387771729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7dc101960 0x7fe7dc197e20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.149+0000 7fe7e135a640 1 -- 172.21.15.12:0/387771729 >> 172.21.15.12:0/387771729 conn(0x7fe7dc0fbce0 msgr2=0x7fe7dc103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.149+0000 7fe7e135a640 1 -- 172.21.15.12:0/387771729 shutdown_connections 2024-05-09T09:26:43.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.149+0000 7fe7e135a640 1 -- 172.21.15.12:0/387771729 wait complete. 2024-05-09T09:26:43.164 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-09T09:26:43.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.276+0000 7fd8702ae640 1 -- 172.21.15.12:0/3409822427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd868100770 msgr2=0x7fd868100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.276+0000 7fd8702ae640 1 --2- 172.21.15.12:0/3409822427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd868100770 0x7fd868100b90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd864009860 tx=0x7fd86402a340 comp rx=0 tx=0).stop 2024-05-09T09:26:43.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.276+0000 7fd8702ae640 1 -- 172.21.15.12:0/3409822427 shutdown_connections 2024-05-09T09:26:43.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.276+0000 7fd8702ae640 1 --2- 172.21.15.12:0/3409822427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd868102320 0x7fd868108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.276+0000 7fd8702ae640 1 --2- 172.21.15.12:0/3409822427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd868101960 0x7fd868101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.276+0000 7fd8702ae640 1 --2- 172.21.15.12:0/3409822427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd868100770 0x7fd868100b90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.276+0000 7fd8702ae640 1 -- 172.21.15.12:0/3409822427 >> 172.21.15.12:0/3409822427 conn(0x7fd8680fbce0 msgr2=0x7fd8680fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.276+0000 7fd8702ae640 1 -- 172.21.15.12:0/3409822427 shutdown_connections 2024-05-09T09:26:43.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.276+0000 7fd8702ae640 1 -- 172.21.15.12:0/3409822427 wait complete. 2024-05-09T09:26:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.277+0000 7fd8702ae640 1 Processor -- start 2024-05-09T09:26:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.277+0000 7fd8702ae640 1 -- start start 2024-05-09T09:26:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.277+0000 7fd8702ae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd868100770 0x7fd868197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.277+0000 7fd8702ae640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd868101960 0x7fd868198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.277+0000 7fd8702ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd868102320 0x7fd86819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.277+0000 7fd8702ae640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd86810c3d0 con 0x7fd868101960 2024-05-09T09:26:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.277+0000 7fd8702ae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd86810c250 con 0x7fd868102320 2024-05-09T09:26:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.277+0000 7fd8702ae640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd86810c550 con 0x7fd868100770 2024-05-09T09:26:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd86d849640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd868101960 0x7fd868198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd86d849640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd868101960 0x7fd868198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34760/0 (socket says 172.21.15.12:34760) 2024-05-09T09:26:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd86d849640 1 -- 172.21.15.12:0/3202971890 learned_addr learned my addr 172.21.15.12:0/3202971890 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd86e84b640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd868102320 0x7fd86819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd86e04a640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd868100770 0x7fd868197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd86d849640 1 -- 172.21.15.12:0/3202971890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd868100770 msgr2=0x7fd868197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd86d849640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd868100770 0x7fd868197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd86d849640 1 -- 172.21.15.12:0/3202971890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd868102320 msgr2=0x7fd86819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd86d849640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd868102320 0x7fd86819f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd86d849640 1 -- 172.21.15.12:0/3202971890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86819f850 con 0x7fd868101960 2024-05-09T09:26:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd86e84b640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd868102320 0x7fd86819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd86d849640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd868101960 0x7fd868198180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd85800e9f0 tx=0x7fd85800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd85f7fe640 1 -- 172.21.15.12:0/3202971890 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85800cd80 con 0x7fd868101960 2024-05-09T09:26:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd8702ae640 1 -- 172.21.15.12:0/3202971890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86819fb40 con 0x7fd868101960 2024-05-09T09:26:43.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd85f7fe640 1 -- 172.21.15.12:0/3202971890 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd8580054f0 con 0x7fd868101960 2024-05-09T09:26:43.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd85f7fe640 1 -- 172.21.15.12:0/3202971890 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858010690 con 0x7fd868101960 2024-05-09T09:26:43.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.278+0000 7fd8702ae640 1 -- 172.21.15.12:0/3202971890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8681a0010 con 0x7fd868101960 2024-05-09T09:26:43.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.282+0000 7fd85f7fe640 1 -- 172.21.15.12:0/3202971890 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd858004f20 con 0x7fd868101960 2024-05-09T09:26:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.282+0000 7fd872fb7740 1 Processor -- start 2024-05-09T09:26:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.282+0000 7fd872fb7740 1 -- start start 2024-05-09T09:26:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.282+0000 7fd872fb7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c436f40 0x56551c437360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd872fb7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c437930 0x56551c43c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd872fb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c43caa0 0x56551c43ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd872fb7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56551c39bc40 con 0x56551c437930 2024-05-09T09:26:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd872fb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56551c343580 con 0x56551c43caa0 2024-05-09T09:26:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd872fb7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56551c397c00 con 0x56551c436f40 2024-05-09T09:26:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd86e04a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c437930 0x56551c43c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd86e04a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c437930 0x56551c43c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34766/0 (socket says 172.21.15.12:34766) 2024-05-09T09:26:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd86e04a640 1 -- 172.21.15.12:0/1605448035 learned_addr learned my addr 172.21.15.12:0/1605448035 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd86e84b640 1 --2- 172.21.15.12:0/1605448035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c436f40 0x56551c437360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd86d849640 1 --2- 172.21.15.12:0/1605448035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c43caa0 0x56551c43ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd86e04a640 1 -- 172.21.15.12:0/1605448035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c436f40 msgr2=0x56551c437360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd86e04a640 1 --2- 172.21.15.12:0/1605448035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c436f40 0x56551c437360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd86e04a640 1 -- 172.21.15.12:0/1605448035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c43caa0 msgr2=0x56551c43ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd86e04a640 1 --2- 172.21.15.12:0/1605448035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c43caa0 0x56551c43ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd86e04a640 1 -- 172.21.15.12:0/1605448035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56551c0af3f0 con 0x56551c437930 2024-05-09T09:26:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.283+0000 7fd86e84b640 1 --2- 172.21.15.12:0/1605448035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c436f40 0x56551c437360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.284+0000 7fd85f7fe640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd8540a2c10 0x7fd8540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.284+0000 7fd86e04a640 1 --2- 172.21.15.12:0/1605448035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c437930 0x56551c43c430 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd864009830 tx=0x7fd8640095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c5a18b2ecb7a25 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.284+0000 7fd85f7fe640 1 -- 172.21.15.12:0/3202971890 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd8580cf8f0 con 0x7fd868101960 2024-05-09T09:26:43.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.284+0000 7fd85cbfe640 1 -- 172.21.15.12:0/1605448035 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864037040 con 0x56551c437930 2024-05-09T09:26:43.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.284+0000 7fd86e84b640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd8540a2c10 0x7fd8540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.284+0000 7fd85cbfe640 1 -- 172.21.15.12:0/1605448035 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd864030c20 con 0x56551c437930 2024-05-09T09:26:43.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.284+0000 7fd85cbfe640 1 -- 172.21.15.12:0/1605448035 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640052a0 con 0x56551c437930 2024-05-09T09:26:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.285+0000 7fd872fb7740 1 -- 172.21.15.12:0/1605448035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c437930 msgr2=0x56551c43c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.285+0000 7fd86e84b640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd8540a2c10 0x7fd8540a50d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd860009fd0 tx=0x7fd8600095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.285+0000 7fd872fb7740 1 --2- 172.21.15.12:0/1605448035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c437930 0x56551c43c430 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd864009830 tx=0x7fd8640095a0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.285+0000 7fd872fb7740 1 -- 172.21.15.12:0/1605448035 shutdown_connections 2024-05-09T09:26:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.285+0000 7fd872fb7740 1 --2- 172.21.15.12:0/1605448035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c43caa0 0x56551c43ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.285+0000 7fd872fb7740 1 --2- 172.21.15.12:0/1605448035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c437930 0x56551c43c430 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.285+0000 7fd872fb7740 1 --2- 172.21.15.12:0/1605448035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c436f40 0x56551c437360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.285+0000 7fd872fb7740 1 -- 172.21.15.12:0/1605448035 >> 172.21.15.12:0/1605448035 conn(0x56551c4325b0 msgr2=0x56551c4329b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.285+0000 7fd872fb7740 1 -- 172.21.15.12:0/1605448035 shutdown_connections 2024-05-09T09:26:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.285+0000 7fd872fb7740 1 -- 172.21.15.12:0/1605448035 wait complete. 2024-05-09T09:26:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.285+0000 7fd872fb7740 1 -- start start 2024-05-09T09:26:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd872fb7740 1 Processor -- start 2024-05-09T09:26:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd872fb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c436f40 0x56551c445960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd872fb7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c43caa0 0x56551c445ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd872fb7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c44aed0 0x56551c44d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd872fb7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56551c443e00 con 0x56551c44aed0 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd872fb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56551c356c40 con 0x56551c436f40 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd872fb7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56551c443f80 con 0x56551c43caa0 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd872fb7740 10 client.0 _ll_register_callbacks cb 0x56551c0aebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd86e84b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c44aed0 0x56551c44d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd86e04a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c436f40 0x56551c445960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd86e84b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c44aed0 0x56551c44d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34772/0 (socket says 172.21.15.12:34772) 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd86e04a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c436f40 0x56551c445960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47198/0 (socket says 172.21.15.12:47198) 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd86e84b640 1 -- 172.21.15.12:0/2143087026 learned_addr learned my addr 172.21.15.12:0/2143087026 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd86d849640 1 --2- 172.21.15.12:0/2143087026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c43caa0 0x56551c445ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd86e04a640 1 -- 172.21.15.12:0/2143087026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c43caa0 msgr2=0x56551c445ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd86e04a640 1 --2- 172.21.15.12:0/2143087026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c43caa0 0x56551c445ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd86e04a640 1 -- 172.21.15.12:0/2143087026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c44aed0 msgr2=0x56551c44d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd86e04a640 1 --2- 172.21.15.12:0/2143087026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c44aed0 0x56551c44d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.286+0000 7fd86e04a640 1 -- 172.21.15.12:0/2143087026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56551c1797d0 con 0x56551c436f40 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.287+0000 7fd86e04a640 1 --2- 172.21.15.12:0/2143087026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c436f40 0x56551c445960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x56551c437df0 tx=0x7fd8640045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96dc8564740362d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.287+0000 7fd851ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.287+0000 7fd851ffb640 1 -- 172.21.15.12:0/2143087026 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864037030 con 0x56551c436f40 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.287+0000 7fd872fb7740 1 -- 172.21.15.12:0/2143087026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd864030c20 con 0x56551c436f40 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.287+0000 7fd851ffb640 1 -- 172.21.15.12:0/2143087026 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd864030820 con 0x56551c436f40 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.287+0000 7fd851ffb640 1 -- 172.21.15.12:0/2143087026 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd864055050 con 0x56551c436f40 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.287+0000 7fd872fb7740 10 client.25451 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.287+0000 7fd872fb7740 1 -- 172.21.15.12:0/2143087026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56551c43f650 con 0x56551c436f40 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.288+0000 7fd851ffb640 1 -- 172.21.15.12:0/2143087026 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd864030c20 con 0x56551c436f40 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.288+0000 7fd851ffb640 1 -- 172.21.15.12:0/2143087026 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd864004bc0 con 0x56551c436f40 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.288+0000 7fd872fb7740 10 client.25451 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.288+0000 7fd872fb7740 10 client.25451 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.288+0000 7fd872fb7740 20 client.25451 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.288+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2143087026 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x56551c440a00 0x56551c442e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.288+0000 7fd872fb7740 4 client.25451 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.288+0000 7fd872fb7740 1 -- 172.21.15.12:0/2143087026 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56551c3a9370 con 0x56551c440a00 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.288+0000 7fd86d849640 1 --2- 172.21.15.12:0/2143087026 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x56551c440a00 0x56551c442e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd851ffb640 10 client.25451 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd86d849640 1 --2- 172.21.15.12:0/2143087026 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x56551c440a00 0x56551c442e50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e56da34b6c49318 server_cookie=1f1c7cc812170d06 in_seq=0 out_seq=0 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd851ffb640 1 -- 172.21.15.12:0/2143087026 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56551c3a9370 con 0x56551c440a00 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd851ffb640 10 client.25451 handle_command_reply: tid=0 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd872fb7740 1 client.25451 shutdown 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd872fb7740 1 -- 172.21.15.12:0/2143087026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c436f40 msgr2=0x56551c445960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2143087026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c436f40 0x56551c445960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x56551c437df0 tx=0x7fd8640045e0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd872fb7740 1 -- 172.21.15.12:0/2143087026 shutdown_connections 2024-05-09T09:26:43.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2143087026 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x56551c440a00 0x56551c442e50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2143087026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c44aed0 0x56551c44d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2143087026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c43caa0 0x56551c445ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd851ffb640 0 client.25451 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:26:43.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2143087026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c436f40 0x56551c445960 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.289+0000 7fd872fb7740 1 -- 172.21.15.12:0/2143087026 >> 172.21.15.12:0/2143087026 conn(0x56551c4325b0 msgr2=0x56551c4317b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.290+0000 7fd872fb7740 1 -- 172.21.15.12:0/2143087026 shutdown_connections 2024-05-09T09:26:43.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.290+0000 7fd872fb7740 1 -- 172.21.15.12:0/2143087026 wait complete. 2024-05-09T09:26:43.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.290+0000 7fd872fb7740 20 client.25451 trim_cache size 0 max 16384 2024-05-09T09:26:43.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.303+0000 7fd872fb7740 1 Processor -- start 2024-05-09T09:26:43.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.303+0000 7fd872fb7740 1 -- start start 2024-05-09T09:26:43.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.303+0000 7fd872fb7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 0x56551c430f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.303+0000 7fd872fb7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c43caa0 0x56551c431460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd872fb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c44aed0 0x56551c4319a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd872fb7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd8681a8260 con 0x56551c436f40 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd872fb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8681a9da0 con 0x56551c44aed0 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd872fb7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd8681a8930 con 0x56551c43caa0 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd86e04a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 0x56551c430f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd86e04a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 0x56551c430f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34782/0 (socket says 172.21.15.12:34782) 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd86e04a640 1 -- 172.21.15.12:0/2612783945 learned_addr learned my addr 172.21.15.12:0/2612783945 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd86e84b640 1 --2- 172.21.15.12:0/2612783945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c44aed0 0x56551c4319a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd86d849640 1 --2- 172.21.15.12:0/2612783945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c43caa0 0x56551c431460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd86e04a640 1 -- 172.21.15.12:0/2612783945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c43caa0 msgr2=0x56551c431460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd86e04a640 1 --2- 172.21.15.12:0/2612783945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c43caa0 0x56551c431460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd86e04a640 1 -- 172.21.15.12:0/2612783945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c44aed0 msgr2=0x56551c4319a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd86e04a640 1 --2- 172.21.15.12:0/2612783945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c44aed0 0x56551c4319a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd86e04a640 1 -- 172.21.15.12:0/2612783945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56551c423d20 con 0x56551c436f40 2024-05-09T09:26:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd86e84b640 1 --2- 172.21.15.12:0/2612783945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c44aed0 0x56551c4319a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:43.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.304+0000 7fd86e04a640 1 --2- 172.21.15.12:0/2612783945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 0x56551c430f20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd864004650 tx=0x7fd864030540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbef211971130ff5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.305+0000 7fd853fff640 1 -- 172.21.15.12:0/2612783945 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86405e070 con 0x56551c436f40 2024-05-09T09:26:43.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.305+0000 7fd853fff640 1 -- 172.21.15.12:0/2612783945 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd864030e20 con 0x56551c436f40 2024-05-09T09:26:43.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.305+0000 7fd853fff640 1 -- 172.21.15.12:0/2612783945 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864037070 con 0x56551c436f40 2024-05-09T09:26:43.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.306+0000 7fd872fb7740 1 -- 172.21.15.12:0/2612783945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 msgr2=0x56551c430f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.306+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2612783945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 0x56551c430f20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd864004650 tx=0x7fd864030540 comp rx=0 tx=0).stop 2024-05-09T09:26:43.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.306+0000 7fd872fb7740 1 -- 172.21.15.12:0/2612783945 shutdown_connections 2024-05-09T09:26:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.306+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2612783945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c44aed0 0x56551c4319a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.306+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2612783945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c43caa0 0x56551c431460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.306+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2612783945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 0x56551c430f20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.306+0000 7fd872fb7740 1 -- 172.21.15.12:0/2612783945 >> 172.21.15.12:0/2612783945 conn(0x56551c4325b0 msgr2=0x56551c444320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.306+0000 7fd872fb7740 1 -- 172.21.15.12:0/2612783945 shutdown_connections 2024-05-09T09:26:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.306+0000 7fd872fb7740 1 -- 172.21.15.12:0/2612783945 wait complete. 2024-05-09T09:26:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.306+0000 7fd872fb7740 1 -- start start 2024-05-09T09:26:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.306+0000 7fd872fb7740 1 Processor -- start 2024-05-09T09:26:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd872fb7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 0x56551c454f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd872fb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c43caa0 0x56551c4554b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd872fb7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c44aed0 0x56551c4b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd86e04a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 0x56551c454f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd86e04a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 0x56551c454f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34792/0 (socket says 172.21.15.12:34792) 2024-05-09T09:26:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd86e04a640 1 -- 172.21.15.12:0/2692203930 learned_addr learned my addr 172.21.15.12:0/2692203930 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd872fb7740 1 -- 172.21.15.12:0/2692203930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd8681a96d0 con 0x56551c436f40 2024-05-09T09:26:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd872fb7740 1 -- 172.21.15.12:0/2692203930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8681a9000 con 0x56551c43caa0 2024-05-09T09:26:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd872fb7740 1 -- 172.21.15.12:0/2692203930 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd8681a6240 con 0x56551c44aed0 2024-05-09T09:26:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd872fb7740 10 client.0 _ll_register_callbacks cb 0x7fd8580d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd86d849640 1 --2- 172.21.15.12:0/2692203930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c43caa0 0x56551c4554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd86e84b640 1 --2- 172.21.15.12:0/2692203930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c44aed0 0x56551c4b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd86e04a640 1 -- 172.21.15.12:0/2692203930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c44aed0 msgr2=0x56551c4b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd86e04a640 1 --2- 172.21.15.12:0/2692203930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c44aed0 0x56551c4b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd86e04a640 1 -- 172.21.15.12:0/2692203930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c43caa0 msgr2=0x56551c4554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd86e04a640 1 --2- 172.21.15.12:0/2692203930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c43caa0 0x56551c4554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.307+0000 7fd86e04a640 1 -- 172.21.15.12:0/2692203930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56551c4b2850 con 0x56551c436f40 2024-05-09T09:26:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.308+0000 7fd86e04a640 1 --2- 172.21.15.12:0/2692203930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 0x56551c454f70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd8640059f0 tx=0x7fd86402a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4ef3332948156 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.308+0000 7fd851ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.308+0000 7fd851ffb640 1 -- 172.21.15.12:0/2692203930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86405e030 con 0x56551c436f40 2024-05-09T09:26:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.308+0000 7fd851ffb640 1 -- 172.21.15.12:0/2692203930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd864030c20 con 0x56551c436f40 2024-05-09T09:26:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.308+0000 7fd872fb7740 1 -- 172.21.15.12:0/2692203930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd864030e20 con 0x56551c436f40 2024-05-09T09:26:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.308+0000 7fd851ffb640 1 -- 172.21.15.12:0/2692203930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd86406b050 con 0x56551c436f40 2024-05-09T09:26:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.308+0000 7fd872fb7740 10 client.15948 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.308+0000 7fd872fb7740 1 -- 172.21.15.12:0/2692203930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56551c4b3b50 con 0x56551c436f40 2024-05-09T09:26:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.308+0000 7fd851ffb640 1 -- 172.21.15.12:0/2692203930 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd86403edf0 con 0x56551c436f40 2024-05-09T09:26:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.308+0000 7fd851ffb640 1 -- 172.21.15.12:0/2692203930 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd8640046c0 con 0x56551c436f40 2024-05-09T09:26:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.309+0000 7fd872fb7740 10 client.15948 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.309+0000 7fd872fb7740 10 client.15948 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:26:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.309+0000 7fd872fb7740 20 client.15948 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.309+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2692203930 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x56551c4b44a0 0x56551c4b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.309+0000 7fd872fb7740 4 client.15948 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.309+0000 7fd872fb7740 1 -- 172.21.15.12:0/2692203930 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd864030e20 con 0x56551c4b44a0 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.309+0000 7fd86d849640 1 --2- 172.21.15.12:0/2692203930 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x56551c4b44a0 0x56551c4b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.309+0000 7fd86d849640 1 --2- 172.21.15.12:0/2692203930 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x56551c4b44a0 0x56551c4b68c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2791a873f94eb96b server_cookie=ae2dd4afcd184abb in_seq=0 out_seq=0 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.309+0000 7fd851ffb640 10 client.15948 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.309+0000 7fd851ffb640 1 -- 172.21.15.12:0/2692203930 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd864030e20 con 0x56551c4b44a0 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.309+0000 7fd851ffb640 10 client.15948 handle_command_reply: tid=0 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.309+0000 7fd872fb7740 1 client.15948 shutdown 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.310+0000 7fd872fb7740 1 -- 172.21.15.12:0/2692203930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 msgr2=0x56551c454f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.310+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2692203930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 0x56551c454f70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd8640059f0 tx=0x7fd86402a340 comp rx=0 tx=0).stop 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.310+0000 7fd872fb7740 1 -- 172.21.15.12:0/2692203930 shutdown_connections 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.310+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2692203930 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x56551c4b44a0 0x56551c4b68c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.310+0000 7fd851ffb640 0 client.15948 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.310+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2692203930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56551c44aed0 0x56551c4b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.310+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2692203930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551c43caa0 0x56551c4554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.310+0000 7fd872fb7740 1 --2- 172.21.15.12:0/2692203930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56551c436f40 0x56551c454f70 secure :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd8640059f0 tx=0x7fd86402a340 comp rx=0 tx=0).stop 2024-05-09T09:26:43.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.310+0000 7fd872fb7740 1 -- 172.21.15.12:0/2692203930 >> 172.21.15.12:0/2692203930 conn(0x56551c4325b0 msgr2=0x56551c430f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.310+0000 7fd872fb7740 1 -- 172.21.15.12:0/2692203930 shutdown_connections 2024-05-09T09:26:43.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.310+0000 7fd872fb7740 1 -- 172.21.15.12:0/2692203930 wait complete. 2024-05-09T09:26:43.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.310+0000 7fd872fb7740 20 client.15948 trim_cache size 0 max 16384 2024-05-09T09:26:43.315 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:43.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.311+0000 7fd8702ae640 1 -- 172.21.15.12:0/3202971890 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd8540a2c10 msgr2=0x7fd8540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.311+0000 7fd8702ae640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd8540a2c10 0x7fd8540a50d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd860009fd0 tx=0x7fd8600095c0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.311+0000 7fd8702ae640 1 -- 172.21.15.12:0/3202971890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd868101960 msgr2=0x7fd868198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.311+0000 7fd8702ae640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd868101960 0x7fd868198180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd85800e9f0 tx=0x7fd85800edf0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.311+0000 7fd8702ae640 1 -- 172.21.15.12:0/3202971890 shutdown_connections 2024-05-09T09:26:43.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.311+0000 7fd8702ae640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd8540a2c10 0x7fd8540a50d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.311+0000 7fd8702ae640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd868102320 0x7fd86819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.311+0000 7fd8702ae640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd868101960 0x7fd868198180 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.311+0000 7fd8702ae640 1 --2- 172.21.15.12:0/3202971890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd868100770 0x7fd868197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.311+0000 7fd8702ae640 1 -- 172.21.15.12:0/3202971890 >> 172.21.15.12:0/3202971890 conn(0x7fd8680fbce0 msgr2=0x7fd8680689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.311+0000 7fd8702ae640 1 -- 172.21.15.12:0/3202971890 shutdown_connections 2024-05-09T09:26:43.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.311+0000 7fd8702ae640 1 -- 172.21.15.12:0/3202971890 wait complete. 2024-05-09T09:26:43.325 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-09T09:26:43.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.437+0000 7f15cea3f640 1 -- 172.21.15.12:0/1184835340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15c8100760 msgr2=0x7f15c8100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.437+0000 7f15cea3f640 1 --2- 172.21.15.12:0/1184835340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15c8100760 0x7f15c8100b80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009900 tx=0x7f15b802a410 comp rx=0 tx=0).stop 2024-05-09T09:26:43.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.437+0000 7f15cea3f640 1 -- 172.21.15.12:0/1184835340 shutdown_connections 2024-05-09T09:26:43.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.437+0000 7f15cea3f640 1 --2- 172.21.15.12:0/1184835340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c8102310 0x7f15c8108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.437+0000 7f15cea3f640 1 --2- 172.21.15.12:0/1184835340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15c8101950 0x7f15c8101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.437+0000 7f15cea3f640 1 --2- 172.21.15.12:0/1184835340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15c8100760 0x7f15c8100b80 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.437+0000 7f15cea3f640 1 -- 172.21.15.12:0/1184835340 >> 172.21.15.12:0/1184835340 conn(0x7f15c80fbcf0 msgr2=0x7f15c80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.437+0000 7f15cea3f640 1 -- 172.21.15.12:0/1184835340 shutdown_connections 2024-05-09T09:26:43.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.437+0000 7f15cea3f640 1 -- 172.21.15.12:0/1184835340 wait complete. 2024-05-09T09:26:43.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.438+0000 7f15cea3f640 1 Processor -- start 2024-05-09T09:26:43.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.438+0000 7f15cea3f640 1 -- start start 2024-05-09T09:26:43.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.438+0000 7f15cea3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c8101950 0x7f15c8197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.438+0000 7f15cea3f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15c8102310 0x7f15c8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.438+0000 7f15cea3f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15c819d380 0x7f15c819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.438+0000 7f15cea3f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f15c810d4d0 con 0x7f15c819d380 2024-05-09T09:26:43.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.438+0000 7f15cea3f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15c810d350 con 0x7f15c8101950 2024-05-09T09:26:43.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.438+0000 7f15cea3f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15c810d650 con 0x7f15c8102310 2024-05-09T09:26:43.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.439+0000 7f15ccfdc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15c819d380 0x7f15c819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.439+0000 7f15ccfdc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15c819d380 0x7f15c819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34808/0 (socket says 172.21.15.12:34808) 2024-05-09T09:26:43.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.439+0000 7f15ccfdc640 1 -- 172.21.15.12:0/585637098 learned_addr learned my addr 172.21.15.12:0/585637098 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.439+0000 7f15c7fff640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c8101950 0x7f15c8197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.439+0000 7f15c7fff640 1 -- 172.21.15.12:0/585637098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15c8102310 msgr2=0x7f15c8198320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:26:43.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.439+0000 7f15c7fff640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15c8102310 0x7f15c8198320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.439+0000 7f15c7fff640 1 -- 172.21.15.12:0/585637098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15c819d380 msgr2=0x7f15c819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.439+0000 7f15c7fff640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15c819d380 0x7f15c819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.439+0000 7f15c7fff640 1 -- 172.21.15.12:0/585637098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c8109560 con 0x7f15c8101950 2024-05-09T09:26:43.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.439+0000 7f15ccfdc640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15c819d380 0x7f15c819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:43.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.439+0000 7f15c7fff640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c8101950 0x7f15c8197de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f15c81002a0 tx=0x7f15b80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.440+0000 7f15c57fa640 1 -- 172.21.15.12:0/585637098 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b803f030 con 0x7f15c8101950 2024-05-09T09:26:43.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.440+0000 7f15cea3f640 1 -- 172.21.15.12:0/585637098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c81097f0 con 0x7f15c8101950 2024-05-09T09:26:43.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.440+0000 7f15cea3f640 1 -- 172.21.15.12:0/585637098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c8109cb0 con 0x7f15c8101950 2024-05-09T09:26:43.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.440+0000 7f15c57fa640 1 -- 172.21.15.12:0/585637098 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f15b8003ac0 con 0x7f15c8101950 2024-05-09T09:26:43.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.440+0000 7f15c57fa640 1 -- 172.21.15.12:0/585637098 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8035070 con 0x7f15c8101950 2024-05-09T09:26:43.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.443+0000 7f15d182f740 1 Processor -- start 2024-05-09T09:26:43.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.443+0000 7f15d182f740 1 -- start start 2024-05-09T09:26:43.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.443+0000 7f15d182f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f09c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.443+0000 7f15d182f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0a1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.443+0000 7f15d182f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f0a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.443+0000 7f15d182f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563f8f000c40 con 0x563f8f0a1aa0 2024-05-09T09:26:43.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.443+0000 7f15d182f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563f8efa8580 con 0x563f8f09c930 2024-05-09T09:26:43.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.443+0000 7f15d182f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563f8effcc00 con 0x563f8f09bf40 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.443+0000 7f15c57fa640 1 -- 172.21.15.12:0/585637098 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f15b8042030 con 0x7f15c8101950 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.443+0000 7f15ccfdc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f09c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.443+0000 7f15c77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f0a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.443+0000 7f15ccfdc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f09c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:36880/0 (socket says 172.21.15.12:36880) 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.443+0000 7f15ccfdc640 1 -- 172.21.15.12:0/1828020639 learned_addr learned my addr 172.21.15.12:0/1828020639 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.444+0000 7f15c77fe640 1 -- 172.21.15.12:0/1828020639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 msgr2=0x563f8f09c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.444+0000 7f15c77fe640 1 --2- 172.21.15.12:0/1828020639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f09c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.444+0000 7f15c77fe640 1 -- 172.21.15.12:0/1828020639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 msgr2=0x563f8f0a1430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.444+0000 7f15c77fe640 1 --2- 172.21.15.12:0/1828020639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0a1430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.444+0000 7f15c77fe640 1 -- 172.21.15.12:0/1828020639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8ed143f0 con 0x563f8f0a1aa0 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.444+0000 7f15ccfdc640 1 --2- 172.21.15.12:0/1828020639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f09c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.444+0000 7f15c77fe640 1 --2- 172.21.15.12:0/1828020639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f0a3f30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009aa0 tx=0x7f15bc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d639574944589a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.444+0000 7f15c57fa640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f15b00a2d90 0x7f15b00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.444+0000 7f15ae7fc640 1 -- 172.21.15.12:0/1828020639 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00ece0 con 0x563f8f0a1aa0 2024-05-09T09:26:43.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.444+0000 7f15c57fa640 1 -- 172.21.15.12:0/585637098 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f15b803b720 con 0x7f15c8101950 2024-05-09T09:26:43.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.444+0000 7f15ae7fc640 1 -- 172.21.15.12:0/1828020639 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f15bc00ee80 con 0x563f8f0a1aa0 2024-05-09T09:26:43.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.445+0000 7f15ae7fc640 1 -- 172.21.15.12:0/1828020639 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc004ad0 con 0x563f8f0a1aa0 2024-05-09T09:26:43.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.445+0000 7f15ccfdc640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f15b00a2d90 0x7f15b00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.445+0000 7f15d182f740 1 -- 172.21.15.12:0/1828020639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 msgr2=0x563f8f0a3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.445+0000 7f15ccfdc640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f15b00a2d90 0x7f15b00a5250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x563f8f09d190 tx=0x7f15c0009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.445+0000 7f15d182f740 1 --2- 172.21.15.12:0/1828020639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f0a3f30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009aa0 tx=0x7f15bc009f60 comp rx=0 tx=0).stop 2024-05-09T09:26:43.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.445+0000 7f15d182f740 1 -- 172.21.15.12:0/1828020639 shutdown_connections 2024-05-09T09:26:43.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.445+0000 7f15d182f740 1 --2- 172.21.15.12:0/1828020639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f0a3f30 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.445+0000 7f15d182f740 1 --2- 172.21.15.12:0/1828020639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.445+0000 7f15d182f740 1 --2- 172.21.15.12:0/1828020639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f09c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.445+0000 7f15d182f740 1 -- 172.21.15.12:0/1828020639 >> 172.21.15.12:0/1828020639 conn(0x563f8f0975b0 msgr2=0x563f8f0979b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.445+0000 7f15d182f740 1 -- 172.21.15.12:0/1828020639 shutdown_connections 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.446+0000 7f15d182f740 1 -- 172.21.15.12:0/1828020639 wait complete. 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.446+0000 7f15d182f740 1 -- start start 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.446+0000 7f15d182f740 1 Processor -- start 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.446+0000 7f15d182f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15d182f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0a6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15d182f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f0ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15d182f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563f8f0a4a50 con 0x563f8f0a1aa0 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15d182f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563f8efbbc40 con 0x563f8f09c930 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15ccfdc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f0ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15c7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15c7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f0a63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:36894/0 (socket says 172.21.15.12:36894) 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15c7fff640 1 -- 172.21.15.12:0/1334721792 learned_addr learned my addr 172.21.15.12:0/1334721792 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15d182f740 1 -- 172.21.15.12:0/1334721792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563f8f0a4bd0 con 0x563f8f09bf40 2024-05-09T09:26:43.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15c77fe640 1 --2- 172.21.15.12:0/1334721792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15d182f740 10 client.0 _ll_register_callbacks cb 0x563f8ed13be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15c7fff640 1 -- 172.21.15.12:0/1334721792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 msgr2=0x563f8f0a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15c7fff640 1 --2- 172.21.15.12:0/1334721792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15c7fff640 1 -- 172.21.15.12:0/1334721792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 msgr2=0x563f8f0ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15c7fff640 1 --2- 172.21.15.12:0/1334721792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f0ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15c7fff640 1 -- 172.21.15.12:0/1334721792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8edde7d0 con 0x563f8f09bf40 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.447+0000 7f15ccfdc640 1 --2- 172.21.15.12:0/1334721792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f0ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.448+0000 7f15c7fff640 1 --2- 172.21.15.12:0/1334721792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f0a63e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f15b80ed1b0 tx=0x7f15b80bbb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50022cacbd7e8807 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.448+0000 7f15a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.448+0000 7f15a77fe640 1 -- 172.21.15.12:0/1334721792 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8035030 con 0x563f8f09bf40 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.448+0000 7f15d182f740 1 -- 172.21.15.12:0/1334721792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563f8f00e370 con 0x563f8f09bf40 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.448+0000 7f15a77fe640 1 -- 172.21.15.12:0/1334721792 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f15b80b8580 con 0x563f8f09bf40 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.448+0000 7f15a77fe640 1 -- 172.21.15.12:0/1334721792 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f15b80bbf10 con 0x563f8f09bf40 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.448+0000 7f15d182f740 10 client.25585 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:43.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.448+0000 7f15d182f740 1 -- 172.21.15.12:0/1334721792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563f8f0a4b10 con 0x563f8f09bf40 2024-05-09T09:26:43.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.448+0000 7f15a77fe640 1 -- 172.21.15.12:0/1334721792 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x563f8f00e370 con 0x563f8f09bf40 2024-05-09T09:26:43.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.449+0000 7f15a77fe640 1 -- 172.21.15.12:0/1334721792 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f15b80bc170 con 0x563f8f09bf40 2024-05-09T09:26:43.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.449+0000 7f15d182f740 10 client.25585 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:43.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.449+0000 7f15d182f740 10 client.25585 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:26:43.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.449+0000 7f15d182f740 20 client.25585 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:43.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.449+0000 7f15d182f740 1 --2- 172.21.15.12:0/1334721792 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x563f8f0b23f0 0x563f8f0b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.449+0000 7f15d182f740 4 client.25585 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:43.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.449+0000 7f15d182f740 1 -- 172.21.15.12:0/1334721792 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f8f0086c0 con 0x563f8f0b23f0 2024-05-09T09:26:43.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.450+0000 7f15c77fe640 1 --2- 172.21.15.12:0/1334721792 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x563f8f0b23f0 0x563f8f0b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.450+0000 7f15a77fe640 10 client.25585 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:26:43.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.450+0000 7f15c77fe640 1 --2- 172.21.15.12:0/1334721792 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x563f8f0b23f0 0x563f8f0b4890 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c520319874e540 server_cookie=74873ffa64d76daf in_seq=0 out_seq=0 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15a77fe640 1 -- 172.21.15.12:0/1334721792 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f8f0086c0 con 0x563f8f0b23f0 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15a77fe640 10 client.25585 handle_command_reply: tid=0 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15d182f740 1 client.25585 shutdown 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15d182f740 1 -- 172.21.15.12:0/1334721792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 msgr2=0x563f8f0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15d182f740 1 --2- 172.21.15.12:0/1334721792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f0a63e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f15b80ed1b0 tx=0x7f15b80bbb10 comp rx=0 tx=0).stop 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15d182f740 1 -- 172.21.15.12:0/1334721792 shutdown_connections 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15d182f740 1 --2- 172.21.15.12:0/1334721792 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x563f8f0b23f0 0x563f8f0b4890 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15d182f740 1 --2- 172.21.15.12:0/1334721792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f0ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15d182f740 1 --2- 172.21.15.12:0/1334721792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15d182f740 1 --2- 172.21.15.12:0/1334721792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f0a63e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15d182f740 1 -- 172.21.15.12:0/1334721792 >> 172.21.15.12:0/1334721792 conn(0x563f8f0975b0 msgr2=0x563f8f0967b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15a77fe640 0 client.25585 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15d182f740 1 -- 172.21.15.12:0/1334721792 shutdown_connections 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.451+0000 7f15d182f740 1 -- 172.21.15.12:0/1334721792 wait complete. 2024-05-09T09:26:43.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.452+0000 7f15d182f740 20 client.25585 trim_cache size 0 max 16384 2024-05-09T09:26:43.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.465+0000 7f15d182f740 1 Processor -- start 2024-05-09T09:26:43.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.465+0000 7f15d182f740 1 -- start start 2024-05-09T09:26:43.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.465+0000 7f15d182f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f09bf40 0x563f8f0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.465+0000 7f15d182f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.465+0000 7f15d182f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f0a1aa0 0x563f8f09a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.465+0000 7f15d182f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f15b400fc70 con 0x563f8f09bf40 2024-05-09T09:26:43.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.465+0000 7f15d182f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15b40117b0 con 0x563f8f09c930 2024-05-09T09:26:43.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.465+0000 7f15d182f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4010340 con 0x563f8f0a1aa0 2024-05-09T09:26:43.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.465+0000 7f15c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f09bf40 0x563f8f0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.465+0000 7f15c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f09bf40 0x563f8f0a5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34838/0 (socket says 172.21.15.12:34838) 2024-05-09T09:26:43.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.465+0000 7f15c7fff640 1 -- 172.21.15.12:0/2520461796 learned_addr learned my addr 172.21.15.12:0/2520461796 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.466+0000 7f15c77fe640 1 --2- 172.21.15.12:0/2520461796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.466+0000 7f15ccfdc640 1 --2- 172.21.15.12:0/2520461796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f0a1aa0 0x563f8f09a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.466+0000 7f15c7fff640 1 -- 172.21.15.12:0/2520461796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f0a1aa0 msgr2=0x563f8f09a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.466+0000 7f15c7fff640 1 --2- 172.21.15.12:0/2520461796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f0a1aa0 0x563f8f09a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.466+0000 7f15c7fff640 1 -- 172.21.15.12:0/2520461796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 msgr2=0x563f8f0ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.466+0000 7f15c7fff640 1 --2- 172.21.15.12:0/2520461796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.466+0000 7f15c7fff640 1 -- 172.21.15.12:0/2520461796 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8f098020 con 0x563f8f09bf40 2024-05-09T09:26:43.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.466+0000 7f15ccfdc640 1 --2- 172.21.15.12:0/2520461796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f0a1aa0 0x563f8f09a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:43.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.466+0000 7f15c77fe640 1 --2- 172.21.15.12:0/2520461796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:43.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.466+0000 7f15c7fff640 1 --2- 172.21.15.12:0/2520461796 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f09bf40 0x563f8f0a5710 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f15b8000c00 tx=0x7f15b80ba420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b43f4a4281394 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.466+0000 7f15adffb640 1 -- 172.21.15.12:0/2520461796 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8035070 con 0x563f8f09bf40 2024-05-09T09:26:43.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.466+0000 7f15adffb640 1 -- 172.21.15.12:0/2520461796 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f15b80c4b80 con 0x563f8f09bf40 2024-05-09T09:26:43.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.466+0000 7f15adffb640 1 -- 172.21.15.12:0/2520461796 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b80c46f0 con 0x563f8f09bf40 2024-05-09T09:26:43.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.467+0000 7f15d182f740 1 -- 172.21.15.12:0/2520461796 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f09bf40 msgr2=0x563f8f0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.467+0000 7f15d182f740 1 --2- 172.21.15.12:0/2520461796 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f09bf40 0x563f8f0a5710 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f15b8000c00 tx=0x7f15b80ba420 comp rx=0 tx=0).stop 2024-05-09T09:26:43.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.467+0000 7f15d182f740 1 -- 172.21.15.12:0/2520461796 shutdown_connections 2024-05-09T09:26:43.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.467+0000 7f15d182f740 1 --2- 172.21.15.12:0/2520461796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f0a1aa0 0x563f8f09a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.467+0000 7f15d182f740 1 --2- 172.21.15.12:0/2520461796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.467+0000 7f15d182f740 1 --2- 172.21.15.12:0/2520461796 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f09bf40 0x563f8f0a5710 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.467+0000 7f15d182f740 1 -- 172.21.15.12:0/2520461796 >> 172.21.15.12:0/2520461796 conn(0x563f8f0975b0 msgr2=0x563f8f0a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.467+0000 7f15d182f740 1 -- 172.21.15.12:0/2520461796 shutdown_connections 2024-05-09T09:26:43.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.467+0000 7f15d182f740 1 -- 172.21.15.12:0/2520461796 wait complete. 2024-05-09T09:26:43.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.468+0000 7f15d182f740 1 -- start start 2024-05-09T09:26:43.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.468+0000 7f15d182f740 1 Processor -- start 2024-05-09T09:26:43.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.468+0000 7f15d182f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f0aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.468+0000 7f15d182f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.468+0000 7f15d182f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f116970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.468+0000 7f15d182f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f15b40110e0 con 0x563f8f0a1aa0 2024-05-09T09:26:43.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.468+0000 7f15d182f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4010a10 con 0x563f8f09c930 2024-05-09T09:26:43.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.468+0000 7f15d182f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15b400dc50 con 0x563f8f09bf40 2024-05-09T09:26:43.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.468+0000 7f15d182f740 10 client.0 _ll_register_callbacks cb 0x7f15b8031a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:43.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.468+0000 7f15ccfdc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f116970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.468+0000 7f15ccfdc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f116970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34844/0 (socket says 172.21.15.12:34844) 2024-05-09T09:26:43.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.468+0000 7f15ccfdc640 1 -- 172.21.15.12:0/2389660279 learned_addr learned my addr 172.21.15.12:0/2389660279 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.468+0000 7f15c77fe640 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.469+0000 7f15c7fff640 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f0aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.469+0000 7f15ccfdc640 1 -- 172.21.15.12:0/2389660279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 msgr2=0x563f8f0aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.469+0000 7f15ccfdc640 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f0aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.469+0000 7f15ccfdc640 1 -- 172.21.15.12:0/2389660279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 msgr2=0x563f8f0aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.469+0000 7f15ccfdc640 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.469+0000 7f15ccfdc640 1 -- 172.21.15.12:0/2389660279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8f116eb0 con 0x563f8f0a1aa0 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.469+0000 7f15c7fff640 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.469+0000 7f15ccfdc640 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f116970 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f15c0012fe0 tx=0x7f15c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88fb61da3bf6d4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.469+0000 7f15a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.469+0000 7f15a77fe640 1 -- 172.21.15.12:0/2389660279 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0004b20 con 0x563f8f0a1aa0 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.469+0000 7f15a77fe640 1 -- 172.21.15.12:0/2389660279 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f15c0004cc0 con 0x563f8f0a1aa0 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.469+0000 7f15d182f740 1 -- 172.21.15.12:0/2389660279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15b80c4b80 con 0x563f8f0a1aa0 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.470+0000 7f15a77fe640 1 -- 172.21.15.12:0/2389660279 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f15c0034050 con 0x563f8f0a1aa0 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.470+0000 7f15d182f740 10 client.15966 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.470+0000 7f15d182f740 1 -- 172.21.15.12:0/2389660279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563f8f1181a0 con 0x563f8f0a1aa0 2024-05-09T09:26:43.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.470+0000 7f15a77fe640 1 -- 172.21.15.12:0/2389660279 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f15b80c4b80 con 0x563f8f0a1aa0 2024-05-09T09:26:43.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.470+0000 7f15a77fe640 1 -- 172.21.15.12:0/2389660279 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f15c0028070 con 0x563f8f0a1aa0 2024-05-09T09:26:43.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.470+0000 7f15d182f740 10 client.15966 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:43.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.470+0000 7f15d182f740 10 client.15966 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:26:43.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.470+0000 7f15d182f740 20 client.15966 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:43.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.470+0000 7f15d182f740 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x563f8f118b80 0x563f8f11afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.470+0000 7f15d182f740 4 client.15966 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:43.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.470+0000 7f15d182f740 1 -- 172.21.15.12:0/2389660279 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f8ef85820 con 0x563f8f118b80 2024-05-09T09:26:43.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.471+0000 7f15c7fff640 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x563f8f118b80 0x563f8f11afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.471+0000 7f15c7fff640 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x563f8f118b80 0x563f8f11afa0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15c2b2dbebce9f91 server_cookie=de7474f8e588afa6 in_seq=0 out_seq=0 2024-05-09T09:26:43.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.471+0000 7f15a77fe640 10 client.15966 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.471+0000 7f15a77fe640 1 -- 172.21.15.12:0/2389660279 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f8ef85820 con 0x563f8f118b80 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.471+0000 7f15a77fe640 10 client.15966 handle_command_reply: tid=0 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.471+0000 7f15d182f740 1 client.15966 shutdown 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.471+0000 7f15d182f740 1 -- 172.21.15.12:0/2389660279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 msgr2=0x563f8f116970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.471+0000 7f15d182f740 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f116970 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f15c0012fe0 tx=0x7f15c0004ab0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.471+0000 7f15d182f740 1 -- 172.21.15.12:0/2389660279 shutdown_connections 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.471+0000 7f15d182f740 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x563f8f118b80 0x563f8f11afa0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.471+0000 7f15d182f740 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f8f0a1aa0 0x563f8f116970 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.472+0000 7f15d182f740 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f8f09c930 0x563f8f0aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.472+0000 7f15d182f740 1 --2- 172.21.15.12:0/2389660279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f8f09bf40 0x563f8f0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.472+0000 7f15a77fe640 0 client.15966 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.472+0000 7f15d182f740 1 -- 172.21.15.12:0/2389660279 >> 172.21.15.12:0/2389660279 conn(0x563f8f0975b0 msgr2=0x563f8f0a5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.472+0000 7f15d182f740 1 -- 172.21.15.12:0/2389660279 shutdown_connections 2024-05-09T09:26:43.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.472+0000 7f15d182f740 1 -- 172.21.15.12:0/2389660279 wait complete. 2024-05-09T09:26:43.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.472+0000 7f15d182f740 20 client.15966 trim_cache size 0 max 16384 2024-05-09T09:26:43.478 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:43.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.472+0000 7f15cea3f640 1 -- 172.21.15.12:0/585637098 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f15b00a2d90 msgr2=0x7f15b00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.472+0000 7f15cea3f640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f15b00a2d90 0x7f15b00a5250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x563f8f09d190 tx=0x7f15c0009680 comp rx=0 tx=0).stop 2024-05-09T09:26:43.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.472+0000 7f15cea3f640 1 -- 172.21.15.12:0/585637098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c8101950 msgr2=0x7f15c8197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.472+0000 7f15cea3f640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c8101950 0x7f15c8197de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f15c81002a0 tx=0x7f15b80042f0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.473+0000 7f15cea3f640 1 -- 172.21.15.12:0/585637098 shutdown_connections 2024-05-09T09:26:43.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.473+0000 7f15cea3f640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f15b00a2d90 0x7f15b00a5250 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.473+0000 7f15cea3f640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15c819d380 0x7f15c819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.473+0000 7f15cea3f640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15c8102310 0x7f15c8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.473+0000 7f15cea3f640 1 --2- 172.21.15.12:0/585637098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c8101950 0x7f15c8197de0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.473+0000 7f15cea3f640 1 -- 172.21.15.12:0/585637098 >> 172.21.15.12:0/585637098 conn(0x7f15c80fbcf0 msgr2=0x7f15c8104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.473+0000 7f15cea3f640 1 -- 172.21.15.12:0/585637098 shutdown_connections 2024-05-09T09:26:43.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.473+0000 7f15cea3f640 1 -- 172.21.15.12:0/585637098 wait complete. 2024-05-09T09:26:43.484 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:26:43.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.594+0000 7f8f0f49a640 1 -- 172.21.15.12:0/674941327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f08102320 msgr2=0x7f8f08108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.594+0000 7f8f0f49a640 1 --2- 172.21.15.12:0/674941327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f08102320 0x7f8f08108bb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009ca0 tx=0x7f8f0002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.594+0000 7f8f0f49a640 1 -- 172.21.15.12:0/674941327 shutdown_connections 2024-05-09T09:26:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.594+0000 7f8f0f49a640 1 --2- 172.21.15.12:0/674941327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f08102320 0x7f8f08108bb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.594+0000 7f8f0f49a640 1 --2- 172.21.15.12:0/674941327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f08101960 0x7f8f08101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.594+0000 7f8f0f49a640 1 --2- 172.21.15.12:0/674941327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f08100770 0x7f8f08100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.594+0000 7f8f0f49a640 1 -- 172.21.15.12:0/674941327 >> 172.21.15.12:0/674941327 conn(0x7f8f080fbce0 msgr2=0x7f8f080fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.595+0000 7f8f0f49a640 1 -- 172.21.15.12:0/674941327 shutdown_connections 2024-05-09T09:26:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.595+0000 7f8f0f49a640 1 -- 172.21.15.12:0/674941327 wait complete. 2024-05-09T09:26:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.595+0000 7f8f0f49a640 1 Processor -- start 2024-05-09T09:26:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.595+0000 7f8f0f49a640 1 -- start start 2024-05-09T09:26:43.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.595+0000 7f8f0f49a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f08100770 0x7f8f08104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.595+0000 7f8f0f49a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f08101960 0x7f8f08102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.595+0000 7f8f0f49a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f08102320 0x7f8f08103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0f49a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810c420 con 0x7f8f08102320 2024-05-09T09:26:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0f49a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810c2a0 con 0x7f8f08101960 2024-05-09T09:26:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0f49a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810c5a0 con 0x7f8f08100770 2024-05-09T09:26:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0da37640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f08102320 0x7f8f08103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0da37640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f08102320 0x7f8f08103280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34868/0 (socket says 172.21.15.12:34868) 2024-05-09T09:26:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0da37640 1 -- 172.21.15.12:0/680637105 learned_addr learned my addr 172.21.15.12:0/680637105 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f08101960 0x7f8f08102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0d236640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f08100770 0x7f8f08104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0da37640 1 -- 172.21.15.12:0/680637105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f08100770 msgr2=0x7f8f08104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0da37640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f08100770 0x7f8f08104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0da37640 1 -- 172.21.15.12:0/680637105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f08101960 msgr2=0x7f8f08102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0da37640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f08101960 0x7f8f08102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0da37640 1 -- 172.21.15.12:0/680637105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f08103b10 con 0x7f8f08102320 2024-05-09T09:26:43.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8f0da37640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f08102320 0x7f8f08103280 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8f0002ade0 tx=0x7f8f0002f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.596+0000 7f8efe7fc640 1 -- 172.21.15.12:0/680637105 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00003b70 con 0x7f8f08102320 2024-05-09T09:26:43.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.597+0000 7f8efe7fc640 1 -- 172.21.15.12:0/680637105 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8f00003d10 con 0x7f8f08102320 2024-05-09T09:26:43.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.597+0000 7f8efe7fc640 1 -- 172.21.15.12:0/680637105 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0002fc20 con 0x7f8f08102320 2024-05-09T09:26:43.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.597+0000 7f8f0f49a640 1 -- 172.21.15.12:0/680637105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f081a1cd0 con 0x7f8f08102320 2024-05-09T09:26:43.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.597+0000 7f8f0f49a640 1 -- 172.21.15.12:0/680637105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f081a21d0 con 0x7f8f08102320 2024-05-09T09:26:43.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.599+0000 7f8efe7fc640 1 -- 172.21.15.12:0/680637105 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8f00004040 con 0x7f8f08102320 2024-05-09T09:26:43.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.599+0000 7f8f121a1740 1 Processor -- start 2024-05-09T09:26:43.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f121a1740 1 -- start start 2024-05-09T09:26:43.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f121a1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840cf40 0x55a9b840d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f121a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b840d930 0x55a9b8412430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f121a1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 0x55a9b8414f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f121a1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a9b8371c40 con 0x55a9b840cf40 2024-05-09T09:26:43.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f121a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a9b8319580 con 0x55a9b840d930 2024-05-09T09:26:43.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f121a1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a9b836dc00 con 0x55a9b8412aa0 2024-05-09T09:26:43.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f0ca35640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840cf40 0x55a9b840d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f0ca35640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840cf40 0x55a9b840d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34876/0 (socket says 172.21.15.12:34876) 2024-05-09T09:26:43.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f0ca35640 1 -- 172.21.15.12:0/3863881291 learned_addr learned my addr 172.21.15.12:0/3863881291 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f0d236640 1 --2- 172.21.15.12:0/3863881291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b840d930 0x55a9b8412430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/3863881291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 0x55a9b8414f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8efe7fc640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8ef40a2c90 0x7f8ef40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f0ca35640 1 -- 172.21.15.12:0/3863881291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 msgr2=0x55a9b8414f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/3863881291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 0x55a9b8414f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f0ca35640 1 -- 172.21.15.12:0/3863881291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b840d930 msgr2=0x55a9b8412430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.600+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/3863881291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b840d930 0x55a9b8412430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.601+0000 7f8f0ca35640 1 -- 172.21.15.12:0/3863881291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b80853f0 con 0x55a9b840cf40 2024-05-09T09:26:43.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.601+0000 7f8efe7fc640 1 -- 172.21.15.12:0/680637105 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f8f000f08b0 con 0x7f8f08102320 2024-05-09T09:26:43.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.601+0000 7f8f0da37640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8ef40a2c90 0x7f8ef40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.601+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/3863881291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840cf40 0x55a9b840d360 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80098e0 tx=0x7f8ef8009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1888d2c89b69e246 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.601+0000 7f8f0da37640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8ef40a2c90 0x7f8ef40a5150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8f00031a10 tx=0x7f8f0002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.601+0000 7f8ef37fe640 1 -- 172.21.15.12:0/3863881291 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8026070 con 0x55a9b840cf40 2024-05-09T09:26:43.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.601+0000 7f8ef37fe640 1 -- 172.21.15.12:0/3863881291 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8ef8021960 con 0x55a9b840cf40 2024-05-09T09:26:43.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.601+0000 7f8ef37fe640 1 -- 172.21.15.12:0/3863881291 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80039b0 con 0x55a9b840cf40 2024-05-09T09:26:43.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.602+0000 7f8f121a1740 1 -- 172.21.15.12:0/3863881291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840cf40 msgr2=0x55a9b840d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.602+0000 7f8f121a1740 1 --2- 172.21.15.12:0/3863881291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840cf40 0x55a9b840d360 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80098e0 tx=0x7f8ef8009850 comp rx=0 tx=0).stop 2024-05-09T09:26:43.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.602+0000 7f8f121a1740 1 -- 172.21.15.12:0/3863881291 shutdown_connections 2024-05-09T09:26:43.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.602+0000 7f8f121a1740 1 --2- 172.21.15.12:0/3863881291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 0x55a9b8414f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.602+0000 7f8f121a1740 1 --2- 172.21.15.12:0/3863881291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b840d930 0x55a9b8412430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.602+0000 7f8f121a1740 1 --2- 172.21.15.12:0/3863881291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840cf40 0x55a9b840d360 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.602+0000 7f8f121a1740 1 -- 172.21.15.12:0/3863881291 >> 172.21.15.12:0/3863881291 conn(0x55a9b84085b0 msgr2=0x55a9b84089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.602+0000 7f8f121a1740 1 -- 172.21.15.12:0/3863881291 shutdown_connections 2024-05-09T09:26:43.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.602+0000 7f8f121a1740 1 -- 172.21.15.12:0/3863881291 wait complete. 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.603+0000 7f8f121a1740 1 -- start start 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.603+0000 7f8f121a1740 1 Processor -- start 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.603+0000 7f8f121a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b840d930 0x55a9b84176c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.603+0000 7f8f121a1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 0x55a9b8417c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.603+0000 7f8f121a1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b841cc30 0x55a9b841f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.603+0000 7f8f121a1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a9b8415a50 con 0x55a9b841cc30 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.603+0000 7f8f121a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a9b832cc40 con 0x55a9b840d930 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.603+0000 7f8f121a1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a9b8415bd0 con 0x55a9b8412aa0 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.603+0000 7f8f0ca35640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b841cc30 0x55a9b841f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.603+0000 7f8f0ca35640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b841cc30 0x55a9b841f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34880/0 (socket says 172.21.15.12:34880) 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.603+0000 7f8f0ca35640 1 -- 172.21.15.12:0/221464290 learned_addr learned my addr 172.21.15.12:0/221464290 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.603+0000 7f8f121a1740 10 client.0 _ll_register_callbacks cb 0x55a9b8084be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.603+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/221464290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b840d930 0x55a9b84176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8f0d236640 1 --2- 172.21.15.12:0/221464290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 0x55a9b8417c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8f0ca35640 1 -- 172.21.15.12:0/221464290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 msgr2=0x55a9b8417c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/221464290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 0x55a9b8417c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8f0ca35640 1 -- 172.21.15.12:0/221464290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b840d930 msgr2=0x55a9b84176c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/221464290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b840d930 0x55a9b84176c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8f0ca35640 1 -- 172.21.15.12:0/221464290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b814f7d0 con 0x55a9b841cc30 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/221464290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b841cc30 0x55a9b841f020 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8021f80 tx=0x7f8ef8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f79c07ce6becf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8ef0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8ef0ff9640 1 -- 172.21.15.12:0/221464290 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8026020 con 0x55a9b841cc30 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8f121a1740 1 -- 172.21.15.12:0/221464290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef8021960 con 0x55a9b841cc30 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8ef0ff9640 1 -- 172.21.15.12:0/221464290 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f8ef8021700 con 0x55a9b841cc30 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8ef0ff9640 1 -- 172.21.15.12:0/221464290 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ef8033050 con 0x55a9b841cc30 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8f121a1740 10 client.15978 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.604+0000 7f8f121a1740 1 -- 172.21.15.12:0/221464290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a9b8433380 con 0x55a9b841cc30 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.605+0000 7f8ef0ff9640 1 -- 172.21.15.12:0/221464290 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8ef8021960 con 0x55a9b841cc30 2024-05-09T09:26:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.605+0000 7f8ef0ff9640 1 -- 172.21.15.12:0/221464290 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ef801f020 con 0x55a9b841cc30 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.605+0000 7f8f121a1740 10 client.15978 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.605+0000 7f8f121a1740 10 client.15978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.605+0000 7f8f121a1740 20 client.15978 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.605+0000 7f8f121a1740 1 --2- 172.21.15.12:0/221464290 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55a9b84347d0 0x55a9b8436c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.605+0000 7f8f121a1740 4 client.15978 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.605+0000 7f8f121a1740 1 -- 172.21.15.12:0/221464290 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9b837f370 con 0x55a9b84347d0 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.605+0000 7f8f0d236640 1 --2- 172.21.15.12:0/221464290 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55a9b84347d0 0x55a9b8436c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.606+0000 7f8ef0ff9640 10 client.15978 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.606+0000 7f8f0d236640 1 --2- 172.21.15.12:0/221464290 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55a9b84347d0 0x55a9b8436c10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be99e1be57028d44 server_cookie=adf0f5f39585727f in_seq=0 out_seq=0 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.606+0000 7f8ef0ff9640 1 -- 172.21.15.12:0/221464290 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9b837f370 con 0x55a9b84347d0 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.606+0000 7f8ef0ff9640 10 client.15978 handle_command_reply: tid=0 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.606+0000 7f8f121a1740 1 client.15978 shutdown 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.607+0000 7f8f121a1740 1 -- 172.21.15.12:0/221464290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b841cc30 msgr2=0x55a9b841f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.607+0000 7f8f121a1740 1 --2- 172.21.15.12:0/221464290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b841cc30 0x55a9b841f020 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8021f80 tx=0x7f8ef8009260 comp rx=0 tx=0).stop 2024-05-09T09:26:43.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.607+0000 7f8f121a1740 1 -- 172.21.15.12:0/221464290 shutdown_connections 2024-05-09T09:26:43.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.607+0000 7f8f121a1740 1 --2- 172.21.15.12:0/221464290 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55a9b84347d0 0x55a9b8436c10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.607+0000 7f8f121a1740 1 --2- 172.21.15.12:0/221464290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b841cc30 0x55a9b841f020 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.607+0000 7f8f121a1740 1 --2- 172.21.15.12:0/221464290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 0x55a9b8417c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.607+0000 7f8f121a1740 1 --2- 172.21.15.12:0/221464290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b840d930 0x55a9b84176c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.607+0000 7f8f121a1740 1 -- 172.21.15.12:0/221464290 >> 172.21.15.12:0/221464290 conn(0x55a9b84085b0 msgr2=0x55a9b84077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.607+0000 7f8ef0ff9640 0 client.15978 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:26:43.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.607+0000 7f8f121a1740 1 -- 172.21.15.12:0/221464290 shutdown_connections 2024-05-09T09:26:43.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.607+0000 7f8f121a1740 1 -- 172.21.15.12:0/221464290 wait complete. 2024-05-09T09:26:43.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.607+0000 7f8f121a1740 20 client.15978 trim_cache size 0 max 16384 2024-05-09T09:26:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.620+0000 7f8f121a1740 1 Processor -- start 2024-05-09T09:26:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.620+0000 7f8f121a1740 1 -- start start 2024-05-09T09:26:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f121a1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 0x55a9b84176c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f121a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b8412aa0 0x55a9b8417c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f121a1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b841cc30 0x55a9b840f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f121a1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081a8410 con 0x55a9b840d930 2024-05-09T09:26:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f121a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081a9f50 con 0x55a9b8412aa0 2024-05-09T09:26:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f121a1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081a8ae0 con 0x55a9b841cc30 2024-05-09T09:26:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f0ca35640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 0x55a9b84176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f0ca35640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 0x55a9b84176c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34890/0 (socket says 172.21.15.12:34890) 2024-05-09T09:26:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f0ca35640 1 -- 172.21.15.12:0/954640135 learned_addr learned my addr 172.21.15.12:0/954640135 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f0d236640 1 --2- 172.21.15.12:0/954640135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b8412aa0 0x55a9b8417c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f0ca35640 1 -- 172.21.15.12:0/954640135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b841cc30 msgr2=0x55a9b840f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/954640135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b841cc30 0x55a9b840f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f0ca35640 1 -- 172.21.15.12:0/954640135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b8412aa0 msgr2=0x55a9b8417c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/954640135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b8412aa0 0x55a9b8417c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f0ca35640 1 -- 172.21.15.12:0/954640135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b8403610 con 0x55a9b840d930 2024-05-09T09:26:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/954640135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 0x55a9b84176c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8010ed0 tx=0x7f8ef80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34589f01fafcb592 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.621+0000 7f8ef2ffd640 1 -- 172.21.15.12:0/954640135 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8026070 con 0x55a9b840d930 2024-05-09T09:26:43.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.622+0000 7f8ef2ffd640 1 -- 172.21.15.12:0/954640135 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8ef8009c90 con 0x55a9b840d930 2024-05-09T09:26:43.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.622+0000 7f8ef2ffd640 1 -- 172.21.15.12:0/954640135 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8005330 con 0x55a9b840d930 2024-05-09T09:26:43.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.622+0000 7f8f121a1740 1 -- 172.21.15.12:0/954640135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 msgr2=0x55a9b84176c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.622+0000 7f8f121a1740 1 --2- 172.21.15.12:0/954640135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 0x55a9b84176c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8010ed0 tx=0x7f8ef80035f0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.622+0000 7f8f121a1740 1 -- 172.21.15.12:0/954640135 shutdown_connections 2024-05-09T09:26:43.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.622+0000 7f8f121a1740 1 --2- 172.21.15.12:0/954640135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b841cc30 0x55a9b840f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.622+0000 7f8f121a1740 1 --2- 172.21.15.12:0/954640135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b8412aa0 0x55a9b8417c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.622+0000 7f8f121a1740 1 --2- 172.21.15.12:0/954640135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 0x55a9b84176c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.622+0000 7f8f121a1740 1 -- 172.21.15.12:0/954640135 >> 172.21.15.12:0/954640135 conn(0x55a9b84085b0 msgr2=0x55a9b8408990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.622+0000 7f8f121a1740 1 -- 172.21.15.12:0/954640135 shutdown_connections 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.622+0000 7f8f121a1740 1 -- 172.21.15.12:0/954640135 wait complete. 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.623+0000 7f8f121a1740 1 -- start start 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.623+0000 7f8f121a1740 1 Processor -- start 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.623+0000 7f8f121a1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 0x55a9b84456c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.623+0000 7f8f121a1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 0x55a9b8445c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.623+0000 7f8f121a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b841cc30 0x55a9b8446bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.623+0000 7f8f121a1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081a9880 con 0x55a9b840d930 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.623+0000 7f8f121a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081a91b0 con 0x55a9b841cc30 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.623+0000 7f8f121a1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081a63f0 con 0x55a9b8412aa0 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.623+0000 7f8f121a1740 10 client.0 _ll_register_callbacks cb 0x7f8ef8024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.623+0000 7f8f0ca35640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 0x55a9b84456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.623+0000 7f8f0ca35640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 0x55a9b84456c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34906/0 (socket says 172.21.15.12:34906) 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.623+0000 7f8f0ca35640 1 -- 172.21.15.12:0/3264538191 learned_addr learned my addr 172.21.15.12:0/3264538191 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.623+0000 7f8f0d236640 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 0x55a9b8445c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.624+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b841cc30 0x55a9b8446bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.624+0000 7f8f0d236640 1 -- 172.21.15.12:0/3264538191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b841cc30 msgr2=0x55a9b8446bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.624+0000 7f8f0d236640 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b841cc30 0x55a9b8446bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.624+0000 7f8f0d236640 1 -- 172.21.15.12:0/3264538191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 msgr2=0x55a9b84456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.624+0000 7f8f0d236640 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 0x55a9b84456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.624+0000 7f8f0d236640 1 -- 172.21.15.12:0/3264538191 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b84471b0 con 0x55a9b8412aa0 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.624+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 0x55a9b84456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.624+0000 7f8f0d236640 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 0x55a9b8445c00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55a9b8437480 tx=0x7f8f0401bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cc2df52289d3b82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.624+0000 7f8ef0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.624+0000 7f8ef0ff9640 1 -- 172.21.15.12:0/3264538191 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0401b290 con 0x55a9b8412aa0 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.624+0000 7f8ef0ff9640 1 -- 172.21.15.12:0/3264538191 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f8f0401b430 con 0x55a9b8412aa0 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.624+0000 7f8f121a1740 1 -- 172.21.15.12:0/3264538191 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a9b83796c0 con 0x55a9b8412aa0 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.625+0000 7f8ef0ff9640 1 -- 172.21.15.12:0/3264538191 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f0402d050 con 0x55a9b8412aa0 2024-05-09T09:26:43.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.625+0000 7f8f121a1740 10 client.25600 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:43.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.625+0000 7f8f121a1740 1 -- 172.21.15.12:0/3264538191 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a9b849a830 con 0x55a9b8412aa0 2024-05-09T09:26:43.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.625+0000 7f8ef0ff9640 1 -- 172.21.15.12:0/3264538191 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55a9b83796c0 con 0x55a9b8412aa0 2024-05-09T09:26:43.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.625+0000 7f8ef0ff9640 1 -- 172.21.15.12:0/3264538191 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8f0400f040 con 0x55a9b8412aa0 2024-05-09T09:26:43.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.625+0000 7f8f121a1740 10 client.25600 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:43.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.625+0000 7f8f121a1740 10 client.25600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:26:43.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.625+0000 7f8f121a1740 20 client.25600 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:43.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.625+0000 7f8f121a1740 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55a9b849b230 0x55a9b849d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.625+0000 7f8f121a1740 4 client.25600 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:43.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.625+0000 7f8f121a1740 1 -- 172.21.15.12:0/3264538191 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9b82f6820 con 0x55a9b849b230 2024-05-09T09:26:43.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.625+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55a9b849b230 0x55a9b849d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8f0ca35640 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55a9b849b230 0x55a9b849d650 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30bd157158a1fc9a server_cookie=3ddd9aa6c832ec08 in_seq=0 out_seq=0 2024-05-09T09:26:43.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8ef0ff9640 10 client.25600 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8ef0ff9640 1 -- 172.21.15.12:0/3264538191 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a9b82f6820 con 0x55a9b849b230 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8ef0ff9640 10 client.25600 handle_command_reply: tid=0 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8f121a1740 1 client.25600 shutdown 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8f121a1740 1 -- 172.21.15.12:0/3264538191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 msgr2=0x55a9b8445c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8f121a1740 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 0x55a9b8445c00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55a9b8437480 tx=0x7f8f0401bf90 comp rx=0 tx=0).stop 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8f121a1740 1 -- 172.21.15.12:0/3264538191 shutdown_connections 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8f121a1740 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55a9b849b230 0x55a9b849d650 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8f121a1740 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b841cc30 0x55a9b8446bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8ef0ff9640 0 client.25600 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8f121a1740 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9b8412aa0 0x55a9b8445c00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8f121a1740 1 --2- 172.21.15.12:0/3264538191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a9b840d930 0x55a9b84456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.626+0000 7f8f121a1740 1 -- 172.21.15.12:0/3264538191 >> 172.21.15.12:0/3264538191 conn(0x55a9b84085b0 msgr2=0x55a9b8417600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.627+0000 7f8f121a1740 1 -- 172.21.15.12:0/3264538191 shutdown_connections 2024-05-09T09:26:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.627+0000 7f8f121a1740 1 -- 172.21.15.12:0/3264538191 wait complete. 2024-05-09T09:26:43.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.627+0000 7f8f121a1740 20 client.25600 trim_cache size 0 max 16384 2024-05-09T09:26:43.632 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.627+0000 7f8f0f49a640 1 -- 172.21.15.12:0/680637105 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8ef40a2c90 msgr2=0x7f8ef40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.627+0000 7f8f0f49a640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8ef40a2c90 0x7f8ef40a5150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8f00031a10 tx=0x7f8f0002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.627+0000 7f8f0f49a640 1 -- 172.21.15.12:0/680637105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f08102320 msgr2=0x7f8f08103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.627+0000 7f8f0f49a640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f08102320 0x7f8f08103280 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8f0002ade0 tx=0x7f8f0002f3e0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.627+0000 7f8f0f49a640 1 -- 172.21.15.12:0/680637105 shutdown_connections 2024-05-09T09:26:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.627+0000 7f8f0f49a640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8ef40a2c90 0x7f8ef40a5150 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.627+0000 7f8f0f49a640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f08102320 0x7f8f08103280 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.627+0000 7f8f0f49a640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f08101960 0x7f8f08102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.627+0000 7f8f0f49a640 1 --2- 172.21.15.12:0/680637105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f08100770 0x7f8f08104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.627+0000 7f8f0f49a640 1 -- 172.21.15.12:0/680637105 >> 172.21.15.12:0/680637105 conn(0x7f8f080fbce0 msgr2=0x7f8f080fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.628+0000 7f8f0f49a640 1 -- 172.21.15.12:0/680637105 shutdown_connections 2024-05-09T09:26:43.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.628+0000 7f8f0f49a640 1 -- 172.21.15.12:0/680637105 wait complete. 2024-05-09T09:26:43.638 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:26:43.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:43 smithi012 ceph-mon[30140]: pgmap v402: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:43.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:43 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/387771729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:43.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.771+0000 7f23e08ca640 1 -- 172.21.15.12:0/3995410696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23dc101960 msgr2=0x7f23dc101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.771+0000 7f23e08ca640 1 --2- 172.21.15.12:0/3995410696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23dc101960 0x7f23dc101de0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009900 tx=0x7f23d002a410 comp rx=0 tx=0).stop 2024-05-09T09:26:43.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.772+0000 7f23e08ca640 1 -- 172.21.15.12:0/3995410696 shutdown_connections 2024-05-09T09:26:43.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.772+0000 7f23e08ca640 1 --2- 172.21.15.12:0/3995410696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23dc102320 0x7f23dc108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.772+0000 7f23e08ca640 1 --2- 172.21.15.12:0/3995410696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23dc101960 0x7f23dc101de0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.772+0000 7f23e08ca640 1 --2- 172.21.15.12:0/3995410696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23dc100770 0x7f23dc100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.772+0000 7f23e08ca640 1 -- 172.21.15.12:0/3995410696 >> 172.21.15.12:0/3995410696 conn(0x7f23dc0fbce0 msgr2=0x7f23dc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.772+0000 7f23e08ca640 1 -- 172.21.15.12:0/3995410696 shutdown_connections 2024-05-09T09:26:43.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.772+0000 7f23e08ca640 1 -- 172.21.15.12:0/3995410696 wait complete. 2024-05-09T09:26:43.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.772+0000 7f23e08ca640 1 Processor -- start 2024-05-09T09:26:43.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.772+0000 7f23e08ca640 1 -- start start 2024-05-09T09:26:43.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23e08ca640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23dc100770 0x7f23dc197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23e08ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23dc101960 0x7f23dc198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23e08ca640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23dc102320 0x7f23dc19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23e08ca640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc10c420 con 0x7f23dc100770 2024-05-09T09:26:43.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23e08ca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc10c2a0 con 0x7f23dc101960 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23e08ca640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc10c5a0 con 0x7f23dc102320 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23da59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23dc100770 0x7f23dc197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23da59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23dc100770 0x7f23dc197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34922/0 (socket says 172.21.15.12:34922) 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23da59c640 1 -- 172.21.15.12:0/1511577919 learned_addr learned my addr 172.21.15.12:0/1511577919 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23dad9d640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23dc102320 0x7f23dc19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23d9d9b640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23dc101960 0x7f23dc198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23da59c640 1 -- 172.21.15.12:0/1511577919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23dc102320 msgr2=0x7f23dc19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23da59c640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23dc102320 0x7f23dc19f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23da59c640 1 -- 172.21.15.12:0/1511577919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23dc101960 msgr2=0x7f23dc198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23da59c640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23dc101960 0x7f23dc198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23da59c640 1 -- 172.21.15.12:0/1511577919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc19f7f0 con 0x7f23dc100770 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.773+0000 7f23d9d9b640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23dc101960 0x7f23dc198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.774+0000 7f23da59c640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23dc100770 0x7f23dc197c40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00d950 tx=0x7f23cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.774+0000 7f23cb7fe640 1 -- 172.21.15.12:0/1511577919 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc017070 con 0x7f23dc100770 2024-05-09T09:26:43.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.774+0000 7f23e08ca640 1 -- 172.21.15.12:0/1511577919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc19fae0 con 0x7f23dc100770 2024-05-09T09:26:43.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.774+0000 7f23cb7fe640 1 -- 172.21.15.12:0/1511577919 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f23cc009c30 con 0x7f23dc100770 2024-05-09T09:26:43.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.774+0000 7f23e08ca640 1 -- 172.21.15.12:0/1511577919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc1a0000 con 0x7f23dc100770 2024-05-09T09:26:43.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.774+0000 7f23cb7fe640 1 -- 172.21.15.12:0/1511577919 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc021060 con 0x7f23dc100770 2024-05-09T09:26:43.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.776+0000 7f23e35d6740 1 Processor -- start 2024-05-09T09:26:43.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.777+0000 7f23e35d6740 1 -- start start 2024-05-09T09:26:43.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.777+0000 7f23e35d6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce01af40 0x55b3ce01b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.777+0000 7f23e35d6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce01b930 0x55b3ce020430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.777+0000 7f23e35d6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce020aa0 0x55b3ce022f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.777+0000 7f23e35d6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b3cdf7fc40 con 0x55b3ce020aa0 2024-05-09T09:26:43.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.777+0000 7f23e35d6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3cdf27580 con 0x55b3ce01b930 2024-05-09T09:26:43.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.777+0000 7f23e35d6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b3cdf7bc00 con 0x55b3ce01af40 2024-05-09T09:26:43.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.777+0000 7f23d9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce020aa0 0x55b3ce022f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.777+0000 7f23d9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce020aa0 0x55b3ce022f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34930/0 (socket says 172.21.15.12:34930) 2024-05-09T09:26:43.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.777+0000 7f23d9d9b640 1 -- 172.21.15.12:0/1852795232 learned_addr learned my addr 172.21.15.12:0/1852795232 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.777+0000 7f23dad9d640 1 --2- 172.21.15.12:0/1852795232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce01af40 0x55b3ce01b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.778+0000 7f23dad9d640 1 -- 172.21.15.12:0/1852795232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce01b930 msgr2=0x55b3ce020430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:26:43.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.778+0000 7f23dad9d640 1 --2- 172.21.15.12:0/1852795232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce01b930 0x55b3ce020430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.778+0000 7f23dad9d640 1 -- 172.21.15.12:0/1852795232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce020aa0 msgr2=0x55b3ce022f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.778+0000 7f23cb7fe640 1 -- 172.21.15.12:0/1511577919 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f23cc01a070 con 0x7f23dc100770 2024-05-09T09:26:43.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.778+0000 7f23dad9d640 1 --2- 172.21.15.12:0/1852795232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce020aa0 0x55b3ce022f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.778+0000 7f23dad9d640 1 -- 172.21.15.12:0/1852795232 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3cdc933f0 con 0x55b3ce01af40 2024-05-09T09:26:43.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.778+0000 7f23d9d9b640 1 --2- 172.21.15.12:0/1852795232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce020aa0 0x55b3ce022f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:43.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.778+0000 7f23dad9d640 1 --2- 172.21.15.12:0/1852795232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce01af40 0x55b3ce01b360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009f10 tx=0x7f23d4010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=656e601c4d17cc61 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.778+0000 7f23c8ff9640 1 -- 172.21.15.12:0/1852795232 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4016070 con 0x55b3ce01af40 2024-05-09T09:26:43.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.779+0000 7f23c8ff9640 1 -- 172.21.15.12:0/1852795232 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f23d401bb30 con 0x55b3ce01af40 2024-05-09T09:26:43.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.779+0000 7f23c8ff9640 1 -- 172.21.15.12:0/1852795232 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4012ce0 con 0x55b3ce01af40 2024-05-09T09:26:43.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.779+0000 7f23cb7fe640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f23c40a2be0 0x7f23c40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.779+0000 7f23cb7fe640 1 -- 172.21.15.12:0/1511577919 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f23cc0cf230 con 0x7f23dc100770 2024-05-09T09:26:43.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.779+0000 7f23da59c640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f23c40a2be0 0x7f23c40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.779+0000 7f23e35d6740 1 -- 172.21.15.12:0/1852795232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce01af40 msgr2=0x55b3ce01b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.779+0000 7f23e35d6740 1 --2- 172.21.15.12:0/1852795232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce01af40 0x55b3ce01b360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009f10 tx=0x7f23d4010740 comp rx=0 tx=0).stop 2024-05-09T09:26:43.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 -- 172.21.15.12:0/1852795232 shutdown_connections 2024-05-09T09:26:43.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 --2- 172.21.15.12:0/1852795232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce020aa0 0x55b3ce022f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 --2- 172.21.15.12:0/1852795232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce01b930 0x55b3ce020430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 --2- 172.21.15.12:0/1852795232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce01af40 0x55b3ce01b360 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 -- 172.21.15.12:0/1852795232 >> 172.21.15.12:0/1852795232 conn(0x55b3ce0165b0 msgr2=0x55b3ce0169b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23da59c640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f23c40a2be0 0x7f23c40a50a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f23cc097b20 tx=0x7f23cc086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 -- 172.21.15.12:0/1852795232 shutdown_connections 2024-05-09T09:26:43.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 -- 172.21.15.12:0/1852795232 wait complete. 2024-05-09T09:26:43.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 -- start start 2024-05-09T09:26:43.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 Processor -- start 2024-05-09T09:26:43.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce01af40 0x55b3ce02dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01b930 0x55b3ce02e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce020aa0 0x55b3ce035160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b3ce023a50 con 0x55b3ce01b930 2024-05-09T09:26:43.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3cdf3ac40 con 0x55b3ce01af40 2024-05-09T09:26:43.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b3ce023bd0 con 0x55b3ce020aa0 2024-05-09T09:26:43.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.780+0000 7f23e35d6740 10 client.0 _ll_register_callbacks cb 0x55b3cdc92be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:43.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.781+0000 7f23d9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01b930 0x55b3ce02e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.781+0000 7f23dad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce01af40 0x55b3ce02dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.781+0000 7f23d9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01b930 0x55b3ce02e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34934/0 (socket says 172.21.15.12:34934) 2024-05-09T09:26:43.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.781+0000 7f23d9d9b640 1 -- 172.21.15.12:0/3036353230 learned_addr learned my addr 172.21.15.12:0/3036353230 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.781+0000 7f23dad9d640 1 --2- 172.21.15.12:0/3036353230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce020aa0 0x55b3ce035160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.781+0000 7f23d9d9b640 1 -- 172.21.15.12:0/3036353230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce020aa0 msgr2=0x55b3ce035160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.781+0000 7f23d9d9b640 1 --2- 172.21.15.12:0/3036353230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce020aa0 0x55b3ce035160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.781+0000 7f23d9d9b640 1 -- 172.21.15.12:0/3036353230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce01af40 msgr2=0x55b3ce02dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.781+0000 7f23d9d9b640 1 --2- 172.21.15.12:0/3036353230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce01af40 0x55b3ce02dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.781+0000 7f23d9d9b640 1 -- 172.21.15.12:0/3036353230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3cdd5d7d0 con 0x55b3ce01b930 2024-05-09T09:26:43.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.781+0000 7f23d9d9b640 1 --2- 172.21.15.12:0/3036353230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01b930 0x55b3ce02e120 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009230 tx=0x7f23d0004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5df4cd516e0a070 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.781+0000 7f23c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:43.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.782+0000 7f23c1ffb640 1 -- 172.21.15.12:0/3036353230 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0035030 con 0x55b3ce01b930 2024-05-09T09:26:43.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.782+0000 7f23e35d6740 1 -- 172.21.15.12:0/3036353230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23d401bb30 con 0x55b3ce01b930 2024-05-09T09:26:43.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.782+0000 7f23c1ffb640 1 -- 172.21.15.12:0/3036353230 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f23d0005460 con 0x55b3ce01b930 2024-05-09T09:26:43.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.782+0000 7f23c1ffb640 1 -- 172.21.15.12:0/3036353230 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f23d00046c0 con 0x55b3ce01b930 2024-05-09T09:26:43.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.782+0000 7f23e35d6740 10 client.15996 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:43.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.782+0000 7f23e35d6740 1 -- 172.21.15.12:0/3036353230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b3ce038590 con 0x55b3ce01b930 2024-05-09T09:26:43.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.782+0000 7f23c1ffb640 1 -- 172.21.15.12:0/3036353230 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f23d401bb30 con 0x55b3ce01b930 2024-05-09T09:26:43.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.782+0000 7f23c1ffb640 1 -- 172.21.15.12:0/3036353230 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f23d0031bb0 con 0x55b3ce01b930 2024-05-09T09:26:43.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.783+0000 7f23e35d6740 10 client.15996 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:43.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.783+0000 7f23e35d6740 10 client.15996 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:26:43.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.783+0000 7f23e35d6740 20 client.15996 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:43.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.783+0000 7f23e35d6740 1 --2- 172.21.15.12:0/3036353230 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55b3ce039b90 0x55b3ce03bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.783+0000 7f23e35d6740 4 client.15996 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:43.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.783+0000 7f23e35d6740 1 -- 172.21.15.12:0/3036353230 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3cdf8d370 con 0x55b3ce039b90 2024-05-09T09:26:43.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.783+0000 7f23dad9d640 1 --2- 172.21.15.12:0/3036353230 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55b3ce039b90 0x55b3ce03bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.783+0000 7f23c1ffb640 10 client.15996 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:26:43.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.783+0000 7f23dad9d640 1 --2- 172.21.15.12:0/3036353230 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55b3ce039b90 0x55b3ce03bff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10631f0df58b7682 server_cookie=e1f1aef47c676a13 in_seq=0 out_seq=0 2024-05-09T09:26:43.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.784+0000 7f23c1ffb640 1 -- 172.21.15.12:0/3036353230 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3cdf8d370 con 0x55b3ce039b90 2024-05-09T09:26:43.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.784+0000 7f23c1ffb640 10 client.15996 handle_command_reply: tid=0 2024-05-09T09:26:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.784+0000 7f23e35d6740 1 client.15996 shutdown 2024-05-09T09:26:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.784+0000 7f23e35d6740 1 -- 172.21.15.12:0/3036353230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01b930 msgr2=0x55b3ce02e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.784+0000 7f23e35d6740 1 --2- 172.21.15.12:0/3036353230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01b930 0x55b3ce02e120 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009230 tx=0x7f23d0004440 comp rx=0 tx=0).stop 2024-05-09T09:26:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.784+0000 7f23e35d6740 1 -- 172.21.15.12:0/3036353230 shutdown_connections 2024-05-09T09:26:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.784+0000 7f23e35d6740 1 --2- 172.21.15.12:0/3036353230 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55b3ce039b90 0x55b3ce03bff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.784+0000 7f23e35d6740 1 --2- 172.21.15.12:0/3036353230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce020aa0 0x55b3ce035160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.784+0000 7f23c1ffb640 0 client.15996 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:26:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.784+0000 7f23e35d6740 1 --2- 172.21.15.12:0/3036353230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01b930 0x55b3ce02e120 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.784+0000 7f23e35d6740 1 --2- 172.21.15.12:0/3036353230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce01af40 0x55b3ce02dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.784+0000 7f23e35d6740 1 -- 172.21.15.12:0/3036353230 >> 172.21.15.12:0/3036353230 conn(0x55b3ce0165b0 msgr2=0x55b3ce0157b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.784+0000 7f23e35d6740 1 -- 172.21.15.12:0/3036353230 shutdown_connections 2024-05-09T09:26:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.785+0000 7f23e35d6740 1 -- 172.21.15.12:0/3036353230 wait complete. 2024-05-09T09:26:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.785+0000 7f23e35d6740 20 client.15996 trim_cache size 0 max 16384 2024-05-09T09:26:43.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.811+0000 7f23e35d6740 1 Processor -- start 2024-05-09T09:26:43.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.811+0000 7f23e35d6740 1 -- start start 2024-05-09T09:26:43.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.811+0000 7f23e35d6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01af40 0x55b3ce01c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.811+0000 7f23e35d6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce020aa0 0x55b3ce035120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.811+0000 7f23e35d6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce039b90 0x55b3ce014f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.811+0000 7f23e35d6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1a8390 con 0x55b3ce01af40 2024-05-09T09:26:43.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.811+0000 7f23e35d6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1a9ed0 con 0x55b3ce039b90 2024-05-09T09:26:43.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.811+0000 7f23e35d6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1a8a60 con 0x55b3ce020aa0 2024-05-09T09:26:43.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.811+0000 7f23dad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01af40 0x55b3ce01c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.811+0000 7f23dad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01af40 0x55b3ce01c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34946/0 (socket says 172.21.15.12:34946) 2024-05-09T09:26:43.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.811+0000 7f23dad9d640 1 -- 172.21.15.12:0/3403366372 learned_addr learned my addr 172.21.15.12:0/3403366372 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.811+0000 7f23d9d9b640 1 --2- 172.21.15.12:0/3403366372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce020aa0 0x55b3ce035120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.811+0000 7f23dad9d640 1 --2- 172.21.15.12:0/3403366372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce039b90 0x55b3ce014f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.812+0000 7f23dad9d640 1 -- 172.21.15.12:0/3403366372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce020aa0 msgr2=0x55b3ce035120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.812+0000 7f23dad9d640 1 --2- 172.21.15.12:0/3403366372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce020aa0 0x55b3ce035120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.812+0000 7f23dad9d640 1 -- 172.21.15.12:0/3403366372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce039b90 msgr2=0x55b3ce014f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.812+0000 7f23dad9d640 1 --2- 172.21.15.12:0/3403366372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce039b90 0x55b3ce014f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.812+0000 7f23dad9d640 1 -- 172.21.15.12:0/3403366372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ce02ccc0 con 0x55b3ce01af40 2024-05-09T09:26:43.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.812+0000 7f23d9d9b640 1 --2- 172.21.15.12:0/3403366372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce020aa0 0x55b3ce035120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:43.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.812+0000 7f23dad9d640 1 --2- 172.21.15.12:0/3403366372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01af40 0x55b3ce01c3d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f23d4000f80 tx=0x7f23d4009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d02bc65c938699 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.812+0000 7f23c3fff640 1 -- 172.21.15.12:0/3403366372 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d402ee60 con 0x55b3ce01af40 2024-05-09T09:26:43.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.812+0000 7f23c3fff640 1 -- 172.21.15.12:0/3403366372 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f23d4003e40 con 0x55b3ce01af40 2024-05-09T09:26:43.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.812+0000 7f23c3fff640 1 -- 172.21.15.12:0/3403366372 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4016040 con 0x55b3ce01af40 2024-05-09T09:26:43.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.813+0000 7f23e35d6740 1 -- 172.21.15.12:0/3403366372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01af40 msgr2=0x55b3ce01c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.813+0000 7f23e35d6740 1 --2- 172.21.15.12:0/3403366372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01af40 0x55b3ce01c3d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f23d4000f80 tx=0x7f23d4009600 comp rx=0 tx=0).stop 2024-05-09T09:26:43.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.813+0000 7f23e35d6740 1 -- 172.21.15.12:0/3403366372 shutdown_connections 2024-05-09T09:26:43.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.813+0000 7f23e35d6740 1 --2- 172.21.15.12:0/3403366372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce039b90 0x55b3ce014f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.813+0000 7f23e35d6740 1 --2- 172.21.15.12:0/3403366372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce020aa0 0x55b3ce035120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.813+0000 7f23e35d6740 1 --2- 172.21.15.12:0/3403366372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce01af40 0x55b3ce01c3d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.813+0000 7f23e35d6740 1 -- 172.21.15.12:0/3403366372 >> 172.21.15.12:0/3403366372 conn(0x55b3ce0165b0 msgr2=0x55b3ce01cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.813+0000 7f23e35d6740 1 -- 172.21.15.12:0/3403366372 shutdown_connections 2024-05-09T09:26:43.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.813+0000 7f23e35d6740 1 -- 172.21.15.12:0/3403366372 wait complete. 2024-05-09T09:26:43.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.813+0000 7f23e35d6740 1 -- start start 2024-05-09T09:26:43.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23e35d6740 1 Processor -- start 2024-05-09T09:26:43.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23e35d6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce01af40 0x55b3ce032830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23e35d6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce020aa0 0x55b3ce032d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23e35d6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce039b90 0x55b3ce033d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23e35d6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1a9800 con 0x55b3ce039b90 2024-05-09T09:26:43.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23e35d6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1a9130 con 0x55b3ce020aa0 2024-05-09T09:26:43.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23e35d6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1a6370 con 0x55b3ce01af40 2024-05-09T09:26:43.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23e35d6740 10 client.0 _ll_register_callbacks cb 0x7f23d0009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:43.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23dad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce039b90 0x55b3ce033d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23dad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce039b90 0x55b3ce033d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34962/0 (socket says 172.21.15.12:34962) 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23dad9d640 1 -- 172.21.15.12:0/2186049736 learned_addr learned my addr 172.21.15.12:0/2186049736 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23dad9d640 1 --2- 172.21.15.12:0/2186049736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce01af40 0x55b3ce032830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23d9d9b640 1 --2- 172.21.15.12:0/2186049736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce020aa0 0x55b3ce032d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23dad9d640 1 -- 172.21.15.12:0/2186049736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce01af40 msgr2=0x55b3ce032830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23dad9d640 1 --2- 172.21.15.12:0/2186049736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce01af40 0x55b3ce032830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23dad9d640 1 -- 172.21.15.12:0/2186049736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce020aa0 msgr2=0x55b3ce032d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23dad9d640 1 --2- 172.21.15.12:0/2186049736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce020aa0 0x55b3ce032d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.814+0000 7f23dad9d640 1 -- 172.21.15.12:0/2186049736 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ce034370 con 0x55b3ce039b90 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.815+0000 7f23dad9d640 1 --2- 172.21.15.12:0/2186049736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce039b90 0x55b3ce033d80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f23d4005fd0 tx=0x7f23d4009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7814804db396c3f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.815+0000 7f23c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.815+0000 7f23c1ffb640 1 -- 172.21.15.12:0/2186049736 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4016040 con 0x55b3ce039b90 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.815+0000 7f23c1ffb640 1 -- 172.21.15.12:0/2186049736 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f23d401b4f0 con 0x55b3ce039b90 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.815+0000 7f23e35d6740 1 -- 172.21.15.12:0/2186049736 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b3cdf876c0 con 0x55b3ce039b90 2024-05-09T09:26:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.815+0000 7f23c1ffb640 1 -- 172.21.15.12:0/2186049736 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f23d404a050 con 0x55b3ce039b90 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.815+0000 7f23e35d6740 10 client.16002 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.815+0000 7f23e35d6740 1 -- 172.21.15.12:0/2186049736 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b3ce0a0190 con 0x55b3ce039b90 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.816+0000 7f23c1ffb640 1 -- 172.21.15.12:0/2186049736 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55b3cdf876c0 con 0x55b3ce039b90 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.816+0000 7f23c1ffb640 1 -- 172.21.15.12:0/2186049736 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f23d4043070 con 0x55b3ce039b90 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.816+0000 7f23e35d6740 10 client.16002 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.816+0000 7f23e35d6740 10 client.16002 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.816+0000 7f23e35d6740 20 client.16002 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.816+0000 7f23e35d6740 1 --2- 172.21.15.12:0/2186049736 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55b3ce0a0a20 0x55b3ce0a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.816+0000 7f23e35d6740 4 client.16002 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.816+0000 7f23e35d6740 1 -- 172.21.15.12:0/2186049736 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3cdf04820 con 0x55b3ce0a0a20 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.816+0000 7f23d9d9b640 1 --2- 172.21.15.12:0/2186049736 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55b3ce0a0a20 0x55b3ce0a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23d9d9b640 1 --2- 172.21.15.12:0/2186049736 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55b3ce0a0a20 0x55b3ce0a2e40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78c37f91f2cc5a52 server_cookie=df09b854f2263cd7 in_seq=0 out_seq=0 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23c1ffb640 10 client.16002 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:26:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23c1ffb640 1 -- 172.21.15.12:0/2186049736 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3cdf04820 con 0x55b3ce0a0a20 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23c1ffb640 10 client.16002 handle_command_reply: tid=0 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23e35d6740 1 client.16002 shutdown 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23e35d6740 1 -- 172.21.15.12:0/2186049736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce039b90 msgr2=0x55b3ce033d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23e35d6740 1 --2- 172.21.15.12:0/2186049736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce039b90 0x55b3ce033d80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f23d4005fd0 tx=0x7f23d4009e40 comp rx=0 tx=0).stop 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23e35d6740 1 -- 172.21.15.12:0/2186049736 shutdown_connections 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23e35d6740 1 --2- 172.21.15.12:0/2186049736 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55b3ce0a0a20 0x55b3ce0a2e40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23e35d6740 1 --2- 172.21.15.12:0/2186049736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ce039b90 0x55b3ce033d80 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23e35d6740 1 --2- 172.21.15.12:0/2186049736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3ce020aa0 0x55b3ce032d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23e35d6740 1 --2- 172.21.15.12:0/2186049736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ce01af40 0x55b3ce032830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23c1ffb640 0 client.16002 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23e35d6740 1 -- 172.21.15.12:0/2186049736 >> 172.21.15.12:0/2186049736 conn(0x55b3ce0165b0 msgr2=0x55b3ce019330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23e35d6740 1 -- 172.21.15.12:0/2186049736 shutdown_connections 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23e35d6740 1 -- 172.21.15.12:0/2186049736 wait complete. 2024-05-09T09:26:43.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.817+0000 7f23e35d6740 20 client.16002 trim_cache size 0 max 16384 2024-05-09T09:26:43.822 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:43.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.818+0000 7f23e08ca640 1 -- 172.21.15.12:0/1511577919 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f23c40a2be0 msgr2=0x7f23c40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.818+0000 7f23e08ca640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f23c40a2be0 0x7f23c40a50a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f23cc097b20 tx=0x7f23cc086040 comp rx=0 tx=0).stop 2024-05-09T09:26:43.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.818+0000 7f23e08ca640 1 -- 172.21.15.12:0/1511577919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23dc100770 msgr2=0x7f23dc197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.818+0000 7f23e08ca640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23dc100770 0x7f23dc197c40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00d950 tx=0x7f23cc00de10 comp rx=0 tx=0).stop 2024-05-09T09:26:43.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.818+0000 7f23e08ca640 1 -- 172.21.15.12:0/1511577919 shutdown_connections 2024-05-09T09:26:43.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.818+0000 7f23e08ca640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f23c40a2be0 0x7f23c40a50a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.818+0000 7f23e08ca640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23dc102320 0x7f23dc19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.818+0000 7f23e08ca640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23dc101960 0x7f23dc198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.818+0000 7f23e08ca640 1 --2- 172.21.15.12:0/1511577919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23dc100770 0x7f23dc197c40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.818+0000 7f23e08ca640 1 -- 172.21.15.12:0/1511577919 >> 172.21.15.12:0/1511577919 conn(0x7f23dc0fbce0 msgr2=0x7f23dc0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.818+0000 7f23e08ca640 1 -- 172.21.15.12:0/1511577919 shutdown_connections 2024-05-09T09:26:43.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.818+0000 7f23e08ca640 1 -- 172.21.15.12:0/1511577919 wait complete. 2024-05-09T09:26:43.832 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:26:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.986+0000 7f100152a640 1 -- 172.21.15.12:0/4224424057 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec035020 con 0x7f0ffc102a70 2024-05-09T09:26:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.986+0000 7f1004790640 1 -- 172.21.15.12:0/4224424057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ffc102a70 msgr2=0x7f0ffc102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.986+0000 7f1004790640 1 --2- 172.21.15.12:0/4224424057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ffc102a70 0x7f0ffc102ef0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009900 tx=0x7f0fec02a410 comp rx=0 tx=0).stop 2024-05-09T09:26:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.986+0000 7f1004790640 1 -- 172.21.15.12:0/4224424057 shutdown_connections 2024-05-09T09:26:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.986+0000 7f1004790640 1 --2- 172.21.15.12:0/4224424057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ffc103430 0x7f0ffc109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.986+0000 7f1004790640 1 --2- 172.21.15.12:0/4224424057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ffc102a70 0x7f0ffc102ef0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.986+0000 7f1004790640 1 --2- 172.21.15.12:0/4224424057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc101880 0x7f0ffc101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.986+0000 7f1004790640 1 -- 172.21.15.12:0/4224424057 >> 172.21.15.12:0/4224424057 conn(0x7f0ffc0fce10 msgr2=0x7f0ffc0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.986+0000 7f1004790640 1 -- 172.21.15.12:0/4224424057 shutdown_connections 2024-05-09T09:26:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.986+0000 7f1004790640 1 -- 172.21.15.12:0/4224424057 wait complete. 2024-05-09T09:26:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.987+0000 7f1004790640 1 Processor -- start 2024-05-09T09:26:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.987+0000 7f1004790640 1 -- start start 2024-05-09T09:26:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.987+0000 7f1004790640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc101880 0x7f0ffc197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.987+0000 7f1004790640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ffc102a70 0x7f0ffc1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.987+0000 7f1004790640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ffc103430 0x7f0ffc19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.987+0000 7f1004790640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc10d530 con 0x7f0ffc102a70 2024-05-09T09:26:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.987+0000 7f1004790640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc10d3b0 con 0x7f0ffc101880 2024-05-09T09:26:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.987+0000 7f1004790640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc10d6b0 con 0x7f0ffc103430 2024-05-09T09:26:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.987+0000 7f1001d2b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ffc102a70 0x7f0ffc1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.987+0000 7f1001d2b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ffc102a70 0x7f0ffc1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34988/0 (socket says 172.21.15.12:34988) 2024-05-09T09:26:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.987+0000 7f1001d2b640 1 -- 172.21.15.12:0/1385037770 learned_addr learned my addr 172.21.15.12:0/1385037770 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.987+0000 7f100252c640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc101880 0x7f0ffc197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f1001d2b640 1 -- 172.21.15.12:0/1385037770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ffc103430 msgr2=0x7f0ffc19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f1002d2d640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ffc103430 0x7f0ffc19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f1001d2b640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ffc103430 0x7f0ffc19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f1001d2b640 1 -- 172.21.15.12:0/1385037770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc101880 msgr2=0x7f0ffc197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f1001d2b640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc101880 0x7f0ffc197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f1001d2b640 1 -- 172.21.15.12:0/1385037770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc19f7a0 con 0x7f0ffc102a70 2024-05-09T09:26:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f100252c640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc101880 0x7f0ffc197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f1002d2d640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ffc103430 0x7f0ffc19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f1001d2b640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ffc102a70 0x7f0ffc1980c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0fec005d90 tx=0x7f0fec004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f0ff37fe640 1 -- 172.21.15.12:0/1385037770 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec040070 con 0x7f0ffc102a70 2024-05-09T09:26:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f0ff37fe640 1 -- 172.21.15.12:0/1385037770 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0fec004d50 con 0x7f0ffc102a70 2024-05-09T09:26:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f1004790640 1 -- 172.21.15.12:0/1385037770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc19fa30 con 0x7f0ffc102a70 2024-05-09T09:26:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f0ff37fe640 1 -- 172.21.15.12:0/1385037770 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec035020 con 0x7f0ffc102a70 2024-05-09T09:26:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.988+0000 7f1004790640 1 -- 172.21.15.12:0/1385037770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc073c40 con 0x7f0ffc102a70 2024-05-09T09:26:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.992+0000 7f1007578740 1 Processor -- start 2024-05-09T09:26:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.992+0000 7f0ff37fe640 1 -- 172.21.15.12:0/1385037770 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0fec043070 con 0x7f0ffc102a70 2024-05-09T09:26:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.992+0000 7f1007578740 1 -- start start 2024-05-09T09:26:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.992+0000 7f1007578740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de528f40 0x5637de529360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.992+0000 7f1007578740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de529930 0x5637de52e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.992+0000 7f1007578740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de52eaa0 0x5637de530f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.992+0000 7f1007578740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5637de48dc40 con 0x5637de529930 2024-05-09T09:26:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.992+0000 7f1007578740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5637de435580 con 0x5637de528f40 2024-05-09T09:26:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.992+0000 7f1007578740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5637de489c00 con 0x5637de52eaa0 2024-05-09T09:26:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.993+0000 7f100252c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de529930 0x5637de52e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.993+0000 7f1002d2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de528f40 0x5637de529360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.993+0000 7f1002d2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de528f40 0x5637de529360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47362/0 (socket says 172.21.15.12:47362) 2024-05-09T09:26:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.993+0000 7f1002d2d640 1 -- 172.21.15.12:0/3918988405 learned_addr learned my addr 172.21.15.12:0/3918988405 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.993+0000 7f1001d2b640 1 --2- 172.21.15.12:0/3918988405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de52eaa0 0x5637de530f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:43 smithi170 ceph-mon[33632]: pgmap v402: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:43.994 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:43 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/387771729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:43.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.993+0000 7f1002d2d640 1 -- 172.21.15.12:0/3918988405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de52eaa0 msgr2=0x5637de530f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.993+0000 7f1002d2d640 1 --2- 172.21.15.12:0/3918988405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de52eaa0 0x5637de530f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.993+0000 7f1002d2d640 1 -- 172.21.15.12:0/3918988405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de529930 msgr2=0x5637de52e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:43.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.993+0000 7f1002d2d640 1 --2- 172.21.15.12:0/3918988405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de529930 0x5637de52e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.993+0000 7f1002d2d640 1 -- 172.21.15.12:0/3918988405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637de1a13f0 con 0x5637de528f40 2024-05-09T09:26:43.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.993+0000 7f100252c640 1 --2- 172.21.15.12:0/3918988405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de529930 0x5637de52e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:43.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.993+0000 7f1002d2d640 1 --2- 172.21.15.12:0/3918988405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de528f40 0x5637de529360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009f10 tx=0x7f0ff400e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c00afb1121e207 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.993+0000 7f0ff37fe640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0fe80a7020 0x7f0fe80a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:43.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.994+0000 7f100252c640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0fe80a7020 0x7f0fe80a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:43.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.994+0000 7f0ff37fe640 1 -- 172.21.15.12:0/1385037770 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f0fec0f0110 con 0x7f0ffc102a70 2024-05-09T09:26:43.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.994+0000 7f0ff0ff9640 1 -- 172.21.15.12:0/3918988405 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4016070 con 0x5637de528f40 2024-05-09T09:26:43.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.994+0000 7f100252c640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0fe80a7020 0x7f0fe80a94e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5637de529d30 tx=0x7f0ff800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:43.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.994+0000 7f0ff0ff9640 1 -- 172.21.15.12:0/3918988405 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0ff4004d20 con 0x5637de528f40 2024-05-09T09:26:43.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.995+0000 7f1007578740 1 -- 172.21.15.12:0/3918988405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de528f40 msgr2=0x5637de529360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:43.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.995+0000 7f1007578740 1 --2- 172.21.15.12:0/3918988405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de528f40 0x5637de529360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009f10 tx=0x7f0ff400e6c0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.995+0000 7f1007578740 1 -- 172.21.15.12:0/3918988405 shutdown_connections 2024-05-09T09:26:43.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.995+0000 7f1007578740 1 --2- 172.21.15.12:0/3918988405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de52eaa0 0x5637de530f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.995+0000 7f1007578740 1 --2- 172.21.15.12:0/3918988405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de529930 0x5637de52e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.995+0000 7f1007578740 1 --2- 172.21.15.12:0/3918988405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de528f40 0x5637de529360 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:43.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.995+0000 7f1007578740 1 -- 172.21.15.12:0/3918988405 >> 172.21.15.12:0/3918988405 conn(0x5637de5245b0 msgr2=0x5637de5249b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:43.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.996+0000 7f1007578740 1 -- 172.21.15.12:0/3918988405 shutdown_connections 2024-05-09T09:26:43.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.996+0000 7f1007578740 1 -- 172.21.15.12:0/3918988405 wait complete. 2024-05-09T09:26:43.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.996+0000 7f1007578740 1 -- start start 2024-05-09T09:26:43.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.996+0000 7f1007578740 1 Processor -- start 2024-05-09T09:26:44.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.997+0000 7f1007578740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de528f40 0x5637de533390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:44.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.997+0000 7f1007578740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de529930 0x5637de5338d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:44.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.997+0000 7f1007578740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de52eaa0 0x5637de53a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:44.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.997+0000 7f1007578740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5637de531af0 con 0x5637de52eaa0 2024-05-09T09:26:44.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.997+0000 7f1007578740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5637de448c40 con 0x5637de528f40 2024-05-09T09:26:44.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.997+0000 7f1007578740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5637de531c70 con 0x5637de529930 2024-05-09T09:26:44.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.997+0000 7f1002d2d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de52eaa0 0x5637de53a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:44.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.997+0000 7f1002d2d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de52eaa0 0x5637de53a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35002/0 (socket says 172.21.15.12:35002) 2024-05-09T09:26:44.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.997+0000 7f1002d2d640 1 -- 172.21.15.12:0/368955734 learned_addr learned my addr 172.21.15.12:0/368955734 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:44.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.997+0000 7f100252c640 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de528f40 0x5637de533390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:44.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.997+0000 7f1007578740 10 client.0 _ll_register_callbacks cb 0x5637de1a0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:44.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.997+0000 7f1001d2b640 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de529930 0x5637de5338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:44.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f100252c640 1 -- 172.21.15.12:0/368955734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de529930 msgr2=0x5637de5338d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:44.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f100252c640 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de529930 0x5637de5338d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f100252c640 1 -- 172.21.15.12:0/368955734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de52eaa0 msgr2=0x5637de53a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:44.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f100252c640 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de52eaa0 0x5637de53a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f100252c640 1 -- 172.21.15.12:0/368955734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637de26b7d0 con 0x5637de528f40 2024-05-09T09:26:44.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f1002d2d640 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de52eaa0 0x5637de53a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:44.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f100252c640 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de528f40 0x5637de533390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8005d00 tx=0x7f0ff8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5241967856f6834e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:44.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f0fe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:26:44.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f0fe5ffb640 1 -- 172.21.15.12:0/368955734 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8010e70 con 0x5637de528f40 2024-05-09T09:26:44.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f1007578740 1 -- 172.21.15.12:0/368955734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff4004d20 con 0x5637de528f40 2024-05-09T09:26:44.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f0fe5ffb640 1 -- 172.21.15.12:0/368955734 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f0ff8017030 con 0x5637de528f40 2024-05-09T09:26:44.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f0fe5ffb640 1 -- 172.21.15.12:0/368955734 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ff8032050 con 0x5637de528f40 2024-05-09T09:26:44.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f1007578740 10 client.25517 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:44.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.998+0000 7f1007578740 1 -- 172.21.15.12:0/368955734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5637de53dc40 con 0x5637de528f40 2024-05-09T09:26:44.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:43.999+0000 7f0fe5ffb640 1 -- 172.21.15.12:0/368955734 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0ff4004d20 con 0x5637de528f40 2024-05-09T09:26:44.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.000+0000 7f0fe5ffb640 1 -- 172.21.15.12:0/368955734 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ff801fb10 con 0x5637de528f40 2024-05-09T09:26:44.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.000+0000 7f1007578740 10 client.25517 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:44.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.000+0000 7f1007578740 10 client.25517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:44.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.000+0000 7f1007578740 20 client.25517 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:44.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.000+0000 7f1007578740 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5637de53f770 0x5637de541be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:44.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.000+0000 7f1007578740 4 client.25517 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:44.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.000+0000 7f1007578740 1 -- 172.21.15.12:0/368955734 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637de49b370 con 0x5637de53f770 2024-05-09T09:26:44.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.000+0000 7f1001d2b640 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5637de53f770 0x5637de541be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:44.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.001+0000 7f0fe5ffb640 10 client.25517 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:44.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.001+0000 7f1001d2b640 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5637de53f770 0x5637de541be0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb2e6e505de2e88e server_cookie=292f8b833c230a69 in_seq=0 out_seq=0 2024-05-09T09:26:44.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.001+0000 7f0fe5ffb640 1 -- 172.21.15.12:0/368955734 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637de49b370 con 0x5637de53f770 2024-05-09T09:26:44.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.001+0000 7f0fe5ffb640 10 client.25517 handle_command_reply: tid=0 2024-05-09T09:26:44.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.001+0000 7f1007578740 1 client.25517 shutdown 2024-05-09T09:26:44.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.002+0000 7f1007578740 1 -- 172.21.15.12:0/368955734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de528f40 msgr2=0x5637de533390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:44.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.002+0000 7f1007578740 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de528f40 0x5637de533390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8005d00 tx=0x7f0ff8004730 comp rx=0 tx=0).stop 2024-05-09T09:26:44.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.002+0000 7f1007578740 1 -- 172.21.15.12:0/368955734 shutdown_connections 2024-05-09T09:26:44.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.002+0000 7f1007578740 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5637de53f770 0x5637de541be0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.002+0000 7f1007578740 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de52eaa0 0x5637de53a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.002+0000 7f1007578740 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de529930 0x5637de5338d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.002+0000 7f1007578740 1 --2- 172.21.15.12:0/368955734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de528f40 0x5637de533390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.002+0000 7f1007578740 1 -- 172.21.15.12:0/368955734 >> 172.21.15.12:0/368955734 conn(0x5637de5245b0 msgr2=0x5637de5237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:44.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.002+0000 7f0fe5ffb640 0 client.25517 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:44.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.002+0000 7f1007578740 1 -- 172.21.15.12:0/368955734 shutdown_connections 2024-05-09T09:26:44.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.002+0000 7f1007578740 1 -- 172.21.15.12:0/368955734 wait complete. 2024-05-09T09:26:44.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.002+0000 7f1007578740 20 client.25517 trim_cache size 0 max 16384 2024-05-09T09:26:44.007 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:43 smithi016 ceph-mon[33335]: pgmap v402: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:26:44.007 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:43 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/387771729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:44.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.029+0000 7f1007578740 1 Processor -- start 2024-05-09T09:26:44.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.029+0000 7f1007578740 1 -- start start 2024-05-09T09:26:44.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.029+0000 7f1007578740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de528f40 0x5637de527ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:44.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.029+0000 7f1007578740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de529930 0x5637de53a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:44.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.029+0000 7f1007578740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de52eaa0 0x5637de5329b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:44.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.029+0000 7f1007578740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1a84f0 con 0x5637de528f40 2024-05-09T09:26:44.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.029+0000 7f1007578740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1aa030 con 0x5637de52eaa0 2024-05-09T09:26:44.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.029+0000 7f1007578740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1a8bc0 con 0x5637de529930 2024-05-09T09:26:44.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.029+0000 7f100252c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de528f40 0x5637de527ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:44.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.029+0000 7f100252c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de528f40 0x5637de527ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35014/0 (socket says 172.21.15.12:35014) 2024-05-09T09:26:44.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.029+0000 7f100252c640 1 -- 172.21.15.12:0/3717601378 learned_addr learned my addr 172.21.15.12:0/3717601378 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:44.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.029+0000 7f1001d2b640 1 --2- 172.21.15.12:0/3717601378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de529930 0x5637de53a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:44.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.030+0000 7f1002d2d640 1 --2- 172.21.15.12:0/3717601378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de52eaa0 0x5637de5329b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:44.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.030+0000 7f1001d2b640 1 -- 172.21.15.12:0/3717601378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de52eaa0 msgr2=0x5637de5329b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:44.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.030+0000 7f1001d2b640 1 --2- 172.21.15.12:0/3717601378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de52eaa0 0x5637de5329b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.030+0000 7f1001d2b640 1 -- 172.21.15.12:0/3717601378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de528f40 msgr2=0x5637de527ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:44.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.030+0000 7f1001d2b640 1 --2- 172.21.15.12:0/3717601378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de528f40 0x5637de527ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.030+0000 7f1001d2b640 1 -- 172.21.15.12:0/3717601378 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637de515d20 con 0x5637de529930 2024-05-09T09:26:44.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.030+0000 7f1001d2b640 1 --2- 172.21.15.12:0/3717601378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de529930 0x5637de53a830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0ed890 tx=0x7f0fec0f2620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db65261d2dc18e90 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:44.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.030+0000 7f0fe7fff640 1 -- 172.21.15.12:0/3717601378 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec04a040 con 0x5637de529930 2024-05-09T09:26:44.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.030+0000 7f0fe7fff640 1 -- 172.21.15.12:0/3717601378 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0fec0bcca0 con 0x5637de529930 2024-05-09T09:26:44.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.030+0000 7f0fe7fff640 1 -- 172.21.15.12:0/3717601378 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec003d10 con 0x5637de529930 2024-05-09T09:26:44.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.031+0000 7f1007578740 1 -- 172.21.15.12:0/3717601378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de529930 msgr2=0x5637de53a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:44.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.031+0000 7f1007578740 1 --2- 172.21.15.12:0/3717601378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de529930 0x5637de53a830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0ed890 tx=0x7f0fec0f2620 comp rx=0 tx=0).stop 2024-05-09T09:26:44.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.031+0000 7f1007578740 1 -- 172.21.15.12:0/3717601378 shutdown_connections 2024-05-09T09:26:44.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.031+0000 7f1007578740 1 --2- 172.21.15.12:0/3717601378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de52eaa0 0x5637de5329b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.031+0000 7f1007578740 1 --2- 172.21.15.12:0/3717601378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de529930 0x5637de53a830 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.031+0000 7f1007578740 1 --2- 172.21.15.12:0/3717601378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de528f40 0x5637de527ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.031+0000 7f1007578740 1 -- 172.21.15.12:0/3717601378 >> 172.21.15.12:0/3717601378 conn(0x5637de5245b0 msgr2=0x5637de527490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:44.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.031+0000 7f1007578740 1 -- 172.21.15.12:0/3717601378 shutdown_connections 2024-05-09T09:26:44.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.032+0000 7f1007578740 1 -- 172.21.15.12:0/3717601378 wait complete. 2024-05-09T09:26:44.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.032+0000 7f1007578740 1 -- start start 2024-05-09T09:26:44.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.032+0000 7f1007578740 1 Processor -- start 2024-05-09T09:26:44.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.032+0000 7f1007578740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de528f40 0x5637de537490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:44.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.032+0000 7f1007578740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de529930 0x5637de5379d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:44.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.032+0000 7f1007578740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de52eaa0 0x5637de5a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:44.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.032+0000 7f1007578740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1a9960 con 0x5637de52eaa0 2024-05-09T09:26:44.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.032+0000 7f1007578740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1a9290 con 0x5637de529930 2024-05-09T09:26:44.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.032+0000 7f1007578740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1a64d0 con 0x5637de528f40 2024-05-09T09:26:44.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f1001d2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de529930 0x5637de5379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:44.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f1002d2d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de52eaa0 0x5637de5a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:44.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f1001d2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de529930 0x5637de5379d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47394/0 (socket says 172.21.15.12:47394) 2024-05-09T09:26:44.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f1001d2b640 1 -- 172.21.15.12:0/4282014163 learned_addr learned my addr 172.21.15.12:0/4282014163 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:44.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f1007578740 10 client.0 _ll_register_callbacks cb 0x7f0ff8005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:44.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f100252c640 1 --2- 172.21.15.12:0/4282014163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de528f40 0x5637de537490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:44.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f1001d2b640 1 -- 172.21.15.12:0/4282014163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de528f40 msgr2=0x5637de537490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:44.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f1001d2b640 1 --2- 172.21.15.12:0/4282014163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de528f40 0x5637de537490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f1001d2b640 1 -- 172.21.15.12:0/4282014163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de52eaa0 msgr2=0x5637de5a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:44.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f1001d2b640 1 --2- 172.21.15.12:0/4282014163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de52eaa0 0x5637de5a3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f1001d2b640 1 -- 172.21.15.12:0/4282014163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637de53aa50 con 0x5637de529930 2024-05-09T09:26:44.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f1002d2d640 1 --2- 172.21.15.12:0/4282014163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de52eaa0 0x5637de5a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:44.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f1001d2b640 1 --2- 172.21.15.12:0/4282014163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de529930 0x5637de5379d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0fec006010 tx=0x7f0fec0bc1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4580c038f9bf947 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:44.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f0fe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:26:44.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f0fe5ffb640 1 -- 172.21.15.12:0/4282014163 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec04a070 con 0x5637de529930 2024-05-09T09:26:44.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f0fe5ffb640 1 -- 172.21.15.12:0/4282014163 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f0fec0b8b80 con 0x5637de529930 2024-05-09T09:26:44.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.033+0000 7f1007578740 1 -- 172.21.15.12:0/4282014163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fec0bcca0 con 0x5637de529930 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.034+0000 7f0fe5ffb640 1 -- 172.21.15.12:0/4282014163 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0fec0ba6e0 con 0x5637de529930 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.034+0000 7f1007578740 10 client.25523 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.034+0000 7f1007578740 1 -- 172.21.15.12:0/4282014163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5637de5a5260 con 0x5637de529930 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.034+0000 7f0fe5ffb640 1 -- 172.21.15.12:0/4282014163 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0fec0bc5a0 con 0x5637de529930 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.035+0000 7f0fe5ffb640 1 -- 172.21.15.12:0/4282014163 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0fec03a040 con 0x5637de529930 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.035+0000 7f1007578740 10 client.25523 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.035+0000 7f1007578740 10 client.25523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.035+0000 7f1007578740 20 client.25523 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.035+0000 7f1007578740 1 --2- 172.21.15.12:0/4282014163 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5637de5a5b50 0x5637de5a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.035+0000 7f1007578740 4 client.25523 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.035+0000 7f1007578740 1 -- 172.21.15.12:0/4282014163 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0fec0bcca0 con 0x5637de5a5b50 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.035+0000 7f100252c640 1 --2- 172.21.15.12:0/4282014163 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5637de5a5b50 0x5637de5a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f0fe5ffb640 10 client.25523 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:44.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f100252c640 1 --2- 172.21.15.12:0/4282014163 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5637de5a5b50 0x5637de5a7f40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f2b788b6163cec server_cookie=f584b516e10aeba0 in_seq=0 out_seq=0 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f0fe5ffb640 1 -- 172.21.15.12:0/4282014163 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0fec0bcca0 con 0x5637de5a5b50 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f0fe5ffb640 10 client.25523 handle_command_reply: tid=0 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f1007578740 1 client.25523 shutdown 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f1007578740 1 -- 172.21.15.12:0/4282014163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de529930 msgr2=0x5637de5379d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f1007578740 1 --2- 172.21.15.12:0/4282014163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de529930 0x5637de5379d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0fec006010 tx=0x7f0fec0bc1e0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f1007578740 1 -- 172.21.15.12:0/4282014163 shutdown_connections 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f1007578740 1 --2- 172.21.15.12:0/4282014163 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5637de5a5b50 0x5637de5a7f40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f1007578740 1 --2- 172.21.15.12:0/4282014163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637de52eaa0 0x5637de5a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f0fe5ffb640 0 client.25523 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f1007578740 1 --2- 172.21.15.12:0/4282014163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637de529930 0x5637de5379d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f1007578740 1 --2- 172.21.15.12:0/4282014163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5637de528f40 0x5637de537490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f1007578740 1 -- 172.21.15.12:0/4282014163 >> 172.21.15.12:0/4282014163 conn(0x5637de5245b0 msgr2=0x5637de527d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f1007578740 1 -- 172.21.15.12:0/4282014163 shutdown_connections 2024-05-09T09:26:44.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f1007578740 1 -- 172.21.15.12:0/4282014163 wait complete. 2024-05-09T09:26:44.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.036+0000 7f1007578740 20 client.25523 trim_cache size 0 max 16384 2024-05-09T09:26:44.042 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:44.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.037+0000 7f1004790640 1 -- 172.21.15.12:0/1385037770 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0fe80a7020 msgr2=0x7f0fe80a94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:44.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.037+0000 7f1004790640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0fe80a7020 0x7f0fe80a94e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5637de529d30 tx=0x7f0ff800b390 comp rx=0 tx=0).stop 2024-05-09T09:26:44.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.037+0000 7f1004790640 1 -- 172.21.15.12:0/1385037770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ffc102a70 msgr2=0x7f0ffc1980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:44.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.037+0000 7f1004790640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ffc102a70 0x7f0ffc1980c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0fec005d90 tx=0x7f0fec004950 comp rx=0 tx=0).stop 2024-05-09T09:26:44.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.037+0000 7f1004790640 1 -- 172.21.15.12:0/1385037770 shutdown_connections 2024-05-09T09:26:44.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.037+0000 7f1004790640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0fe80a7020 0x7f0fe80a94e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.037+0000 7f1004790640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ffc103430 0x7f0ffc19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.037+0000 7f1004790640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ffc102a70 0x7f0ffc1980c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.037+0000 7f1004790640 1 --2- 172.21.15.12:0/1385037770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc101880 0x7f0ffc197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:44.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.037+0000 7f1004790640 1 -- 172.21.15.12:0/1385037770 >> 172.21.15.12:0/1385037770 conn(0x7f0ffc0fce10 msgr2=0x7f0ffc105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:44.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.037+0000 7f1004790640 1 -- 172.21.15.12:0/1385037770 shutdown_connections 2024-05-09T09:26:44.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:44.037+0000 7f1004790640 1 -- 172.21.15.12:0/1385037770 wait complete. 2024-05-09T09:26:45.053 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:26:45.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:26:45.053 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:26:45.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.177+0000 7f756d409640 1 -- 172.21.15.12:0/1064171411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7568100770 msgr2=0x7f7568100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.177+0000 7f756d409640 1 --2- 172.21.15.12:0/1064171411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7568100770 0x7f7568100b90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7558009900 tx=0x7f755802a410 comp rx=0 tx=0).stop 2024-05-09T09:26:45.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.177+0000 7f756d409640 1 -- 172.21.15.12:0/1064171411 shutdown_connections 2024-05-09T09:26:45.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.177+0000 7f756d409640 1 --2- 172.21.15.12:0/1064171411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7568102320 0x7f7568108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.177+0000 7f756d409640 1 --2- 172.21.15.12:0/1064171411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7568101960 0x7f7568101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.177+0000 7f756d409640 1 --2- 172.21.15.12:0/1064171411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7568100770 0x7f7568100b90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.177+0000 7f756d409640 1 -- 172.21.15.12:0/1064171411 >> 172.21.15.12:0/1064171411 conn(0x7f75680fbce0 msgr2=0x7f75680fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.177+0000 7f756d409640 1 -- 172.21.15.12:0/1064171411 shutdown_connections 2024-05-09T09:26:45.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.177+0000 7f756d409640 1 -- 172.21.15.12:0/1064171411 wait complete. 2024-05-09T09:26:45.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.178+0000 7f756d409640 1 Processor -- start 2024-05-09T09:26:45.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.178+0000 7f756d409640 1 -- start start 2024-05-09T09:26:45.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.178+0000 7f756d409640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7568100770 0x7f7568069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.178+0000 7f756d409640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7568101960 0x7f756819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.178+0000 7f756d409640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7568102320 0x7f756819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.178+0000 7f756d409640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f756810c3d0 con 0x7f7568102320 2024-05-09T09:26:45.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.178+0000 7f756d409640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f756810c250 con 0x7f7568101960 2024-05-09T09:26:45.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.178+0000 7f756d409640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f756810c550 con 0x7f7568100770 2024-05-09T09:26:45.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.178+0000 7f75667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7568101960 0x7f756819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.178+0000 7f75677fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7568102320 0x7f756819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.178+0000 7f75667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7568101960 0x7f756819c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47420/0 (socket says 172.21.15.12:47420) 2024-05-09T09:26:45.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.178+0000 7f75667fc640 1 -- 172.21.15.12:0/4244662443 learned_addr learned my addr 172.21.15.12:0/4244662443 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.178+0000 7f7566ffd640 1 --2- 172.21.15.12:0/4244662443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7568100770 0x7f7568069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.179+0000 7f75667fc640 1 -- 172.21.15.12:0/4244662443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7568100770 msgr2=0x7f7568069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.179+0000 7f75667fc640 1 --2- 172.21.15.12:0/4244662443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7568100770 0x7f7568069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.179+0000 7f75667fc640 1 -- 172.21.15.12:0/4244662443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7568102320 msgr2=0x7f756819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.179+0000 7f75667fc640 1 --2- 172.21.15.12:0/4244662443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7568102320 0x7f756819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.179+0000 7f75667fc640 1 -- 172.21.15.12:0/4244662443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756819f880 con 0x7f7568101960 2024-05-09T09:26:45.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.179+0000 7f75667fc640 1 --2- 172.21.15.12:0/4244662443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7568101960 0x7f756819c950 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f755c00d770 tx=0x7f755c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.179+0000 7f754ffff640 1 -- 172.21.15.12:0/4244662443 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c017070 con 0x7f7568101960 2024-05-09T09:26:45.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.179+0000 7f756d409640 1 -- 172.21.15.12:0/4244662443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756819fb10 con 0x7f7568101960 2024-05-09T09:26:45.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.179+0000 7f754ffff640 1 -- 172.21.15.12:0/4244662443 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f755c0054f0 con 0x7f7568101960 2024-05-09T09:26:45.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.179+0000 7f754ffff640 1 -- 172.21.15.12:0/4244662443 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c004d60 con 0x7f7568101960 2024-05-09T09:26:45.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.179+0000 7f756d409640 1 -- 172.21.15.12:0/4244662443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75681a0060 con 0x7f7568101960 2024-05-09T09:26:45.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.182+0000 7f75701f0740 1 Processor -- start 2024-05-09T09:26:45.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f75701f0740 1 -- start start 2024-05-09T09:26:45.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f754ffff640 1 -- 172.21.15.12:0/4244662443 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f755c010460 con 0x7f7568101960 2024-05-09T09:26:45.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f75701f0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a7fb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f75701f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 0x56148a7fe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f75701f0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 0x56148a80be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f75701f0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56148a75f990 con 0x56148a7fb4c0 2024-05-09T09:26:45.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f75701f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56148a7075b0 con 0x56148a7fbeb0 2024-05-09T09:26:45.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f75701f0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56148a75b950 con 0x56148a7ff010 2024-05-09T09:26:45.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f75677fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a7fb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f75677fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a7fb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35040/0 (socket says 172.21.15.12:35040) 2024-05-09T09:26:45.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f75677fe640 1 -- 172.21.15.12:0/4065865632 learned_addr learned my addr 172.21.15.12:0/4065865632 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f75667fc640 1 --2- 172.21.15.12:0/4065865632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 0x56148a80be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f7566ffd640 1 --2- 172.21.15.12:0/4065865632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 0x56148a7fe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f75677fe640 1 -- 172.21.15.12:0/4065865632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 msgr2=0x56148a80be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.183+0000 7f75677fe640 1 --2- 172.21.15.12:0/4065865632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 0x56148a80be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.184+0000 7f75677fe640 1 -- 172.21.15.12:0/4065865632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 msgr2=0x56148a7fe9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.184+0000 7f75677fe640 1 --2- 172.21.15.12:0/4065865632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 0x56148a7fe9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.184+0000 7f75677fe640 1 -- 172.21.15.12:0/4065865632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56148a53b7e0 con 0x56148a7fb4c0 2024-05-09T09:26:45.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.184+0000 7f75667fc640 1 --2- 172.21.15.12:0/4065865632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 0x56148a80be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:45.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.184+0000 7f75677fe640 1 --2- 172.21.15.12:0/4065865632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a7fb8e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7560009fd0 tx=0x7f75600118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f162da275fcfa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.184+0000 7f754d7fa640 1 -- 172.21.15.12:0/4065865632 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560004d20 con 0x56148a7fb4c0 2024-05-09T09:26:45.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.184+0000 7f754ffff640 1 --2- 172.21.15.12:0/4244662443 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f75500a2c10 0x7f75500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.184+0000 7f754d7fa640 1 -- 172.21.15.12:0/4065865632 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7560004050 con 0x56148a7fb4c0 2024-05-09T09:26:45.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.184+0000 7f754d7fa640 1 -- 172.21.15.12:0/4065865632 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560005240 con 0x56148a7fb4c0 2024-05-09T09:26:45.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.184+0000 7f7566ffd640 1 --2- 172.21.15.12:0/4244662443 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f75500a2c10 0x7f75500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.184+0000 7f754ffff640 1 -- 172.21.15.12:0/4244662443 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f755c0ced20 con 0x7f7568101960 2024-05-09T09:26:45.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.185+0000 7f75701f0740 1 -- 172.21.15.12:0/4065865632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 msgr2=0x56148a7fb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.185+0000 7f7566ffd640 1 --2- 172.21.15.12:0/4244662443 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f75500a2c10 0x7f75500a50d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7558006010 tx=0x7f7558031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.185+0000 7f75701f0740 1 --2- 172.21.15.12:0/4065865632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a7fb8e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7560009fd0 tx=0x7f75600118d0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.185+0000 7f75701f0740 1 -- 172.21.15.12:0/4065865632 shutdown_connections 2024-05-09T09:26:45.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.185+0000 7f75701f0740 1 --2- 172.21.15.12:0/4065865632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 0x56148a80be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.185+0000 7f75701f0740 1 --2- 172.21.15.12:0/4065865632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 0x56148a7fe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.185+0000 7f75701f0740 1 --2- 172.21.15.12:0/4065865632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a7fb8e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.185+0000 7f75701f0740 1 -- 172.21.15.12:0/4065865632 >> 172.21.15.12:0/4065865632 conn(0x56148a7c30a0 msgr2=0x56148a7f6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.185+0000 7f75701f0740 1 -- 172.21.15.12:0/4065865632 shutdown_connections 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.185+0000 7f75701f0740 1 -- 172.21.15.12:0/4065865632 wait complete. 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.185+0000 7f75701f0740 1 -- start start 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.186+0000 7f75701f0740 1 Processor -- start 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.186+0000 7f75701f0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a80e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.186+0000 7f75701f0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7fbeb0 0x56148a80e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.186+0000 7f75701f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7ff010 0x56148a8156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.186+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a80e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.186+0000 7f75701f0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56148a80c890 con 0x56148a7fb4c0 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.186+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a80e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35054/0 (socket says 172.21.15.12:35054) 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.186+0000 7f7566ffd640 1 -- 172.21.15.12:0/1673721364 learned_addr learned my addr 172.21.15.12:0/1673721364 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.186+0000 7f75701f0740 1 -- 172.21.15.12:0/1673721364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56148a71ab40 con 0x56148a7ff010 2024-05-09T09:26:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.186+0000 7f75701f0740 1 -- 172.21.15.12:0/1673721364 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56148a80ca10 con 0x56148a7fbeb0 2024-05-09T09:26:45.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.186+0000 7f75701f0740 10 client.0 _ll_register_callbacks cb 0x56148a70f0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:45.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.186+0000 7f75667fc640 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7fbeb0 0x56148a80e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.186+0000 7f75677fe640 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7ff010 0x56148a8156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.187+0000 7f75677fe640 1 -- 172.21.15.12:0/1673721364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7fbeb0 msgr2=0x56148a80e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.187+0000 7f75677fe640 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7fbeb0 0x56148a80e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.187+0000 7f75677fe640 1 -- 172.21.15.12:0/1673721364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 msgr2=0x56148a80e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.187+0000 7f75677fe640 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a80e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.187+0000 7f75677fe640 1 -- 172.21.15.12:0/1673721364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56148a59bf70 con 0x56148a7ff010 2024-05-09T09:26:45.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.187+0000 7f7566ffd640 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a80e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.187+0000 7f75677fe640 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7ff010 0x56148a8156c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f756001b040 tx=0x7f7560004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed14e1e9b7a78f51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.187+0000 7f754aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:26:45.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.187+0000 7f754aff5640 1 -- 172.21.15.12:0/1673721364 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560029020 con 0x56148a7ff010 2024-05-09T09:26:45.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.187+0000 7f754aff5640 1 -- 172.21.15.12:0/1673721364 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f7560005020 con 0x56148a7ff010 2024-05-09T09:26:45.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.187+0000 7f75701f0740 1 -- 172.21.15.12:0/1673721364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7560004050 con 0x56148a7ff010 2024-05-09T09:26:45.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.188+0000 7f754aff5640 1 -- 172.21.15.12:0/1673721364 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7560034050 con 0x56148a7ff010 2024-05-09T09:26:45.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.188+0000 7f75701f0740 10 client.25535 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:45.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.188+0000 7f75701f0740 1 -- 172.21.15.12:0/1673721364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56148a433170 con 0x56148a7ff010 2024-05-09T09:26:45.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.188+0000 7f754aff5640 1 -- 172.21.15.12:0/1673721364 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7560004050 con 0x56148a7ff010 2024-05-09T09:26:45.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.188+0000 7f754aff5640 1 -- 172.21.15.12:0/1673721364 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f756000f670 con 0x56148a7ff010 2024-05-09T09:26:45.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.188+0000 7f75701f0740 10 client.25535 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:45.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.188+0000 7f75701f0740 10 client.25535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:45.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.188+0000 7f75701f0740 20 client.25535 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.188+0000 7f75701f0740 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56148a8199f0 0x56148a81be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.188+0000 7f75701f0740 4 client.25535 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.188+0000 7f75701f0740 1 -- 172.21.15.12:0/1673721364 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56148a76d0c0 con 0x56148a8199f0 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.189+0000 7f7566ffd640 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56148a8199f0 0x56148a81be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.189+0000 7f754aff5640 10 client.25535 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.189+0000 7f7566ffd640 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56148a8199f0 0x56148a81be00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6fa781defe0cb24 server_cookie=28100243c5c9de67 in_seq=0 out_seq=0 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f754aff5640 1 -- 172.21.15.12:0/1673721364 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56148a76d0c0 con 0x56148a8199f0 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f754aff5640 10 client.25535 handle_command_reply: tid=0 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f75701f0740 1 client.25535 shutdown 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f75701f0740 1 -- 172.21.15.12:0/1673721364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7ff010 msgr2=0x56148a8156c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f75701f0740 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7ff010 0x56148a8156c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f756001b040 tx=0x7f7560004c70 comp rx=0 tx=0).stop 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f75701f0740 1 -- 172.21.15.12:0/1673721364 shutdown_connections 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f75701f0740 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56148a8199f0 0x56148a81be00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f754aff5640 0 client.25535 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:45.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f75701f0740 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7ff010 0x56148a8156c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f75701f0740 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7fbeb0 0x56148a80e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f75701f0740 1 --2- 172.21.15.12:0/1673721364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a80e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f75701f0740 1 -- 172.21.15.12:0/1673721364 >> 172.21.15.12:0/1673721364 conn(0x56148a7c30a0 msgr2=0x56148a7f6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f75701f0740 1 -- 172.21.15.12:0/1673721364 shutdown_connections 2024-05-09T09:26:45.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f75701f0740 1 -- 172.21.15.12:0/1673721364 wait complete. 2024-05-09T09:26:45.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.190+0000 7f75701f0740 20 client.25535 trim_cache size 0 max 16384 2024-05-09T09:26:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.208+0000 7f75701f0740 1 Processor -- start 2024-05-09T09:26:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.208+0000 7f75701f0740 1 -- start start 2024-05-09T09:26:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f75701f0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a7f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f75701f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 0x56148a8155e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f75701f0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 0x56148a7fa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f75701f0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f75681a8440 con 0x56148a7fb4c0 2024-05-09T09:26:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f75701f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75681a9f80 con 0x56148a7fbeb0 2024-05-09T09:26:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f75701f0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75681a8b10 con 0x56148a7ff010 2024-05-09T09:26:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a7f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a7f9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35062/0 (socket says 172.21.15.12:35062) 2024-05-09T09:26:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f7566ffd640 1 -- 172.21.15.12:0/2144734536 learned_addr learned my addr 172.21.15.12:0/2144734536 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f75667fc640 1 --2- 172.21.15.12:0/2144734536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 0x56148a8155e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f75677fe640 1 --2- 172.21.15.12:0/2144734536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 0x56148a7fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f7566ffd640 1 -- 172.21.15.12:0/2144734536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 msgr2=0x56148a7fa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f7566ffd640 1 --2- 172.21.15.12:0/2144734536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 0x56148a7fa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f7566ffd640 1 -- 172.21.15.12:0/2144734536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 msgr2=0x56148a8155e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f7566ffd640 1 --2- 172.21.15.12:0/2144734536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 0x56148a8155e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f7566ffd640 1 -- 172.21.15.12:0/2144734536 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56148a80b950 con 0x56148a7fb4c0 2024-05-09T09:26:45.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.209+0000 7f75667fc640 1 --2- 172.21.15.12:0/2144734536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 0x56148a8155e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.210+0000 7f7566ffd640 1 --2- 172.21.15.12:0/2144734536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a7f9e00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f75681017c0 tx=0x7f755803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43117c6ed5bbced5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.210+0000 7f754cff9640 1 -- 172.21.15.12:0/2144734536 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558035040 con 0x56148a7fb4c0 2024-05-09T09:26:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.210+0000 7f754cff9640 1 -- 172.21.15.12:0/2144734536 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7558058070 con 0x56148a7fb4c0 2024-05-09T09:26:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.210+0000 7f754cff9640 1 -- 172.21.15.12:0/2144734536 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558003e20 con 0x56148a7fb4c0 2024-05-09T09:26:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.211+0000 7f75701f0740 1 -- 172.21.15.12:0/2144734536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 msgr2=0x56148a7f9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.211+0000 7f75701f0740 1 --2- 172.21.15.12:0/2144734536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a7f9e00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f75681017c0 tx=0x7f755803f000 comp rx=0 tx=0).stop 2024-05-09T09:26:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.211+0000 7f75701f0740 1 -- 172.21.15.12:0/2144734536 shutdown_connections 2024-05-09T09:26:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.211+0000 7f75701f0740 1 --2- 172.21.15.12:0/2144734536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 0x56148a7fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.211+0000 7f75701f0740 1 --2- 172.21.15.12:0/2144734536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 0x56148a8155e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.211+0000 7f75701f0740 1 --2- 172.21.15.12:0/2144734536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a7f9e00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.211+0000 7f75701f0740 1 -- 172.21.15.12:0/2144734536 >> 172.21.15.12:0/2144734536 conn(0x56148a7c30a0 msgr2=0x56148a80e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.211+0000 7f75701f0740 1 -- 172.21.15.12:0/2144734536 shutdown_connections 2024-05-09T09:26:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.211+0000 7f75701f0740 1 -- 172.21.15.12:0/2144734536 wait complete. 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.211+0000 7f75701f0740 1 -- start start 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.212+0000 7f75701f0740 1 Processor -- start 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.212+0000 7f75701f0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a812ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.212+0000 7f75701f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 0x56148a8133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.212+0000 7f75701f0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 0x56148a8143f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.212+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a812ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.212+0000 7f75701f0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f75681a98b0 con 0x56148a7fb4c0 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.212+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a812ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35064/0 (socket says 172.21.15.12:35064) 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.212+0000 7f7566ffd640 1 -- 172.21.15.12:0/3286910924 learned_addr learned my addr 172.21.15.12:0/3286910924 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.212+0000 7f75701f0740 1 -- 172.21.15.12:0/3286910924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75681a91e0 con 0x56148a7fbeb0 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.212+0000 7f75701f0740 1 -- 172.21.15.12:0/3286910924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75681a6420 con 0x56148a7ff010 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.212+0000 7f75701f0740 10 client.0 _ll_register_callbacks cb 0x7f7558009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.212+0000 7f75667fc640 1 --2- 172.21.15.12:0/3286910924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 0x56148a8133e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.212+0000 7f75677fe640 1 --2- 172.21.15.12:0/3286910924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 0x56148a8143f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.213+0000 7f7566ffd640 1 -- 172.21.15.12:0/3286910924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 msgr2=0x56148a8143f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.213+0000 7f7566ffd640 1 --2- 172.21.15.12:0/3286910924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 0x56148a8143f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.213+0000 7f7566ffd640 1 -- 172.21.15.12:0/3286910924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 msgr2=0x56148a8133e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.213+0000 7f7566ffd640 1 --2- 172.21.15.12:0/3286910924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 0x56148a8133e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.213+0000 7f7566ffd640 1 -- 172.21.15.12:0/3286910924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56148a8149e0 con 0x56148a7fb4c0 2024-05-09T09:26:45.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.213+0000 7f7566ffd640 1 --2- 172.21.15.12:0/3286910924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a812ea0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7558004630 tx=0x7f75580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f36f8c78758457 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.213+0000 7f754aff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:45.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.213+0000 7f754aff5640 1 -- 172.21.15.12:0/3286910924 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558003ba0 con 0x56148a7fb4c0 2024-05-09T09:26:45.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.213+0000 7f754aff5640 1 -- 172.21.15.12:0/3286910924 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f7558003d40 con 0x56148a7fb4c0 2024-05-09T09:26:45.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.213+0000 7f75701f0740 1 -- 172.21.15.12:0/3286910924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7558058070 con 0x56148a7fb4c0 2024-05-09T09:26:45.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.213+0000 7f754aff5640 1 -- 172.21.15.12:0/3286910924 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f755806e050 con 0x56148a7fb4c0 2024-05-09T09:26:45.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.213+0000 7f75701f0740 10 client.16038 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:45.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.213+0000 7f75701f0740 1 -- 172.21.15.12:0/3286910924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56148a87ffe0 con 0x56148a7fb4c0 2024-05-09T09:26:45.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.214+0000 7f754aff5640 1 -- 172.21.15.12:0/3286910924 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7558058070 con 0x56148a7fb4c0 2024-05-09T09:26:45.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.214+0000 7f754aff5640 1 -- 172.21.15.12:0/3286910924 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f755803dab0 con 0x56148a7fb4c0 2024-05-09T09:26:45.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.214+0000 7f75701f0740 10 client.16038 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:45.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.214+0000 7f75701f0740 10 client.16038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:45.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.214+0000 7f75701f0740 20 client.16038 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:45.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.214+0000 7f75701f0740 1 --2- 172.21.15.12:0/3286910924 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56148a8199f0 0x56148a882950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.214+0000 7f75701f0740 4 client.16038 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:26:45.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.214+0000 7f75701f0740 1 -- 172.21.15.12:0/3286910924 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56148a7fed80 con 0x56148a8199f0 2024-05-09T09:26:45.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.214+0000 7f75667fc640 1 --2- 172.21.15.12:0/3286910924 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56148a8199f0 0x56148a882950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.215+0000 7f75667fc640 1 --2- 172.21.15.12:0/3286910924 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56148a8199f0 0x56148a882950 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc935d960b8031b1 server_cookie=2454f0bf77c03e6c in_seq=0 out_seq=0 2024-05-09T09:26:45.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.215+0000 7f754aff5640 10 client.16038 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:45.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.216+0000 7f754aff5640 1 -- 172.21.15.12:0/3286910924 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56148a7fed80 con 0x56148a8199f0 2024-05-09T09:26:45.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.216+0000 7f754aff5640 10 client.16038 handle_command_reply: tid=0 2024-05-09T09:26:45.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.216+0000 7f75701f0740 1 client.16038 shutdown 2024-05-09T09:26:45.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.216+0000 7f75701f0740 1 -- 172.21.15.12:0/3286910924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 msgr2=0x56148a812ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.216+0000 7f75701f0740 1 --2- 172.21.15.12:0/3286910924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a812ea0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7558004630 tx=0x7f75580046c0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.216+0000 7f75701f0740 1 -- 172.21.15.12:0/3286910924 shutdown_connections 2024-05-09T09:26:45.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.216+0000 7f75701f0740 1 --2- 172.21.15.12:0/3286910924 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56148a8199f0 0x56148a882950 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.216+0000 7f754aff5640 0 client.16038 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:45.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.216+0000 7f75701f0740 1 --2- 172.21.15.12:0/3286910924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56148a7ff010 0x56148a8143f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.216+0000 7f75701f0740 1 --2- 172.21.15.12:0/3286910924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56148a7fbeb0 0x56148a8133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.216+0000 7f75701f0740 1 --2- 172.21.15.12:0/3286910924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56148a7fb4c0 0x56148a812ea0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.217+0000 7f75701f0740 1 -- 172.21.15.12:0/3286910924 >> 172.21.15.12:0/3286910924 conn(0x56148a7c30a0 msgr2=0x56148a7f57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.217+0000 7f75701f0740 1 -- 172.21.15.12:0/3286910924 shutdown_connections 2024-05-09T09:26:45.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.217+0000 7f75701f0740 1 -- 172.21.15.12:0/3286910924 wait complete. 2024-05-09T09:26:45.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.217+0000 7f75701f0740 20 client.16038 trim_cache size 0 max 16384 2024-05-09T09:26:45.220 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:26:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T09:26:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "c8ad7dd1-25b8-45e3-b512-b2a42b5b5279", 2024-05-09T09:26:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T09:26:45.220 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:26:45.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.217+0000 7f756d409640 1 -- 172.21.15.12:0/4244662443 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f75500a2c10 msgr2=0x7f75500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.217+0000 7f756d409640 1 --2- 172.21.15.12:0/4244662443 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f75500a2c10 0x7f75500a50d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7558006010 tx=0x7f7558031510 comp rx=0 tx=0).stop 2024-05-09T09:26:45.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.217+0000 7f756d409640 1 -- 172.21.15.12:0/4244662443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7568101960 msgr2=0x7f756819c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.217+0000 7f756d409640 1 --2- 172.21.15.12:0/4244662443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7568101960 0x7f756819c950 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f755c00d770 tx=0x7f755c00dc30 comp rx=0 tx=0).stop 2024-05-09T09:26:45.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.218+0000 7f756d409640 1 -- 172.21.15.12:0/4244662443 shutdown_connections 2024-05-09T09:26:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.218+0000 7f756d409640 1 --2- 172.21.15.12:0/4244662443 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f75500a2c10 0x7f75500a50d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.218+0000 7f756d409640 1 --2- 172.21.15.12:0/4244662443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7568102320 0x7f756819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.218+0000 7f756d409640 1 --2- 172.21.15.12:0/4244662443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7568101960 0x7f756819c950 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.218+0000 7f756d409640 1 --2- 172.21.15.12:0/4244662443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7568100770 0x7f7568069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.218+0000 7f756d409640 1 -- 172.21.15.12:0/4244662443 >> 172.21.15.12:0/4244662443 conn(0x7f75680fbce0 msgr2=0x7f7568104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.218+0000 7f756d409640 1 -- 172.21.15.12:0/4244662443 shutdown_connections 2024-05-09T09:26:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.218+0000 7f756d409640 1 -- 172.21.15.12:0/4244662443 wait complete. 2024-05-09T09:26:45.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:26:45.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:26:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.307+0000 7f40892c1640 1 -- 172.21.15.12:0/3005391154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084101980 msgr2=0x7f4084101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.307+0000 7f40892c1640 1 --2- 172.21.15.12:0/3005391154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084101980 0x7f4084101e00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4078009900 tx=0x7f407802a410 comp rx=0 tx=0).stop 2024-05-09T09:26:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.307+0000 7f40892c1640 1 -- 172.21.15.12:0/3005391154 shutdown_connections 2024-05-09T09:26:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.307+0000 7f40892c1640 1 --2- 172.21.15.12:0/3005391154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4084102340 0x7f4084108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.307+0000 7f40892c1640 1 --2- 172.21.15.12:0/3005391154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084101980 0x7f4084101e00 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.307+0000 7f40892c1640 1 --2- 172.21.15.12:0/3005391154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4084100790 0x7f4084100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.307+0000 7f40892c1640 1 -- 172.21.15.12:0/3005391154 >> 172.21.15.12:0/3005391154 conn(0x7f40840fbce0 msgr2=0x7f40840fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.308+0000 7f40892c1640 1 -- 172.21.15.12:0/3005391154 shutdown_connections 2024-05-09T09:26:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.308+0000 7f40892c1640 1 -- 172.21.15.12:0/3005391154 wait complete. 2024-05-09T09:26:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.309+0000 7f40892c1640 1 Processor -- start 2024-05-09T09:26:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.309+0000 7f40892c1640 1 -- start start 2024-05-09T09:26:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.310+0000 7f40892c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084100790 0x7f4084195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.310+0000 7f40892c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4084101980 0x7f4084195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.310+0000 7f4082ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084100790 0x7f4084195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.310+0000 7f4082ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084100790 0x7f4084195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35084/0 (socket says 172.21.15.12:35084) 2024-05-09T09:26:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.310+0000 7f4082ffd640 1 -- 172.21.15.12:0/3616833854 learned_addr learned my addr 172.21.15.12:0/3616833854 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.311+0000 7f40892c1640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4084102340 0x7f408419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.311+0000 7f40892c1640 1 -- 172.21.15.12:0/3616833854 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f408410b330 con 0x7f4084100790 2024-05-09T09:26:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.311+0000 7f40827fc640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4084101980 0x7f4084195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.311+0000 7f40837fe640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4084102340 0x7f408419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.311+0000 7f40892c1640 1 -- 172.21.15.12:0/3616833854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f408410b1b0 con 0x7f4084102340 2024-05-09T09:26:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.311+0000 7f40892c1640 1 -- 172.21.15.12:0/3616833854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f408410b4b0 con 0x7f4084101980 2024-05-09T09:26:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.312+0000 7f40837fe640 1 -- 172.21.15.12:0/3616833854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4084101980 msgr2=0x7f4084195bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.312+0000 7f40837fe640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4084101980 0x7f4084195bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.312+0000 7f40837fe640 1 -- 172.21.15.12:0/3616833854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084100790 msgr2=0x7f4084195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.312+0000 7f40837fe640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084100790 0x7f4084195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.312+0000 7f40837fe640 1 -- 172.21.15.12:0/3616833854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408419d290 con 0x7f4084102340 2024-05-09T09:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.312+0000 7f40837fe640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4084102340 0x7f408419cc20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f407c00e940 tx=0x7f407c00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.312+0000 7f406bfff640 1 -- 172.21.15.12:0/3616833854 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c004d20 con 0x7f4084102340 2024-05-09T09:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.312+0000 7f40892c1640 1 -- 172.21.15.12:0/3616833854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408419d580 con 0x7f4084102340 2024-05-09T09:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.312+0000 7f40892c1640 1 -- 172.21.15.12:0/3616833854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408419dad0 con 0x7f4084102340 2024-05-09T09:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.313+0000 7f406bfff640 1 -- 172.21.15.12:0/3616833854 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f407c005020 con 0x7f4084102340 2024-05-09T09:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.313+0000 7f406bfff640 1 -- 172.21.15.12:0/3616833854 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c011690 con 0x7f4084102340 2024-05-09T09:26:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.317+0000 7f408bfcd740 1 Processor -- start 2024-05-09T09:26:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.317+0000 7f406bfff640 1 -- 172.21.15.12:0/3616833854 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f407c00cb20 con 0x7f4084102340 2024-05-09T09:26:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.317+0000 7f408bfcd740 1 -- start start 2024-05-09T09:26:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.317+0000 7f408bfcd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.317+0000 7f408bfcd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.317+0000 7f408bfcd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02b2db0 0x564bc02b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.317+0000 7f408bfcd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564bc0211340 con 0x564bc02b2db0 2024-05-09T09:26:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.317+0000 7f408bfcd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bc01b9290 con 0x564bc02ad250 2024-05-09T09:26:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.317+0000 7f408bfcd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564bc020d300 con 0x564bc02adc40 2024-05-09T09:26:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.318+0000 7f4082ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.318+0000 7f4082ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:37182/0 (socket says 172.21.15.12:37182) 2024-05-09T09:26:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.318+0000 7f4082ffd640 1 -- 172.21.15.12:0/104860647 learned_addr learned my addr 172.21.15.12:0/104860647 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.318+0000 7f40827fc640 1 --2- 172.21.15.12:0/104860647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02b2db0 0x564bc02b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.318+0000 7f406bfff640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f406c0a2d80 0x7f406c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.318+0000 7f4082ffd640 1 -- 172.21.15.12:0/104860647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 msgr2=0x564bc02ad670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:26:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.318+0000 7f4082ffd640 1 --2- 172.21.15.12:0/104860647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02ad670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.318+0000 7f4082ffd640 1 -- 172.21.15.12:0/104860647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02b2db0 msgr2=0x564bc02b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.318+0000 7f4082ffd640 1 --2- 172.21.15.12:0/104860647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02b2db0 0x564bc02b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.318+0000 7f4082ffd640 1 -- 172.21.15.12:0/104860647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc01dd8e0 con 0x564bc02adc40 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.318+0000 7f40827fc640 1 --2- 172.21.15.12:0/104860647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02b2db0 0x564bc02b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.318+0000 7f406bfff640 1 -- 172.21.15.12:0/3616833854 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f407c0cf390 con 0x7f4084102340 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.319+0000 7f4082ffd640 1 --2- 172.21.15.12:0/104860647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02b2740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f407400bf00 tx=0x7f4074009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2616cb5b2ca703ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.319+0000 7f40837fe640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f406c0a2d80 0x7f406c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.319+0000 7f40697fa640 1 -- 172.21.15.12:0/104860647 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074009e40 con 0x564bc02adc40 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.319+0000 7f40697fa640 1 -- 172.21.15.12:0/104860647 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f40740035f0 con 0x564bc02adc40 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.319+0000 7f40697fa640 1 -- 172.21.15.12:0/104860647 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074015830 con 0x564bc02adc40 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.320+0000 7f408bfcd740 1 -- 172.21.15.12:0/104860647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 msgr2=0x564bc02b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.320+0000 7f40837fe640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f406c0a2d80 0x7f406c0a5240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f407c00efd0 tx=0x7f407c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.320+0000 7f408bfcd740 1 --2- 172.21.15.12:0/104860647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02b2740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f407400bf00 tx=0x7f4074009610 comp rx=0 tx=0).stop 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.320+0000 7f408bfcd740 1 -- 172.21.15.12:0/104860647 shutdown_connections 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.320+0000 7f408bfcd740 1 --2- 172.21.15.12:0/104860647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02b2db0 0x564bc02b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.320+0000 7f408bfcd740 1 --2- 172.21.15.12:0/104860647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02b2740 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.320+0000 7f408bfcd740 1 --2- 172.21.15.12:0/104860647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.320+0000 7f408bfcd740 1 -- 172.21.15.12:0/104860647 >> 172.21.15.12:0/104860647 conn(0x564bc02a88c0 msgr2=0x564bc02a8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.320+0000 7f408bfcd740 1 -- 172.21.15.12:0/104860647 shutdown_connections 2024-05-09T09:26:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.320+0000 7f408bfcd740 1 -- 172.21.15.12:0/104860647 wait complete. 2024-05-09T09:26:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.321+0000 7f408bfcd740 1 -- start start 2024-05-09T09:26:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.321+0000 7f408bfcd740 1 Processor -- start 2024-05-09T09:26:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.321+0000 7f408bfcd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.322+0000 7f408bfcd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.322+0000 7f408bfcd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02b2db0 0x564bc02b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.322+0000 7f4082ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.322+0000 7f4082ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:37188/0 (socket says 172.21.15.12:37188) 2024-05-09T09:26:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.322+0000 7f40827fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02b2db0 0x564bc02b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.322+0000 7f40827fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02b2db0 0x564bc02b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35104/0 (socket says 172.21.15.12:35104) 2024-05-09T09:26:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.322+0000 7f4082ffd640 1 -- 172.21.15.12:0/919944880 learned_addr learned my addr 172.21.15.12:0/919944880 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.322+0000 7f408bfcd740 1 -- 172.21.15.12:0/919944880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564bc02b5b40 con 0x564bc02b2db0 2024-05-09T09:26:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.322+0000 7f408bfcd740 1 -- 172.21.15.12:0/919944880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bc01cc790 con 0x564bc02ad250 2024-05-09T09:26:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.322+0000 7f408bfcd740 1 -- 172.21.15.12:0/919944880 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564bc02b5cc0 con 0x564bc02adc40 2024-05-09T09:26:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.323+0000 7f40827fc640 1 -- 172.21.15.12:0/919944880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 msgr2=0x564bc02b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.323+0000 7f40827fc640 1 --2- 172.21.15.12:0/919944880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.323+0000 7f40827fc640 1 -- 172.21.15.12:0/919944880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 msgr2=0x564bc02b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.323+0000 7f40827fc640 1 --2- 172.21.15.12:0/919944880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.323+0000 7f40827fc640 1 -- 172.21.15.12:0/919944880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bbff25210 con 0x564bc02b2db0 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.323+0000 7f408bfcd740 10 client.0 _ll_register_callbacks cb 0x564bc0207a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.323+0000 7f408bfcd740 1 -- 172.21.15.12:0/919944880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40740035f0 con 0x564bc02b2db0 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.323+0000 7f40827fc640 1 --2- 172.21.15.12:0/919944880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02b2db0 0x564bc02b1810 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f407802aa00 tx=0x7f40780049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc5b7d9b43d17c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.323+0000 7f4066ff5640 10 client.16050 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.324+0000 7f4066ff5640 1 -- 172.21.15.12:0/919944880 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078035040 con 0x564bc02b2db0 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.324+0000 7f4066ff5640 1 -- 172.21.15.12:0/919944880 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f4078004d50 con 0x564bc02b2db0 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.324+0000 7f4066ff5640 1 -- 172.21.15.12:0/919944880 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4078004050 con 0x564bc02b2db0 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.324+0000 7f408bfcd740 10 client.16050 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.324+0000 7f408bfcd740 1 -- 172.21.15.12:0/919944880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({fsmap=29}) -- 0x564bbfff1620 con 0x564bc02b2db0 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.324+0000 7f4066ff5640 1 -- 172.21.15.12:0/919944880 <== mon.0 v2:172.21.15.12:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f40780041f0 con 0x564bc02b2db0 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.324+0000 7f408bfcd740 10 client.16050 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.324+0000 7f408bfcd740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.325+0000 7f408bfcd740 20 client.16050 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.325+0000 7f408bfcd740 1 --2- 172.21.15.12:0/919944880 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564bc02cbb10 0x564bc02cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.325+0000 7f408bfcd740 4 client.16050 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.325+0000 7f408bfcd740 1 -- 172.21.15.12:0/919944880 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bc021ea70 con 0x564bc02cbb10 2024-05-09T09:26:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.325+0000 7f4082ffd640 1 --2- 172.21.15.12:0/919944880 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564bc02cbb10 0x564bc02cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.325+0000 7f4066ff5640 10 client.16050 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.325+0000 7f4082ffd640 1 --2- 172.21.15.12:0/919944880 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564bc02cbb10 0x564bc02cdf90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157ff1862f93fc16 server_cookie=2d0c03ca2b6cb87 in_seq=0 out_seq=0 2024-05-09T09:26:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.326+0000 7f4066ff5640 1 -- 172.21.15.12:0/919944880 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564bc021ea70 con 0x564bc02cbb10 2024-05-09T09:26:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.326+0000 7f4066ff5640 10 client.16050 handle_command_reply: tid=0 2024-05-09T09:26:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.326+0000 7f408bfcd740 1 client.16050 shutdown 2024-05-09T09:26:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.326+0000 7f408bfcd740 1 -- 172.21.15.12:0/919944880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02b2db0 msgr2=0x564bc02b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.326+0000 7f408bfcd740 1 --2- 172.21.15.12:0/919944880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02b2db0 0x564bc02b1810 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f407802aa00 tx=0x7f40780049a0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.326+0000 7f408bfcd740 1 -- 172.21.15.12:0/919944880 shutdown_connections 2024-05-09T09:26:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.326+0000 7f408bfcd740 1 --2- 172.21.15.12:0/919944880 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564bc02cbb10 0x564bc02cdf90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.326+0000 7f408bfcd740 1 --2- 172.21.15.12:0/919944880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02b2db0 0x564bc02b1810 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.326+0000 7f408bfcd740 1 --2- 172.21.15.12:0/919944880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.326+0000 7f408bfcd740 1 --2- 172.21.15.12:0/919944880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.326+0000 7f408bfcd740 1 -- 172.21.15.12:0/919944880 >> 172.21.15.12:0/919944880 conn(0x564bc02a88c0 msgr2=0x564bc02a7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.326+0000 7f4066ff5640 0 client.16050 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.327+0000 7f408bfcd740 1 -- 172.21.15.12:0/919944880 shutdown_connections 2024-05-09T09:26:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.327+0000 7f408bfcd740 1 -- 172.21.15.12:0/919944880 wait complete. 2024-05-09T09:26:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.327+0000 7f408bfcd740 20 client.16050 trim_cache size 0 max 16384 2024-05-09T09:26:45.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.357+0000 7f95a8d1b640 1 -- 172.21.15.12:0/1642388235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95a4102320 msgr2=0x7f95a4108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.357+0000 7f95a8d1b640 1 --2- 172.21.15.12:0/1642388235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95a4102320 0x7f95a4108bb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f959c009ca0 tx=0x7f959c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.357+0000 7f95a8d1b640 1 -- 172.21.15.12:0/1642388235 shutdown_connections 2024-05-09T09:26:45.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.357+0000 7f95a8d1b640 1 --2- 172.21.15.12:0/1642388235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95a4102320 0x7f95a4108bb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.357+0000 7f95a8d1b640 1 --2- 172.21.15.12:0/1642388235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a4101960 0x7f95a4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.357+0000 7f95a8d1b640 1 --2- 172.21.15.12:0/1642388235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95a4100770 0x7f95a4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.357+0000 7f95a8d1b640 1 -- 172.21.15.12:0/1642388235 >> 172.21.15.12:0/1642388235 conn(0x7f95a40fbce0 msgr2=0x7f95a40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.357+0000 7f95a8d1b640 1 -- 172.21.15.12:0/1642388235 shutdown_connections 2024-05-09T09:26:45.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.357+0000 7f95a8d1b640 1 -- 172.21.15.12:0/1642388235 wait complete. 2024-05-09T09:26:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.357+0000 7f408bfcd740 1 Processor -- start 2024-05-09T09:26:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f408bfcd740 1 -- start start 2024-05-09T09:26:45.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f95a8d1b640 1 Processor -- start 2024-05-09T09:26:45.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f95a8d1b640 1 -- start start 2024-05-09T09:26:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f408bfcd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f95a8d1b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95a4100770 0x7f95a4199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f408bfcd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02a8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f95a8d1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a4101960 0x7f95a419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f95a8d1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95a4102320 0x7f95a41a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f95a8d1b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f95a410d4c0 con 0x7f95a4102320 2024-05-09T09:26:45.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f95a8d1b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95a410d340 con 0x7f95a4101960 2024-05-09T09:26:45.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f95a8d1b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f95a410d640 con 0x7f95a4100770 2024-05-09T09:26:45.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f95a2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95a4102320 0x7f95a41a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f95a2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95a4102320 0x7f95a41a1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35114/0 (socket says 172.21.15.12:35114) 2024-05-09T09:26:45.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f95a2d9d640 1 -- 172.21.15.12:0/118889826 learned_addr learned my addr 172.21.15.12:0/118889826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f40827fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f40827fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47510/0 (socket says 172.21.15.12:47510) 2024-05-09T09:26:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f40827fc640 1 -- 172.21.15.12:0/3502979614 learned_addr learned my addr 172.21.15.12:0/3502979614 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f408bfcd740 1 --2- 172.21.15.12:0/3502979614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02a9100 0x564bc02ab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f408bfcd740 1 -- 172.21.15.12:0/3502979614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f407000fcb0 con 0x564bc02a9100 2024-05-09T09:26:45.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f95a259c640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95a4100770 0x7f95a4199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f408bfcd740 1 -- 172.21.15.12:0/3502979614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40700117f0 con 0x564bc02ad250 2024-05-09T09:26:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f408bfcd740 1 -- 172.21.15.12:0/3502979614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4070010380 con 0x564bc02adc40 2024-05-09T09:26:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f4082ffd640 1 --2- 172.21.15.12:0/3502979614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.358+0000 7f95a1d9b640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a4101960 0x7f95a419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f95a259c640 1 -- 172.21.15.12:0/118889826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a4101960 msgr2=0x7f95a419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f95a259c640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a4101960 0x7f95a419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f95a259c640 1 -- 172.21.15.12:0/118889826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95a4102320 msgr2=0x7f95a41a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f95a259c640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95a4102320 0x7f95a41a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f95a259c640 1 -- 172.21.15.12:0/118889826 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a41a19d0 con 0x7f95a4100770 2024-05-09T09:26:45.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f95a2d9d640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95a4102320 0x7f95a41a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f40827fc640 1 --2- 172.21.15.12:0/3502979614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02a9100 0x564bc02ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f40827fc640 1 -- 172.21.15.12:0/3502979614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 msgr2=0x564bc02a8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f40827fc640 1 --2- 172.21.15.12:0/3502979614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02a8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f40827fc640 1 -- 172.21.15.12:0/3502979614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02a9100 msgr2=0x564bc02ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f40827fc640 1 --2- 172.21.15.12:0/3502979614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02a9100 0x564bc02ab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f40827fc640 1 -- 172.21.15.12:0/3502979614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc02ce610 con 0x564bc02ad250 2024-05-09T09:26:45.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f95a259c640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95a4100770 0x7f95a4199dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f959400e9c0 tx=0x7f959400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f40827fc640 1 --2- 172.21.15.12:0/3502979614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02afcc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4078009230 tx=0x7f4078041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4492638abb43be1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f95937fe640 1 -- 172.21.15.12:0/118889826 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959400ccc0 con 0x7f95a4100770 2024-05-09T09:26:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f4068ff9640 1 -- 172.21.15.12:0/3502979614 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078058040 con 0x564bc02ad250 2024-05-09T09:26:45.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f95a8d1b640 1 -- 172.21.15.12:0/118889826 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a41a1cc0 con 0x7f95a4100770 2024-05-09T09:26:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f4068ff9640 1 -- 172.21.15.12:0/3502979614 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4078059040 con 0x564bc02ad250 2024-05-09T09:26:45.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f95937fe640 1 -- 172.21.15.12:0/118889826 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f95940035f0 con 0x7f95a4100770 2024-05-09T09:26:45.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.359+0000 7f95937fe640 1 -- 172.21.15.12:0/118889826 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959400ce60 con 0x7f95a4100770 2024-05-09T09:26:45.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.360+0000 7f95a8d1b640 1 -- 172.21.15.12:0/118889826 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a41a21b0 con 0x7f95a4100770 2024-05-09T09:26:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.360+0000 7f4068ff9640 1 -- 172.21.15.12:0/3502979614 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078035040 con 0x564bc02ad250 2024-05-09T09:26:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.360+0000 7f408bfcd740 1 -- 172.21.15.12:0/3502979614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 msgr2=0x564bc02afcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.360+0000 7f408bfcd740 1 --2- 172.21.15.12:0/3502979614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02afcc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4078009230 tx=0x7f4078041000 comp rx=0 tx=0).stop 2024-05-09T09:26:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.361+0000 7f408bfcd740 1 -- 172.21.15.12:0/3502979614 shutdown_connections 2024-05-09T09:26:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.361+0000 7f408bfcd740 1 --2- 172.21.15.12:0/3502979614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02a9100 0x564bc02ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.361+0000 7f408bfcd740 1 --2- 172.21.15.12:0/3502979614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.361+0000 7f408bfcd740 1 --2- 172.21.15.12:0/3502979614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02afcc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.361+0000 7f408bfcd740 1 -- 172.21.15.12:0/3502979614 >> 172.21.15.12:0/3502979614 conn(0x564bc02a88c0 msgr2=0x564bc02af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.361+0000 7f408bfcd740 1 -- 172.21.15.12:0/3502979614 shutdown_connections 2024-05-09T09:26:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.361+0000 7f408bfcd740 1 -- 172.21.15.12:0/3502979614 wait complete. 2024-05-09T09:26:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.362+0000 7f408bfcd740 1 -- start start 2024-05-09T09:26:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.362+0000 7f408bfcd740 1 Processor -- start 2024-05-09T09:26:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.362+0000 7f408bfcd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02a9100 0x564bc02c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f408bfcd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f40827fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02a9100 0x564bc02c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f40827fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02a9100 0x564bc02c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35124/0 (socket says 172.21.15.12:35124) 2024-05-09T09:26:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f40827fc640 1 -- 172.21.15.12:0/1170428618 learned_addr learned my addr 172.21.15.12:0/1170428618 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f408bfcd740 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f95937fe640 1 -- 172.21.15.12:0/118889826 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9594010690 con 0x7f95a4100770 2024-05-09T09:26:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f408bfcd740 1 -- 172.21.15.12:0/1170428618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4070011120 con 0x564bc02a9100 2024-05-09T09:26:45.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f95aba2e740 1 Processor -- start 2024-05-09T09:26:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f408bfcd740 1 -- 172.21.15.12:0/1170428618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4070010a50 con 0x564bc02ad250 2024-05-09T09:26:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f408bfcd740 1 -- 172.21.15.12:0/1170428618 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f407000dc90 con 0x564bc02adc40 2024-05-09T09:26:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f4082ffd640 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f95aba2e740 1 -- start start 2024-05-09T09:26:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f4082ffd640 1 -- 172.21.15.12:0/1170428618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 msgr2=0x564bc02c7090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:26:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f4082ffd640 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02c7090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.363+0000 7f4082ffd640 1 -- 172.21.15.12:0/1170428618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02a9100 msgr2=0x564bc02c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f4082ffd640 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02a9100 0x564bc02c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f4082ffd640 1 -- 172.21.15.12:0/1170428618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc02b1c30 con 0x564bc02ad250 2024-05-09T09:26:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f40827fc640 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02a9100 0x564bc02c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f95aba2e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b1740 0x555d493b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f95aba2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b2130 0x555d493b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f95aba2e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f95aba2e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555d4930dd00 con 0x555d493b72a0 2024-05-09T09:26:45.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f95aba2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555d492b54c0 con 0x555d493b2130 2024-05-09T09:26:45.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f95aba2e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555d49309cc0 con 0x555d493b1740 2024-05-09T09:26:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f408bfcd740 10 client.0 _ll_register_callbacks cb 0x7f4078009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f4082ffd640 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02c6080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x564bc02ce840 tx=0x7f4074009e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=993ce6fe75d47da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f4066ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:26:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f408bfcd740 1 -- 172.21.15.12:0/1170428618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4078059040 con 0x564bc02ad250 2024-05-09T09:26:45.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f95a1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f95a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b2130 0x555d493b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f4066ff5640 1 -- 172.21.15.12:0/1170428618 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407402c020 con 0x564bc02ad250 2024-05-09T09:26:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f4066ff5640 1 -- 172.21.15.12:0/1170428618 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f4074028b50 con 0x564bc02ad250 2024-05-09T09:26:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f95a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b2130 0x555d493b6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47546/0 (socket says 172.21.15.12:47546) 2024-05-09T09:26:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f95a259c640 1 -- 172.21.15.12:0/2611345225 learned_addr learned my addr 172.21.15.12:0/2611345225 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f4066ff5640 1 -- 172.21.15.12:0/1170428618 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4074028db0 con 0x564bc02ad250 2024-05-09T09:26:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.364+0000 7f95a2d9d640 1 --2- 172.21.15.12:0/2611345225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b1740 0x555d493b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f408bfcd740 10 client.25559 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f408bfcd740 1 -- 172.21.15.12:0/1170428618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=29}) -- 0x564bc02c8580 con 0x564bc02ad250 2024-05-09T09:26:45.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f95937fe640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f958c0a2b90 0x7f958c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f95937fe640 1 -- 172.21.15.12:0/118889826 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f95940cf8e0 con 0x7f95a4100770 2024-05-09T09:26:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f4066ff5640 1 -- 172.21.15.12:0/1170428618 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4074005040 con 0x564bc02ad250 2024-05-09T09:26:45.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f95a2d9d640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f958c0a2b90 0x7f958c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f408bfcd740 10 client.25559 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f408bfcd740 10 client.25559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f408bfcd740 20 client.25559 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:45.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f95a2d9d640 1 -- 172.21.15.12:0/2611345225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b2130 msgr2=0x555d493b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f95a2d9d640 1 --2- 172.21.15.12:0/2611345225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b2130 0x555d493b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f95a2d9d640 1 -- 172.21.15.12:0/2611345225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 msgr2=0x555d493b9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f95a2d9d640 1 --2- 172.21.15.12:0/2611345225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493b9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f95a2d9d640 1 -- 172.21.15.12:0/2611345225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d490eb7d0 con 0x555d493b1740 2024-05-09T09:26:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f408bfcd740 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564bc0332450 0x564bc0334910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f408bfcd740 4 client.25559 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:26:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f408bfcd740 1 -- 172.21.15.12:0/1170428618 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564bc0218dc0 con 0x564bc0332450 2024-05-09T09:26:45.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.365+0000 7f95a1d9b640 1 --2- 172.21.15.12:0/2611345225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493b9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.366+0000 7f95a2d9d640 1 --2- 172.21.15.12:0/2611345225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b1740 0x555d493b1b60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f959c033040 tx=0x7f959c031560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c124ee15ba87fb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.366+0000 7f40827fc640 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564bc0332450 0x564bc0334910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.366+0000 7f95a2d9d640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f958c0a2b90 0x7f958c0a5050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f959c049000 tx=0x7f959c04b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.366+0000 7f9590ff9640 1 -- 172.21.15.12:0/2611345225 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c003e00 con 0x555d493b1740 2024-05-09T09:26:45.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.366+0000 7f9590ff9640 1 -- 172.21.15.12:0/2611345225 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f959c04f020 con 0x555d493b1740 2024-05-09T09:26:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.366+0000 7f40827fc640 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564bc0332450 0x564bc0334910 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f8a19ef4967085 server_cookie=9a16f1c1c85a91ae in_seq=0 out_seq=0 2024-05-09T09:26:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.366+0000 7f4066ff5640 10 client.25559 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:45.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.367+0000 7f95aba2e740 1 -- 172.21.15.12:0/2611345225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b1740 msgr2=0x555d493b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.367+0000 7f95aba2e740 1 --2- 172.21.15.12:0/2611345225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b1740 0x555d493b1b60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f959c033040 tx=0x7f959c031560 comp rx=0 tx=0).stop 2024-05-09T09:26:45.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.367+0000 7f95aba2e740 1 -- 172.21.15.12:0/2611345225 shutdown_connections 2024-05-09T09:26:45.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.367+0000 7f95aba2e740 1 --2- 172.21.15.12:0/2611345225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493b9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.367+0000 7f95aba2e740 1 --2- 172.21.15.12:0/2611345225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b2130 0x555d493b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.367+0000 7f95aba2e740 1 --2- 172.21.15.12:0/2611345225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b1740 0x555d493b1b60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.367+0000 7f95aba2e740 1 -- 172.21.15.12:0/2611345225 >> 172.21.15.12:0/2611345225 conn(0x555d493acdb0 msgr2=0x555d493ad1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.367+0000 7f95aba2e740 1 -- 172.21.15.12:0/2611345225 shutdown_connections 2024-05-09T09:26:45.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.368+0000 7f95aba2e740 1 -- 172.21.15.12:0/2611345225 wait complete. 2024-05-09T09:26:45.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.368+0000 7f95aba2e740 1 -- start start 2024-05-09T09:26:45.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.368+0000 7f95aba2e740 1 Processor -- start 2024-05-09T09:26:45.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.368+0000 7f95aba2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b1740 0x555d493bba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.368+0000 7f95aba2e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b2130 0x555d493bbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.368+0000 7f95aba2e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.368+0000 7f95aba2e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555d493ba0c0 con 0x555d493b72a0 2024-05-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.368+0000 7f95aba2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555d492c8b90 con 0x555d493b1740 2024-05-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.368+0000 7f95aba2e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555d493ba240 con 0x555d493b2130 2024-05-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.368+0000 7f95aba2e740 10 client.0 _ll_register_callbacks cb 0x555d49304da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.369+0000 7f95a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b1740 0x555d493bba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.369+0000 7f95a1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b2130 0x555d493bbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.369+0000 7f95a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b1740 0x555d493bba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47554/0 (socket says 172.21.15.12:47554) 2024-05-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.369+0000 7f95a259c640 1 -- 172.21.15.12:0/3312565072 learned_addr learned my addr 172.21.15.12:0/3312565072 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.369+0000 7f95a2d9d640 1 --2- 172.21.15.12:0/3312565072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.369+0000 7f95a259c640 1 -- 172.21.15.12:0/3312565072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b2130 msgr2=0x555d493bbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.369+0000 7f95a259c640 1 --2- 172.21.15.12:0/3312565072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b2130 0x555d493bbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.369+0000 7f95a259c640 1 -- 172.21.15.12:0/3312565072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 msgr2=0x555d493c2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.369+0000 7f95a259c640 1 --2- 172.21.15.12:0/3312565072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493c2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.369+0000 7f95a259c640 1 -- 172.21.15.12:0/3312565072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d490213f0 con 0x555d493b1740 2024-05-09T09:26:45.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.370+0000 7f95a2d9d640 1 --2- 172.21.15.12:0/3312565072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493c2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.370+0000 7f9589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:26:45.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.370+0000 7f95a259c640 1 --2- 172.21.15.12:0/3312565072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b1740 0x555d493bba80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f95940986b0 tx=0x7f9594098040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=231057c6ffa4dbca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.370+0000 7f9589ffb640 1 -- 172.21.15.12:0/3312565072 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959409b3c0 con 0x555d493b1740 2024-05-09T09:26:45.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.370+0000 7f95aba2e740 1 -- 172.21.15.12:0/3312565072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f959c04f020 con 0x555d493b1740 2024-05-09T09:26:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.370+0000 7f4066ff5640 1 -- 172.21.15.12:0/1170428618 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564bc0218dc0 con 0x564bc0332450 2024-05-09T09:26:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.370+0000 7f4066ff5640 10 client.25559 handle_command_reply: tid=0 2024-05-09T09:26:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.370+0000 7f408bfcd740 1 client.25559 shutdown 2024-05-09T09:26:45.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.370+0000 7f9589ffb640 1 -- 172.21.15.12:0/3312565072 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f959409b1c0 con 0x555d493b1740 2024-05-09T09:26:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.370+0000 7f408bfcd740 1 -- 172.21.15.12:0/1170428618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 msgr2=0x564bc02c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.370+0000 7f408bfcd740 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02c6080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x564bc02ce840 tx=0x7f4074009e40 comp rx=0 tx=0).stop 2024-05-09T09:26:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.370+0000 7f408bfcd740 1 -- 172.21.15.12:0/1170428618 shutdown_connections 2024-05-09T09:26:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f408bfcd740 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564bc0332450 0x564bc0334910 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f408bfcd740 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564bc02adc40 0x564bc02c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f408bfcd740 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc02ad250 0x564bc02c6080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f408bfcd740 1 --2- 172.21.15.12:0/1170428618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564bc02a9100 0x564bc02c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f408bfcd740 1 -- 172.21.15.12:0/1170428618 >> 172.21.15.12:0/1170428618 conn(0x564bc02a88c0 msgr2=0x564bc02ab950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f4066ff5640 0 client.25559 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:45.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f9589ffb640 1 -- 172.21.15.12:0/3312565072 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95940d1e60 con 0x555d493b1740 2024-05-09T09:26:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f408bfcd740 1 -- 172.21.15.12:0/1170428618 shutdown_connections 2024-05-09T09:26:45.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f95aba2e740 10 client.25565 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:45.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f95aba2e740 1 -- 172.21.15.12:0/3312565072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555d48fe10f0 con 0x555d493b1740 2024-05-09T09:26:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f408bfcd740 1 -- 172.21.15.12:0/1170428618 wait complete. 2024-05-09T09:26:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f408bfcd740 20 client.25559 trim_cache size 0 max 16384 2024-05-09T09:26:45.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f9589ffb640 1 -- 172.21.15.12:0/3312565072 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f959c04f020 con 0x555d493b1740 2024-05-09T09:26:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:26:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:26:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:26:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:26:45.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f9589ffb640 1 -- 172.21.15.12:0/3312565072 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f95940b2070 con 0x555d493b1740 2024-05-09T09:26:45.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f95aba2e740 10 client.25565 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:45.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f95aba2e740 10 client.25565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:45.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f95aba2e740 20 client.25565 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:45.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f95aba2e740 1 --2- 172.21.15.12:0/3312565072 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x555d493c7bf0 0x555d493ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f95aba2e740 4 client.25565 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:45.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.371+0000 7f95aba2e740 1 -- 172.21.15.12:0/3312565072 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d4931b430 con 0x555d493c7bf0 2024-05-09T09:26:45.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f95a1d9b640 1 --2- 172.21.15.12:0/3312565072 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x555d493c7bf0 0x555d493ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f40892c1640 1 -- 172.21.15.12:0/3616833854 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f406c0a2d80 msgr2=0x7f406c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f40892c1640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f406c0a2d80 0x7f406c0a5240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f407c00efd0 tx=0x7f407c004a40 comp rx=0 tx=0).stop 2024-05-09T09:26:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f40892c1640 1 -- 172.21.15.12:0/3616833854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4084102340 msgr2=0x7f408419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f40892c1640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4084102340 0x7f408419cc20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f407c00e940 tx=0x7f407c00ee00 comp rx=0 tx=0).stop 2024-05-09T09:26:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f40892c1640 1 -- 172.21.15.12:0/3616833854 shutdown_connections 2024-05-09T09:26:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f40892c1640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f406c0a2d80 0x7f406c0a5240 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f40892c1640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4084102340 0x7f408419cc20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f40892c1640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4084101980 0x7f4084195bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f40892c1640 1 --2- 172.21.15.12:0/3616833854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084100790 0x7f4084195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f40892c1640 1 -- 172.21.15.12:0/3616833854 >> 172.21.15.12:0/3616833854 conn(0x7f40840fbce0 msgr2=0x7f4084104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f9589ffb640 10 client.25565 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f40892c1640 1 -- 172.21.15.12:0/3616833854 shutdown_connections 2024-05-09T09:26:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:45.372+0000 7f40892c1640 1 -- 172.21.15.12:0/3616833854 wait complete. 2024-05-09T09:26:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.373+0000 7f95a1d9b640 1 --2- 172.21.15.12:0/3312565072 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x555d493c7bf0 0x555d493ca070 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36279b596bf26f38 server_cookie=2080749f20cfb799 in_seq=0 out_seq=0 2024-05-09T09:26:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f9589ffb640 1 -- 172.21.15.12:0/3312565072 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d4931b430 con 0x555d493c7bf0 2024-05-09T09:26:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f9589ffb640 10 client.25565 handle_command_reply: tid=0 2024-05-09T09:26:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f95aba2e740 1 client.25565 shutdown 2024-05-09T09:26:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f95aba2e740 1 -- 172.21.15.12:0/3312565072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b1740 msgr2=0x555d493bba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f95aba2e740 1 --2- 172.21.15.12:0/3312565072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b1740 0x555d493bba80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f95940986b0 tx=0x7f9594098040 comp rx=0 tx=0).stop 2024-05-09T09:26:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f95aba2e740 1 -- 172.21.15.12:0/3312565072 shutdown_connections 2024-05-09T09:26:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f95aba2e740 1 --2- 172.21.15.12:0/3312565072 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x555d493c7bf0 0x555d493ca070 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f95aba2e740 1 --2- 172.21.15.12:0/3312565072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493c2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f95aba2e740 1 --2- 172.21.15.12:0/3312565072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b2130 0x555d493bbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f9589ffb640 0 client.25565 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f95aba2e740 1 --2- 172.21.15.12:0/3312565072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b1740 0x555d493bba80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f95aba2e740 1 -- 172.21.15.12:0/3312565072 >> 172.21.15.12:0/3312565072 conn(0x555d493acdb0 msgr2=0x555d493a37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f95aba2e740 1 -- 172.21.15.12:0/3312565072 shutdown_connections 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f95aba2e740 1 -- 172.21.15.12:0/3312565072 wait complete. 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.374+0000 7f95aba2e740 20 client.25565 trim_cache size 0 max 16384 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.393+0000 7f95aba2e740 1 Processor -- start 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.393+0000 7f95aba2e740 1 -- start start 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.393+0000 7f95aba2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b1740 0x555d493a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.393+0000 7f95aba2e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b2130 0x555d493a3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.393+0000 7f95aba2e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b72a0 0x555d493a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.393+0000 7f95aba2e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41a86d0 con 0x555d493b2130 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.393+0000 7f95aba2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41aa210 con 0x555d493b1740 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.393+0000 7f95aba2e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41a8da0 con 0x555d493b72a0 2024-05-09T09:26:45.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.393+0000 7f95a1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b2130 0x555d493a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.393+0000 7f95a1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b2130 0x555d493a3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35168/0 (socket says 172.21.15.12:35168) 2024-05-09T09:26:45.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.394+0000 7f95a1d9b640 1 -- 172.21.15.12:0/2204208313 learned_addr learned my addr 172.21.15.12:0/2204208313 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.394+0000 7f95a259c640 1 --2- 172.21.15.12:0/2204208313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b1740 0x555d493a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.394+0000 7f95a2d9d640 1 --2- 172.21.15.12:0/2204208313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b72a0 0x555d493a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.394+0000 7f95a1d9b640 1 -- 172.21.15.12:0/2204208313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b72a0 msgr2=0x555d493a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.394+0000 7f95a1d9b640 1 --2- 172.21.15.12:0/2204208313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b72a0 0x555d493a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.394+0000 7f95a1d9b640 1 -- 172.21.15.12:0/2204208313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b1740 msgr2=0x555d493a2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.394+0000 7f95a1d9b640 1 --2- 172.21.15.12:0/2204208313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b1740 0x555d493a2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.394+0000 7f95a1d9b640 1 -- 172.21.15.12:0/2204208313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d493ca6f0 con 0x555d493b2130 2024-05-09T09:26:45.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.394+0000 7f95a1d9b640 1 --2- 172.21.15.12:0/2204208313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b2130 0x555d493a3490 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9598009e50 tx=0x7f9598005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637a55bf5b3f347e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.394+0000 7f958bfff640 1 -- 172.21.15.12:0/2204208313 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95980200f0 con 0x555d493b2130 2024-05-09T09:26:45.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.394+0000 7f958bfff640 1 -- 172.21.15.12:0/2204208313 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9598020760 con 0x555d493b2130 2024-05-09T09:26:45.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.394+0000 7f958bfff640 1 -- 172.21.15.12:0/2204208313 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598010c70 con 0x555d493b2130 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.395+0000 7f95aba2e740 1 -- 172.21.15.12:0/2204208313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b2130 msgr2=0x555d493a3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.395+0000 7f95aba2e740 1 --2- 172.21.15.12:0/2204208313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b2130 0x555d493a3490 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9598009e50 tx=0x7f9598005f90 comp rx=0 tx=0).stop 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.395+0000 7f95aba2e740 1 -- 172.21.15.12:0/2204208313 shutdown_connections 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.395+0000 7f95aba2e740 1 --2- 172.21.15.12:0/2204208313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b72a0 0x555d493a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.395+0000 7f95aba2e740 1 --2- 172.21.15.12:0/2204208313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b2130 0x555d493a3490 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.395+0000 7f95aba2e740 1 --2- 172.21.15.12:0/2204208313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b1740 0x555d493a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.395+0000 7f95aba2e740 1 -- 172.21.15.12:0/2204208313 >> 172.21.15.12:0/2204208313 conn(0x555d493acdb0 msgr2=0x555d493b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.395+0000 7f95aba2e740 1 -- 172.21.15.12:0/2204208313 shutdown_connections 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.396+0000 7f95aba2e740 1 -- 172.21.15.12:0/2204208313 wait complete. 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.396+0000 7f95aba2e740 1 -- start start 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.396+0000 7f95aba2e740 1 Processor -- start 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.396+0000 7f95aba2e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b1740 0x555d493bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.396+0000 7f95aba2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b2130 0x555d493bff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.396+0000 7f95aba2e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493c0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.396+0000 7f95aba2e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41a9b40 con 0x555d493b72a0 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.396+0000 7f95aba2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41a9470 con 0x555d493b2130 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95aba2e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41a6650 con 0x555d493b1740 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95a2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493c0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95aba2e740 10 client.0 _ll_register_callbacks cb 0x7f959409bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95a2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493c0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35184/0 (socket says 172.21.15.12:35184) 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95a2d9d640 1 -- 172.21.15.12:0/4285741100 learned_addr learned my addr 172.21.15.12:0/4285741100 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95a1d9b640 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b2130 0x555d493bff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95a259c640 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b1740 0x555d493bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95a2d9d640 1 -- 172.21.15.12:0/4285741100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b1740 msgr2=0x555d493bfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95a2d9d640 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b1740 0x555d493bfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95a2d9d640 1 -- 172.21.15.12:0/4285741100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b2130 msgr2=0x555d493bff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95a2d9d640 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b2130 0x555d493bff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95a2d9d640 1 -- 172.21.15.12:0/4285741100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d493b1380 con 0x555d493b72a0 2024-05-09T09:26:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95a1d9b640 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b2130 0x555d493bff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95a2d9d640 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493c0f60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f959c02ffc0 tx=0x7f959c02fff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ff5f57b19e00f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f9589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f9589ffb640 1 -- 172.21.15.12:0/4285741100 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c061060 con 0x555d493b72a0 2024-05-09T09:26:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f9589ffb640 1 -- 172.21.15.12:0/4285741100 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f959c050040 con 0x555d493b72a0 2024-05-09T09:26:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.397+0000 7f95aba2e740 1 -- 172.21.15.12:0/4285741100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9598020760 con 0x555d493b72a0 2024-05-09T09:26:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.398+0000 7f9589ffb640 1 -- 172.21.15.12:0/4285741100 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f959c003e00 con 0x555d493b72a0 2024-05-09T09:26:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.398+0000 7f95aba2e740 10 client.16074 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.398+0000 7f95aba2e740 1 -- 172.21.15.12:0/4285741100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555d493acbe0 con 0x555d493b72a0 2024-05-09T09:26:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.398+0000 7f9589ffb640 1 -- 172.21.15.12:0/4285741100 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9598020760 con 0x555d493b72a0 2024-05-09T09:26:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.398+0000 7f9589ffb640 1 -- 172.21.15.12:0/4285741100 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f959c030c50 con 0x555d493b72a0 2024-05-09T09:26:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.398+0000 7f95aba2e740 10 client.16074 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.398+0000 7f95aba2e740 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.398+0000 7f95aba2e740 20 client.16074 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.398+0000 7f95aba2e740 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x555d4942e4c0 0x555d494308e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.398+0000 7f95aba2e740 4 client.16074 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:26:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.398+0000 7f95aba2e740 1 -- 172.21.15.12:0/4285741100 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555d49315780 con 0x555d4942e4c0 2024-05-09T09:26:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.399+0000 7f95a259c640 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x555d4942e4c0 0x555d494308e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.399+0000 7f95a259c640 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x555d4942e4c0 0x555d494308e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b02d960e4112f0d server_cookie=b88a76f4b2e88c4e in_seq=0 out_seq=0 2024-05-09T09:26:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.399+0000 7f9589ffb640 10 client.16074 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.399+0000 7f9589ffb640 1 -- 172.21.15.12:0/4285741100 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555d49315780 con 0x555d4942e4c0 2024-05-09T09:26:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.399+0000 7f9589ffb640 10 client.16074 handle_command_reply: tid=0 2024-05-09T09:26:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.399+0000 7f95aba2e740 1 client.16074 shutdown 2024-05-09T09:26:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.399+0000 7f95aba2e740 1 -- 172.21.15.12:0/4285741100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 msgr2=0x555d493c0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.399+0000 7f95aba2e740 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493c0f60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f959c02ffc0 tx=0x7f959c02fff0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.399+0000 7f95aba2e740 1 -- 172.21.15.12:0/4285741100 shutdown_connections 2024-05-09T09:26:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.399+0000 7f95aba2e740 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x555d4942e4c0 0x555d494308e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95aba2e740 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d493b72a0 0x555d493c0f60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95aba2e740 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d493b2130 0x555d493bff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f9589ffb640 0 client.16074 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95aba2e740 1 --2- 172.21.15.12:0/4285741100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d493b1740 0x555d493bfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95aba2e740 1 -- 172.21.15.12:0/4285741100 >> 172.21.15.12:0/4285741100 conn(0x555d493acdb0 msgr2=0x555d493a2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95aba2e740 1 -- 172.21.15.12:0/4285741100 shutdown_connections 2024-05-09T09:26:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95aba2e740 1 -- 172.21.15.12:0/4285741100 wait complete. 2024-05-09T09:26:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95aba2e740 20 client.16074 trim_cache size 0 max 16384 2024-05-09T09:26:45.411 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:26:45.411 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T09:26:45.411 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T09:26:45.411 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:26:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95a8d1b640 1 -- 172.21.15.12:0/118889826 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f958c0a2b90 msgr2=0x7f958c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95a8d1b640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f958c0a2b90 0x7f958c0a5050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f959c049000 tx=0x7f959c04b000 comp rx=0 tx=0).stop 2024-05-09T09:26:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95a8d1b640 1 -- 172.21.15.12:0/118889826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95a4100770 msgr2=0x7f95a4199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95a8d1b640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95a4100770 0x7f95a4199dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f959400e9c0 tx=0x7f959400ee80 comp rx=0 tx=0).stop 2024-05-09T09:26:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95a8d1b640 1 -- 172.21.15.12:0/118889826 shutdown_connections 2024-05-09T09:26:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95a8d1b640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f958c0a2b90 0x7f958c0a5050 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95a8d1b640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95a4102320 0x7f95a41a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.400+0000 7f95a8d1b640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a4101960 0x7f95a419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.401+0000 7f95a8d1b640 1 --2- 172.21.15.12:0/118889826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95a4100770 0x7f95a4199dc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.401+0000 7f95a8d1b640 1 -- 172.21.15.12:0/118889826 >> 172.21.15.12:0/118889826 conn(0x7f95a40fbce0 msgr2=0x7f95a4109570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.401+0000 7f95a8d1b640 1 -- 172.21.15.12:0/118889826 shutdown_connections 2024-05-09T09:26:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.401+0000 7f95a8d1b640 1 -- 172.21.15.12:0/118889826 wait complete. 2024-05-09T09:26:45.413 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:26:45.414 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:26:45.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.560+0000 7f15e124d640 1 -- 172.21.15.12:0/93424385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15dc100770 msgr2=0x7f15dc100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.560+0000 7f15e124d640 1 --2- 172.21.15.12:0/93424385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15dc100770 0x7f15dc100b90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009900 tx=0x7f15cc02a410 comp rx=0 tx=0).stop 2024-05-09T09:26:45.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.560+0000 7f15e124d640 1 -- 172.21.15.12:0/93424385 shutdown_connections 2024-05-09T09:26:45.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.560+0000 7f15e124d640 1 --2- 172.21.15.12:0/93424385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc102320 0x7f15dc108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.560+0000 7f15e124d640 1 --2- 172.21.15.12:0/93424385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc101960 0x7f15dc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.560+0000 7f15e124d640 1 --2- 172.21.15.12:0/93424385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15dc100770 0x7f15dc100b90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.560+0000 7f15e124d640 1 -- 172.21.15.12:0/93424385 >> 172.21.15.12:0/93424385 conn(0x7f15dc0fbce0 msgr2=0x7f15dc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.560+0000 7f15e124d640 1 -- 172.21.15.12:0/93424385 shutdown_connections 2024-05-09T09:26:45.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.560+0000 7f15e124d640 1 -- 172.21.15.12:0/93424385 wait complete. 2024-05-09T09:26:45.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.561+0000 7f15e124d640 1 Processor -- start 2024-05-09T09:26:45.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.561+0000 7f15e124d640 1 -- start start 2024-05-09T09:26:45.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.561+0000 7f15e124d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc100770 0x7f15dc104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.561+0000 7f15e124d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc101960 0x7f15dc102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.561+0000 7f15e124d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15dc102320 0x7f15dc103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.561+0000 7f15e124d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc10c420 con 0x7f15dc102320 2024-05-09T09:26:45.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.561+0000 7f15e124d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc10c2a0 con 0x7f15dc100770 2024-05-09T09:26:45.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.561+0000 7f15e124d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc10c5a0 con 0x7f15dc101960 2024-05-09T09:26:45.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15db59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15dc102320 0x7f15dc103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15dad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc100770 0x7f15dc104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15db59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15dc102320 0x7f15dc103280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35200/0 (socket says 172.21.15.12:35200) 2024-05-09T09:26:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15db59e640 1 -- 172.21.15.12:0/1515970222 learned_addr learned my addr 172.21.15.12:0/1515970222 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15da59c640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc101960 0x7f15dc102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15db59e640 1 -- 172.21.15.12:0/1515970222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc101960 msgr2=0x7f15dc102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15db59e640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc101960 0x7f15dc102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15db59e640 1 -- 172.21.15.12:0/1515970222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc100770 msgr2=0x7f15dc104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15db59e640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc100770 0x7f15dc104700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15db59e640 1 -- 172.21.15.12:0/1515970222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15dc103b10 con 0x7f15dc102320 2024-05-09T09:26:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15dad9d640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc100770 0x7f15dc104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15da59c640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc101960 0x7f15dc102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15db59e640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15dc102320 0x7f15dc103280 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f15d400eb10 tx=0x7f15d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.562+0000 7f15c3fff640 1 -- 172.21.15.12:0/1515970222 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4004b20 con 0x7f15dc102320 2024-05-09T09:26:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.563+0000 7f15e124d640 1 -- 172.21.15.12:0/1515970222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15dc1a1cd0 con 0x7f15dc102320 2024-05-09T09:26:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.563+0000 7f15c3fff640 1 -- 172.21.15.12:0/1515970222 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f15d4004cc0 con 0x7f15dc102320 2024-05-09T09:26:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.563+0000 7f15c3fff640 1 -- 172.21.15.12:0/1515970222 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4010700 con 0x7f15dc102320 2024-05-09T09:26:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.563+0000 7f15e124d640 1 -- 172.21.15.12:0/1515970222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15dc1a21d0 con 0x7f15dc102320 2024-05-09T09:26:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.564+0000 7f15e124d640 1 -- 172.21.15.12:0/1515970222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15dc1a2460 con 0x7f15dc102320 2024-05-09T09:26:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.566+0000 7f15c3fff640 1 -- 172.21.15.12:0/1515970222 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f15d4004e70 con 0x7f15dc102320 2024-05-09T09:26:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.567+0000 7f15c3fff640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f15c40a2b90 0x7f15c40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.567+0000 7f15c3fff640 1 -- 172.21.15.12:0/1515970222 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f15d40cfbf0 con 0x7f15dc102320 2024-05-09T09:26:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.568+0000 7f15dad9d640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f15c40a2b90 0x7f15c40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.568+0000 7f15dad9d640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f15c40a2b90 0x7f15c40a5050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f15cc004770 tx=0x7f15cc031530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.570+0000 7f15c3fff640 1 -- 172.21.15.12:0/1515970222 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f15d4053740 con 0x7f15dc102320 2024-05-09T09:26:45.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.801+0000 7f15e124d640 1 -- 172.21.15.12:0/1515970222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15dc1a2630 con 0x7f15dc102320 2024-05-09T09:26:45.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.802+0000 7f15c3fff640 1 -- 172.21.15.12:0/1515970222 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11931 (secure 0 0 0) 0x7f15d409c780 con 0x7f15dc102320 2024-05-09T09:26:45.802 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:26:45.802 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:25:31.411273+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[14628,24484,24515,24523,24502]},"id":1}]} 2024-05-09T09:26:45.803 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-05-09T09:26:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.805+0000 7f15e124d640 1 -- 172.21.15.12:0/1515970222 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f15c40a2b90 msgr2=0x7f15c40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.805+0000 7f15e124d640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f15c40a2b90 0x7f15c40a5050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f15cc004770 tx=0x7f15cc031530 comp rx=0 tx=0).stop 2024-05-09T09:26:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.805+0000 7f15e124d640 1 -- 172.21.15.12:0/1515970222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15dc102320 msgr2=0x7f15dc103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.805+0000 7f15e124d640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15dc102320 0x7f15dc103280 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f15d400eb10 tx=0x7f15d400c2d0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.805+0000 7f15e124d640 1 -- 172.21.15.12:0/1515970222 shutdown_connections 2024-05-09T09:26:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.805+0000 7f15e124d640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f15c40a2b90 0x7f15c40a5050 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.805+0000 7f15e124d640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15dc102320 0x7f15dc103280 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.805+0000 7f15e124d640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc101960 0x7f15dc102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.805+0000 7f15e124d640 1 --2- 172.21.15.12:0/1515970222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc100770 0x7f15dc104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.805+0000 7f15e124d640 1 -- 172.21.15.12:0/1515970222 >> 172.21.15.12:0/1515970222 conn(0x7f15dc0fbce0 msgr2=0x7f15dc0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.805+0000 7f15e124d640 1 -- 172.21.15.12:0/1515970222 shutdown_connections 2024-05-09T09:26:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.805+0000 7f15e124d640 1 -- 172.21.15.12:0/1515970222 wait complete. 2024-05-09T09:26:45.820 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-09T09:26:45.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.932+0000 7f6782838640 1 -- 172.21.15.12:0/3120920082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f677c075810 msgr2=0x7f677c073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.932+0000 7f6782838640 1 --2- 172.21.15.12:0/3120920082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f677c075810 0x7f677c073c60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f676c009900 tx=0x7f676c02a480 comp rx=0 tx=0).stop 2024-05-09T09:26:45.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.932+0000 7f6782838640 1 -- 172.21.15.12:0/3120920082 shutdown_connections 2024-05-09T09:26:45.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.932+0000 7f6782838640 1 --2- 172.21.15.12:0/3120920082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c109710 0x7f677c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.932+0000 7f6782838640 1 --2- 172.21.15.12:0/3120920082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c074230 0x7f677c074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.932+0000 7f6782838640 1 --2- 172.21.15.12:0/3120920082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f677c075810 0x7f677c073c60 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.932+0000 7f6782838640 1 -- 172.21.15.12:0/3120920082 >> 172.21.15.12:0/3120920082 conn(0x7f677c0fce10 msgr2=0x7f677c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.932+0000 7f6782838640 1 -- 172.21.15.12:0/3120920082 shutdown_connections 2024-05-09T09:26:45.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.932+0000 7f6782838640 1 -- 172.21.15.12:0/3120920082 wait complete. 2024-05-09T09:26:45.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.933+0000 7f6782838640 1 Processor -- start 2024-05-09T09:26:45.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.933+0000 7f6782838640 1 -- start start 2024-05-09T09:26:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.933+0000 7f6782838640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f677c074230 0x7f677c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f6782838640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c075810 0x7f677c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f6782838640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c109710 0x7f677c1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f677bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f677c074230 0x7f677c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f6782838640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f677c10f730 con 0x7f677c074230 2024-05-09T09:26:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f6782838640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f677c10f5b0 con 0x7f677c109710 2024-05-09T09:26:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f6782838640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f677c10f8b0 con 0x7f677c075810 2024-05-09T09:26:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f677bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f677c074230 0x7f677c199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35220/0 (socket says 172.21.15.12:35220) 2024-05-09T09:26:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f677bfff640 1 -- 172.21.15.12:0/3421690664 learned_addr learned my addr 172.21.15.12:0/3421690664 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f677b7fe640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c075810 0x7f677c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f677bfff640 1 -- 172.21.15.12:0/3421690664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c075810 msgr2=0x7f677c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f677bfff640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c075810 0x7f677c19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f677bfff640 1 -- 172.21.15.12:0/3421690664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c109710 msgr2=0x7f677c1a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f6780dd5640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c109710 0x7f677c1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f677bfff640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c109710 0x7f677c1a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f677bfff640 1 -- 172.21.15.12:0/3421690664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c1a19d0 con 0x7f677c074230 2024-05-09T09:26:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f677b7fe640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c075810 0x7f677c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f6780dd5640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c109710 0x7f677c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.934+0000 7f677bfff640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f677c074230 0x7f677c199dc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f676c02a9a0 tx=0x7f676c004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.935+0000 7f67797fa640 1 -- 172.21.15.12:0/3421690664 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c03f070 con 0x7f677c074230 2024-05-09T09:26:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.935+0000 7f67797fa640 1 -- 172.21.15.12:0/3421690664 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f676c005430 con 0x7f677c074230 2024-05-09T09:26:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.935+0000 7f6782838640 1 -- 172.21.15.12:0/3421690664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c1a1c60 con 0x7f677c074230 2024-05-09T09:26:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.935+0000 7f67797fa640 1 -- 172.21.15.12:0/3421690664 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c035030 con 0x7f677c074230 2024-05-09T09:26:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.935+0000 7f6782838640 1 -- 172.21.15.12:0/3421690664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c1a21a0 con 0x7f677c074230 2024-05-09T09:26:45.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.937+0000 7f67797fa640 1 -- 172.21.15.12:0/3421690664 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f676c042070 con 0x7f677c074230 2024-05-09T09:26:45.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.938+0000 7f67797fa640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67640a2c10 0x7f67640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.938+0000 7f67797fa640 1 -- 172.21.15.12:0/3421690664 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f676c0ef8d0 con 0x7f677c074230 2024-05-09T09:26:45.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.938+0000 7f6785622740 1 Processor -- start 2024-05-09T09:26:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.939+0000 7f677b7fe640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67640a2c10 0x7f67640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.939+0000 7f6785622740 1 -- start start 2024-05-09T09:26:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.939+0000 7f677b7fe640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67640a2c10 0x7f67640a50d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6770009fc0 tx=0x7f6770005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.939+0000 7f6785622740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f1f40 0x55c45e4f2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.939+0000 7f6785622740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e4f7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.939+0000 7f6785622740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f7aa0 0x55c45e4f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.939+0000 7f6785622740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c45e456c40 con 0x55c45e4f2930 2024-05-09T09:26:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f6785622740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c45e3fe580 con 0x55c45e4f7aa0 2024-05-09T09:26:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f677b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e4f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f677bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f1f40 0x55c45e4f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f677bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f1f40 0x55c45e4f2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:37290/0 (socket says 172.21.15.12:37290) 2024-05-09T09:26:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f677b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e4f7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35236/0 (socket says 172.21.15.12:35236) 2024-05-09T09:26:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f677bfff640 1 -- 172.21.15.12:0/1259733336 learned_addr learned my addr 172.21.15.12:0/1259733336 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f6785622740 1 -- 172.21.15.12:0/1259733336 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c45e452c00 con 0x55c45e4f1f40 2024-05-09T09:26:45.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f6780dd5640 1 --2- 172.21.15.12:0/1259733336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f7aa0 0x55c45e4f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f677b7fe640 1 -- 172.21.15.12:0/1259733336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f1f40 msgr2=0x55c45e4f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f677b7fe640 1 --2- 172.21.15.12:0/1259733336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f1f40 0x55c45e4f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f677b7fe640 1 -- 172.21.15.12:0/1259733336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f7aa0 msgr2=0x55c45e4f9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f677b7fe640 1 --2- 172.21.15.12:0/1259733336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f7aa0 0x55c45e4f9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f677b7fe640 1 -- 172.21.15.12:0/1259733336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c45e16a3f0 con 0x55c45e4f2930 2024-05-09T09:26:45.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f677bfff640 1 --2- 172.21.15.12:0/1259733336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f1f40 0x55c45e4f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.940+0000 7f677b7fe640 1 --2- 172.21.15.12:0/1259733336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e4f7430 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6770011ca0 tx=0x7f6770014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5467dd158dd64c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.941+0000 7f67627fc640 1 -- 172.21.15.12:0/1259733336 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677000f480 con 0x55c45e4f2930 2024-05-09T09:26:45.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.941+0000 7f67627fc640 1 -- 172.21.15.12:0/1259733336 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f677000faf0 con 0x55c45e4f2930 2024-05-09T09:26:45.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.941+0000 7f67627fc640 1 -- 172.21.15.12:0/1259733336 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677001cb80 con 0x55c45e4f2930 2024-05-09T09:26:45.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.941+0000 7f6785622740 1 -- 172.21.15.12:0/1259733336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 msgr2=0x55c45e4f7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.941+0000 7f6785622740 1 --2- 172.21.15.12:0/1259733336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e4f7430 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6770011ca0 tx=0x7f6770014000 comp rx=0 tx=0).stop 2024-05-09T09:26:45.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.942+0000 7f6785622740 1 -- 172.21.15.12:0/1259733336 shutdown_connections 2024-05-09T09:26:45.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.942+0000 7f6785622740 1 --2- 172.21.15.12:0/1259733336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f7aa0 0x55c45e4f9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.942+0000 7f6785622740 1 --2- 172.21.15.12:0/1259733336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e4f7430 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.942+0000 7f6785622740 1 --2- 172.21.15.12:0/1259733336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f1f40 0x55c45e4f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.942+0000 7f6785622740 1 -- 172.21.15.12:0/1259733336 >> 172.21.15.12:0/1259733336 conn(0x55c45e4ed5b0 msgr2=0x55c45e4ed9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.942+0000 7f6785622740 1 -- 172.21.15.12:0/1259733336 shutdown_connections 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.942+0000 7f6785622740 1 -- 172.21.15.12:0/1259733336 wait complete. 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.942+0000 7f6785622740 1 -- start start 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.942+0000 7f6785622740 1 Processor -- start 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f6785622740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e4fc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f6785622740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e4fc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f677b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e4fc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f677b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e4fc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35252/0 (socket says 172.21.15.12:35252) 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f677b7fe640 1 -- 172.21.15.12:0/3687000520 learned_addr learned my addr 172.21.15.12:0/3687000520 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f6785622740 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f7aa0 0x55c45e5038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f6785622740 1 -- 172.21.15.12:0/3687000520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c45e4faa50 con 0x55c45e4f2930 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f6785622740 1 -- 172.21.15.12:0/3687000520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c45e411c40 con 0x55c45e4f1f40 2024-05-09T09:26:45.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f6785622740 1 -- 172.21.15.12:0/3687000520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c45e4fabd0 con 0x55c45e4f7aa0 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f677bfff640 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e4fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f6780dd5640 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f7aa0 0x55c45e5038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f677bfff640 1 -- 172.21.15.12:0/3687000520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f7aa0 msgr2=0x55c45e5038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f677bfff640 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f7aa0 0x55c45e5038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f677bfff640 1 -- 172.21.15.12:0/3687000520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 msgr2=0x55c45e4fc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f677bfff640 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e4fc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f677bfff640 1 -- 172.21.15.12:0/3687000520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c45e2347d0 con 0x55c45e4f1f40 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f677b7fe640 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e4fc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.943+0000 7f6785622740 10 client.0 _ll_register_callbacks cb 0x55c45e169be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.944+0000 7f6785622740 1 -- 172.21.15.12:0/3687000520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f677000faf0 con 0x55c45e4f1f40 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.944+0000 7f677bfff640 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e4fc360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f676c0b81c0 tx=0x7f676c0bbd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a7ec81389082cf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.944+0000 7f675b7fe640 10 client.25586 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.944+0000 7f675b7fe640 1 -- 172.21.15.12:0/3687000520 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c049040 con 0x55c45e4f1f40 2024-05-09T09:26:45.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.944+0000 7f675b7fe640 1 -- 172.21.15.12:0/3687000520 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f676c0b83d0 con 0x55c45e4f1f40 2024-05-09T09:26:45.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.944+0000 7f675b7fe640 1 -- 172.21.15.12:0/3687000520 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f676c0c49d0 con 0x55c45e4f1f40 2024-05-09T09:26:45.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.944+0000 7f6785622740 10 client.25586 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:45.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.944+0000 7f6785622740 1 -- 172.21.15.12:0/3687000520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55c45e506950 con 0x55c45e4f1f40 2024-05-09T09:26:45.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.944+0000 7f675b7fe640 1 -- 172.21.15.12:0/3687000520 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f676c0a6040 con 0x55c45e4f1f40 2024-05-09T09:26:45.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.944+0000 7f6785622740 10 client.25586 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:45.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.944+0000 7f6785622740 10 client.25586 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:26:45.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.944+0000 7f6785622740 20 client.25586 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:45.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.944+0000 7f6785622740 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55c45e5080b0 0x55c45e50a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.945+0000 7f6785622740 4 client.25586 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:45.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.945+0000 7f6785622740 1 -- 172.21.15.12:0/3687000520 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c45e464370 con 0x55c45e5080b0 2024-05-09T09:26:45.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.945+0000 7f677b7fe640 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55c45e5080b0 0x55c45e50a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.945+0000 7f677b7fe640 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55c45e5080b0 0x55c45e50a570 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4bcad8dbb59c564a server_cookie=a8cd6e91ebb32 in_seq=0 out_seq=0 2024-05-09T09:26:45.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.945+0000 7f675b7fe640 10 client.25586 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f675b7fe640 1 -- 172.21.15.12:0/3687000520 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c45e464370 con 0x55c45e5080b0 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f675b7fe640 10 client.25586 handle_command_reply: tid=0 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f6785622740 1 client.25586 shutdown 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f6785622740 1 -- 172.21.15.12:0/3687000520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 msgr2=0x55c45e4fc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f6785622740 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e4fc360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f676c0b81c0 tx=0x7f676c0bbd60 comp rx=0 tx=0).stop 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f6785622740 1 -- 172.21.15.12:0/3687000520 shutdown_connections 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f6785622740 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55c45e5080b0 0x55c45e50a570 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f675b7fe640 0 client.25586 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f6785622740 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f7aa0 0x55c45e5038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f6785622740 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e4fc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f6785622740 1 --2- 172.21.15.12:0/3687000520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e4fc360 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f6785622740 1 -- 172.21.15.12:0/3687000520 >> 172.21.15.12:0/3687000520 conn(0x55c45e4ed5b0 msgr2=0x55c45e4ec7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f6785622740 1 -- 172.21.15.12:0/3687000520 shutdown_connections 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f6785622740 1 -- 172.21.15.12:0/3687000520 wait complete. 2024-05-09T09:26:45.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.946+0000 7f6785622740 20 client.25586 trim_cache size 0 max 16384 2024-05-09T09:26:45.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.972+0000 7f6785622740 1 Processor -- start 2024-05-09T09:26:45.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.972+0000 7f6785622740 1 -- start start 2024-05-09T09:26:45.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6785622740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e4ed990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6785622740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f2930 0x55c45e5038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6785622740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f7aa0 0x55c45e4fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6785622740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1ac7a0 con 0x55c45e4f7aa0 2024-05-09T09:26:45.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6785622740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1ae2e0 con 0x55c45e4f1f40 2024-05-09T09:26:45.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6785622740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1ace70 con 0x55c45e4f2930 2024-05-09T09:26:45.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6780dd5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f7aa0 0x55c45e4fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6780dd5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f7aa0 0x55c45e4fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35254/0 (socket says 172.21.15.12:35254) 2024-05-09T09:26:45.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6780dd5640 1 -- 172.21.15.12:0/3871939811 learned_addr learned my addr 172.21.15.12:0/3871939811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f677bfff640 1 --2- 172.21.15.12:0/3871939811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e4ed990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f677b7fe640 1 --2- 172.21.15.12:0/3871939811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f2930 0x55c45e5038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6780dd5640 1 -- 172.21.15.12:0/3871939811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f2930 msgr2=0x55c45e5038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6780dd5640 1 --2- 172.21.15.12:0/3871939811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f2930 0x55c45e5038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6780dd5640 1 -- 172.21.15.12:0/3871939811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 msgr2=0x55c45e4ed990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6780dd5640 1 --2- 172.21.15.12:0/3871939811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e4ed990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.973+0000 7f6780dd5640 1 -- 172.21.15.12:0/3871939811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c45e4ee320 con 0x55c45e4f7aa0 2024-05-09T09:26:45.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.974+0000 7f6780dd5640 1 --2- 172.21.15.12:0/3871939811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f7aa0 0x55c45e4fc680 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6774009aa0 tx=0x7f6774009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d5cf5a70ea4824 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.974+0000 7f6761ffb640 1 -- 172.21.15.12:0/3871939811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774012450 con 0x55c45e4f7aa0 2024-05-09T09:26:45.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.974+0000 7f6761ffb640 1 -- 172.21.15.12:0/3871939811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6774012ac0 con 0x55c45e4f7aa0 2024-05-09T09:26:45.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.974+0000 7f6761ffb640 1 -- 172.21.15.12:0/3871939811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774011ca0 con 0x55c45e4f7aa0 2024-05-09T09:26:45.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.974+0000 7f6785622740 1 -- 172.21.15.12:0/3871939811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f7aa0 msgr2=0x55c45e4fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.974+0000 7f6785622740 1 --2- 172.21.15.12:0/3871939811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f7aa0 0x55c45e4fc680 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6774009aa0 tx=0x7f6774009f60 comp rx=0 tx=0).stop 2024-05-09T09:26:45.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.975+0000 7f6785622740 1 -- 172.21.15.12:0/3871939811 shutdown_connections 2024-05-09T09:26:45.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.975+0000 7f6785622740 1 --2- 172.21.15.12:0/3871939811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f7aa0 0x55c45e4fc680 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.975+0000 7f6785622740 1 --2- 172.21.15.12:0/3871939811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f2930 0x55c45e5038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.975+0000 7f6785622740 1 --2- 172.21.15.12:0/3871939811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e4ed990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.975+0000 7f6785622740 1 -- 172.21.15.12:0/3871939811 >> 172.21.15.12:0/3871939811 conn(0x55c45e4ed5b0 msgr2=0x55c45e4fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.975+0000 7f6785622740 1 -- 172.21.15.12:0/3871939811 shutdown_connections 2024-05-09T09:26:45.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.975+0000 7f6785622740 1 -- 172.21.15.12:0/3871939811 wait complete. 2024-05-09T09:26:45.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.975+0000 7f6785622740 1 -- start start 2024-05-09T09:26:45.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.975+0000 7f6785622740 1 Processor -- start 2024-05-09T09:26:45.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.975+0000 7f6785622740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e501480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.975+0000 7f6785622740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e5019c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.975+0000 7f6785622740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f7aa0 0x55c45e5029d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f6785622740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1adc10 con 0x55c45e4f2930 2024-05-09T09:26:45.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f6785622740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1ad540 con 0x55c45e4f1f40 2024-05-09T09:26:45.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f6785622740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1aa780 con 0x55c45e4f7aa0 2024-05-09T09:26:45.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f6785622740 10 client.0 _ll_register_callbacks cb 0x7f676c005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:45.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f677bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e501480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f677b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e5019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f677bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e501480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47640/0 (socket says 172.21.15.12:47640) 2024-05-09T09:26:45.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f677bfff640 1 -- 172.21.15.12:0/2359596014 learned_addr learned my addr 172.21.15.12:0/2359596014 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:45.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f677b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e5019c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35266/0 (socket says 172.21.15.12:35266) 2024-05-09T09:26:45.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f6780dd5640 1 --2- 172.21.15.12:0/2359596014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f7aa0 0x55c45e5029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f677b7fe640 1 -- 172.21.15.12:0/2359596014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f7aa0 msgr2=0x55c45e5029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f677b7fe640 1 --2- 172.21.15.12:0/2359596014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f7aa0 0x55c45e5029d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f677b7fe640 1 -- 172.21.15.12:0/2359596014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 msgr2=0x55c45e501480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:45.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f677b7fe640 1 --2- 172.21.15.12:0/2359596014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e501480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f677b7fe640 1 -- 172.21.15.12:0/2359596014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c45e502fc0 con 0x55c45e4f2930 2024-05-09T09:26:45.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f677bfff640 1 --2- 172.21.15.12:0/2359596014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e501480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:45.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f677b7fe640 1 --2- 172.21.15.12:0/2359596014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e5019c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55c45e50ae20 tx=0x7f6770005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ce4fbc2a72e839 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:45.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f675b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:45.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.976+0000 7f675b7fe640 1 -- 172.21.15.12:0/2359596014 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677000fd20 con 0x55c45e4f2930 2024-05-09T09:26:45.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.977+0000 7f675b7fe640 1 -- 172.21.15.12:0/2359596014 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f677002b040 con 0x55c45e4f2930 2024-05-09T09:26:45.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.977+0000 7f6785622740 1 -- 172.21.15.12:0/2359596014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6774012ac0 con 0x55c45e4f2930 2024-05-09T09:26:45.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.977+0000 7f675b7fe640 1 -- 172.21.15.12:0/2359596014 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f677000f480 con 0x55c45e4f2930 2024-05-09T09:26:45.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.977+0000 7f6785622740 10 client.16098 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:45.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.977+0000 7f6785622740 1 -- 172.21.15.12:0/2359596014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c45e56eae0 con 0x55c45e4f2930 2024-05-09T09:26:45.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.977+0000 7f675b7fe640 1 -- 172.21.15.12:0/2359596014 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6774012ac0 con 0x55c45e4f2930 2024-05-09T09:26:45.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.977+0000 7f675b7fe640 1 -- 172.21.15.12:0/2359596014 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f677001a070 con 0x55c45e4f2930 2024-05-09T09:26:45.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.977+0000 7f6785622740 10 client.16098 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:45.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.977+0000 7f6785622740 10 client.16098 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:26:45.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.977+0000 7f6785622740 20 client.16098 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:45.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.977+0000 7f6785622740 1 --2- 172.21.15.12:0/2359596014 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55c45e5080b0 0x55c45e571110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:45.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.977+0000 7f6785622740 4 client.16098 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:45.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.977+0000 7f6785622740 1 -- 172.21.15.12:0/2359596014 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c45e45e6c0 con 0x55c45e5080b0 2024-05-09T09:26:45.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.978+0000 7f677bfff640 1 --2- 172.21.15.12:0/2359596014 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55c45e5080b0 0x55c45e571110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:45.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.978+0000 7f675b7fe640 10 client.16098 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:26:45.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.978+0000 7f677bfff640 1 --2- 172.21.15.12:0/2359596014 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55c45e5080b0 0x55c45e571110 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=11f3e1d77d10e2dc server_cookie=6d4acf005f8ecfff in_seq=0 out_seq=0 2024-05-09T09:26:45.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.978+0000 7f675b7fe640 1 -- 172.21.15.12:0/2359596014 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c45e45e6c0 con 0x55c45e5080b0 2024-05-09T09:26:45.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.978+0000 7f675b7fe640 10 client.16098 handle_command_reply: tid=0 2024-05-09T09:26:45.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.978+0000 7f6785622740 1 client.16098 shutdown 2024-05-09T09:26:45.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.978+0000 7f6785622740 1 -- 172.21.15.12:0/2359596014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 msgr2=0x55c45e5019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.978+0000 7f6785622740 1 --2- 172.21.15.12:0/2359596014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e5019c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55c45e50ae20 tx=0x7f6770005bb0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.979+0000 7f6785622740 1 -- 172.21.15.12:0/2359596014 shutdown_connections 2024-05-09T09:26:45.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.979+0000 7f6785622740 1 --2- 172.21.15.12:0/2359596014 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55c45e5080b0 0x55c45e571110 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.979+0000 7f6785622740 1 --2- 172.21.15.12:0/2359596014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c45e4f7aa0 0x55c45e5029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.979+0000 7f6785622740 1 --2- 172.21.15.12:0/2359596014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c45e4f2930 0x55c45e5019c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.979+0000 7f6785622740 1 --2- 172.21.15.12:0/2359596014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c45e4f1f40 0x55c45e501480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.979+0000 7f6785622740 1 -- 172.21.15.12:0/2359596014 >> 172.21.15.12:0/2359596014 conn(0x55c45e4ed5b0 msgr2=0x55c45e4f0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.979+0000 7f675b7fe640 0 client.16098 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:26:45.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.979+0000 7f6785622740 1 -- 172.21.15.12:0/2359596014 shutdown_connections 2024-05-09T09:26:45.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.979+0000 7f6785622740 1 -- 172.21.15.12:0/2359596014 wait complete. 2024-05-09T09:26:45.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.979+0000 7f6785622740 20 client.16098 trim_cache size 0 max 16384 2024-05-09T09:26:45.993 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:45.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.980+0000 7f6782838640 1 -- 172.21.15.12:0/3421690664 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67640a2c10 msgr2=0x7f67640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.980+0000 7f6782838640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67640a2c10 0x7f67640a50d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6770009fc0 tx=0x7f6770005f90 comp rx=0 tx=0).stop 2024-05-09T09:26:45.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.980+0000 7f6782838640 1 -- 172.21.15.12:0/3421690664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f677c074230 msgr2=0x7f677c199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:45.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.980+0000 7f6782838640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f677c074230 0x7f677c199dc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f676c02a9a0 tx=0x7f676c004f70 comp rx=0 tx=0).stop 2024-05-09T09:26:45.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.980+0000 7f6782838640 1 -- 172.21.15.12:0/3421690664 shutdown_connections 2024-05-09T09:26:45.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.980+0000 7f6782838640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f67640a2c10 0x7f67640a50d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.980+0000 7f6782838640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c109710 0x7f677c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.980+0000 7f6782838640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c075810 0x7f677c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.980+0000 7f6782838640 1 --2- 172.21.15.12:0/3421690664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f677c074230 0x7f677c199dc0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:45.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.980+0000 7f6782838640 1 -- 172.21.15.12:0/3421690664 >> 172.21.15.12:0/3421690664 conn(0x7f677c0fce10 msgr2=0x7f677c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:45.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.980+0000 7f6782838640 1 -- 172.21.15.12:0/3421690664 shutdown_connections 2024-05-09T09:26:45.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:45.980+0000 7f6782838640 1 -- 172.21.15.12:0/3421690664 wait complete. 2024-05-09T09:26:45.995 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:45 smithi012 ceph-mon[30140]: pgmap v403: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:45.995 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:45 smithi012 ceph-mon[30140]: scrub queued for path: / 2024-05-09T09:26:45.995 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:45 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:45.995 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:45 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:26:45.995 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:45 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:45.995 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:45 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:26:45.996 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-09T09:26:45.997 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:45 smithi170 ceph-mon[33632]: pgmap v403: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:45.997 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:45 smithi170 ceph-mon[33632]: scrub queued for path: / 2024-05-09T09:26:45.997 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:45 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:45.997 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:45 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:26:45.997 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:45 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:45.997 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:45 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:26:46.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:45 smithi016 ceph-mon[33335]: pgmap v403: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:46.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:45 smithi016 ceph-mon[33335]: scrub queued for path: / 2024-05-09T09:26:46.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:45 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:46.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:45 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:26:46.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:45 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:46.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:45 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:26:46.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.108+0000 7f4481309640 1 -- 172.21.15.12:0/2954982237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c1034c0 msgr2=0x7f447c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.108+0000 7f4481309640 1 --2- 172.21.15.12:0/2954982237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c1034c0 0x7f447c109d50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4474009cd0 tx=0x7f447402a8e0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.108+0000 7f4481309640 1 -- 172.21.15.12:0/2954982237 shutdown_connections 2024-05-09T09:26:46.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.108+0000 7f4481309640 1 --2- 172.21.15.12:0/2954982237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c1034c0 0x7f447c109d50 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.108+0000 7f4481309640 1 --2- 172.21.15.12:0/2954982237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c102b00 0x7f447c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.108+0000 7f4481309640 1 --2- 172.21.15.12:0/2954982237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f447c0ffb30 0x7f447c0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.108+0000 7f4481309640 1 -- 172.21.15.12:0/2954982237 >> 172.21.15.12:0/2954982237 conn(0x7f447c0fb460 msgr2=0x7f447c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.108+0000 7f4481309640 1 -- 172.21.15.12:0/2954982237 shutdown_connections 2024-05-09T09:26:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.108+0000 7f4481309640 1 -- 172.21.15.12:0/2954982237 wait complete. 2024-05-09T09:26:46.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.109+0000 7f4481309640 1 Processor -- start 2024-05-09T09:26:46.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.109+0000 7f4481309640 1 -- start start 2024-05-09T09:26:46.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f4481309640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0ffb30 0x7f447c197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f4481309640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f447c102b00 0x7f447c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f4481309640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c1034c0 0x7f447c19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f4481309640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f447c10d550 con 0x7f447c0ffb30 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f4481309640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f447c10d3d0 con 0x7f447c1034c0 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f4481309640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f447c10d6d0 con 0x7f447c102b00 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f447affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0ffb30 0x7f447c197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f447a7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f447c102b00 0x7f447c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f447affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0ffb30 0x7f447c197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35284/0 (socket says 172.21.15.12:35284) 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f447affd640 1 -- 172.21.15.12:0/2007693638 learned_addr learned my addr 172.21.15.12:0/2007693638 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f447b7fe640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c1034c0 0x7f447c19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f447a7fc640 1 -- 172.21.15.12:0/2007693638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c1034c0 msgr2=0x7f447c19f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f447a7fc640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c1034c0 0x7f447c19f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f447a7fc640 1 -- 172.21.15.12:0/2007693638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0ffb30 msgr2=0x7f447c197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f447a7fc640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0ffb30 0x7f447c197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f447a7fc640 1 -- 172.21.15.12:0/2007693638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c19f8d0 con 0x7f447c102b00 2024-05-09T09:26:46.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f447b7fe640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c1034c0 0x7f447c19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:46.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f447affd640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0ffb30 0x7f447c197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:46.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.110+0000 7f447a7fc640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f447c102b00 0x7f447c1981d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f447000b3f0 tx=0x7f447000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.111+0000 7f4463fff640 1 -- 172.21.15.12:0/2007693638 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4470015020 con 0x7f447c102b00 2024-05-09T09:26:46.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.111+0000 7f4481309640 1 -- 172.21.15.12:0/2007693638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c19fbc0 con 0x7f447c102b00 2024-05-09T09:26:46.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.111+0000 7f4481309640 1 -- 172.21.15.12:0/2007693638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c1a0160 con 0x7f447c102b00 2024-05-09T09:26:46.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.111+0000 7f4463fff640 1 -- 172.21.15.12:0/2007693638 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4470011d00 con 0x7f447c102b00 2024-05-09T09:26:46.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.111+0000 7f4463fff640 1 -- 172.21.15.12:0/2007693638 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447001ad70 con 0x7f447c102b00 2024-05-09T09:26:46.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.113+0000 7f4463fff640 1 -- 172.21.15.12:0/2007693638 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4470018020 con 0x7f447c102b00 2024-05-09T09:26:46.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.114+0000 7f4463fff640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f44640a2c10 0x7f44640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.114+0000 7f4463fff640 1 -- 172.21.15.12:0/2007693638 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f44700cec10 con 0x7f447c102b00 2024-05-09T09:26:46.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.114+0000 7f447affd640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f44640a2c10 0x7f44640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.114+0000 7f4484015740 1 Processor -- start 2024-05-09T09:26:46.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f4484015740 1 -- start start 2024-05-09T09:26:46.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f4484015740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff6932e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f4484015740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 0x558dff6963a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f4484015740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff696a10 0x558dff6a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f4484015740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558dff5f7c40 con 0x558dff6938b0 2024-05-09T09:26:46.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f4484015740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558dff59f580 con 0x558dff692ec0 2024-05-09T09:26:46.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f4484015740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558dff5f3c00 con 0x558dff696a10 2024-05-09T09:26:46.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f447affd640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f44640a2c10 0x7f44640a50d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f447c198b60 tx=0x7f446c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f447a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 0x558dff6963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f447b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff696a10 0x558dff6a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f447a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 0x558dff6963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35294/0 (socket says 172.21.15.12:35294) 2024-05-09T09:26:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f447a7fc640 1 -- 172.21.15.12:0/1820273084 learned_addr learned my addr 172.21.15.12:0/1820273084 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f447affd640 1 --2- 172.21.15.12:0/1820273084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff6932e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f447a7fc640 1 -- 172.21.15.12:0/1820273084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff696a10 msgr2=0x558dff6a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f447a7fc640 1 --2- 172.21.15.12:0/1820273084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff696a10 0x558dff6a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f447a7fc640 1 -- 172.21.15.12:0/1820273084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 msgr2=0x558dff6932e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f447a7fc640 1 --2- 172.21.15.12:0/1820273084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff6932e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f447a7fc640 1 -- 172.21.15.12:0/1820273084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dff30b3f0 con 0x558dff6938b0 2024-05-09T09:26:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.115+0000 7f447affd640 1 --2- 172.21.15.12:0/1820273084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff6932e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.116+0000 7f447a7fc640 1 --2- 172.21.15.12:0/1820273084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 0x558dff6963a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4470097a50 tx=0x7f4470000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a3a99e951e90ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.116+0000 7f44617fa640 1 -- 172.21.15.12:0/1820273084 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700bb0b0 con 0x558dff6938b0 2024-05-09T09:26:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.116+0000 7f44617fa640 1 -- 172.21.15.12:0/1820273084 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4470099380 con 0x558dff6938b0 2024-05-09T09:26:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.116+0000 7f44617fa640 1 -- 172.21.15.12:0/1820273084 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447009a4d0 con 0x558dff6938b0 2024-05-09T09:26:46.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.117+0000 7f4484015740 1 -- 172.21.15.12:0/1820273084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 msgr2=0x558dff6963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.117+0000 7f4484015740 1 --2- 172.21.15.12:0/1820273084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 0x558dff6963a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4470097a50 tx=0x7f4470000bc0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.117+0000 7f4484015740 1 -- 172.21.15.12:0/1820273084 shutdown_connections 2024-05-09T09:26:46.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.117+0000 7f4484015740 1 --2- 172.21.15.12:0/1820273084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff696a10 0x558dff6a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.117+0000 7f4484015740 1 --2- 172.21.15.12:0/1820273084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 0x558dff6963a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.117+0000 7f4484015740 1 --2- 172.21.15.12:0/1820273084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff6932e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.117+0000 7f4484015740 1 -- 172.21.15.12:0/1820273084 >> 172.21.15.12:0/1820273084 conn(0x558dff68e5b0 msgr2=0x558dff68e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.117+0000 7f4484015740 1 -- 172.21.15.12:0/1820273084 shutdown_connections 2024-05-09T09:26:46.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.117+0000 7f4484015740 1 -- 172.21.15.12:0/1820273084 wait complete. 2024-05-09T09:26:46.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.117+0000 7f4484015740 1 -- start start 2024-05-09T09:26:46.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.117+0000 7f4484015740 1 Processor -- start 2024-05-09T09:26:46.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f4484015740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff6a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f4484015740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 0x558dff6a6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f4484015740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff696a10 0x558dff6ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f4484015740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558dff6a41c0 con 0x558dff6938b0 2024-05-09T09:26:46.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f4484015740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558dff5b2c40 con 0x558dff692ec0 2024-05-09T09:26:46.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f4484015740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558dff6a4340 con 0x558dff696a10 2024-05-09T09:26:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f447a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 0x558dff6a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f4484015740 10 client.0 _ll_register_callbacks cb 0x558dff30abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f447a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 0x558dff6a6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35296/0 (socket says 172.21.15.12:35296) 2024-05-09T09:26:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f447a7fc640 1 -- 172.21.15.12:0/1803173565 learned_addr learned my addr 172.21.15.12:0/1803173565 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f447b7fe640 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff696a10 0x558dff6ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f447affd640 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff6a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f447a7fc640 1 -- 172.21.15.12:0/1803173565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff696a10 msgr2=0x558dff6ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f447a7fc640 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff696a10 0x558dff6ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f447a7fc640 1 -- 172.21.15.12:0/1803173565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 msgr2=0x558dff6a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f447a7fc640 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff6a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f447a7fc640 1 -- 172.21.15.12:0/1803173565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dff3d57d0 con 0x558dff6938b0 2024-05-09T09:26:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.118+0000 7f447b7fe640 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff696a10 0x558dff6ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.119+0000 7f447a7fc640 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 0x558dff6a6040 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f447009b010 tx=0x7f447009b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fe614f33579be5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.119+0000 7f445eff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.119+0000 7f445eff5640 1 -- 172.21.15.12:0/1803173565 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700b3070 con 0x558dff6938b0 2024-05-09T09:26:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.119+0000 7f4484015740 1 -- 172.21.15.12:0/1803173565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4470099380 con 0x558dff6938b0 2024-05-09T09:26:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.119+0000 7f445eff5640 1 -- 172.21.15.12:0/1803173565 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f447001a490 con 0x558dff6938b0 2024-05-09T09:26:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.119+0000 7f445eff5640 1 -- 172.21.15.12:0/1803173565 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44700b9810 con 0x558dff6938b0 2024-05-09T09:26:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.119+0000 7f4484015740 10 client.16110 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.119+0000 7f4484015740 1 -- 172.21.15.12:0/1803173565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558dff6a41c0 con 0x558dff6938b0 2024-05-09T09:26:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.119+0000 7f445eff5640 1 -- 172.21.15.12:0/1803173565 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4470099380 con 0x558dff6938b0 2024-05-09T09:26:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.119+0000 7f445eff5640 1 -- 172.21.15.12:0/1803173565 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4470096dd0 con 0x558dff6938b0 2024-05-09T09:26:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.119+0000 7f4484015740 10 client.16110 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.119+0000 7f4484015740 10 client.16110 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:26:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.120+0000 7f4484015740 20 client.16110 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.120+0000 7f4484015740 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x558dff6981d0 0x558dff69a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.120+0000 7f4484015740 4 client.16110 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.120+0000 7f4484015740 1 -- 172.21.15.12:0/1803173565 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dff605370 con 0x558dff6981d0 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.120+0000 7f447affd640 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x558dff6981d0 0x558dff69a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.120+0000 7f447affd640 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x558dff6981d0 0x558dff69a5e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=468e199b1c2f06c6 server_cookie=58fa61efd87f813e in_seq=0 out_seq=0 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.120+0000 7f445eff5640 10 client.16110 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.121+0000 7f445eff5640 1 -- 172.21.15.12:0/1803173565 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558dff605370 con 0x558dff6981d0 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.121+0000 7f445eff5640 10 client.16110 handle_command_reply: tid=0 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.121+0000 7f4484015740 1 client.16110 shutdown 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.121+0000 7f4484015740 1 -- 172.21.15.12:0/1803173565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 msgr2=0x558dff6a6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.121+0000 7f4484015740 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 0x558dff6a6040 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f447009b010 tx=0x7f447009b200 comp rx=0 tx=0).stop 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.121+0000 7f4484015740 1 -- 172.21.15.12:0/1803173565 shutdown_connections 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.121+0000 7f4484015740 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x558dff6981d0 0x558dff69a5e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.121+0000 7f4484015740 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff696a10 0x558dff6ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.121+0000 7f445eff5640 0 client.16110 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:26:46.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.121+0000 7f4484015740 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff6938b0 0x558dff6a6040 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.121+0000 7f4484015740 1 --2- 172.21.15.12:0/1803173565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff6a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.121+0000 7f4484015740 1 -- 172.21.15.12:0/1803173565 >> 172.21.15.12:0/1803173565 conn(0x558dff68e5b0 msgr2=0x558dff68d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.122+0000 7f4484015740 1 -- 172.21.15.12:0/1803173565 shutdown_connections 2024-05-09T09:26:46.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.122+0000 7f4484015740 1 -- 172.21.15.12:0/1803173565 wait complete. 2024-05-09T09:26:46.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.122+0000 7f4484015740 20 client.16110 trim_cache size 0 max 16384 2024-05-09T09:26:46.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.148+0000 7f4484015740 1 Processor -- start 2024-05-09T09:26:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.148+0000 7f4484015740 1 -- start start 2024-05-09T09:26:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.148+0000 7f4484015740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff69eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.148+0000 7f4484015740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff6938b0 0x558dff69f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f4484015740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff696a10 0x558dff691560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f4484015740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1a85d0 con 0x558dff696a10 2024-05-09T09:26:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f4484015740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1aa110 con 0x558dff692ec0 2024-05-09T09:26:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f4484015740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1a8ca0 con 0x558dff6938b0 2024-05-09T09:26:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f447b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff696a10 0x558dff691560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f447affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff69eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f447affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff69eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47680/0 (socket says 172.21.15.12:47680) 2024-05-09T09:26:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f447affd640 1 -- 172.21.15.12:0/619711655 learned_addr learned my addr 172.21.15.12:0/619711655 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f447a7fc640 1 --2- 172.21.15.12:0/619711655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff6938b0 0x558dff69f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f447affd640 1 -- 172.21.15.12:0/619711655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff6938b0 msgr2=0x558dff69f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f447affd640 1 --2- 172.21.15.12:0/619711655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff6938b0 0x558dff69f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f447affd640 1 -- 172.21.15.12:0/619711655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff696a10 msgr2=0x558dff691560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f447affd640 1 --2- 172.21.15.12:0/619711655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff696a10 0x558dff691560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f447affd640 1 -- 172.21.15.12:0/619711655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dff67fd20 con 0x558dff692ec0 2024-05-09T09:26:46.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f447b7fe640 1 --2- 172.21.15.12:0/619711655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff696a10 0x558dff691560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:46.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.149+0000 7f447affd640 1 --2- 172.21.15.12:0/619711655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff69eea0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f446c009fd0 tx=0x7f446c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f4917c3f2891ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.150+0000 7f4460ff9640 1 -- 172.21.15.12:0/619711655 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c029250 con 0x558dff692ec0 2024-05-09T09:26:46.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.150+0000 7f4460ff9640 1 -- 172.21.15.12:0/619711655 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f446c0298c0 con 0x558dff692ec0 2024-05-09T09:26:46.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.150+0000 7f4460ff9640 1 -- 172.21.15.12:0/619711655 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c01d450 con 0x558dff692ec0 2024-05-09T09:26:46.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.150+0000 7f4484015740 1 -- 172.21.15.12:0/619711655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 msgr2=0x558dff69eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.150+0000 7f4484015740 1 --2- 172.21.15.12:0/619711655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff69eea0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f446c009fd0 tx=0x7f446c014000 comp rx=0 tx=0).stop 2024-05-09T09:26:46.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.150+0000 7f4484015740 1 -- 172.21.15.12:0/619711655 shutdown_connections 2024-05-09T09:26:46.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.150+0000 7f4484015740 1 --2- 172.21.15.12:0/619711655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff696a10 0x558dff691560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.150+0000 7f4484015740 1 --2- 172.21.15.12:0/619711655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff6938b0 0x558dff69f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.150+0000 7f4484015740 1 --2- 172.21.15.12:0/619711655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff692ec0 0x558dff69eea0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.150+0000 7f4484015740 1 -- 172.21.15.12:0/619711655 >> 172.21.15.12:0/619711655 conn(0x558dff68e5b0 msgr2=0x558dff6a0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.150+0000 7f4484015740 1 -- 172.21.15.12:0/619711655 shutdown_connections 2024-05-09T09:26:46.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f4484015740 1 -- 172.21.15.12:0/619711655 wait complete. 2024-05-09T09:26:46.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f4484015740 1 -- start start 2024-05-09T09:26:46.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f4484015740 1 Processor -- start 2024-05-09T09:26:46.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f4484015740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff692ec0 0x558dff6b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f4484015740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff6938b0 0x558dff6b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f4484015740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff696a10 0x558dff70dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f4484015740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1a9a40 con 0x558dff692ec0 2024-05-09T09:26:46.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f4484015740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1a9370 con 0x558dff696a10 2024-05-09T09:26:46.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f4484015740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1a65b0 con 0x558dff6938b0 2024-05-09T09:26:46.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f4484015740 10 client.0 _ll_register_callbacks cb 0x7f446c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:46.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f447affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff692ec0 0x558dff6b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f447affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff692ec0 0x558dff6b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35316/0 (socket says 172.21.15.12:35316) 2024-05-09T09:26:46.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f447affd640 1 -- 172.21.15.12:0/1180509318 learned_addr learned my addr 172.21.15.12:0/1180509318 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.151+0000 7f447a7fc640 1 --2- 172.21.15.12:0/1180509318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff6938b0 0x558dff6b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f447b7fe640 1 --2- 172.21.15.12:0/1180509318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff696a10 0x558dff70dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f447affd640 1 -- 172.21.15.12:0/1180509318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff6938b0 msgr2=0x558dff6b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f447affd640 1 --2- 172.21.15.12:0/1180509318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff6938b0 0x558dff6b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f447affd640 1 -- 172.21.15.12:0/1180509318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff696a10 msgr2=0x558dff70dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f447affd640 1 --2- 172.21.15.12:0/1180509318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff696a10 0x558dff70dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f447affd640 1 -- 172.21.15.12:0/1180509318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dff70e110 con 0x558dff692ec0 2024-05-09T09:26:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f447affd640 1 --2- 172.21.15.12:0/1180509318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff692ec0 0x558dff6b4510 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f446c0058c0 tx=0x7f446c004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e10d3c34d5d81c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f445eff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f445eff5640 1 -- 172.21.15.12:0/1180509318 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c00f4e0 con 0x558dff692ec0 2024-05-09T09:26:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f445eff5640 1 -- 172.21.15.12:0/1180509318 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f446c00fc30 con 0x558dff692ec0 2024-05-09T09:26:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f4484015740 1 -- 172.21.15.12:0/1180509318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f446c0298c0 con 0x558dff692ec0 2024-05-09T09:26:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f445eff5640 1 -- 172.21.15.12:0/1180509318 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f446c00fe30 con 0x558dff692ec0 2024-05-09T09:26:46.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f4484015740 10 client.16116 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:46.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.152+0000 7f4484015740 1 -- 172.21.15.12:0/1180509318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558dff70f2d0 con 0x558dff692ec0 2024-05-09T09:26:46.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.153+0000 7f445eff5640 1 -- 172.21.15.12:0/1180509318 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f446c0298c0 con 0x558dff692ec0 2024-05-09T09:26:46.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.153+0000 7f445eff5640 1 -- 172.21.15.12:0/1180509318 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f446c039070 con 0x558dff692ec0 2024-05-09T09:26:46.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.153+0000 7f4484015740 10 client.16116 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:46.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.153+0000 7f4484015740 10 client.16116 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:26:46.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.153+0000 7f4484015740 20 client.16116 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:46.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.153+0000 7f4484015740 1 --2- 172.21.15.12:0/1180509318 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x558dff6981d0 0x558dff711be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.153+0000 7f4484015740 4 client.16116 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:46.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.153+0000 7f4484015740 1 -- 172.21.15.12:0/1180509318 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558dff6a58f0 con 0x558dff6981d0 2024-05-09T09:26:46.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.153+0000 7f447a7fc640 1 --2- 172.21.15.12:0/1180509318 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x558dff6981d0 0x558dff711be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f445eff5640 10 client.16116 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:26:46.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f447a7fc640 1 --2- 172.21.15.12:0/1180509318 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x558dff6981d0 0x558dff711be0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cc7d10dc1bcad5d6 server_cookie=2d4515b5f0151a58 in_seq=0 out_seq=0 2024-05-09T09:26:46.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f445eff5640 1 -- 172.21.15.12:0/1180509318 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558dff6a58f0 con 0x558dff6981d0 2024-05-09T09:26:46.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f445eff5640 10 client.16116 handle_command_reply: tid=0 2024-05-09T09:26:46.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f4484015740 1 client.16116 shutdown 2024-05-09T09:26:46.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f4484015740 1 -- 172.21.15.12:0/1180509318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff692ec0 msgr2=0x558dff6b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f4484015740 1 --2- 172.21.15.12:0/1180509318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff692ec0 0x558dff6b4510 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f446c0058c0 tx=0x7f446c004140 comp rx=0 tx=0).stop 2024-05-09T09:26:46.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f4484015740 1 -- 172.21.15.12:0/1180509318 shutdown_connections 2024-05-09T09:26:46.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f4484015740 1 --2- 172.21.15.12:0/1180509318 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x558dff6981d0 0x558dff711be0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f445eff5640 0 client.16116 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:26:46.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f4484015740 1 --2- 172.21.15.12:0/1180509318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dff696a10 0x558dff70dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f4484015740 1 --2- 172.21.15.12:0/1180509318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558dff6938b0 0x558dff6b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f4484015740 1 --2- 172.21.15.12:0/1180509318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558dff692ec0 0x558dff6b4510 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.154+0000 7f4484015740 1 -- 172.21.15.12:0/1180509318 >> 172.21.15.12:0/1180509318 conn(0x558dff68e5b0 msgr2=0x558dff6a5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.155+0000 7f4484015740 1 -- 172.21.15.12:0/1180509318 shutdown_connections 2024-05-09T09:26:46.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.155+0000 7f4484015740 1 -- 172.21.15.12:0/1180509318 wait complete. 2024-05-09T09:26:46.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.155+0000 7f4484015740 20 client.16116 trim_cache size 0 max 16384 2024-05-09T09:26:46.161 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:46.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.155+0000 7f4481309640 1 -- 172.21.15.12:0/2007693638 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f44640a2c10 msgr2=0x7f44640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.155+0000 7f4481309640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f44640a2c10 0x7f44640a50d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f447c198b60 tx=0x7f446c005f90 comp rx=0 tx=0).stop 2024-05-09T09:26:46.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.155+0000 7f4481309640 1 -- 172.21.15.12:0/2007693638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f447c102b00 msgr2=0x7f447c1981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.155+0000 7f4481309640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f447c102b00 0x7f447c1981d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f447000b3f0 tx=0x7f447000b8b0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.155+0000 7f4481309640 1 -- 172.21.15.12:0/2007693638 shutdown_connections 2024-05-09T09:26:46.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.155+0000 7f4481309640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f44640a2c10 0x7f44640a50d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.155+0000 7f4481309640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c1034c0 0x7f447c19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.155+0000 7f4481309640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f447c102b00 0x7f447c1981d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.155+0000 7f4481309640 1 --2- 172.21.15.12:0/2007693638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0ffb30 0x7f447c197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.155+0000 7f4481309640 1 -- 172.21.15.12:0/2007693638 >> 172.21.15.12:0/2007693638 conn(0x7f447c0fb460 msgr2=0x7f447c105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.156+0000 7f4481309640 1 -- 172.21.15.12:0/2007693638 shutdown_connections 2024-05-09T09:26:46.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.156+0000 7f4481309640 1 -- 172.21.15.12:0/2007693638 wait complete. 2024-05-09T09:26:46.169 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:26:46.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.281+0000 7ffb09672640 1 -- 172.21.15.12:0/3971806844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb04102340 msgr2=0x7ffb04108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.281+0000 7ffb09672640 1 --2- 172.21.15.12:0/3971806844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb04102340 0x7ffb04108bd0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009ca0 tx=0x7ffafc02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.281+0000 7ffb09672640 1 -- 172.21.15.12:0/3971806844 shutdown_connections 2024-05-09T09:26:46.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.281+0000 7ffb09672640 1 --2- 172.21.15.12:0/3971806844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb04102340 0x7ffb04108bd0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.281+0000 7ffb09672640 1 --2- 172.21.15.12:0/3971806844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb04101980 0x7ffb04101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.281+0000 7ffb09672640 1 --2- 172.21.15.12:0/3971806844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb04100790 0x7ffb04100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.281+0000 7ffb09672640 1 -- 172.21.15.12:0/3971806844 >> 172.21.15.12:0/3971806844 conn(0x7ffb040fbd20 msgr2=0x7ffb040fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.281+0000 7ffb09672640 1 -- 172.21.15.12:0/3971806844 shutdown_connections 2024-05-09T09:26:46.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.281+0000 7ffb09672640 1 -- 172.21.15.12:0/3971806844 wait complete. 2024-05-09T09:26:46.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.281+0000 7ffb09672640 1 Processor -- start 2024-05-09T09:26:46.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb09672640 1 -- start start 2024-05-09T09:26:46.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb09672640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb04100790 0x7ffb04104720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb09672640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb04101980 0x7ffb04102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb09672640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb04102340 0x7ffb041032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb09672640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0410c440 con 0x7ffb04100790 2024-05-09T09:26:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb09672640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0410c2c0 con 0x7ffb04101980 2024-05-09T09:26:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb09672640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0410c5c0 con 0x7ffb04102340 2024-05-09T09:26:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb02ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb04100790 0x7ffb04104720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb02ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb04100790 0x7ffb04104720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35346/0 (socket says 172.21.15.12:35346) 2024-05-09T09:26:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb02ffd640 1 -- 172.21.15.12:0/1552583966 learned_addr learned my addr 172.21.15.12:0/1552583966 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb027fc640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb04101980 0x7ffb04102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb037fe640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb04102340 0x7ffb041032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb02ffd640 1 -- 172.21.15.12:0/1552583966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb04102340 msgr2=0x7ffb041032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb02ffd640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb04102340 0x7ffb041032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb02ffd640 1 -- 172.21.15.12:0/1552583966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb04101980 msgr2=0x7ffb04102d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb02ffd640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb04101980 0x7ffb04102d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.282+0000 7ffb02ffd640 1 -- 172.21.15.12:0/1552583966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb04103b30 con 0x7ffb04100790 2024-05-09T09:26:46.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.283+0000 7ffb02ffd640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb04100790 0x7ffb04104720 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400e9c0 tx=0x7ffaf400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.283+0000 7ffaebfff640 1 -- 172.21.15.12:0/1552583966 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400cd80 con 0x7ffb04100790 2024-05-09T09:26:46.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.283+0000 7ffb09672640 1 -- 172.21.15.12:0/1552583966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb041a1cf0 con 0x7ffb04100790 2024-05-09T09:26:46.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.283+0000 7ffb09672640 1 -- 172.21.15.12:0/1552583966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb041a2190 con 0x7ffb04100790 2024-05-09T09:26:46.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.283+0000 7ffaebfff640 1 -- 172.21.15.12:0/1552583966 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ffaf40035f0 con 0x7ffb04100790 2024-05-09T09:26:46.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.283+0000 7ffaebfff640 1 -- 172.21.15.12:0/1552583966 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4010690 con 0x7ffb04100790 2024-05-09T09:26:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.286+0000 7ffaebfff640 1 -- 172.21.15.12:0/1552583966 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffaf4004000 con 0x7ffb04100790 2024-05-09T09:26:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.286+0000 7ffb0c37f740 1 Processor -- start 2024-05-09T09:26:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffb0c37f740 1 -- start start 2024-05-09T09:26:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffb0c37f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8ef40 0x556c63b8f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffb0c37f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8f930 0x556c63b94430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffb0c37f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b94aa0 0x556c63b96f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffb02ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8f930 0x556c63b94430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffaebfff640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ffaec0a2b90 0x7ffaec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffb02ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8f930 0x556c63b94430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35348/0 (socket says 172.21.15.12:35348) 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffb037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8ef40 0x556c63b8f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffb037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8ef40 0x556c63b8f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47718/0 (socket says 172.21.15.12:47718) 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffb02ffd640 1 -- 172.21.15.12:0/2982654857 learned_addr learned my addr 172.21.15.12:0/2982654857 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffb0c37f740 1 -- 172.21.15.12:0/2982654857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556c63af3c40 con 0x556c63b8f930 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffb0c37f740 1 -- 172.21.15.12:0/2982654857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c63a9b580 con 0x556c63b8ef40 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffb0c37f740 1 -- 172.21.15.12:0/2982654857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556c63aefc00 con 0x556c63b94aa0 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.287+0000 7ffaebfff640 1 -- 172.21.15.12:0/1552583966 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7ffaf40cf830 con 0x7ffb04100790 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.288+0000 7ffb027fc640 1 --2- 172.21.15.12:0/2982654857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b94aa0 0x556c63b96f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.288+0000 7ffb037fe640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ffaec0a2b90 0x7ffaec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.288+0000 7ffb037fe640 1 -- 172.21.15.12:0/2982654857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b94aa0 msgr2=0x556c63b96f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.288+0000 7ffb037fe640 1 --2- 172.21.15.12:0/2982654857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b94aa0 0x556c63b96f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.288+0000 7ffb037fe640 1 -- 172.21.15.12:0/2982654857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8f930 msgr2=0x556c63b94430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.288+0000 7ffb037fe640 1 --2- 172.21.15.12:0/2982654857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8f930 0x556c63b94430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.288+0000 7ffb037fe640 1 -- 172.21.15.12:0/2982654857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c638073f0 con 0x556c63b8ef40 2024-05-09T09:26:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.288+0000 7ffb02ffd640 1 --2- 172.21.15.12:0/2982654857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8f930 0x556c63b94430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.288+0000 7ffb037fe640 1 --2- 172.21.15.12:0/2982654857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8ef40 0x556c63b8f360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffafc004af0 tx=0x7ffafc003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99040de94e4e747 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.288+0000 7ffb037fe640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ffaec0a2b90 0x7ffaec0a5050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffafc047000 tx=0x7ffafc049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.288+0000 7ffae97fa640 1 -- 172.21.15.12:0/2982654857 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc004610 con 0x556c63b8ef40 2024-05-09T09:26:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.289+0000 7ffae97fa640 1 -- 172.21.15.12:0/2982654857 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ffafc036040 con 0x556c63b8ef40 2024-05-09T09:26:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.289+0000 7ffae97fa640 1 -- 172.21.15.12:0/2982654857 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc030820 con 0x556c63b8ef40 2024-05-09T09:26:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.289+0000 7ffb0c37f740 1 -- 172.21.15.12:0/2982654857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8ef40 msgr2=0x556c63b8f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.289+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/2982654857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8ef40 0x556c63b8f360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffafc004af0 tx=0x7ffafc003c00 comp rx=0 tx=0).stop 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.289+0000 7ffb0c37f740 1 -- 172.21.15.12:0/2982654857 shutdown_connections 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.289+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/2982654857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b94aa0 0x556c63b96f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.289+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/2982654857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8f930 0x556c63b94430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.290+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/2982654857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8ef40 0x556c63b8f360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.290+0000 7ffb0c37f740 1 -- 172.21.15.12:0/2982654857 >> 172.21.15.12:0/2982654857 conn(0x556c63b8a5b0 msgr2=0x556c63b8a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.290+0000 7ffb0c37f740 1 -- 172.21.15.12:0/2982654857 shutdown_connections 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.290+0000 7ffb0c37f740 1 -- 172.21.15.12:0/2982654857 wait complete. 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.290+0000 7ffb0c37f740 1 -- start start 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.290+0000 7ffb0c37f740 1 Processor -- start 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.290+0000 7ffb0c37f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b8ef40 0x556c63b991b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.290+0000 7ffb0c37f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8f930 0x556c63b996f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.290+0000 7ffb0c37f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 0x556c63ba0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb0c37f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556c63b97990 con 0x556c63b94aa0 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb0c37f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c63aaec40 con 0x556c63b8f930 2024-05-09T09:26:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 0x556c63ba0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 0x556c63ba0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35352/0 (socket says 172.21.15.12:35352) 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb037fe640 1 -- 172.21.15.12:0/1998938987 learned_addr learned my addr 172.21.15.12:0/1998938987 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb0c37f740 1 -- 172.21.15.12:0/1998938987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556c63b97b10 con 0x556c63b8ef40 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb027fc640 1 --2- 172.21.15.12:0/1998938987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8f930 0x556c63b996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb0c37f740 10 client.0 _ll_register_callbacks cb 0x556c63806be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb037fe640 1 -- 172.21.15.12:0/1998938987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b8ef40 msgr2=0x556c63b991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb037fe640 1 --2- 172.21.15.12:0/1998938987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b8ef40 0x556c63b991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb037fe640 1 -- 172.21.15.12:0/1998938987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8f930 msgr2=0x556c63b996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb037fe640 1 --2- 172.21.15.12:0/1998938987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8f930 0x556c63b996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb037fe640 1 -- 172.21.15.12:0/1998938987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c638d17d0 con 0x556c63b94aa0 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb037fe640 1 --2- 172.21.15.12:0/1998938987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 0x556c63ba0730 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ffafc031be0 tx=0x7ffafc030870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb42a57ebbf6a0fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffae6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffae6ff5640 1 -- 172.21.15.12:0/1998938987 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc057030 con 0x556c63b94aa0 2024-05-09T09:26:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffae6ff5640 1 -- 172.21.15.12:0/1998938987 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7ffafc004620 con 0x556c63b94aa0 2024-05-09T09:26:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.291+0000 7ffb0c37f740 1 -- 172.21.15.12:0/1998938987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffafc036040 con 0x556c63b94aa0 2024-05-09T09:26:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.292+0000 7ffae6ff5640 1 -- 172.21.15.12:0/1998938987 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffafc064050 con 0x556c63b94aa0 2024-05-09T09:26:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.292+0000 7ffb0c37f740 10 client.16128 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.292+0000 7ffb0c37f740 1 -- 172.21.15.12:0/1998938987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c63b97990 con 0x556c63b94aa0 2024-05-09T09:26:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.292+0000 7ffae6ff5640 1 -- 172.21.15.12:0/1998938987 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ffafc004820 con 0x556c63b94aa0 2024-05-09T09:26:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.293+0000 7ffae6ff5640 1 -- 172.21.15.12:0/1998938987 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffafc02f880 con 0x556c63b94aa0 2024-05-09T09:26:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.293+0000 7ffb0c37f740 10 client.16128 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.293+0000 7ffb0c37f740 10 client.16128 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:26:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.293+0000 7ffb0c37f740 20 client.16128 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.293+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/1998938987 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556c63ba4d10 0x556c63ba7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.293+0000 7ffb0c37f740 4 client.16128 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.293+0000 7ffb0c37f740 1 -- 172.21.15.12:0/1998938987 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffafc036040 con 0x556c63ba4d10 2024-05-09T09:26:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.293+0000 7ffb02ffd640 1 --2- 172.21.15.12:0/1998938987 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556c63ba4d10 0x556c63ba7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.293+0000 7ffb02ffd640 1 --2- 172.21.15.12:0/1998938987 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556c63ba4d10 0x556c63ba7180 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a509ccf6b8ba193b server_cookie=1197f2c357268ab6 in_seq=0 out_seq=0 2024-05-09T09:26:46.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.293+0000 7ffae6ff5640 10 client.16128 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:26:46.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.294+0000 7ffae6ff5640 1 -- 172.21.15.12:0/1998938987 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ffafc036040 con 0x556c63ba4d10 2024-05-09T09:26:46.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.294+0000 7ffae6ff5640 10 client.16128 handle_command_reply: tid=0 2024-05-09T09:26:46.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.294+0000 7ffb0c37f740 1 client.16128 shutdown 2024-05-09T09:26:46.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.294+0000 7ffb0c37f740 1 -- 172.21.15.12:0/1998938987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 msgr2=0x556c63ba0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.294+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/1998938987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 0x556c63ba0730 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ffafc031be0 tx=0x7ffafc030870 comp rx=0 tx=0).stop 2024-05-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.295+0000 7ffb0c37f740 1 -- 172.21.15.12:0/1998938987 shutdown_connections 2024-05-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.295+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/1998938987 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556c63ba4d10 0x556c63ba7180 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.295+0000 7ffae6ff5640 0 client.16128 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.295+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/1998938987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 0x556c63ba0730 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.295+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/1998938987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8f930 0x556c63b996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.295+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/1998938987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b8ef40 0x556c63b991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.295+0000 7ffb0c37f740 1 -- 172.21.15.12:0/1998938987 >> 172.21.15.12:0/1998938987 conn(0x556c63b8a5b0 msgr2=0x556c63b897b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.295+0000 7ffb0c37f740 1 -- 172.21.15.12:0/1998938987 shutdown_connections 2024-05-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.295+0000 7ffb0c37f740 1 -- 172.21.15.12:0/1998938987 wait complete. 2024-05-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.295+0000 7ffb0c37f740 20 client.16128 trim_cache size 0 max 16384 2024-05-09T09:26:46.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb0c37f740 1 Processor -- start 2024-05-09T09:26:46.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb0c37f740 1 -- start start 2024-05-09T09:26:46.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb0c37f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8ef40 0x556c63b8dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb0c37f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b8f930 0x556c63ba06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb0c37f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 0x556c63b97ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb0c37f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf000fcc0 con 0x556c63b94aa0 2024-05-09T09:26:46.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb0c37f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf0011800 con 0x556c63b8ef40 2024-05-09T09:26:46.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb0c37f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf0010390 con 0x556c63b8f930 2024-05-09T09:26:46.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 0x556c63b97ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 0x556c63b97ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35366/0 (socket says 172.21.15.12:35366) 2024-05-09T09:26:46.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb037fe640 1 -- 172.21.15.12:0/2262443549 learned_addr learned my addr 172.21.15.12:0/2262443549 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb02ffd640 1 --2- 172.21.15.12:0/2262443549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8ef40 0x556c63b8dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb027fc640 1 --2- 172.21.15.12:0/2262443549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b8f930 0x556c63ba06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb037fe640 1 -- 172.21.15.12:0/2262443549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b8f930 msgr2=0x556c63ba06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb037fe640 1 --2- 172.21.15.12:0/2262443549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b8f930 0x556c63ba06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.309+0000 7ffb037fe640 1 -- 172.21.15.12:0/2262443549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8ef40 msgr2=0x556c63b8dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.310+0000 7ffb037fe640 1 --2- 172.21.15.12:0/2262443549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8ef40 0x556c63b8dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.310+0000 7ffb037fe640 1 -- 172.21.15.12:0/2262443549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c63b8b320 con 0x556c63b94aa0 2024-05-09T09:26:46.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.310+0000 7ffb02ffd640 1 --2- 172.21.15.12:0/2262443549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8ef40 0x556c63b8dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:46.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.310+0000 7ffb037fe640 1 --2- 172.21.15.12:0/2262443549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 0x556c63b97ee0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009430 tx=0x7ffafc005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1577b9707b44cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.310+0000 7ffae8ff9640 1 -- 172.21.15.12:0/2262443549 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc057020 con 0x556c63b94aa0 2024-05-09T09:26:46.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.310+0000 7ffae8ff9640 1 -- 172.21.15.12:0/2262443549 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ffafc036030 con 0x556c63b94aa0 2024-05-09T09:26:46.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.310+0000 7ffae8ff9640 1 -- 172.21.15.12:0/2262443549 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc004620 con 0x556c63b94aa0 2024-05-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.311+0000 7ffb0c37f740 1 -- 172.21.15.12:0/2262443549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 msgr2=0x556c63b97ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.311+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/2262443549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 0x556c63b97ee0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009430 tx=0x7ffafc005f60 comp rx=0 tx=0).stop 2024-05-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.311+0000 7ffb0c37f740 1 -- 172.21.15.12:0/2262443549 shutdown_connections 2024-05-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.311+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/2262443549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b94aa0 0x556c63b97ee0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.311+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/2262443549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b8f930 0x556c63ba06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.311+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/2262443549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8ef40 0x556c63b8dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.311+0000 7ffb0c37f740 1 -- 172.21.15.12:0/2262443549 >> 172.21.15.12:0/2262443549 conn(0x556c63b8a5b0 msgr2=0x556c63b8d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.311+0000 7ffb0c37f740 1 -- 172.21.15.12:0/2262443549 shutdown_connections 2024-05-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.311+0000 7ffb0c37f740 1 -- 172.21.15.12:0/2262443549 wait complete. 2024-05-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.312+0000 7ffb0c37f740 1 -- start start 2024-05-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.312+0000 7ffb0c37f740 1 Processor -- start 2024-05-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.312+0000 7ffb0c37f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8ef40 0x556c63b9e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.312+0000 7ffb0c37f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8f930 0x556c63b9e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.312+0000 7ffb0c37f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b94aa0 0x556c63b9f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb0c37f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf0011130 con 0x556c63b8ef40 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb02ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8ef40 0x556c63b9e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb02ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8ef40 0x556c63b9e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35372/0 (socket says 172.21.15.12:35372) 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb02ffd640 1 -- 172.21.15.12:0/979552366 learned_addr learned my addr 172.21.15.12:0/979552366 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb0c37f740 1 -- 172.21.15.12:0/979552366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf0010a60 con 0x556c63b8f930 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb0c37f740 1 -- 172.21.15.12:0/979552366 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf000dca0 con 0x556c63b94aa0 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb027fc640 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8f930 0x556c63b9e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb0c37f740 10 client.0 _ll_register_callbacks cb 0x7ffaf800b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb037fe640 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b94aa0 0x556c63b9f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb02ffd640 1 -- 172.21.15.12:0/979552366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b94aa0 msgr2=0x556c63b9f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb02ffd640 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b94aa0 0x556c63b9f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb02ffd640 1 -- 172.21.15.12:0/979552366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8f930 msgr2=0x556c63b9e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb02ffd640 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8f930 0x556c63b9e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb02ffd640 1 -- 172.21.15.12:0/979552366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c63b9fba0 con 0x556c63b8ef40 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb027fc640 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8f930 0x556c63b9e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb02ffd640 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8ef40 0x556c63b9e060 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40a43e0 tx=0x7ffaf40cfb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27164652b26d36bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffae6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffae6ff5640 1 -- 172.21.15.12:0/979552366 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40a4910 con 0x556c63b8ef40 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffae6ff5640 1 -- 172.21.15.12:0/979552366 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7ffaf4098310 con 0x556c63b8ef40 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.313+0000 7ffb0c37f740 1 -- 172.21.15.12:0/979552366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffafc036030 con 0x556c63b8ef40 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.314+0000 7ffae6ff5640 1 -- 172.21.15.12:0/979552366 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffaf40042f0 con 0x556c63b8ef40 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.314+0000 7ffb0c37f740 10 client.16134 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.314+0000 7ffb0c37f740 1 -- 172.21.15.12:0/979552366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c63c0b680 con 0x556c63b8ef40 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.314+0000 7ffae6ff5640 1 -- 172.21.15.12:0/979552366 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ffaf4004490 con 0x556c63b8ef40 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.314+0000 7ffae6ff5640 1 -- 172.21.15.12:0/979552366 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffaf40979f0 con 0x556c63b8ef40 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.314+0000 7ffb0c37f740 10 client.16134 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.314+0000 7ffb0c37f740 10 client.16134 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.314+0000 7ffb0c37f740 20 client.16134 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.314+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556c63ba4d10 0x556c63c0e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.314+0000 7ffb0c37f740 4 client.16134 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.314+0000 7ffb0c37f740 1 -- 172.21.15.12:0/979552366 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ffafc036030 con 0x556c63ba4d10 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.314+0000 7ffb027fc640 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556c63ba4d10 0x556c63c0e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.315+0000 7ffae6ff5640 10 client.16134 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.315+0000 7ffb027fc640 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556c63ba4d10 0x556c63c0e070 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2a036279e09a8ed server_cookie=d84fad2f4caacc9a in_seq=0 out_seq=0 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.315+0000 7ffae6ff5640 1 -- 172.21.15.12:0/979552366 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ffafc036030 con 0x556c63ba4d10 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.315+0000 7ffae6ff5640 10 client.16134 handle_command_reply: tid=0 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.315+0000 7ffb0c37f740 1 client.16134 shutdown 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.316+0000 7ffb0c37f740 1 -- 172.21.15.12:0/979552366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8ef40 msgr2=0x556c63b9e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.316+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8ef40 0x556c63b9e060 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40a43e0 tx=0x7ffaf40cfb70 comp rx=0 tx=0).stop 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.316+0000 7ffb0c37f740 1 -- 172.21.15.12:0/979552366 shutdown_connections 2024-05-09T09:26:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.316+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556c63ba4d10 0x556c63c0e070 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.316+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c63b94aa0 0x556c63b9f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.316+0000 7ffae6ff5640 0 client.16134 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:26:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.316+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c63b8f930 0x556c63b9e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.316+0000 7ffb0c37f740 1 --2- 172.21.15.12:0/979552366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c63b8ef40 0x556c63b9e060 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.316+0000 7ffb0c37f740 1 -- 172.21.15.12:0/979552366 >> 172.21.15.12:0/979552366 conn(0x556c63b8a5b0 msgr2=0x556c63b88f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.316+0000 7ffb0c37f740 1 -- 172.21.15.12:0/979552366 shutdown_connections 2024-05-09T09:26:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.316+0000 7ffb0c37f740 1 -- 172.21.15.12:0/979552366 wait complete. 2024-05-09T09:26:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.316+0000 7ffb0c37f740 20 client.16134 trim_cache size 0 max 16384 2024-05-09T09:26:46.320 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.317+0000 7ffb09672640 1 -- 172.21.15.12:0/1552583966 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ffaec0a2b90 msgr2=0x7ffaec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.317+0000 7ffb09672640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ffaec0a2b90 0x7ffaec0a5050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffafc047000 tx=0x7ffafc049000 comp rx=0 tx=0).stop 2024-05-09T09:26:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.317+0000 7ffb09672640 1 -- 172.21.15.12:0/1552583966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb04100790 msgr2=0x7ffb04104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.317+0000 7ffb09672640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb04100790 0x7ffb04104720 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400e9c0 tx=0x7ffaf400ee80 comp rx=0 tx=0).stop 2024-05-09T09:26:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.317+0000 7ffb09672640 1 -- 172.21.15.12:0/1552583966 shutdown_connections 2024-05-09T09:26:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.317+0000 7ffb09672640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ffaec0a2b90 0x7ffaec0a5050 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.317+0000 7ffb09672640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb04102340 0x7ffb041032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.317+0000 7ffb09672640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb04101980 0x7ffb04102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.317+0000 7ffb09672640 1 --2- 172.21.15.12:0/1552583966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb04100790 0x7ffb04104720 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.317+0000 7ffb09672640 1 -- 172.21.15.12:0/1552583966 >> 172.21.15.12:0/1552583966 conn(0x7ffb040fbd20 msgr2=0x7ffb040fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.317+0000 7ffb09672640 1 -- 172.21.15.12:0/1552583966 shutdown_connections 2024-05-09T09:26:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.317+0000 7ffb09672640 1 -- 172.21.15.12:0/1552583966 wait complete. 2024-05-09T09:26:46.331 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:26:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.447+0000 7f37a933b640 1 -- 172.21.15.12:0/566448229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37a4102320 msgr2=0x7f37a4108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.447+0000 7f37a933b640 1 --2- 172.21.15.12:0/566448229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37a4102320 0x7f37a4108bb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f379c009ca0 tx=0x7f379c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.447+0000 7f37a933b640 1 -- 172.21.15.12:0/566448229 shutdown_connections 2024-05-09T09:26:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.447+0000 7f37a933b640 1 --2- 172.21.15.12:0/566448229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37a4102320 0x7f37a4108bb0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.447+0000 7f37a933b640 1 --2- 172.21.15.12:0/566448229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a4101960 0x7f37a4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.447+0000 7f37a933b640 1 --2- 172.21.15.12:0/566448229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37a4100770 0x7f37a4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.447+0000 7f37a933b640 1 -- 172.21.15.12:0/566448229 >> 172.21.15.12:0/566448229 conn(0x7f37a40fbce0 msgr2=0x7f37a40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.447+0000 7f37a933b640 1 -- 172.21.15.12:0/566448229 shutdown_connections 2024-05-09T09:26:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.447+0000 7f37a933b640 1 -- 172.21.15.12:0/566448229 wait complete. 2024-05-09T09:26:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.447+0000 7f37a933b640 1 Processor -- start 2024-05-09T09:26:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a933b640 1 -- start start 2024-05-09T09:26:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a933b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37a4100770 0x7f37a410b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a933b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37a4101960 0x7f37a4109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a933b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a4102320 0x7f37a4109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a933b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f37a410d4e0 con 0x7f37a4100770 2024-05-09T09:26:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a933b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37a410d360 con 0x7f37a4102320 2024-05-09T09:26:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a933b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f37a410d660 con 0x7f37a4101960 2024-05-09T09:26:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37a4100770 0x7f37a410b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37a4100770 0x7f37a410b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35390/0 (socket says 172.21.15.12:35390) 2024-05-09T09:26:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a2ffd640 1 -- 172.21.15.12:0/2820794941 learned_addr learned my addr 172.21.15.12:0/2820794941 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a27fc640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37a4101960 0x7f37a4109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a37fe640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a4102320 0x7f37a4109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a2ffd640 1 -- 172.21.15.12:0/2820794941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37a4101960 msgr2=0x7f37a4109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37a4101960 0x7f37a4109730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a2ffd640 1 -- 172.21.15.12:0/2820794941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a4102320 msgr2=0x7f37a4109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a4102320 0x7f37a4109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.448+0000 7f37a2ffd640 1 -- 172.21.15.12:0/2820794941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a410a500 con 0x7f37a4100770 2024-05-09T09:26:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.449+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37a4100770 0x7f37a410b0f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f379400e9c0 tx=0x7f379400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.449+0000 7f378bfff640 1 -- 172.21.15.12:0/2820794941 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379400cd80 con 0x7f37a4100770 2024-05-09T09:26:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.449+0000 7f37a933b640 1 -- 172.21.15.12:0/2820794941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a41a1ca0 con 0x7f37a4100770 2024-05-09T09:26:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.449+0000 7f37a933b640 1 -- 172.21.15.12:0/2820794941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a41a21a0 con 0x7f37a4100770 2024-05-09T09:26:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.449+0000 7f378bfff640 1 -- 172.21.15.12:0/2820794941 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f37940035f0 con 0x7f37a4100770 2024-05-09T09:26:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.449+0000 7f378bfff640 1 -- 172.21.15.12:0/2820794941 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794010690 con 0x7f37a4100770 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.451+0000 7f37ac04a740 1 Processor -- start 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.451+0000 7f37ac04a740 1 -- start start 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f378bfff640 1 -- 172.21.15.12:0/2820794941 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3794010830 con 0x7f37a4100770 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37ac04a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb34f40 0x55bc6eb35360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37ac04a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 0x55bc6eb3a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37ac04a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb3aaa0 0x55bc6eb3cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37ac04a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bc6ea99c40 con 0x55bc6eb35930 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37ac04a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bc6ea41580 con 0x55bc6eb3aaa0 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37ac04a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bc6ea95c00 con 0x55bc6eb34f40 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37a2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 0x55bc6eb3a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37a2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 0x55bc6eb3a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35396/0 (socket says 172.21.15.12:35396) 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37a2ffd640 1 -- 172.21.15.12:0/324379133 learned_addr learned my addr 172.21.15.12:0/324379133 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37a37fe640 1 --2- 172.21.15.12:0/324379133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb34f40 0x55bc6eb35360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37a27fc640 1 --2- 172.21.15.12:0/324379133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb3aaa0 0x55bc6eb3cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37a2ffd640 1 -- 172.21.15.12:0/324379133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb34f40 msgr2=0x55bc6eb35360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/324379133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb34f40 0x55bc6eb35360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37a2ffd640 1 -- 172.21.15.12:0/324379133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb3aaa0 msgr2=0x55bc6eb3cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/324379133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb3aaa0 0x55bc6eb3cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37a2ffd640 1 -- 172.21.15.12:0/324379133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6e7ad3f0 con 0x55bc6eb35930 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.452+0000 7f37a37fe640 1 --2- 172.21.15.12:0/324379133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb34f40 0x55bc6eb35360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.453+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/324379133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 0x55bc6eb3a430 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f37940d5a90 tx=0x7f37940d5b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5b4a16d34ee811 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.453+0000 7f37897fa640 1 -- 172.21.15.12:0/324379133 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37940de070 con 0x55bc6eb35930 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.453+0000 7f37897fa640 1 -- 172.21.15.12:0/324379133 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f37940d10d0 con 0x55bc6eb35930 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.453+0000 7f378bfff640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f378c0a2c10 0x7f378c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.453+0000 7f37897fa640 1 -- 172.21.15.12:0/324379133 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37940da680 con 0x55bc6eb35930 2024-05-09T09:26:46.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.453+0000 7f378bfff640 1 -- 172.21.15.12:0/2820794941 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f37940cf820 con 0x7f37a4100770 2024-05-09T09:26:46.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.453+0000 7f37a37fe640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f378c0a2c10 0x7f378c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.454+0000 7f37ac04a740 1 -- 172.21.15.12:0/324379133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 msgr2=0x55bc6eb3a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.454+0000 7f37ac04a740 1 --2- 172.21.15.12:0/324379133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 0x55bc6eb3a430 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f37940d5a90 tx=0x7f37940d5b80 comp rx=0 tx=0).stop 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.454+0000 7f37a37fe640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f378c0a2c10 0x7f378c0a50d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55bc6eb36190 tx=0x7f379c031b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.454+0000 7f37ac04a740 1 -- 172.21.15.12:0/324379133 shutdown_connections 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.454+0000 7f37ac04a740 1 --2- 172.21.15.12:0/324379133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb3aaa0 0x55bc6eb3cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.454+0000 7f37ac04a740 1 --2- 172.21.15.12:0/324379133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 0x55bc6eb3a430 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.454+0000 7f37ac04a740 1 --2- 172.21.15.12:0/324379133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb34f40 0x55bc6eb35360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.454+0000 7f37ac04a740 1 -- 172.21.15.12:0/324379133 >> 172.21.15.12:0/324379133 conn(0x55bc6eb305b0 msgr2=0x55bc6eb309b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.454+0000 7f37ac04a740 1 -- 172.21.15.12:0/324379133 shutdown_connections 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.454+0000 7f37ac04a740 1 -- 172.21.15.12:0/324379133 wait complete. 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.454+0000 7f37ac04a740 1 -- start start 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37ac04a740 1 Processor -- start 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37ac04a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 0x55bc6eb3f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37ac04a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb35930 0x55bc6eb3f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37ac04a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb3aaa0 0x55bc6eb46900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37ac04a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bc6eb3da50 con 0x55bc6eb34f40 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37ac04a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bc6ea54c40 con 0x55bc6eb35930 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37ac04a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bc6eb3dbd0 con 0x55bc6eb3aaa0 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37ac04a740 10 client.0 _ll_register_callbacks cb 0x55bc6e7acbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37a2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 0x55bc6eb3f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37a2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 0x55bc6eb3f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35410/0 (socket says 172.21.15.12:35410) 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37a2ffd640 1 -- 172.21.15.12:0/3901341244 learned_addr learned my addr 172.21.15.12:0/3901341244 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37a27fc640 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb35930 0x55bc6eb3f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37a37fe640 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb3aaa0 0x55bc6eb46900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37a2ffd640 1 -- 172.21.15.12:0/3901341244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb3aaa0 msgr2=0x55bc6eb46900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb3aaa0 0x55bc6eb46900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37a2ffd640 1 -- 172.21.15.12:0/3901341244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb35930 msgr2=0x55bc6eb3f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb35930 0x55bc6eb3f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37a2ffd640 1 -- 172.21.15.12:0/3901341244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6e8777d0 con 0x55bc6eb34f40 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37a27fc640 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb35930 0x55bc6eb3f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.455+0000 7f37a37fe640 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb3aaa0 0x55bc6eb46900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.456+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 0x55bc6eb3f3f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f37940d5a60 tx=0x7f3794003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aaaaf30ecfe0a5a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.456+0000 7f3786ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.456+0000 7f3786ff5640 1 -- 172.21.15.12:0/3901341244 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37940b1020 con 0x55bc6eb34f40 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.456+0000 7f3786ff5640 1 -- 172.21.15.12:0/3901341244 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f37940cd000 con 0x55bc6eb34f40 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.456+0000 7f37ac04a740 1 -- 172.21.15.12:0/3901341244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37940d10d0 con 0x55bc6eb34f40 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.456+0000 7f3786ff5640 1 -- 172.21.15.12:0/3901341244 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37940b9050 con 0x55bc6eb34f40 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.456+0000 7f37ac04a740 10 client.16146 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.456+0000 7f37ac04a740 1 -- 172.21.15.12:0/3901341244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bc6eb49c70 con 0x55bc6eb34f40 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.456+0000 7f3786ff5640 1 -- 172.21.15.12:0/3901341244 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f37940d10d0 con 0x55bc6eb34f40 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.456+0000 7f3786ff5640 1 -- 172.21.15.12:0/3901341244 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f379409beb0 con 0x55bc6eb34f40 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.457+0000 7f37ac04a740 10 client.16146 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.457+0000 7f37ac04a740 10 client.16146 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:26:46.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.457+0000 7f37ac04a740 20 client.16146 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:46.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.457+0000 7f37ac04a740 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55bc6eb4afc0 0x55bc6eb4d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.457+0000 7f37ac04a740 4 client.16146 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:46.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.457+0000 7f37ac04a740 1 -- 172.21.15.12:0/3901341244 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc6eaa7370 con 0x55bc6eb4afc0 2024-05-09T09:26:46.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.457+0000 7f37a27fc640 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55bc6eb4afc0 0x55bc6eb4d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.457+0000 7f3786ff5640 10 client.16146 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:26:46.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.457+0000 7f37a27fc640 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55bc6eb4afc0 0x55bc6eb4d3b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8e5e3fc9621edee server_cookie=7f713a33c785ff0 in_seq=0 out_seq=0 2024-05-09T09:26:46.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f3786ff5640 1 -- 172.21.15.12:0/3901341244 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc6eaa7370 con 0x55bc6eb4afc0 2024-05-09T09:26:46.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f3786ff5640 10 client.16146 handle_command_reply: tid=0 2024-05-09T09:26:46.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f37ac04a740 1 client.16146 shutdown 2024-05-09T09:26:46.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f37ac04a740 1 -- 172.21.15.12:0/3901341244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 msgr2=0x55bc6eb3f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f37ac04a740 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 0x55bc6eb3f3f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f37940d5a60 tx=0x7f3794003ec0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f37ac04a740 1 -- 172.21.15.12:0/3901341244 shutdown_connections 2024-05-09T09:26:46.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f37ac04a740 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55bc6eb4afc0 0x55bc6eb4d3b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f37ac04a740 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb3aaa0 0x55bc6eb46900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f37ac04a740 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb35930 0x55bc6eb3f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f3786ff5640 0 client.16146 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:26:46.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f37ac04a740 1 --2- 172.21.15.12:0/3901341244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 0x55bc6eb3f3f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f37ac04a740 1 -- 172.21.15.12:0/3901341244 >> 172.21.15.12:0/3901341244 conn(0x55bc6eb305b0 msgr2=0x55bc6eb2f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f37ac04a740 1 -- 172.21.15.12:0/3901341244 shutdown_connections 2024-05-09T09:26:46.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.458+0000 7f37ac04a740 1 -- 172.21.15.12:0/3901341244 wait complete. 2024-05-09T09:26:46.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.459+0000 7f37ac04a740 20 client.16146 trim_cache size 0 max 16384 2024-05-09T09:26:46.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.485+0000 7f37ac04a740 1 Processor -- start 2024-05-09T09:26:46.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.485+0000 7f37ac04a740 1 -- start start 2024-05-09T09:26:46.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.485+0000 7f37ac04a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb305b0 0x55bc6eb3f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.485+0000 7f37ac04a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 0x55bc6eb46890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.485+0000 7f37ac04a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb35930 0x55bc6eb2ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.485+0000 7f37ac04a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f379000fcc0 con 0x55bc6eb34f40 2024-05-09T09:26:46.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.485+0000 7f37ac04a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3790011800 con 0x55bc6eb305b0 2024-05-09T09:26:46.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.485+0000 7f37ac04a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3790010390 con 0x55bc6eb35930 2024-05-09T09:26:46.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.485+0000 7f37a27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 0x55bc6eb46890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.485+0000 7f37a27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 0x55bc6eb46890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35418/0 (socket says 172.21.15.12:35418) 2024-05-09T09:26:46.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.485+0000 7f37a27fc640 1 -- 172.21.15.12:0/855934960 learned_addr learned my addr 172.21.15.12:0/855934960 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.486+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/855934960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb305b0 0x55bc6eb3f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.486+0000 7f37a37fe640 1 --2- 172.21.15.12:0/855934960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb35930 0x55bc6eb2ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.486+0000 7f37a27fc640 1 -- 172.21.15.12:0/855934960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb35930 msgr2=0x55bc6eb2ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.486+0000 7f37a27fc640 1 --2- 172.21.15.12:0/855934960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb35930 0x55bc6eb2ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.486+0000 7f37a27fc640 1 -- 172.21.15.12:0/855934960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb305b0 msgr2=0x55bc6eb3f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.486+0000 7f37a27fc640 1 --2- 172.21.15.12:0/855934960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb305b0 0x55bc6eb3f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.486+0000 7f37a27fc640 1 -- 172.21.15.12:0/855934960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6eb21d20 con 0x55bc6eb34f40 2024-05-09T09:26:46.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.486+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/855934960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb305b0 0x55bc6eb3f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:46.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.486+0000 7f37a27fc640 1 --2- 172.21.15.12:0/855934960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 0x55bc6eb46890 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3798005bc0 tx=0x7f3798005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4f739c2b2457bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.486+0000 7f3788ff9640 1 -- 172.21.15.12:0/855934960 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798004050 con 0x55bc6eb34f40 2024-05-09T09:26:46.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.486+0000 7f3788ff9640 1 -- 172.21.15.12:0/855934960 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f37980054f0 con 0x55bc6eb34f40 2024-05-09T09:26:46.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.486+0000 7f3788ff9640 1 -- 172.21.15.12:0/855934960 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798004da0 con 0x55bc6eb34f40 2024-05-09T09:26:46.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.487+0000 7f37ac04a740 1 -- 172.21.15.12:0/855934960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 msgr2=0x55bc6eb46890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.487+0000 7f37ac04a740 1 --2- 172.21.15.12:0/855934960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 0x55bc6eb46890 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3798005bc0 tx=0x7f3798005f90 comp rx=0 tx=0).stop 2024-05-09T09:26:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.487+0000 7f37ac04a740 1 -- 172.21.15.12:0/855934960 shutdown_connections 2024-05-09T09:26:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.487+0000 7f37ac04a740 1 --2- 172.21.15.12:0/855934960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb35930 0x55bc6eb2ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.487+0000 7f37ac04a740 1 --2- 172.21.15.12:0/855934960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb34f40 0x55bc6eb46890 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.487+0000 7f37ac04a740 1 --2- 172.21.15.12:0/855934960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb305b0 0x55bc6eb3f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.487+0000 7f37ac04a740 1 -- 172.21.15.12:0/855934960 >> 172.21.15.12:0/855934960 conn(0x55bc6eb36610 msgr2=0x55bc6eb3f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.487+0000 7f37ac04a740 1 -- 172.21.15.12:0/855934960 shutdown_connections 2024-05-09T09:26:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.487+0000 7f37ac04a740 1 -- 172.21.15.12:0/855934960 wait complete. 2024-05-09T09:26:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.487+0000 7f37ac04a740 1 -- start start 2024-05-09T09:26:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.487+0000 7f37ac04a740 1 Processor -- start 2024-05-09T09:26:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37ac04a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb305b0 0x55bc6eb43d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37ac04a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb34f40 0x55bc6eb44250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37ac04a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 0x55bc6eb45260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37ac04a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3790011130 con 0x55bc6eb35930 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37ac04a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3790010a60 con 0x55bc6eb34f40 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37ac04a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f379000dca0 con 0x55bc6eb305b0 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37ac04a740 10 client.0 _ll_register_callbacks cb 0x7f3798009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37a37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 0x55bc6eb45260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37a37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 0x55bc6eb45260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35426/0 (socket says 172.21.15.12:35426) 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37a37fe640 1 -- 172.21.15.12:0/1540996125 learned_addr learned my addr 172.21.15.12:0/1540996125 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37a27fc640 1 --2- 172.21.15.12:0/1540996125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb34f40 0x55bc6eb44250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/1540996125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb305b0 0x55bc6eb43d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37a37fe640 1 -- 172.21.15.12:0/1540996125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb305b0 msgr2=0x55bc6eb43d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37a37fe640 1 --2- 172.21.15.12:0/1540996125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb305b0 0x55bc6eb43d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37a37fe640 1 -- 172.21.15.12:0/1540996125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb34f40 msgr2=0x55bc6eb44250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37a37fe640 1 --2- 172.21.15.12:0/1540996125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb34f40 0x55bc6eb44250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.488+0000 7f37a37fe640 1 -- 172.21.15.12:0/1540996125 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6eb45850 con 0x55bc6eb35930 2024-05-09T09:26:46.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.489+0000 7f37a37fe640 1 --2- 172.21.15.12:0/1540996125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 0x55bc6eb45260 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f379c035000 tx=0x7f379c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfed3b8d8010052c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.489+0000 7f3786ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:46.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.489+0000 7f3786ff5640 1 -- 172.21.15.12:0/1540996125 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c059060 con 0x55bc6eb35930 2024-05-09T09:26:46.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.489+0000 7f37ac04a740 1 -- 172.21.15.12:0/1540996125 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37980054f0 con 0x55bc6eb35930 2024-05-09T09:26:46.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.489+0000 7f3786ff5640 1 -- 172.21.15.12:0/1540996125 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f379c004250 con 0x55bc6eb35930 2024-05-09T09:26:46.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.489+0000 7f3786ff5640 1 -- 172.21.15.12:0/1540996125 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f379c060050 con 0x55bc6eb35930 2024-05-09T09:26:46.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.489+0000 7f37ac04a740 10 client.16152 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:46.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.489+0000 7f37ac04a740 1 -- 172.21.15.12:0/1540996125 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bc6eb46c40 con 0x55bc6eb35930 2024-05-09T09:26:46.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.489+0000 7f3786ff5640 1 -- 172.21.15.12:0/1540996125 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f37980054f0 con 0x55bc6eb35930 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.490+0000 7f3786ff5640 1 -- 172.21.15.12:0/1540996125 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f379c0305c0 con 0x55bc6eb35930 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.490+0000 7f37ac04a740 10 client.16152 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.490+0000 7f37ac04a740 10 client.16152 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.490+0000 7f37ac04a740 20 client.16152 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.490+0000 7f37ac04a740 1 --2- 172.21.15.12:0/1540996125 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55bc6eb3aaa0 0x55bc6ebbc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.490+0000 7f37ac04a740 4 client.16152 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.490+0000 7f37ac04a740 1 -- 172.21.15.12:0/1540996125 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc6eaa16c0 con 0x55bc6eb3aaa0 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.490+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/1540996125 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55bc6eb3aaa0 0x55bc6ebbc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.490+0000 7f37a2ffd640 1 --2- 172.21.15.12:0/1540996125 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55bc6eb3aaa0 0x55bc6ebbc3f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee15a3c36512275d server_cookie=a186bc4ca350467d in_seq=0 out_seq=0 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.490+0000 7f3786ff5640 10 client.16152 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f3786ff5640 1 -- 172.21.15.12:0/1540996125 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc6eaa16c0 con 0x55bc6eb3aaa0 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f3786ff5640 10 client.16152 handle_command_reply: tid=0 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f37ac04a740 1 client.16152 shutdown 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f37ac04a740 1 -- 172.21.15.12:0/1540996125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 msgr2=0x55bc6eb45260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f37ac04a740 1 --2- 172.21.15.12:0/1540996125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 0x55bc6eb45260 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f379c035000 tx=0x7f379c0095c0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f37ac04a740 1 -- 172.21.15.12:0/1540996125 shutdown_connections 2024-05-09T09:26:46.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f37ac04a740 1 --2- 172.21.15.12:0/1540996125 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55bc6eb3aaa0 0x55bc6ebbc3f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f37ac04a740 1 --2- 172.21.15.12:0/1540996125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc6eb35930 0x55bc6eb45260 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f3786ff5640 0 client.16152 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:26:46.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f37ac04a740 1 --2- 172.21.15.12:0/1540996125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc6eb34f40 0x55bc6eb44250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f37ac04a740 1 --2- 172.21.15.12:0/1540996125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc6eb305b0 0x55bc6eb43d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f37ac04a740 1 -- 172.21.15.12:0/1540996125 >> 172.21.15.12:0/1540996125 conn(0x55bc6eb36610 msgr2=0x55bc6eb33560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f37ac04a740 1 -- 172.21.15.12:0/1540996125 shutdown_connections 2024-05-09T09:26:46.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f37ac04a740 1 -- 172.21.15.12:0/1540996125 wait complete. 2024-05-09T09:26:46.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.491+0000 7f37ac04a740 20 client.16152 trim_cache size 0 max 16384 2024-05-09T09:26:46.496 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:46.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.492+0000 7f37a933b640 1 -- 172.21.15.12:0/2820794941 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f378c0a2c10 msgr2=0x7f378c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.492+0000 7f37a933b640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f378c0a2c10 0x7f378c0a50d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55bc6eb36190 tx=0x7f379c031b80 comp rx=0 tx=0).stop 2024-05-09T09:26:46.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.492+0000 7f37a933b640 1 -- 172.21.15.12:0/2820794941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37a4100770 msgr2=0x7f37a410b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.492+0000 7f37a933b640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37a4100770 0x7f37a410b0f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f379400e9c0 tx=0x7f379400ee80 comp rx=0 tx=0).stop 2024-05-09T09:26:46.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.492+0000 7f37a933b640 1 -- 172.21.15.12:0/2820794941 shutdown_connections 2024-05-09T09:26:46.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.492+0000 7f37a933b640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f378c0a2c10 0x7f378c0a50d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.492+0000 7f37a933b640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a4102320 0x7f37a4109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.492+0000 7f37a933b640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37a4101960 0x7f37a4109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.492+0000 7f37a933b640 1 --2- 172.21.15.12:0/2820794941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37a4100770 0x7f37a410b0f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.492+0000 7f37a933b640 1 -- 172.21.15.12:0/2820794941 >> 172.21.15.12:0/2820794941 conn(0x7f37a40fbce0 msgr2=0x7f37a4104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.492+0000 7f37a933b640 1 -- 172.21.15.12:0/2820794941 shutdown_connections 2024-05-09T09:26:46.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.492+0000 7f37a933b640 1 -- 172.21.15.12:0/2820794941 wait complete. 2024-05-09T09:26:46.508 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:26:46.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.619+0000 7ffa4af9e640 1 -- 172.21.15.12:0/1760606660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa44102390 msgr2=0x7ffa44104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.619+0000 7ffa4af9e640 1 --2- 172.21.15.12:0/1760606660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa44102390 0x7ffa44104780 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009900 tx=0x7ffa3802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.619+0000 7ffa4af9e640 1 -- 172.21.15.12:0/1760606660 shutdown_connections 2024-05-09T09:26:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.619+0000 7ffa4af9e640 1 --2- 172.21.15.12:0/1760606660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa441075f0 0x7ffa441099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.619+0000 7ffa4af9e640 1 --2- 172.21.15.12:0/1760606660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa44104cc0 0x7ffa441070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.619+0000 7ffa4af9e640 1 --2- 172.21.15.12:0/1760606660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa44102390 0x7ffa44104780 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.619+0000 7ffa4af9e640 1 -- 172.21.15.12:0/1760606660 >> 172.21.15.12:0/1760606660 conn(0x7ffa440fbce0 msgr2=0x7ffa440fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.619+0000 7ffa4af9e640 1 -- 172.21.15.12:0/1760606660 shutdown_connections 2024-05-09T09:26:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.619+0000 7ffa4af9e640 1 -- 172.21.15.12:0/1760606660 wait complete. 2024-05-09T09:26:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.619+0000 7ffa4af9e640 1 Processor -- start 2024-05-09T09:26:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa4af9e640 1 -- start start 2024-05-09T09:26:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa4af9e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa44102390 0x7ffa44101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa4af9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa44104cc0 0x7ffa44100340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa4af9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa441075f0 0x7ffa44100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa4af9e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4410d4f0 con 0x7ffa44102390 2024-05-09T09:26:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa4af9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4410d370 con 0x7ffa441075f0 2024-05-09T09:26:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa4af9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4410d670 con 0x7ffa44104cc0 2024-05-09T09:26:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa48d3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa44102390 0x7ffa44101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa48d3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa44102390 0x7ffa44101d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35444/0 (socket says 172.21.15.12:35444) 2024-05-09T09:26:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa48d3a640 1 -- 172.21.15.12:0/2692818846 learned_addr learned my addr 172.21.15.12:0/2692818846 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa43fff640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa44104cc0 0x7ffa44100340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa4953b640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa441075f0 0x7ffa44100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa48d3a640 1 -- 172.21.15.12:0/2692818846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa44104cc0 msgr2=0x7ffa44100340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa44104cc0 0x7ffa44100340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa48d3a640 1 -- 172.21.15.12:0/2692818846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa441075f0 msgr2=0x7ffa44100880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa441075f0 0x7ffa44100880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.620+0000 7ffa48d3a640 1 -- 172.21.15.12:0/2692818846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa44101110 con 0x7ffa44102390 2024-05-09T09:26:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.621+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa44102390 0x7ffa44101d00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009230 tx=0x7ffa38031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.621+0000 7ffa41ffb640 1 -- 172.21.15.12:0/2692818846 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3803f030 con 0x7ffa44102390 2024-05-09T09:26:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.621+0000 7ffa4af9e640 1 -- 172.21.15.12:0/2692818846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa441a1cb0 con 0x7ffa44102390 2024-05-09T09:26:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.621+0000 7ffa41ffb640 1 -- 172.21.15.12:0/2692818846 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ffa38003ac0 con 0x7ffa44102390 2024-05-09T09:26:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.621+0000 7ffa41ffb640 1 -- 172.21.15.12:0/2692818846 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38035070 con 0x7ffa44102390 2024-05-09T09:26:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.621+0000 7ffa4af9e640 1 -- 172.21.15.12:0/2692818846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa441a21b0 con 0x7ffa44102390 2024-05-09T09:26:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.624+0000 7ffa4dd88740 1 Processor -- start 2024-05-09T09:26:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.624+0000 7ffa4dd88740 1 -- start start 2024-05-09T09:26:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.624+0000 7ffa4dd88740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564123887f40 0x564123888360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.624+0000 7ffa4dd88740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123888930 0x56412388d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.624+0000 7ffa4dd88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56412388daa0 0x56412388ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.624+0000 7ffa4dd88740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5641237ecc40 con 0x564123888930 2024-05-09T09:26:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.624+0000 7ffa4dd88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564123794580 con 0x56412388daa0 2024-05-09T09:26:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.624+0000 7ffa4dd88740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5641237e8c00 con 0x564123887f40 2024-05-09T09:26:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.624+0000 7ffa41ffb640 1 -- 172.21.15.12:0/2692818846 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa38042030 con 0x7ffa44102390 2024-05-09T09:26:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.624+0000 7ffa43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56412388daa0 0x56412388ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.625+0000 7ffa43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56412388daa0 0x56412388ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47834/0 (socket says 172.21.15.12:47834) 2024-05-09T09:26:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.625+0000 7ffa43fff640 1 -- 172.21.15.12:0/3039909775 learned_addr learned my addr 172.21.15.12:0/3039909775 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.625+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/3039909775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123888930 0x56412388d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.625+0000 7ffa4953b640 1 --2- 172.21.15.12:0/3039909775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564123887f40 0x564123888360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.625+0000 7ffa43fff640 1 -- 172.21.15.12:0/3039909775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564123887f40 msgr2=0x564123888360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.625+0000 7ffa43fff640 1 --2- 172.21.15.12:0/3039909775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564123887f40 0x564123888360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.625+0000 7ffa43fff640 1 -- 172.21.15.12:0/3039909775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123888930 msgr2=0x56412388d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.625+0000 7ffa43fff640 1 --2- 172.21.15.12:0/3039909775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123888930 0x56412388d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.625+0000 7ffa43fff640 1 -- 172.21.15.12:0/3039909775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641235003f0 con 0x56412388daa0 2024-05-09T09:26:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.625+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/3039909775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123888930 0x56412388d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.625+0000 7ffa41ffb640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ffa340a2b90 0x7ffa340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.625+0000 7ffa43fff640 1 --2- 172.21.15.12:0/3039909775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56412388daa0 0x56412388ff30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000bcd0 tx=0x7ffa300107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e84a651ab7418991 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.626+0000 7ffa4953b640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ffa340a2b90 0x7ffa340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.626+0000 7ffa41ffb640 1 -- 172.21.15.12:0/2692818846 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7ffa380efa70 con 0x7ffa44102390 2024-05-09T09:26:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.626+0000 7ffa2effd640 1 -- 172.21.15.12:0/3039909775 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3001c680 con 0x56412388daa0 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.626+0000 7ffa2effd640 1 -- 172.21.15.12:0/3039909775 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ffa3001ccf0 con 0x56412388daa0 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.626+0000 7ffa2effd640 1 -- 172.21.15.12:0/3039909775 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30011d10 con 0x56412388daa0 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.627+0000 7ffa4dd88740 1 -- 172.21.15.12:0/3039909775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56412388daa0 msgr2=0x56412388ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.627+0000 7ffa4953b640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ffa340a2b90 0x7ffa340a5050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x564123889190 tx=0x7ffa3c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.627+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/3039909775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56412388daa0 0x56412388ff30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000bcd0 tx=0x7ffa300107a0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.627+0000 7ffa4dd88740 1 -- 172.21.15.12:0/3039909775 shutdown_connections 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.627+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/3039909775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56412388daa0 0x56412388ff30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.627+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/3039909775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123888930 0x56412388d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.627+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/3039909775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564123887f40 0x564123888360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.627+0000 7ffa4dd88740 1 -- 172.21.15.12:0/3039909775 >> 172.21.15.12:0/3039909775 conn(0x5641238835b0 msgr2=0x5641238839b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.627+0000 7ffa4dd88740 1 -- 172.21.15.12:0/3039909775 shutdown_connections 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.627+0000 7ffa4dd88740 1 -- 172.21.15.12:0/3039909775 wait complete. 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.627+0000 7ffa4dd88740 1 -- start start 2024-05-09T09:26:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.627+0000 7ffa4dd88740 1 Processor -- start 2024-05-09T09:26:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa4dd88740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 0x5641238923b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa4dd88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123888930 0x5641238928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa4dd88740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56412388daa0 0x564123899930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa4dd88740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564123890aa0 con 0x564123887f40 2024-05-09T09:26:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa4dd88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5641237a7c40 con 0x564123888930 2024-05-09T09:26:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa4dd88740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564123890c20 con 0x56412388daa0 2024-05-09T09:26:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa4dd88740 10 client.0 _ll_register_callbacks cb 0x5641234ffbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa48d3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 0x5641238923b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa4953b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56412388daa0 0x564123899930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa48d3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 0x5641238923b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35464/0 (socket says 172.21.15.12:35464) 2024-05-09T09:26:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa48d3a640 1 -- 172.21.15.12:0/964379146 learned_addr learned my addr 172.21.15.12:0/964379146 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa43fff640 1 --2- 172.21.15.12:0/964379146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123888930 0x5641238928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa4953b640 1 -- 172.21.15.12:0/964379146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123888930 msgr2=0x5641238928f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa4953b640 1 --2- 172.21.15.12:0/964379146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123888930 0x5641238928f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa4953b640 1 -- 172.21.15.12:0/964379146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 msgr2=0x5641238923b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa4953b640 1 --2- 172.21.15.12:0/964379146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 0x5641238923b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa4953b640 1 -- 172.21.15.12:0/964379146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641235ca7d0 con 0x56412388daa0 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.628+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/964379146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 0x5641238923b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.629+0000 7ffa4953b640 1 --2- 172.21.15.12:0/964379146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56412388daa0 0x564123899930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c005fd0 tx=0x7ffa3c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77eeab5d4cb024cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.629+0000 7ffa23fff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.629+0000 7ffa23fff640 1 -- 172.21.15.12:0/964379146 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c011650 con 0x56412388daa0 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.629+0000 7ffa4dd88740 1 -- 172.21.15.12:0/964379146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa3001ccf0 con 0x56412388daa0 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.629+0000 7ffa23fff640 1 -- 172.21.15.12:0/964379146 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7ffa3c0117f0 con 0x56412388daa0 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.629+0000 7ffa23fff640 1 -- 172.21.15.12:0/964379146 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa3c0119f0 con 0x56412388daa0 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.629+0000 7ffa4dd88740 10 client.25768 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.629+0000 7ffa4dd88740 1 -- 172.21.15.12:0/964379146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56412389cdc0 con 0x56412388daa0 2024-05-09T09:26:46.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.629+0000 7ffa23fff640 1 -- 172.21.15.12:0/964379146 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ffa3001ccf0 con 0x56412388daa0 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.630+0000 7ffa23fff640 1 -- 172.21.15.12:0/964379146 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffa3c026070 con 0x56412388daa0 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.630+0000 7ffa4dd88740 10 client.25768 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.630+0000 7ffa4dd88740 10 client.25768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.630+0000 7ffa4dd88740 20 client.25768 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.630+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/964379146 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56412389e480 0x5641238a0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.630+0000 7ffa4dd88740 4 client.25768 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.630+0000 7ffa4dd88740 1 -- 172.21.15.12:0/964379146 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641237fa370 con 0x56412389e480 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.630+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/964379146 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56412389e480 0x5641238a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.631+0000 7ffa23fff640 10 client.25768 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.631+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/964379146 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56412389e480 0x5641238a0940 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85d5623460f153da server_cookie=bd56a634121bb815 in_seq=0 out_seq=0 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa23fff640 1 -- 172.21.15.12:0/964379146 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641237fa370 con 0x56412389e480 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa23fff640 10 client.25768 handle_command_reply: tid=0 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa4dd88740 1 client.25768 shutdown 2024-05-09T09:26:46.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa4dd88740 1 -- 172.21.15.12:0/964379146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56412388daa0 msgr2=0x564123899930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/964379146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56412388daa0 0x564123899930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c005fd0 tx=0x7ffa3c004ab0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa4dd88740 1 -- 172.21.15.12:0/964379146 shutdown_connections 2024-05-09T09:26:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/964379146 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56412389e480 0x5641238a0940 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/964379146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56412388daa0 0x564123899930 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/964379146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123888930 0x5641238928f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa23fff640 0 client.25768 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/964379146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 0x5641238923b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa4dd88740 1 -- 172.21.15.12:0/964379146 >> 172.21.15.12:0/964379146 conn(0x5641238835b0 msgr2=0x5641238827b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa4dd88740 1 -- 172.21.15.12:0/964379146 shutdown_connections 2024-05-09T09:26:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa4dd88740 1 -- 172.21.15.12:0/964379146 wait complete. 2024-05-09T09:26:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.632+0000 7ffa4dd88740 20 client.25768 trim_cache size 0 max 16384 2024-05-09T09:26:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.646+0000 7ffa4dd88740 1 Processor -- start 2024-05-09T09:26:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.646+0000 7ffa4dd88740 1 -- start start 2024-05-09T09:26:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.646+0000 7ffa4dd88740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 0x564123889d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.646+0000 7ffa4dd88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123888930 0x56412388a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.646+0000 7ffa4dd88740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56412388daa0 0x5641238998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.646+0000 7ffa4dd88740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441ac880 con 0x564123887f40 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.646+0000 7ffa4dd88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441ae3c0 con 0x564123888930 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.646+0000 7ffa4dd88740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441acf50 con 0x56412388daa0 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.646+0000 7ffa48d3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 0x564123889d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.646+0000 7ffa48d3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 0x564123889d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35470/0 (socket says 172.21.15.12:35470) 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.646+0000 7ffa48d3a640 1 -- 172.21.15.12:0/1140427838 learned_addr learned my addr 172.21.15.12:0/1140427838 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.646+0000 7ffa43fff640 1 --2- 172.21.15.12:0/1140427838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123888930 0x56412388a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.647+0000 7ffa4953b640 1 --2- 172.21.15.12:0/1140427838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56412388daa0 0x5641238998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.647+0000 7ffa48d3a640 1 -- 172.21.15.12:0/1140427838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56412388daa0 msgr2=0x5641238998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.647+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/1140427838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56412388daa0 0x5641238998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.647+0000 7ffa48d3a640 1 -- 172.21.15.12:0/1140427838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123888930 msgr2=0x56412388a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.647+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/1140427838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123888930 0x56412388a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.647+0000 7ffa48d3a640 1 -- 172.21.15.12:0/1140427838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564123883150 con 0x564123887f40 2024-05-09T09:26:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.647+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/1140427838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 0x564123889d50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ffa380ed1f0 tx=0x7ffa380f3550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdd47e0a9c209ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.647+0000 7ffa2e7fc640 1 -- 172.21.15.12:0/1140427838 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa380f6070 con 0x564123887f40 2024-05-09T09:26:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.647+0000 7ffa2e7fc640 1 -- 172.21.15.12:0/1140427838 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ffa380bc600 con 0x564123887f40 2024-05-09T09:26:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.647+0000 7ffa2e7fc640 1 -- 172.21.15.12:0/1140427838 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38049040 con 0x564123887f40 2024-05-09T09:26:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.648+0000 7ffa4dd88740 1 -- 172.21.15.12:0/1140427838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 msgr2=0x564123889d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.648+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/1140427838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 0x564123889d50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ffa380ed1f0 tx=0x7ffa380f3550 comp rx=0 tx=0).stop 2024-05-09T09:26:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.648+0000 7ffa4dd88740 1 -- 172.21.15.12:0/1140427838 shutdown_connections 2024-05-09T09:26:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.648+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/1140427838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56412388daa0 0x5641238998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.648+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/1140427838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123888930 0x56412388a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.648+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/1140427838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564123887f40 0x564123889d50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.648+0000 7ffa4dd88740 1 -- 172.21.15.12:0/1140427838 >> 172.21.15.12:0/1140427838 conn(0x5641238835b0 msgr2=0x5641238923b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.648+0000 7ffa4dd88740 1 -- 172.21.15.12:0/1140427838 shutdown_connections 2024-05-09T09:26:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.648+0000 7ffa4dd88740 1 -- 172.21.15.12:0/1140427838 wait complete. 2024-05-09T09:26:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.648+0000 7ffa4dd88740 1 -- start start 2024-05-09T09:26:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.649+0000 7ffa4dd88740 1 Processor -- start 2024-05-09T09:26:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.649+0000 7ffa4dd88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123887f40 0x564123896a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.649+0000 7ffa4dd88740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564123888930 0x564123896f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.649+0000 7ffa4dd88740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56412388daa0 0x564123897f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.649+0000 7ffa4dd88740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441adcf0 con 0x56412388daa0 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.649+0000 7ffa4dd88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441ad620 con 0x564123887f40 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.649+0000 7ffa4dd88740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441aa860 con 0x564123888930 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.649+0000 7ffa4dd88740 10 client.0 _ll_register_callbacks cb 0x7ffa3c003f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.649+0000 7ffa48d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123887f40 0x564123896a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.649+0000 7ffa48d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123887f40 0x564123896a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47860/0 (socket says 172.21.15.12:47860) 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.649+0000 7ffa48d3a640 1 -- 172.21.15.12:0/2468115788 learned_addr learned my addr 172.21.15.12:0/2468115788 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.649+0000 7ffa4953b640 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56412388daa0 0x564123897f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.650+0000 7ffa43fff640 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564123888930 0x564123896f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.650+0000 7ffa48d3a640 1 -- 172.21.15.12:0/2468115788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564123888930 msgr2=0x564123896f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.650+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564123888930 0x564123896f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.650+0000 7ffa48d3a640 1 -- 172.21.15.12:0/2468115788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56412388daa0 msgr2=0x564123897f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.650+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56412388daa0 0x564123897f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.650+0000 7ffa48d3a640 1 -- 172.21.15.12:0/2468115788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564123898540 con 0x564123887f40 2024-05-09T09:26:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.650+0000 7ffa4953b640 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56412388daa0 0x564123897f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.650+0000 7ffa48d3a640 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123887f40 0x564123896a50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa38032fd0 tx=0x7ffa3802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa1decd3fcd9acf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.650+0000 7ffa23fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:26:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.650+0000 7ffa23fff640 1 -- 172.21.15.12:0/2468115788 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa380f6050 con 0x564123887f40 2024-05-09T09:26:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.650+0000 7ffa4dd88740 1 -- 172.21.15.12:0/2468115788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa380bc600 con 0x564123887f40 2024-05-09T09:26:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.650+0000 7ffa23fff640 1 -- 172.21.15.12:0/2468115788 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7ffa380b8570 con 0x564123887f40 2024-05-09T09:26:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.651+0000 7ffa23fff640 1 -- 172.21.15.12:0/2468115788 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa38049070 con 0x564123887f40 2024-05-09T09:26:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.651+0000 7ffa4dd88740 10 client.25658 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.651+0000 7ffa4dd88740 1 -- 172.21.15.12:0/2468115788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564123899840 con 0x564123887f40 2024-05-09T09:26:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.651+0000 7ffa23fff640 1 -- 172.21.15.12:0/2468115788 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ffa380bc600 con 0x564123887f40 2024-05-09T09:26:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.651+0000 7ffa23fff640 1 -- 172.21.15.12:0/2468115788 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffa38003c70 con 0x564123887f40 2024-05-09T09:26:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.651+0000 7ffa4dd88740 10 client.25658 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.651+0000 7ffa4dd88740 10 client.25658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.651+0000 7ffa4dd88740 20 client.25658 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.651+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564123904c20 0x564123907010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.651+0000 7ffa4dd88740 4 client.25658 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:26:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.651+0000 7ffa4dd88740 1 -- 172.21.15.12:0/2468115788 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641237f46c0 con 0x564123904c20 2024-05-09T09:26:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.652+0000 7ffa43fff640 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564123904c20 0x564123907010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.652+0000 7ffa23fff640 10 client.25658 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.652+0000 7ffa43fff640 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564123904c20 0x564123907010 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ab018ec73a8e8f3 server_cookie=e82df1d2a00a5eb in_seq=0 out_seq=0 2024-05-09T09:26:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.652+0000 7ffa23fff640 1 -- 172.21.15.12:0/2468115788 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641237f46c0 con 0x564123904c20 2024-05-09T09:26:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.652+0000 7ffa23fff640 10 client.25658 handle_command_reply: tid=0 2024-05-09T09:26:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.652+0000 7ffa4dd88740 1 client.25658 shutdown 2024-05-09T09:26:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.653+0000 7ffa4dd88740 1 -- 172.21.15.12:0/2468115788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123887f40 msgr2=0x564123896a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.653+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123887f40 0x564123896a50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa38032fd0 tx=0x7ffa3802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.653+0000 7ffa4dd88740 1 -- 172.21.15.12:0/2468115788 shutdown_connections 2024-05-09T09:26:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.653+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564123904c20 0x564123907010 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.653+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56412388daa0 0x564123897f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.653+0000 7ffa23fff640 0 client.25658 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.653+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564123888930 0x564123896f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.653+0000 7ffa4dd88740 1 --2- 172.21.15.12:0/2468115788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564123887f40 0x564123896a50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.653+0000 7ffa4dd88740 1 -- 172.21.15.12:0/2468115788 >> 172.21.15.12:0/2468115788 conn(0x5641238835b0 msgr2=0x5641238923b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.653+0000 7ffa4dd88740 1 -- 172.21.15.12:0/2468115788 shutdown_connections 2024-05-09T09:26:46.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.653+0000 7ffa4dd88740 1 -- 172.21.15.12:0/2468115788 wait complete. 2024-05-09T09:26:46.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.653+0000 7ffa4dd88740 20 client.25658 trim_cache size 0 max 16384 2024-05-09T09:26:46.658 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:26:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.654+0000 7ffa4af9e640 1 -- 172.21.15.12:0/2692818846 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ffa340a2b90 msgr2=0x7ffa340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.654+0000 7ffa4af9e640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ffa340a2b90 0x7ffa340a5050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x564123889190 tx=0x7ffa3c009500 comp rx=0 tx=0).stop 2024-05-09T09:26:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.654+0000 7ffa4af9e640 1 -- 172.21.15.12:0/2692818846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa44102390 msgr2=0x7ffa44101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.654+0000 7ffa4af9e640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa44102390 0x7ffa44101d00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009230 tx=0x7ffa38031ac0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.654+0000 7ffa4af9e640 1 -- 172.21.15.12:0/2692818846 shutdown_connections 2024-05-09T09:26:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.654+0000 7ffa4af9e640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ffa340a2b90 0x7ffa340a5050 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.654+0000 7ffa4af9e640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa441075f0 0x7ffa44100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.654+0000 7ffa4af9e640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa44104cc0 0x7ffa44100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.654+0000 7ffa4af9e640 1 --2- 172.21.15.12:0/2692818846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa44102390 0x7ffa44101d00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.654+0000 7ffa4af9e640 1 -- 172.21.15.12:0/2692818846 >> 172.21.15.12:0/2692818846 conn(0x7ffa440fbce0 msgr2=0x7ffa44102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.654+0000 7ffa4af9e640 1 -- 172.21.15.12:0/2692818846 shutdown_connections 2024-05-09T09:26:46.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:46.654+0000 7ffa4af9e640 1 -- 172.21.15.12:0/2692818846 wait complete. 2024-05-09T09:26:46.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:46 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1515970222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:46.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:46 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1515970222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:47.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:46 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1515970222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:47.669 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:26:47.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:26:47.670 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:26:47.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.841+0000 7f90aabd5640 1 -- 172.21.15.12:0/3371245844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a40ff8c0 msgr2=0x7f90a40ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.841+0000 7f90aabd5640 1 --2- 172.21.15.12:0/3371245844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a40ff8c0 0x7f90a40ffd40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9090009900 tx=0x7f909002a410 comp rx=0 tx=0).stop 2024-05-09T09:26:47.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.841+0000 7f90aabd5640 1 -- 172.21.15.12:0/3371245844 shutdown_connections 2024-05-09T09:26:47.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.841+0000 7f90aabd5640 1 --2- 172.21.15.12:0/3371245844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4100280 0x7f90a41006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.841+0000 7f90aabd5640 1 --2- 172.21.15.12:0/3371245844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a40ff8c0 0x7f90a40ffd40 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.841+0000 7f90aabd5640 1 --2- 172.21.15.12:0/3371245844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a4101d40 0x7f90a4102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.841+0000 7f90aabd5640 1 -- 172.21.15.12:0/3371245844 >> 172.21.15.12:0/3371245844 conn(0x7f90a40fb460 msgr2=0x7f90a40fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:47.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.841+0000 7f90aabd5640 1 -- 172.21.15.12:0/3371245844 shutdown_connections 2024-05-09T09:26:47.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.841+0000 7f90aabd5640 1 -- 172.21.15.12:0/3371245844 wait complete. 2024-05-09T09:26:47.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.842+0000 7f90aabd5640 1 Processor -- start 2024-05-09T09:26:47.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.842+0000 7f90aabd5640 1 -- start start 2024-05-09T09:26:47.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.842+0000 7f90aabd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a40ff8c0 0x7f90a4199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.842+0000 7f90aabd5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a4100280 0x7f90a419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.842+0000 7f90aabd5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4101d40 0x7f90a41a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.842+0000 7f90aabd5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f90a410f780 con 0x7f90a4100280 2024-05-09T09:26:47.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.842+0000 7f90aabd5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90a410f600 con 0x7f90a40ff8c0 2024-05-09T09:26:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.842+0000 7f90aabd5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90a410f900 con 0x7f90a4101d40 2024-05-09T09:26:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.843+0000 7f90a8971640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a40ff8c0 0x7f90a4199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.843+0000 7f90a9172640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4101d40 0x7f90a41a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.843+0000 7f90a8971640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a40ff8c0 0x7f90a4199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47884/0 (socket says 172.21.15.12:47884) 2024-05-09T09:26:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.843+0000 7f90a8971640 1 -- 172.21.15.12:0/3473478411 learned_addr learned my addr 172.21.15.12:0/3473478411 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.843+0000 7f90a3fff640 1 --2- 172.21.15.12:0/3473478411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a4100280 0x7f90a419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.843+0000 7f90a9172640 1 -- 172.21.15.12:0/3473478411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a40ff8c0 msgr2=0x7f90a4199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.843+0000 7f90a9172640 1 --2- 172.21.15.12:0/3473478411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a40ff8c0 0x7f90a4199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.843+0000 7f90a9172640 1 -- 172.21.15.12:0/3473478411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a4100280 msgr2=0x7f90a419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.843+0000 7f90a9172640 1 --2- 172.21.15.12:0/3473478411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a4100280 0x7f90a419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.843+0000 7f90a9172640 1 -- 172.21.15.12:0/3473478411 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a41a1a10 con 0x7f90a4101d40 2024-05-09T09:26:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.843+0000 7f90a9172640 1 --2- 172.21.15.12:0/3473478411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4101d40 0x7f90a41a13a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f909c00be20 tx=0x7f909c00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.843+0000 7f90a1ffb640 1 -- 172.21.15.12:0/3473478411 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c019070 con 0x7f90a4101d40 2024-05-09T09:26:47.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.844+0000 7f90a1ffb640 1 -- 172.21.15.12:0/3473478411 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f909c014910 con 0x7f90a4101d40 2024-05-09T09:26:47.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.844+0000 7f90aabd5640 1 -- 172.21.15.12:0/3473478411 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a41a1d00 con 0x7f90a4101d40 2024-05-09T09:26:47.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.844+0000 7f90aabd5640 1 -- 172.21.15.12:0/3473478411 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a41a2250 con 0x7f90a4101d40 2024-05-09T09:26:47.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.844+0000 7f90a1ffb640 1 -- 172.21.15.12:0/3473478411 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c012890 con 0x7f90a4101d40 2024-05-09T09:26:47.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.847+0000 7f90a1ffb640 1 -- 172.21.15.12:0/3473478411 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f909c01c070 con 0x7f90a4101d40 2024-05-09T09:26:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.847+0000 7f90ad9b6740 1 Processor -- start 2024-05-09T09:26:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.847+0000 7f90ad9b6740 1 -- start start 2024-05-09T09:26:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90ad9b6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90ad9b6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac51eb0 0x55662ac569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90ad9b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac57020 0x55662ac594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90ad9b6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55662abb5990 con 0x55662ac51eb0 2024-05-09T09:26:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90ad9b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55662ab5d5b0 con 0x55662ac57020 2024-05-09T09:26:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90ad9b6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55662abb1950 con 0x55662ac514c0 2024-05-09T09:26:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90a8971640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac51eb0 0x55662ac569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac518e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:37534/0 (socket says 172.21.15.12:37534) 2024-05-09T09:26:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90a3fff640 1 -- 172.21.15.12:0/3948009373 learned_addr learned my addr 172.21.15.12:0/3948009373 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:47.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90a3fff640 1 --2- 172.21.15.12:0/3948009373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac57020 0x55662ac594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90a8971640 1 -- 172.21.15.12:0/3948009373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 msgr2=0x55662ac518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:47.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90a8971640 1 --2- 172.21.15.12:0/3948009373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.848+0000 7f90a8971640 1 -- 172.21.15.12:0/3948009373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac57020 msgr2=0x55662ac594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:47.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.849+0000 7f90a1ffb640 1 --2- 172.21.15.12:0/3473478411 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f90940a2c10 0x7f90940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.849+0000 7f90a8971640 1 --2- 172.21.15.12:0/3948009373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac57020 0x55662ac594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.849+0000 7f90a8971640 1 -- 172.21.15.12:0/3948009373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55662a9917e0 con 0x55662ac51eb0 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.849+0000 7f90a3fff640 1 --2- 172.21.15.12:0/3948009373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac57020 0x55662ac594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.849+0000 7f90a9172640 1 --2- 172.21.15.12:0/3473478411 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f90940a2c10 0x7f90940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.849+0000 7f90a8971640 1 --2- 172.21.15.12:0/3948009373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac51eb0 0x55662ac569b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9098009c80 tx=0x7f9098011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e57ade5a371ce4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.849+0000 7f908effd640 1 -- 172.21.15.12:0/3948009373 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098004230 con 0x55662ac51eb0 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.849+0000 7f908effd640 1 -- 172.21.15.12:0/3948009373 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f90980043d0 con 0x55662ac51eb0 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.849+0000 7f908effd640 1 -- 172.21.15.12:0/3948009373 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098004dc0 con 0x55662ac51eb0 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.850+0000 7f90a1ffb640 1 -- 172.21.15.12:0/3473478411 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f909c0cec20 con 0x7f90a4101d40 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.850+0000 7f90ad9b6740 1 -- 172.21.15.12:0/3948009373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac51eb0 msgr2=0x55662ac569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.850+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/3948009373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac51eb0 0x55662ac569b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9098009c80 tx=0x7f9098011810 comp rx=0 tx=0).stop 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.850+0000 7f90a9172640 1 --2- 172.21.15.12:0/3473478411 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f90940a2c10 0x7f90940a50d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f909c0cc3a0 tx=0x7f909c0d2410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.850+0000 7f90ad9b6740 1 -- 172.21.15.12:0/3948009373 shutdown_connections 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.851+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/3948009373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac57020 0x55662ac594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.851+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/3948009373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac51eb0 0x55662ac569b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.851+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/3948009373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.851+0000 7f90ad9b6740 1 -- 172.21.15.12:0/3948009373 >> 172.21.15.12:0/3948009373 conn(0x55662ac190a0 msgr2=0x55662ac4cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:47.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.851+0000 7f90ad9b6740 1 -- 172.21.15.12:0/3948009373 shutdown_connections 2024-05-09T09:26:47.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.851+0000 7f90ad9b6740 1 -- 172.21.15.12:0/3948009373 wait complete. 2024-05-09T09:26:47.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.851+0000 7f90ad9b6740 1 -- start start 2024-05-09T09:26:47.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.851+0000 7f90ad9b6740 1 Processor -- start 2024-05-09T09:26:47.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.852+0000 7f90ad9b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac514c0 0x55662ac5b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.852+0000 7f90ad9b6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac51eb0 0x55662ac5bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.852+0000 7f90ad9b6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac57020 0x55662ac62f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.852+0000 7f90ad9b6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55662ac59fd0 con 0x55662ac51eb0 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.852+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac514c0 0x55662ac5b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.852+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac514c0 0x55662ac5b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47904/0 (socket says 172.21.15.12:47904) 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.852+0000 7f90a3fff640 1 -- 172.21.15.12:0/627612461 learned_addr learned my addr 172.21.15.12:0/627612461 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.853+0000 7f90a8971640 1 --2- 172.21.15.12:0/627612461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac51eb0 0x55662ac5bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.853+0000 7f90ad9b6740 1 -- 172.21.15.12:0/627612461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55662ab70b40 con 0x55662ac514c0 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.853+0000 7f90ad9b6740 1 -- 172.21.15.12:0/627612461 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55662ac5a150 con 0x55662ac57020 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.853+0000 7f90a3fff640 1 --2- 172.21.15.12:0/627612461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac57020 0x55662ac62f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.853+0000 7f90ad9b6740 10 client.0 _ll_register_callbacks cb 0x55662ab650f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.853+0000 7f90a8971640 1 -- 172.21.15.12:0/627612461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac57020 msgr2=0x55662ac62f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.853+0000 7f90a8971640 1 --2- 172.21.15.12:0/627612461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac57020 0x55662ac62f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.853+0000 7f90a8971640 1 -- 172.21.15.12:0/627612461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac514c0 msgr2=0x55662ac5b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.853+0000 7f90a8971640 1 --2- 172.21.15.12:0/627612461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac514c0 0x55662ac5b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.853+0000 7f90a8971640 1 -- 172.21.15.12:0/627612461 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55662a9f1f70 con 0x55662ac51eb0 2024-05-09T09:26:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.854+0000 7f90a8971640 1 --2- 172.21.15.12:0/627612461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac51eb0 0x55662ac5bec0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x55662ac4ddf0 tx=0x7f90980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc21e6103a8ac708 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.854+0000 7f9083fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.854+0000 7f9083fff640 1 -- 172.21.15.12:0/627612461 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098028020 con 0x55662ac51eb0 2024-05-09T09:26:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.854+0000 7f9083fff640 1 -- 172.21.15.12:0/627612461 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f9098005430 con 0x55662ac51eb0 2024-05-09T09:26:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.854+0000 7f90ad9b6740 1 -- 172.21.15.12:0/627612461 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90980043d0 con 0x55662ac51eb0 2024-05-09T09:26:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.855+0000 7f9083fff640 1 -- 172.21.15.12:0/627612461 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9098024420 con 0x55662ac51eb0 2024-05-09T09:26:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.855+0000 7f90ad9b6740 10 client.16182 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.855+0000 7f90ad9b6740 1 -- 172.21.15.12:0/627612461 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55662a889170 con 0x55662ac51eb0 2024-05-09T09:26:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.855+0000 7f9083fff640 1 -- 172.21.15.12:0/627612461 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f90980043d0 con 0x55662ac51eb0 2024-05-09T09:26:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.855+0000 7f9083fff640 1 -- 172.21.15.12:0/627612461 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9098018070 con 0x55662ac51eb0 2024-05-09T09:26:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.855+0000 7f90ad9b6740 10 client.16182 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.856+0000 7f90ad9b6740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.856+0000 7f90ad9b6740 20 client.16182 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.856+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/627612461 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55662ac66fb0 0x55662ac69400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.856+0000 7f90ad9b6740 4 client.16182 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.856+0000 7f90ad9b6740 1 -- 172.21.15.12:0/627612461 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55662abc30c0 con 0x55662ac66fb0 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.856+0000 7f90a3fff640 1 --2- 172.21.15.12:0/627612461 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55662ac66fb0 0x55662ac69400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.857+0000 7f9083fff640 10 client.16182 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.857+0000 7f90a3fff640 1 --2- 172.21.15.12:0/627612461 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55662ac66fb0 0x55662ac69400 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4de0d7989186c0c server_cookie=d3e46c04a350f0a3 in_seq=0 out_seq=0 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.857+0000 7f9083fff640 1 -- 172.21.15.12:0/627612461 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55662abc30c0 con 0x55662ac66fb0 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.857+0000 7f9083fff640 10 client.16182 handle_command_reply: tid=0 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.857+0000 7f90ad9b6740 1 client.16182 shutdown 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.857+0000 7f90ad9b6740 1 -- 172.21.15.12:0/627612461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac51eb0 msgr2=0x55662ac5bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.857+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/627612461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac51eb0 0x55662ac5bec0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x55662ac4ddf0 tx=0x7f90980046c0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.858+0000 7f90ad9b6740 1 -- 172.21.15.12:0/627612461 shutdown_connections 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.858+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/627612461 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55662ac66fb0 0x55662ac69400 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.858+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/627612461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac57020 0x55662ac62f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.858+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/627612461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac51eb0 0x55662ac5bec0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.858+0000 7f9083fff640 0 client.16182 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:47.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.858+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/627612461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac514c0 0x55662ac5b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.858+0000 7f90ad9b6740 1 -- 172.21.15.12:0/627612461 >> 172.21.15.12:0/627612461 conn(0x55662ac190a0 msgr2=0x55662ac4c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:47.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.858+0000 7f90ad9b6740 1 -- 172.21.15.12:0/627612461 shutdown_connections 2024-05-09T09:26:47.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.858+0000 7f90ad9b6740 1 -- 172.21.15.12:0/627612461 wait complete. 2024-05-09T09:26:47.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.858+0000 7f90ad9b6740 20 client.16182 trim_cache size 0 max 16384 2024-05-09T09:26:47.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.889+0000 7f90ad9b6740 1 Processor -- start 2024-05-09T09:26:47.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.889+0000 7f90ad9b6740 1 -- start start 2024-05-09T09:26:47.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90ad9b6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac4fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90ad9b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac57020 0x55662ac62ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90ad9b6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac66fb0 0x55662ac50370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90ad9b6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41ac880 con 0x55662ac66fb0 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90ad9b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41ae3c0 con 0x55662ac57020 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90a8971640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac57020 0x55662ac62ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac4fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90a8971640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac57020 0x55662ac62ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47916/0 (socket says 172.21.15.12:47916) 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90a8971640 1 -- 172.21.15.12:0/2258975433 learned_addr learned my addr 172.21.15.12:0/2258975433 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90ad9b6740 1 -- 172.21.15.12:0/2258975433 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41acf50 con 0x55662ac514c0 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90a3fff640 1 --2- 172.21.15.12:0/2258975433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac66fb0 0x55662ac50370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90a8971640 1 -- 172.21.15.12:0/2258975433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 msgr2=0x55662ac4fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90a8971640 1 --2- 172.21.15.12:0/2258975433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac4fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90a8971640 1 -- 172.21.15.12:0/2258975433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac66fb0 msgr2=0x55662ac50370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.890+0000 7f90a8971640 1 --2- 172.21.15.12:0/2258975433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac66fb0 0x55662ac50370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.891+0000 7f90a8971640 1 -- 172.21.15.12:0/2258975433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55662ac56de0 con 0x55662ac57020 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.891+0000 7f90a3fff640 1 --2- 172.21.15.12:0/2258975433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac4fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.891+0000 7f90a8971640 1 --2- 172.21.15.12:0/2258975433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac57020 0x55662ac62ec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f90980097f0 tx=0x7f9098011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc236808c07c1fe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.891+0000 7f908e7fc640 1 -- 172.21.15.12:0/2258975433 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098028040 con 0x55662ac57020 2024-05-09T09:26:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.891+0000 7f908e7fc640 1 -- 172.21.15.12:0/2258975433 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f909801f040 con 0x55662ac57020 2024-05-09T09:26:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.891+0000 7f908e7fc640 1 -- 172.21.15.12:0/2258975433 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90980193f0 con 0x55662ac57020 2024-05-09T09:26:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.892+0000 7f90ad9b6740 1 -- 172.21.15.12:0/2258975433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac57020 msgr2=0x55662ac62ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.892+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/2258975433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac57020 0x55662ac62ec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f90980097f0 tx=0x7f9098011810 comp rx=0 tx=0).stop 2024-05-09T09:26:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.892+0000 7f90ad9b6740 1 -- 172.21.15.12:0/2258975433 shutdown_connections 2024-05-09T09:26:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.892+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/2258975433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac66fb0 0x55662ac50370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.892+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/2258975433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac57020 0x55662ac62ec0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.892+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/2258975433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac4fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.892+0000 7f90ad9b6740 1 -- 172.21.15.12:0/2258975433 >> 172.21.15.12:0/2258975433 conn(0x55662ac190a0 msgr2=0x55662ac53350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.892+0000 7f90ad9b6740 1 -- 172.21.15.12:0/2258975433 shutdown_connections 2024-05-09T09:26:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.892+0000 7f90ad9b6740 1 -- 172.21.15.12:0/2258975433 wait complete. 2024-05-09T09:26:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.893+0000 7f90ad9b6740 1 -- start start 2024-05-09T09:26:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.893+0000 7f90ad9b6740 1 Processor -- start 2024-05-09T09:26:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90ad9b6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac60e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90ad9b6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac57020 0x55662ac61380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac60e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90ad9b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac66fb0 0x55662ac62320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90ad9b6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41adcf0 con 0x55662ac57020 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90ad9b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41ad620 con 0x55662ac66fb0 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90ad9b6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41aa860 con 0x55662ac514c0 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac60e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:37542/0 (socket says 172.21.15.12:37542) 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90a3fff640 1 -- 172.21.15.12:0/3373634867 learned_addr learned my addr 172.21.15.12:0/3373634867 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90a8971640 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac57020 0x55662ac61380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90ad9b6740 10 client.0 _ll_register_callbacks cb 0x7f9098009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90a8971640 1 -- 172.21.15.12:0/3373634867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 msgr2=0x55662ac60e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90a8971640 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac60e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90a8971640 1 -- 172.21.15.12:0/3373634867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac66fb0 msgr2=0x55662ac62320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.894+0000 7f90a3fff640 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac66fb0 0x55662ac62320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.895+0000 7f90a8971640 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac66fb0 0x55662ac62320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.895+0000 7f90a8971640 1 -- 172.21.15.12:0/3373634867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55662ac62910 con 0x55662ac57020 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.895+0000 7f90a3fff640 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac66fb0 0x55662ac62320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.895+0000 7f90a8971640 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac57020 0x55662ac61380 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9098019a70 tx=0x7f9098004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c16e55b95feed2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.895+0000 7f9083fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:47.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.898+0000 7f9083fff640 1 -- 172.21.15.12:0/3373634867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098028030 con 0x55662ac57020 2024-05-09T09:26:47.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.898+0000 7f90ad9b6740 1 -- 172.21.15.12:0/3373634867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f909801f040 con 0x55662ac57020 2024-05-09T09:26:47.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.898+0000 7f9083fff640 1 -- 172.21.15.12:0/3373634867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f9098019600 con 0x55662ac57020 2024-05-09T09:26:47.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.898+0000 7f9083fff640 1 -- 172.21.15.12:0/3373634867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9098011d10 con 0x55662ac57020 2024-05-09T09:26:47.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.898+0000 7f90ad9b6740 10 client.16188 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:47.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.898+0000 7f90ad9b6740 1 -- 172.21.15.12:0/3373634867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55662accddc0 con 0x55662ac57020 2024-05-09T09:26:47.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.899+0000 7f9083fff640 1 -- 172.21.15.12:0/3373634867 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f909801f040 con 0x55662ac57020 2024-05-09T09:26:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.899+0000 7f9083fff640 1 -- 172.21.15.12:0/3373634867 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9098018030 con 0x55662ac57020 2024-05-09T09:26:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.899+0000 7f90ad9b6740 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.899+0000 7f90ad9b6740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.899+0000 7f90ad9b6740 20 client.16188 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.899+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55662acce570 0x55662acd0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.899+0000 7f90ad9b6740 4 client.16188 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:26:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.899+0000 7f90ad9b6740 1 -- 172.21.15.12:0/3373634867 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55662abbd410 con 0x55662acce570 2024-05-09T09:26:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.899+0000 7f90a3fff640 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55662acce570 0x55662acd0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.900+0000 7f90a3fff640 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55662acce570 0x55662acd0990 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b9e11fcfefe338 server_cookie=87eecded7e6a6621 in_seq=0 out_seq=0 2024-05-09T09:26:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.900+0000 7f9083fff640 10 client.16188 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:47.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f9083fff640 1 -- 172.21.15.12:0/3373634867 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55662abbd410 con 0x55662acce570 2024-05-09T09:26:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f9083fff640 10 client.16188 handle_command_reply: tid=0 2024-05-09T09:26:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f90ad9b6740 1 client.16188 shutdown 2024-05-09T09:26:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f90ad9b6740 1 -- 172.21.15.12:0/3373634867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac57020 msgr2=0x55662ac61380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac57020 0x55662ac61380 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9098019a70 tx=0x7f9098004870 comp rx=0 tx=0).stop 2024-05-09T09:26:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f90ad9b6740 1 -- 172.21.15.12:0/3373634867 shutdown_connections 2024-05-09T09:26:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55662acce570 0x55662acd0990 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55662ac66fb0 0x55662ac62320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f9083fff640 0 client.16188 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55662ac57020 0x55662ac61380 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f90ad9b6740 1 --2- 172.21.15.12:0/3373634867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55662ac514c0 0x55662ac60e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f90ad9b6740 1 -- 172.21.15.12:0/3373634867 >> 172.21.15.12:0/3373634867 conn(0x55662ac190a0 msgr2=0x55662ac4fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f90ad9b6740 1 -- 172.21.15.12:0/3373634867 shutdown_connections 2024-05-09T09:26:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.901+0000 7f90ad9b6740 1 -- 172.21.15.12:0/3373634867 wait complete. 2024-05-09T09:26:47.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.902+0000 7f90ad9b6740 20 client.16188 trim_cache size 0 max 16384 2024-05-09T09:26:47.903 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:26:47.903 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T09:26:47.903 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "c1349654-8bc1-4849-b9cd-83fb883e65e5", 2024-05-09T09:26:47.903 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T09:26:47.903 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:26:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.902+0000 7f90aabd5640 1 -- 172.21.15.12:0/3473478411 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f90940a2c10 msgr2=0x7f90940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.902+0000 7f90aabd5640 1 --2- 172.21.15.12:0/3473478411 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f90940a2c10 0x7f90940a50d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f909c0cc3a0 tx=0x7f909c0d2410 comp rx=0 tx=0).stop 2024-05-09T09:26:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.902+0000 7f90aabd5640 1 -- 172.21.15.12:0/3473478411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4101d40 msgr2=0x7f90a41a13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.902+0000 7f90aabd5640 1 --2- 172.21.15.12:0/3473478411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4101d40 0x7f90a41a13a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f909c00be20 tx=0x7f909c00bf10 comp rx=0 tx=0).stop 2024-05-09T09:26:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.902+0000 7f90aabd5640 1 -- 172.21.15.12:0/3473478411 shutdown_connections 2024-05-09T09:26:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.902+0000 7f90aabd5640 1 --2- 172.21.15.12:0/3473478411 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f90940a2c10 0x7f90940a50d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.902+0000 7f90aabd5640 1 --2- 172.21.15.12:0/3473478411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4101d40 0x7f90a41a13a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.902+0000 7f90aabd5640 1 --2- 172.21.15.12:0/3473478411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a4100280 0x7f90a419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.902+0000 7f90aabd5640 1 --2- 172.21.15.12:0/3473478411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a40ff8c0 0x7f90a4199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.902+0000 7f90aabd5640 1 -- 172.21.15.12:0/3473478411 >> 172.21.15.12:0/3473478411 conn(0x7f90a40fb460 msgr2=0x7f90a40fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.903+0000 7f90aabd5640 1 -- 172.21.15.12:0/3473478411 shutdown_connections 2024-05-09T09:26:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:47.903+0000 7f90aabd5640 1 -- 172.21.15.12:0/3473478411 wait complete. 2024-05-09T09:26:47.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:26:47.914 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:26:47.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:47 smithi012 ceph-mon[30140]: pgmap v404: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-09T09:26:47.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:47 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:26:47.992 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:47 smithi170 ceph-mon[33632]: pgmap v404: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-09T09:26:47.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:47 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:26:48.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:47 smithi016 ceph-mon[33335]: pgmap v404: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-09T09:26:48.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:47 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:26:48.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.081+0000 7f4155161640 1 -- 172.21.15.12:0/181281816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4150102a70 msgr2=0x7f4150102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.081+0000 7f4155161640 1 --2- 172.21.15.12:0/181281816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4150102a70 0x7f4150102ef0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4144009860 tx=0x7f414402a340 comp rx=0 tx=0).stop 2024-05-09T09:26:48.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.081+0000 7f4155161640 1 -- 172.21.15.12:0/181281816 shutdown_connections 2024-05-09T09:26:48.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.081+0000 7f4155161640 1 --2- 172.21.15.12:0/181281816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4150103430 0x7f4150109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.081+0000 7f4155161640 1 --2- 172.21.15.12:0/181281816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4150102a70 0x7f4150102ef0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.081+0000 7f4155161640 1 --2- 172.21.15.12:0/181281816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4150101880 0x7f4150101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.081+0000 7f4155161640 1 -- 172.21.15.12:0/181281816 >> 172.21.15.12:0/181281816 conn(0x7f41500fce10 msgr2=0x7f41500ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:48.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.081+0000 7f4155161640 1 -- 172.21.15.12:0/181281816 shutdown_connections 2024-05-09T09:26:48.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.081+0000 7f4155161640 1 -- 172.21.15.12:0/181281816 wait complete. 2024-05-09T09:26:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.082+0000 7f4155161640 1 Processor -- start 2024-05-09T09:26:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.082+0000 7f4155161640 1 -- start start 2024-05-09T09:26:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.083+0000 7f4155161640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4150101880 0x7f4150197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.083+0000 7f4155161640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4150102a70 0x7f41501980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.083+0000 7f4155161640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4150103430 0x7f415019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.083+0000 7f4155161640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f415010d4e0 con 0x7f4150101880 2024-05-09T09:26:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.083+0000 7f4155161640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f415010d360 con 0x7f4150102a70 2024-05-09T09:26:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.083+0000 7f4155161640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f415010d660 con 0x7f4150103430 2024-05-09T09:26:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.083+0000 7f414ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4150101880 0x7f4150197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.083+0000 7f414f59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4150103430 0x7f415019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.083+0000 7f414e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4150102a70 0x7f41501980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.083+0000 7f414ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4150101880 0x7f4150197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35568/0 (socket says 172.21.15.12:35568) 2024-05-09T09:26:48.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.083+0000 7f414f59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4150103430 0x7f415019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:37556/0 (socket says 172.21.15.12:37556) 2024-05-09T09:26:48.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.083+0000 7f414ed9d640 1 -- 172.21.15.12:0/351997363 learned_addr learned my addr 172.21.15.12:0/351997363 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.084+0000 7f414e59c640 1 -- 172.21.15.12:0/351997363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4150103430 msgr2=0x7f415019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.084+0000 7f414e59c640 1 --2- 172.21.15.12:0/351997363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4150103430 0x7f415019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.084+0000 7f414e59c640 1 -- 172.21.15.12:0/351997363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4150101880 msgr2=0x7f4150197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.084+0000 7f414e59c640 1 --2- 172.21.15.12:0/351997363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4150101880 0x7f4150197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.084+0000 7f414e59c640 1 -- 172.21.15.12:0/351997363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4150073ae0 con 0x7f4150102a70 2024-05-09T09:26:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.084+0000 7f414ed9d640 1 --2- 172.21.15.12:0/351997363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4150101880 0x7f4150197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.084+0000 7f414e59c640 1 --2- 172.21.15.12:0/351997363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4150102a70 0x7f41501980c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f414402a930 tx=0x7f4144032740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.084+0000 7f4137fff640 1 -- 172.21.15.12:0/351997363 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144037040 con 0x7f4150102a70 2024-05-09T09:26:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.084+0000 7f4137fff640 1 -- 172.21.15.12:0/351997363 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4144030820 con 0x7f4150102a70 2024-05-09T09:26:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.084+0000 7f4137fff640 1 -- 172.21.15.12:0/351997363 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144004920 con 0x7f4150102a70 2024-05-09T09:26:48.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.084+0000 7f4155161640 1 -- 172.21.15.12:0/351997363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4150073d70 con 0x7f4150102a70 2024-05-09T09:26:48.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.084+0000 7f4155161640 1 -- 172.21.15.12:0/351997363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4150074150 con 0x7f4150102a70 2024-05-09T09:26:48.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.087+0000 7f4137fff640 1 -- 172.21.15.12:0/351997363 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4144005050 con 0x7f4150102a70 2024-05-09T09:26:48.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.087+0000 7f4137fff640 1 --2- 172.21.15.12:0/351997363 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f41380a2c10 0x7f41380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.087+0000 7f4137fff640 1 -- 172.21.15.12:0/351997363 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f41440ef4e0 con 0x7f4150102a70 2024-05-09T09:26:48.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.088+0000 7f414ed9d640 1 --2- 172.21.15.12:0/351997363 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f41380a2c10 0x7f41380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.088+0000 7f4157e6e740 1 Processor -- start 2024-05-09T09:26:48.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.088+0000 7f4157e6e740 1 -- start start 2024-05-09T09:26:48.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.088+0000 7f4157e6e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b510b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.088+0000 7f4157e6e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b515c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.088+0000 7f4157e6e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 0x56360b518730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.088+0000 7f4157e6e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56360b46cd00 con 0x56360b511130 2024-05-09T09:26:48.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.088+0000 7f4157e6e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56360b4144c0 con 0x56360b510740 2024-05-09T09:26:48.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.088+0000 7f4157e6e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56360b468cc0 con 0x56360b5162a0 2024-05-09T09:26:48.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.088+0000 7f414ed9d640 1 --2- 172.21.15.12:0/351997363 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f41380a2c10 0x7f41380a50d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f41400104e0 tx=0x7f414000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.089+0000 7f414ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b510b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.089+0000 7f414ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b510b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47962/0 (socket says 172.21.15.12:47962) 2024-05-09T09:26:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.089+0000 7f414ed9d640 1 -- 172.21.15.12:0/1024041402 learned_addr learned my addr 172.21.15.12:0/1024041402 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.089+0000 7f414e59c640 1 --2- 172.21.15.12:0/1024041402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b515c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.089+0000 7f414f59e640 1 --2- 172.21.15.12:0/1024041402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 0x56360b518730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.089+0000 7f414ed9d640 1 -- 172.21.15.12:0/1024041402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 msgr2=0x56360b518730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.089+0000 7f414ed9d640 1 --2- 172.21.15.12:0/1024041402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 0x56360b518730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.089+0000 7f414ed9d640 1 -- 172.21.15.12:0/1024041402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 msgr2=0x56360b515c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.089+0000 7f414ed9d640 1 --2- 172.21.15.12:0/1024041402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b515c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.089+0000 7f414ed9d640 1 -- 172.21.15.12:0/1024041402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56360b24a7d0 con 0x56360b510740 2024-05-09T09:26:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.089+0000 7f414e59c640 1 --2- 172.21.15.12:0/1024041402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b515c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.089+0000 7f414ed9d640 1 --2- 172.21.15.12:0/1024041402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b510b60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4140011880 tx=0x7f4140013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6bf3fbf50ea8690 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.090+0000 7f41357fa640 1 -- 172.21.15.12:0/1024041402 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414001c8c0 con 0x56360b510740 2024-05-09T09:26:48.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.090+0000 7f41357fa640 1 -- 172.21.15.12:0/1024041402 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4140027480 con 0x56360b510740 2024-05-09T09:26:48.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.090+0000 7f41357fa640 1 -- 172.21.15.12:0/1024041402 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414001e660 con 0x56360b510740 2024-05-09T09:26:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.090+0000 7f4157e6e740 1 -- 172.21.15.12:0/1024041402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 msgr2=0x56360b510b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.090+0000 7f4157e6e740 1 --2- 172.21.15.12:0/1024041402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b510b60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4140011880 tx=0x7f4140013000 comp rx=0 tx=0).stop 2024-05-09T09:26:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.091+0000 7f4157e6e740 1 -- 172.21.15.12:0/1024041402 shutdown_connections 2024-05-09T09:26:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.091+0000 7f4157e6e740 1 --2- 172.21.15.12:0/1024041402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 0x56360b518730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.091+0000 7f4157e6e740 1 --2- 172.21.15.12:0/1024041402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b515c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.091+0000 7f4157e6e740 1 --2- 172.21.15.12:0/1024041402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b510b60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.091+0000 7f4157e6e740 1 -- 172.21.15.12:0/1024041402 >> 172.21.15.12:0/1024041402 conn(0x56360b50bdb0 msgr2=0x56360b50c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.091+0000 7f4157e6e740 1 -- 172.21.15.12:0/1024041402 shutdown_connections 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.091+0000 7f4157e6e740 1 -- 172.21.15.12:0/1024041402 wait complete. 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.091+0000 7f4157e6e740 1 -- start start 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.091+0000 7f4157e6e740 1 Processor -- start 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f4157e6e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b510740 0x56360b504d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f4157e6e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b511130 0x56360b5052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f4157e6e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b5162a0 0x56360b50a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f414ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b510740 0x56360b504d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f414e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b511130 0x56360b5052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f414e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b511130 0x56360b5052a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47976/0 (socket says 172.21.15.12:47976) 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f414ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b510740 0x56360b504d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:37572/0 (socket says 172.21.15.12:37572) 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f414f59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b5162a0 0x56360b50a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f414f59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b5162a0 0x56360b50a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35592/0 (socket says 172.21.15.12:35592) 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f4157e6e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56360b503350 con 0x56360b5162a0 2024-05-09T09:26:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f4157e6e740 1 -- 172.21.15.12:0/3744477528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56360b427b90 con 0x56360b511130 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f414e59c640 1 -- 172.21.15.12:0/3744477528 learned_addr learned my addr 172.21.15.12:0/3744477528 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f4157e6e740 1 -- 172.21.15.12:0/3744477528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56360b5034d0 con 0x56360b510740 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.092+0000 7f4157e6e740 10 client.0 _ll_register_callbacks cb 0x56360b463da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.093+0000 7f414ed9d640 1 -- 172.21.15.12:0/3744477528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b511130 msgr2=0x56360b5052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.093+0000 7f414ed9d640 1 --2- 172.21.15.12:0/3744477528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b511130 0x56360b5052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.093+0000 7f414ed9d640 1 -- 172.21.15.12:0/3744477528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b5162a0 msgr2=0x56360b50a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.093+0000 7f414ed9d640 1 --2- 172.21.15.12:0/3744477528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b5162a0 0x56360b50a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.093+0000 7f414ed9d640 1 -- 172.21.15.12:0/3744477528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56360b1803f0 con 0x56360b510740 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.093+0000 7f414f59e640 1 --2- 172.21.15.12:0/3744477528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b5162a0 0x56360b50a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.093+0000 7f414ed9d640 1 --2- 172.21.15.12:0/3744477528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b510740 0x56360b504d60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f414001c710 tx=0x7f4140027860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bd333bade876c50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.093+0000 7f4132ff5640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.093+0000 7f4132ff5640 1 -- 172.21.15.12:0/3744477528 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414001c8c0 con 0x56360b510740 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.093+0000 7f4132ff5640 1 -- 172.21.15.12:0/3744477528 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f414001ca60 con 0x56360b510740 2024-05-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.093+0000 7f4157e6e740 1 -- 172.21.15.12:0/3744477528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4140027480 con 0x56360b510740 2024-05-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.094+0000 7f4132ff5640 1 -- 172.21.15.12:0/3744477528 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f414003f050 con 0x56360b510740 2024-05-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.094+0000 7f4157e6e740 10 client.25798 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.094+0000 7f4157e6e740 1 -- 172.21.15.12:0/3744477528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56360b1400f0 con 0x56360b510740 2024-05-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.094+0000 7f4132ff5640 1 -- 172.21.15.12:0/3744477528 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4140027480 con 0x56360b510740 2024-05-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.094+0000 7f4132ff5640 1 -- 172.21.15.12:0/3744477528 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f414003a030 con 0x56360b510740 2024-05-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.094+0000 7f4157e6e740 10 client.25798 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.094+0000 7f4157e6e740 10 client.25798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.094+0000 7f4157e6e740 20 client.25798 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.094+0000 7f4157e6e740 1 --2- 172.21.15.12:0/3744477528 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56360b52ee70 0x56360b531260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.094+0000 7f4157e6e740 4 client.25798 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.094+0000 7f4157e6e740 1 -- 172.21.15.12:0/3744477528 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56360b47a430 con 0x56360b52ee70 2024-05-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.095+0000 7f414e59c640 1 --2- 172.21.15.12:0/3744477528 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56360b52ee70 0x56360b531260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.095+0000 7f4132ff5640 10 client.25798 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.095+0000 7f414e59c640 1 --2- 172.21.15.12:0/3744477528 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56360b52ee70 0x56360b531260 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd8d19393ebb10d server_cookie=a9c7a6b33e068415 in_seq=0 out_seq=0 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4132ff5640 1 -- 172.21.15.12:0/3744477528 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56360b47a430 con 0x56360b52ee70 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4132ff5640 10 client.25798 handle_command_reply: tid=0 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4157e6e740 1 client.25798 shutdown 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4157e6e740 1 -- 172.21.15.12:0/3744477528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b510740 msgr2=0x56360b504d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4157e6e740 1 --2- 172.21.15.12:0/3744477528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b510740 0x56360b504d60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f414001c710 tx=0x7f4140027860 comp rx=0 tx=0).stop 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4157e6e740 1 -- 172.21.15.12:0/3744477528 shutdown_connections 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4157e6e740 1 --2- 172.21.15.12:0/3744477528 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56360b52ee70 0x56360b531260 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4157e6e740 1 --2- 172.21.15.12:0/3744477528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b5162a0 0x56360b50a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4157e6e740 1 --2- 172.21.15.12:0/3744477528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b511130 0x56360b5052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4157e6e740 1 --2- 172.21.15.12:0/3744477528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b510740 0x56360b504d60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4157e6e740 1 -- 172.21.15.12:0/3744477528 >> 172.21.15.12:0/3744477528 conn(0x56360b50bdb0 msgr2=0x56360b5027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4132ff5640 0 client.25798 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4157e6e740 1 -- 172.21.15.12:0/3744477528 shutdown_connections 2024-05-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4157e6e740 1 -- 172.21.15.12:0/3744477528 wait complete. 2024-05-09T09:26:48.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.096+0000 7f4157e6e740 20 client.25798 trim_cache size 0 max 16384 2024-05-09T09:26:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.114+0000 7f4157e6e740 1 Processor -- start 2024-05-09T09:26:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.114+0000 7f4157e6e740 1 -- start start 2024-05-09T09:26:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.114+0000 7f4157e6e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b503870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.114+0000 7f4157e6e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b503db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.114+0000 7f4157e6e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 0x56360b512670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.114+0000 7f4157e6e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f41501a83d0 con 0x56360b511130 2024-05-09T09:26:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.114+0000 7f4157e6e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41501a9f10 con 0x56360b510740 2024-05-09T09:26:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.114+0000 7f4157e6e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f41501a8aa0 con 0x56360b5162a0 2024-05-09T09:26:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.114+0000 7f414ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b503870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.114+0000 7f414ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b503870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47988/0 (socket says 172.21.15.12:47988) 2024-05-09T09:26:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.114+0000 7f414ed9d640 1 -- 172.21.15.12:0/4143006393 learned_addr learned my addr 172.21.15.12:0/4143006393 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.114+0000 7f414f59e640 1 --2- 172.21.15.12:0/4143006393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 0x56360b512670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.115+0000 7f414e59c640 1 --2- 172.21.15.12:0/4143006393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b503db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.115+0000 7f414ed9d640 1 -- 172.21.15.12:0/4143006393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 msgr2=0x56360b512670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.115+0000 7f414ed9d640 1 --2- 172.21.15.12:0/4143006393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 0x56360b512670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.115+0000 7f414ed9d640 1 -- 172.21.15.12:0/4143006393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 msgr2=0x56360b503db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.115+0000 7f414ed9d640 1 --2- 172.21.15.12:0/4143006393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b503db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.115+0000 7f414ed9d640 1 -- 172.21.15.12:0/4143006393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56360b50ca80 con 0x56360b510740 2024-05-09T09:26:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.115+0000 7f414ed9d640 1 --2- 172.21.15.12:0/4143006393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b503870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4140011600 tx=0x7f4140032610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a29d9fe42a6497e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.115+0000 7f4134ff9640 1 -- 172.21.15.12:0/4143006393 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4140032710 con 0x56360b510740 2024-05-09T09:26:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.116+0000 7f4134ff9640 1 -- 172.21.15.12:0/4143006393 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4140004020 con 0x56360b510740 2024-05-09T09:26:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.116+0000 7f4134ff9640 1 -- 172.21.15.12:0/4143006393 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414001cd00 con 0x56360b510740 2024-05-09T09:26:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.117+0000 7f4157e6e740 1 -- 172.21.15.12:0/4143006393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 msgr2=0x56360b503870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.117+0000 7f4157e6e740 1 --2- 172.21.15.12:0/4143006393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b503870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4140011600 tx=0x7f4140032610 comp rx=0 tx=0).stop 2024-05-09T09:26:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.117+0000 7f4157e6e740 1 -- 172.21.15.12:0/4143006393 shutdown_connections 2024-05-09T09:26:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.117+0000 7f4157e6e740 1 --2- 172.21.15.12:0/4143006393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 0x56360b512670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.117+0000 7f4157e6e740 1 --2- 172.21.15.12:0/4143006393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b503db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.117+0000 7f4157e6e740 1 --2- 172.21.15.12:0/4143006393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b503870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.117+0000 7f4157e6e740 1 -- 172.21.15.12:0/4143006393 >> 172.21.15.12:0/4143006393 conn(0x56360b50bdb0 msgr2=0x56360b511a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.118+0000 7f4157e6e740 1 -- 172.21.15.12:0/4143006393 shutdown_connections 2024-05-09T09:26:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.118+0000 7f4157e6e740 1 -- 172.21.15.12:0/4143006393 wait complete. 2024-05-09T09:26:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.118+0000 7f4157e6e740 1 -- start start 2024-05-09T09:26:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.118+0000 7f4157e6e740 1 Processor -- start 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.119+0000 7f4157e6e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b542640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.119+0000 7f4157e6e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b53f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.119+0000 7f414ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b542640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.119+0000 7f4157e6e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 0x56360b53f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.119+0000 7f414ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b542640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:47996/0 (socket says 172.21.15.12:47996) 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.119+0000 7f414e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b53f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.119+0000 7f414e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b53f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35612/0 (socket says 172.21.15.12:35612) 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.119+0000 7f4157e6e740 1 -- 172.21.15.12:0/273136105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f41501a9840 con 0x56360b511130 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.119+0000 7f414ed9d640 1 -- 172.21.15.12:0/273136105 learned_addr learned my addr 172.21.15.12:0/273136105 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.119+0000 7f4157e6e740 1 -- 172.21.15.12:0/273136105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41501a9170 con 0x56360b510740 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.119+0000 7f4157e6e740 1 -- 172.21.15.12:0/273136105 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f41501a63b0 con 0x56360b5162a0 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.119+0000 7f414f59e640 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 0x56360b53f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.119+0000 7f4157e6e740 10 client.0 _ll_register_callbacks cb 0x7f4140011780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f414e59c640 1 -- 172.21.15.12:0/273136105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 msgr2=0x56360b53f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f414e59c640 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 0x56360b53f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f414e59c640 1 -- 172.21.15.12:0/273136105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 msgr2=0x56360b542640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f414e59c640 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b542640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f414e59c640 1 -- 172.21.15.12:0/273136105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56360b592eb0 con 0x56360b511130 2024-05-09T09:26:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f414f59e640 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 0x56360b53f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f414ed9d640 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b542640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f414e59c640 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b53f130 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f414402a860 tx=0x7f41440bb330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe1e13e6d8ecbfb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f4132ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f4132ff5640 1 -- 172.21.15.12:0/273136105 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41440d3070 con 0x56360b511130 2024-05-09T09:26:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f4132ff5640 1 -- 172.21.15.12:0/273136105 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f41440eccc0 con 0x56360b511130 2024-05-09T09:26:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f4157e6e740 1 -- 172.21.15.12:0/273136105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4140004020 con 0x56360b511130 2024-05-09T09:26:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f4132ff5640 1 -- 172.21.15.12:0/273136105 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f41440bfcc0 con 0x56360b511130 2024-05-09T09:26:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f4157e6e740 10 client.16206 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:48.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.120+0000 7f4157e6e740 1 -- 172.21.15.12:0/273136105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56360b594210 con 0x56360b511130 2024-05-09T09:26:48.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.121+0000 7f4132ff5640 1 -- 172.21.15.12:0/273136105 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f41440c4580 con 0x56360b511130 2024-05-09T09:26:48.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.121+0000 7f4132ff5640 1 -- 172.21.15.12:0/273136105 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f41440c40f0 con 0x56360b511130 2024-05-09T09:26:48.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.122+0000 7f4157e6e740 10 client.16206 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:48.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.122+0000 7f4157e6e740 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:48.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.122+0000 7f4157e6e740 20 client.16206 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:48.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.122+0000 7f4157e6e740 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56360b594bc0 0x56360b596fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.122+0000 7f4157e6e740 4 client.16206 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.122+0000 7f4157e6e740 1 -- 172.21.15.12:0/273136105 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f4140004020 con 0x56360b594bc0 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.122+0000 7f414ed9d640 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56360b594bc0 0x56360b596fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.122+0000 7f4132ff5640 10 client.16206 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.122+0000 7f414ed9d640 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56360b594bc0 0x56360b596fe0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c3fc86d2e1239d server_cookie=8f0a7c5136a7c0ae in_seq=0 out_seq=0 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4132ff5640 1 -- 172.21.15.12:0/273136105 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f4140004020 con 0x56360b594bc0 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4132ff5640 10 client.16206 handle_command_reply: tid=0 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4157e6e740 1 client.16206 shutdown 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4157e6e740 1 -- 172.21.15.12:0/273136105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 msgr2=0x56360b53f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4157e6e740 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b53f130 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f414402a860 tx=0x7f41440bb330 comp rx=0 tx=0).stop 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4157e6e740 1 -- 172.21.15.12:0/273136105 shutdown_connections 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4157e6e740 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56360b594bc0 0x56360b596fe0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4157e6e740 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56360b5162a0 0x56360b53f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4157e6e740 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360b511130 0x56360b53f130 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4157e6e740 1 --2- 172.21.15.12:0/273136105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360b510740 0x56360b542640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4132ff5640 0 client.16206 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:48.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4157e6e740 1 -- 172.21.15.12:0/273136105 >> 172.21.15.12:0/273136105 conn(0x56360b50bdb0 msgr2=0x56360b503870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:48.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4157e6e740 1 -- 172.21.15.12:0/273136105 shutdown_connections 2024-05-09T09:26:48.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4157e6e740 1 -- 172.21.15.12:0/273136105 wait complete. 2024-05-09T09:26:48.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.123+0000 7f4157e6e740 20 client.16206 trim_cache size 0 max 16384 2024-05-09T09:26:48.126 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:26:48.126 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:26:48.127 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T09:26:48.127 INFO:teuthology.orchestra.run.smithi012.stdout: "c1349654-8bc1-4849-b9cd-83fb883e65e5": { 2024-05-09T09:26:48.127 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T09:26:48.127 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "c1349654-8bc1-4849-b9cd-83fb883e65e5", 2024-05-09T09:26:48.127 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T09:26:48.127 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:26:48.127 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:26:48.127 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:26:48.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.124+0000 7f4155161640 1 -- 172.21.15.12:0/351997363 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f41380a2c10 msgr2=0x7f41380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.124+0000 7f4155161640 1 --2- 172.21.15.12:0/351997363 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f41380a2c10 0x7f41380a50d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f41400104e0 tx=0x7f414000b410 comp rx=0 tx=0).stop 2024-05-09T09:26:48.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.124+0000 7f4155161640 1 -- 172.21.15.12:0/351997363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4150102a70 msgr2=0x7f41501980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.124+0000 7f4155161640 1 --2- 172.21.15.12:0/351997363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4150102a70 0x7f41501980c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f414402a930 tx=0x7f4144032740 comp rx=0 tx=0).stop 2024-05-09T09:26:48.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.124+0000 7f4155161640 1 -- 172.21.15.12:0/351997363 shutdown_connections 2024-05-09T09:26:48.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.124+0000 7f4155161640 1 --2- 172.21.15.12:0/351997363 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f41380a2c10 0x7f41380a50d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.124+0000 7f4155161640 1 --2- 172.21.15.12:0/351997363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4150103430 0x7f415019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.124+0000 7f4155161640 1 --2- 172.21.15.12:0/351997363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4150102a70 0x7f41501980c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.124+0000 7f4155161640 1 --2- 172.21.15.12:0/351997363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4150101880 0x7f4150197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.124+0000 7f4155161640 1 -- 172.21.15.12:0/351997363 >> 172.21.15.12:0/351997363 conn(0x7f41500fce10 msgr2=0x7f4150105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:48.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.125+0000 7f4155161640 1 -- 172.21.15.12:0/351997363 shutdown_connections 2024-05-09T09:26:48.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:26:48.125+0000 7f4155161640 1 -- 172.21.15.12:0/351997363 wait complete. 2024-05-09T09:26:48.140 INFO:tasks.cephfs.filesystem:scrub status for tag:c1349654-8bc1-4849-b9cd-83fb883e65e5 - {'path': '/', 'tag': 'c1349654-8bc1-4849-b9cd-83fb883e65e5', 'options': 'recursive,force'} 2024-05-09T09:26:49.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:48 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:48 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:48 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:48 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:48 smithi012 ceph-mon[30140]: scrub queued for path: / 2024-05-09T09:26:49.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:48 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:49.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:48 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:26:49.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:48 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:49.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:48 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:48 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:48 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:26:49.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:48 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:26:49.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:48 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:48 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:48 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:48 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:48 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:48 smithi170 ceph-mon[33632]: scrub queued for path: / 2024-05-09T09:26:49.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:48 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:49.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:48 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:26:49.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:48 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:49.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:48 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:48 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:48 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:26:49.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:48 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:26:49.244 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:48 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:48 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:48 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:48 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:48 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:48 smithi016 ceph-mon[33335]: scrub queued for path: / 2024-05-09T09:26:49.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:48 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:49.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:48 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:26:49.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:48 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:26:49.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:48 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:48 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:49.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:48 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:26:49.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:48 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:26:49.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:48 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:26:50.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:49 smithi012 ceph-mon[30140]: pgmap v405: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T09:26:50.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:49 smithi170 ceph-mon[33632]: pgmap v405: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T09:26:50.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:49 smithi016 ceph-mon[33335]: pgmap v405: 129 pgs: 129 active+clean; 666 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T09:26:51.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:50 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:26:51.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:50 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:26:51.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:50 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:26:52.067 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 95% (11975/12535) Updating files: 96% (12034/12535) Updating files: 97% (12159/12535) Updating files: 98% (12285/12535) Updating files: 99% (12410/12535) Updating files: 100% (12535/12535) Updating files: 100% (12535/12535), done. 2024-05-09T09:26:52.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:51 smithi012 ceph-mon[30140]: pgmap v406: 129 pgs: 129 active+clean; 667 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-09T09:26:52.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:51 smithi170 ceph-mon[33632]: pgmap v406: 129 pgs: 129 active+clean; 667 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-09T09:26:52.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:51 smithi016 ceph-mon[33335]: pgmap v406: 129 pgs: 129 active+clean; 667 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-09T09:26:52.317 INFO:tasks.workunit.client.0.smithi012.stderr:Note: switching to 'be85c202969d25af6e130678e7d73a20e9c4ca11'. 2024-05-09T09:26:52.317 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-09T09:26:52.317 INFO:tasks.workunit.client.0.smithi012.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T09:26:52.317 INFO:tasks.workunit.client.0.smithi012.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T09:26:52.317 INFO:tasks.workunit.client.0.smithi012.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T09:26:52.317 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-09T09:26:52.317 INFO:tasks.workunit.client.0.smithi012.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T09:26:52.318 INFO:tasks.workunit.client.0.smithi012.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T09:26:52.318 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-09T09:26:52.318 INFO:tasks.workunit.client.0.smithi012.stderr: git switch -c 2024-05-09T09:26:52.318 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-09T09:26:52.318 INFO:tasks.workunit.client.0.smithi012.stderr:Or undo this operation with: 2024-05-09T09:26:52.318 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-09T09:26:52.318 INFO:tasks.workunit.client.0.smithi012.stderr: git switch - 2024-05-09T09:26:52.318 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-09T09:26:52.318 INFO:tasks.workunit.client.0.smithi012.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T09:26:52.318 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-09T09:26:52.318 INFO:tasks.workunit.client.0.smithi012.stderr:HEAD is now at be85c202969 Merge pull request #57242 from cbodley/wip-65767-squid 2024-05-09T09:26:52.321 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T09:26:52.353 INFO:tasks.workunit.client.0.smithi012.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T09:26:52.355 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T09:26:52.355 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T09:26:52.998 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T09:26:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T09:26:53.093 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T09:26:53.095 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T09:26:53.095 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T09:26:53.135 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T09:26:53.139 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:26:53.139 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T09:26:53.195 INFO:tasks.workunit:Running workunits matching fs/norstats on client.0... 2024-05-09T09:26:53.196 INFO:tasks.workunit:Running workunit fs/norstats/kernel_untar_tar.sh... 2024-05-09T09:26:53.197 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/norstats/kernel_untar_tar.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=be85c202969d25af6e130678e7d73a20e9c4ca11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh 2024-05-09T09:26:53.262 INFO:tasks.workunit.client.0.smithi012.stderr:+ KERNEL=linux-4.0.5 2024-05-09T09:26:53.262 INFO:tasks.workunit.client.0.smithi012.stderr:+ wget -q http://download.ceph.com/qa/linux-4.0.5.tar.xz 2024-05-09T09:26:54.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:53 smithi012 ceph-mon[30140]: pgmap v407: 129 pgs: 129 active+clean; 667 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:54.516 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:53 smithi016 ceph-mon[33335]: pgmap v407: 129 pgs: 129 active+clean; 667 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:54.516 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:53 smithi170 ceph-mon[33632]: pgmap v407: 129 pgs: 129 active+clean; 667 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T09:26:56.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:55 smithi012 ceph-mon[30140]: pgmap v408: 129 pgs: 129 active+clean; 8.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 684 KiB/s wr, 14 op/s 2024-05-09T09:26:56.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:55 smithi170 ceph-mon[33632]: pgmap v408: 129 pgs: 129 active+clean; 8.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 684 KiB/s wr, 14 op/s 2024-05-09T09:26:56.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:55 smithi016 ceph-mon[33335]: pgmap v408: 129 pgs: 129 active+clean; 8.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 684 KiB/s wr, 14 op/s 2024-05-09T09:26:56.392 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir untar_tar 2024-05-09T09:26:56.395 INFO:tasks.workunit.client.0.smithi012.stderr:+ cd untar_tar 2024-05-09T09:26:56.397 INFO:tasks.workunit.client.0.smithi012.stderr:+ tar Jxvf ../linux-4.0.5.tar.xz linux-4.0.5/Documentation/ 2024-05-09T09:26:56.417 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ 2024-05-09T09:26:56.420 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/00-INDEX 2024-05-09T09:26:56.423 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/ 2024-05-09T09:26:56.424 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/README 2024-05-09T09:26:56.426 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/obsolete/ 2024-05-09T09:26:56.428 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/obsolete/proc-sys-vm-nr_pdflush_threads 2024-05-09T09:26:56.431 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-bus-usb 2024-05-09T09:26:56.433 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-class-rfkill 2024-05-09T09:26:56.436 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-koneplus 2024-05-09T09:26:56.438 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-kovaplus 2024-05-09T09:26:56.440 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra 2024-05-09T09:26:56.443 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/removed/ 2024-05-09T09:26:56.445 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/removed/devfs 2024-05-09T09:26:56.447 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/removed/dv1394 2024-05-09T09:26:56.450 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/removed/ip_queue 2024-05-09T09:26:56.452 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/removed/net_dma 2024-05-09T09:26:56.455 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/removed/o2cb 2024-05-09T09:26:56.457 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/removed/raw1394 2024-05-09T09:26:56.459 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/removed/video1394 2024-05-09T09:26:56.462 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/ 2024-05-09T09:26:56.464 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/firewire-cdev 2024-05-09T09:26:56.466 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/o2cb 2024-05-09T09:26:56.469 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/syscalls 2024-05-09T09:26:56.471 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-acpi-pmprofile 2024-05-09T09:26:56.474 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-firewire 2024-05-09T09:26:56.477 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-usb 2024-05-09T09:26:56.479 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-xen-backend 2024-05-09T09:26:56.482 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-backlight 2024-05-09T09:26:56.484 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-rfkill 2024-05-09T09:26:56.487 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-tpm 2024-05-09T09:26:56.490 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-ubi 2024-05-09T09:26:56.492 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-udc 2024-05-09T09:26:56.495 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-node 2024-05-09T09:26:56.498 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-cpu 2024-05-09T09:26:56.500 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-xen_memory 2024-05-09T09:26:56.503 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-ib_srp 2024-05-09T09:26:56.506 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-qla2xxx 2024-05-09T09:26:56.508 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-usb-usbtmc 2024-05-09T09:26:56.511 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-w1_ds28e04 2024-05-09T09:26:56.513 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-efi-vars 2024-05-09T09:26:56.516 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-dump 2024-05-09T09:26:56.519 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-elog 2024-05-09T09:26:56.522 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-module 2024-05-09T09:26:56.525 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-transport-srp 2024-05-09T09:26:56.527 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/thermal-notification 2024-05-09T09:26:56.530 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/stable/vdso 2024-05-09T09:26:56.532 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/ 2024-05-09T09:26:56.535 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-spear-pcie-gadget 2024-05-09T09:26:56.537 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget 2024-05-09T09:26:56.539 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-acm 2024-05-09T09:26:56.541 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ecm 2024-05-09T09:26:56.544 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-eem 2024-05-09T09:26:56.546 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ffs 2024-05-09T09:26:56.548 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-hid 2024-05-09T09:26:56.551 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-loopback 2024-05-09T09:26:56.553 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-mass-storage 2024-05-09T09:26:56.555 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-midi 2024-05-09T09:26:56.557 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ncm 2024-05-09T09:26:56.559 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-obex 2024-05-09T09:26:56.561 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-phonet 2024-05-09T09:26:56.564 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-rndis 2024-05-09T09:26:56.566 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-serial 2024-05-09T09:26:56.568 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-sourcesink 2024-05-09T09:26:56.570 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-subset 2024-05-09T09:26:56.573 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac1 2024-05-09T09:26:56.575 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac2 2024-05-09T09:26:56.577 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uvc 2024-05-09T09:26:56.580 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-driver-genwqe 2024-05-09T09:26:56.582 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ec 2024-05-09T09:26:56.584 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ideapad 2024-05-09T09:26:56.586 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-olpc 2024-05-09T09:26:56.589 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pfo-nx-crypto 2024-05-09T09:26:56.591 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pktcdvd 2024-05-09T09:26:56.593 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/dev-kmsg 2024-05-09T09:26:56.595 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/evm 2024-05-09T09:26:56.598 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/ima_policy 2024-05-09T09:26:56.601 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/procfs-diskstats 2024-05-09T09:26:56.603 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/pstore 2024-05-09T09:26:56.605 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ata 2024-05-09T09:26:56.608 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block 2024-05-09T09:26:56.610 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-bcache 2024-05-09T09:26:56.613 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-dm 2024-05-09T09:26:56.615 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-rssd 2024-05-09T09:26:56.618 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-zram 2024-05-09T09:26:56.620 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-acpi 2024-05-09T09:26:56.623 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-amba 2024-05-09T09:26:56.625 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-bcma 2024-05-09T09:26:56.627 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10 2024-05-09T09:26:56.630 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm3x 2024-05-09T09:26:56.632 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-funnel 2024-05-09T09:26:56.634 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc 2024-05-09T09:26:56.637 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-css 2024-05-09T09:26:56.639 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-events 2024-05-09T09:26:56.642 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-format 2024-05-09T09:26:56.646 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7 2024-05-09T09:26:56.647 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_gpci 2024-05-09T09:26:56.649 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-fcoe 2024-05-09T09:26:56.651 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-hsi 2024-05-09T09:26:56.653 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-fsa9480 2024-05-09T09:26:56.656 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 2024-05-09T09:26:56.658 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-lm3533 2024-05-09T09:26:56.660 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio 2024-05-09T09:26:56.664 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-accel-bmc150 2024-05-09T09:26:56.666 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-ad9523 2024-05-09T09:26:56.669 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-adf4350 2024-05-09T09:26:56.671 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-gyro-bmg160 2024-05-09T09:26:56.674 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-light-lm3533-als 2024-05-09T09:26:56.676 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-mpu6050 2024-05-09T09:26:56.679 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 2024-05-09T09:26:56.681 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-trigger-sysfs 2024-05-09T09:26:56.684 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mdio 2024-05-09T09:26:56.686 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-media 2024-05-09T09:26:56.689 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mei 2024-05-09T09:26:56.692 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci 2024-05-09T09:26:56.694 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss 2024-05-09T09:26:56.697 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-drivers-ehci_hcd 2024-05-09T09:26:56.699 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-platform 2024-05-09T09:26:56.702 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rbd 2024-05-09T09:26:56.704 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rpmsg 2024-05-09T09:26:56.706 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-umc 2024-05-09T09:26:56.708 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb 2024-05-09T09:26:56.711 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg 2024-05-09T09:26:56.714 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-lvstest 2024-05-09T09:26:56.716 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-c2port 2024-05-09T09:26:56.718 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-cfq-target-latency 2024-05-09T09:26:56.720 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class 2024-05-09T09:26:56.722 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-adp8870 2024-05-09T09:26:56.725 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533 2024-05-09T09:26:56.727 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-bdi 2024-05-09T09:26:56.729 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-cxl 2024-05-09T09:26:56.732 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-devfreq 2024-05-09T09:26:56.734 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-extcon 2024-05-09T09:26:56.737 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu 2024-05-09T09:26:56.740 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-amd-iommu 2024-05-09T09:26:56.742 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-intel-iommu 2024-05-09T09:26:56.744 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-lcd 2024-05-09T09:26:56.747 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led 2024-05-09T09:26:56.749 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led-driver-lm3533 2024-05-09T09:26:56.751 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-leds-gt683r 2024-05-09T09:26:56.754 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mei 2024-05-09T09:26:56.756 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mic.txt 2024-05-09T09:26:56.759 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mtd 2024-05-09T09:26:56.761 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net 2024-05-09T09:26:56.764 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-batman-adv 2024-05-09T09:26:56.766 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-cdc_ncm 2024-05-09T09:26:56.769 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-grcan 2024-05-09T09:26:56.772 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-mesh 2024-05-09T09:26:56.774 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-queues 2024-05-09T09:26:56.776 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-statistics 2024-05-09T09:26:56.779 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pktcdvd 2024-05-09T09:26:56.781 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-power 2024-05-09T09:26:56.783 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-powercap 2024-05-09T09:26:56.785 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pwm 2024-05-09T09:26:56.788 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rc 2024-05-09T09:26:56.790 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-regulator 2024-05-09T09:26:56.793 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rtc-rtc0-device-rtc_calibration 2024-05-09T09:26:56.795 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-scsi_host 2024-05-09T09:26:56.798 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc 2024-05-09T09:26:56.800 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc-wusbhc 2024-05-09T09:26:56.803 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-dev 2024-05-09T09:26:56.806 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices 2024-05-09T09:26:56.808 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-edac 2024-05-09T09:26:56.811 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-firmware_node 2024-05-09T09:26:56.814 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-lpss_ltr 2024-05-09T09:26:56.816 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-memory 2024-05-09T09:26:56.819 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-mmc 2024-05-09T09:26:56.821 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-online 2024-05-09T09:26:56.823 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-_UDC_-gadget 2024-05-09T09:26:56.826 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-docg3 2024-05-09T09:26:56.828 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-sh_mobile_lcdc_fb 2024-05-09T09:26:56.830 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power 2024-05-09T09:26:56.833 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D0 2024-05-09T09:26:56.835 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D1 2024-05-09T09:26:56.838 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D2 2024-05-09T09:26:56.840 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D3hot 2024-05-09T09:26:56.842 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_wakeup 2024-05-09T09:26:56.845 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_state 2024-05-09T09:26:56.847 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-real_power_state 2024-05-09T09:26:56.849 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-resource_in_use 2024-05-09T09:26:56.852 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-soc 2024-05-09T09:26:56.854 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-sun 2024-05-09T09:26:56.856 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-cpu 2024-05-09T09:26:56.859 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-ibm-rtl 2024-05-09T09:26:56.861 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-xen_cpu 2024-05-09T09:26:56.863 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-genwqe 2024-05-09T09:26:56.866 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid 2024-05-09T09:26:56.868 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-lenovo 2024-05-09T09:26:56.871 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-logitech-lg4ff 2024-05-09T09:26:56.873 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-multitouch 2024-05-09T09:26:56.875 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-picolcd 2024-05-09T09:26:56.878 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-prodikeys 2024-05-09T09:26:56.880 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-arvo 2024-05-09T09:26:56.883 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-isku 2024-05-09T09:26:56.885 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone 2024-05-09T09:26:56.888 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-koneplus 2024-05-09T09:26:56.890 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-konepure 2024-05-09T09:26:56.892 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kovaplus 2024-05-09T09:26:56.895 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-lua 2024-05-09T09:26:56.897 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-pyra 2024-05-09T09:26:58.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:57 smithi012 ceph-mon[30140]: pgmap v409: 129 pgs: 129 active+clean; 21 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.7 MiB/s wr, 13 op/s 2024-05-09T09:26:58.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:57 smithi170 ceph-mon[33632]: pgmap v409: 129 pgs: 129 active+clean; 21 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.7 MiB/s wr, 13 op/s 2024-05-09T09:26:58.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:57 smithi016 ceph-mon[33335]: pgmap v409: 129 pgs: 129 active+clean; 21 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.7 MiB/s wr, 13 op/s 2024-05-09T09:26:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.477+0000 7f5a3915a640 1 -- 172.21.15.12:0/3280332187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34107620 msgr2=0x7f5a34109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.477+0000 7f5a3915a640 1 --2- 172.21.15.12:0/3280332187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34107620 0x7f5a34109a70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009ca0 tx=0x7f5a2c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.477+0000 7f5a3915a640 1 -- 172.21.15.12:0/3280332187 shutdown_connections 2024-05-09T09:26:58.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.477+0000 7f5a3915a640 1 --2- 172.21.15.12:0/3280332187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34107620 0x7f5a34109a70 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.477+0000 7f5a3915a640 1 --2- 172.21.15.12:0/3280332187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a34104cf0 0x7f5a341070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.477+0000 7f5a3915a640 1 --2- 172.21.15.12:0/3280332187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a341023c0 0x7f5a341047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.477+0000 7f5a3915a640 1 -- 172.21.15.12:0/3280332187 >> 172.21.15.12:0/3280332187 conn(0x7f5a340fbdb0 msgr2=0x7f5a340fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:58.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.477+0000 7f5a3915a640 1 -- 172.21.15.12:0/3280332187 shutdown_connections 2024-05-09T09:26:58.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.477+0000 7f5a3915a640 1 -- 172.21.15.12:0/3280332187 wait complete. 2024-05-09T09:26:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.478+0000 7f5a3915a640 1 Processor -- start 2024-05-09T09:26:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3915a640 1 -- start start 2024-05-09T09:26:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3915a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a341023c0 0x7f5a34101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3915a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34104cf0 0x7f5a34100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3915a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a34107620 0x7f5a341008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3915a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3410d5a0 con 0x7f5a34104cf0 2024-05-09T09:26:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3915a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3410d420 con 0x7f5a34107620 2024-05-09T09:26:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3915a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3410d720 con 0x7f5a341023c0 2024-05-09T09:26:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34104cf0 0x7f5a34100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34104cf0 0x7f5a34100370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56632/0 (socket says 172.21.15.12:56632) 2024-05-09T09:26:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3259c640 1 -- 172.21.15.12:0/1713814409 learned_addr learned my addr 172.21.15.12:0/1713814409 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a32d9d640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a341023c0 0x7f5a34101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3359e640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a34107620 0x7f5a341008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3259c640 1 -- 172.21.15.12:0/1713814409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a341023c0 msgr2=0x7f5a34101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3259c640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a341023c0 0x7f5a34101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3259c640 1 -- 172.21.15.12:0/1713814409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a34107620 msgr2=0x7f5a341008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3259c640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a34107620 0x7f5a341008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3259c640 1 -- 172.21.15.12:0/1713814409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a34101140 con 0x7f5a34104cf0 2024-05-09T09:26:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.479+0000 7f5a3359e640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a34107620 0x7f5a341008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.480+0000 7f5a3259c640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34104cf0 0x7f5a34100370 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00e9f0 tx=0x7f5a1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.480+0000 7f5a2bfff640 1 -- 172.21.15.12:0/1713814409 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c00ccc0 con 0x7f5a34104cf0 2024-05-09T09:26:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.480+0000 7f5a3915a640 1 -- 172.21.15.12:0/1713814409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a341a1bc0 con 0x7f5a34104cf0 2024-05-09T09:26:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.480+0000 7f5a2bfff640 1 -- 172.21.15.12:0/1713814409 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a1c0054f0 con 0x7f5a34104cf0 2024-05-09T09:26:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.480+0000 7f5a2bfff640 1 -- 172.21.15.12:0/1713814409 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c010690 con 0x7f5a34104cf0 2024-05-09T09:26:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.480+0000 7f5a3915a640 1 -- 172.21.15.12:0/1713814409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a341a20c0 con 0x7f5a34104cf0 2024-05-09T09:26:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.482+0000 7f5a3be65740 1 Processor -- start 2024-05-09T09:26:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.482+0000 7f5a2bfff640 1 -- 172.21.15.12:0/1713814409 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5a1c004e80 con 0x7f5a34104cf0 2024-05-09T09:26:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.482+0000 7f5a3be65740 1 -- start start 2024-05-09T09:26:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a3be65740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d2511b250 0x560d2511b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a3be65740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d25120740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a3be65740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d25120db0 0x560d25123240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a3be65740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560d2507f340 con 0x560d2511bc40 2024-05-09T09:26:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a3be65740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d25027290 con 0x560d25120db0 2024-05-09T09:26:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a3be65740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560d2507b300 con 0x560d2511b250 2024-05-09T09:26:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a32d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d25120740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a32d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d25120740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56646/0 (socket says 172.21.15.12:56646) 2024-05-09T09:26:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a32d9d640 1 -- 172.21.15.12:0/3646602972 learned_addr learned my addr 172.21.15.12:0/3646602972 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a3359e640 1 --2- 172.21.15.12:0/3646602972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d2511b250 0x560d2511b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a3259c640 1 --2- 172.21.15.12:0/3646602972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d25120db0 0x560d25123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a32d9d640 1 -- 172.21.15.12:0/3646602972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d2511b250 msgr2=0x560d2511b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a32d9d640 1 --2- 172.21.15.12:0/3646602972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d2511b250 0x560d2511b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a32d9d640 1 -- 172.21.15.12:0/3646602972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d25120db0 msgr2=0x560d25123240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a32d9d640 1 --2- 172.21.15.12:0/3646602972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d25120db0 0x560d25123240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a32d9d640 1 -- 172.21.15.12:0/3646602972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d2504b8e0 con 0x560d2511bc40 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.483+0000 7f5a2bfff640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5a200a2c30 0x7f5a200a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.484+0000 7f5a2bfff640 1 -- 172.21.15.12:0/1713814409 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f5a1c0cf890 con 0x7f5a34104cf0 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.484+0000 7f5a3359e640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5a200a2c30 0x7f5a200a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.484+0000 7f5a32d9d640 1 --2- 172.21.15.12:0/3646602972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d25120740 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400bd10 tx=0x7f5a240107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a6c72ae623aaf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.484+0000 7f5a297fa640 1 -- 172.21.15.12:0/3646602972 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24004030 con 0x560d2511bc40 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.484+0000 7f5a297fa640 1 -- 172.21.15.12:0/3646602972 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a240041d0 con 0x560d2511bc40 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.484+0000 7f5a297fa640 1 -- 172.21.15.12:0/3646602972 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a240117e0 con 0x560d2511bc40 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.485+0000 7f5a3be65740 1 -- 172.21.15.12:0/3646602972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 msgr2=0x560d25120740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.485+0000 7f5a3359e640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5a200a2c30 0x7f5a200a50f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x560d2511c4a0 tx=0x7f5a2c031b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.485+0000 7f5a3be65740 1 --2- 172.21.15.12:0/3646602972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d25120740 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400bd10 tx=0x7f5a240107a0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.485+0000 7f5a3be65740 1 -- 172.21.15.12:0/3646602972 shutdown_connections 2024-05-09T09:26:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.485+0000 7f5a3be65740 1 --2- 172.21.15.12:0/3646602972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d25120db0 0x560d25123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.485+0000 7f5a3be65740 1 --2- 172.21.15.12:0/3646602972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d25120740 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.485+0000 7f5a3be65740 1 --2- 172.21.15.12:0/3646602972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d2511b250 0x560d2511b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.485+0000 7f5a3be65740 1 -- 172.21.15.12:0/3646602972 >> 172.21.15.12:0/3646602972 conn(0x560d251168c0 msgr2=0x560d25116cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.485+0000 7f5a3be65740 1 -- 172.21.15.12:0/3646602972 shutdown_connections 2024-05-09T09:26:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.485+0000 7f5a3be65740 1 -- 172.21.15.12:0/3646602972 wait complete. 2024-05-09T09:26:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.485+0000 7f5a3be65740 1 -- start start 2024-05-09T09:26:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.485+0000 7f5a3be65740 1 Processor -- start 2024-05-09T09:26:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3be65740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2511b250 0x560d25125490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3be65740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d251259d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3be65740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d25120db0 0x560d2512ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3be65740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560d25123d60 con 0x560d2511bc40 2024-05-09T09:26:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3be65740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d2503a790 con 0x560d2511b250 2024-05-09T09:26:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3be65740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560d25123ee0 con 0x560d25120db0 2024-05-09T09:26:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3be65740 10 client.0 _ll_register_callbacks cb 0x560d25075a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d251259d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d251259d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56650/0 (socket says 172.21.15.12:56650) 2024-05-09T09:26:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3259c640 1 -- 172.21.15.12:0/662734426 learned_addr learned my addr 172.21.15.12:0/662734426 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a32d9d640 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2511b250 0x560d25125490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3359e640 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d25120db0 0x560d2512ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3259c640 1 -- 172.21.15.12:0/662734426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d25120db0 msgr2=0x560d2512ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3259c640 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d25120db0 0x560d2512ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3259c640 1 -- 172.21.15.12:0/662734426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2511b250 msgr2=0x560d25125490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3259c640 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2511b250 0x560d25125490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3259c640 1 -- 172.21.15.12:0/662734426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d24d93210 con 0x560d2511bc40 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a32d9d640 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2511b250 0x560d25125490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3259c640 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d251259d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0986f0 tx=0x7f5a1c09b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37589aea56e5d24 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a1a7fc640 1 -- 172.21.15.12:0/662734426 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c09bd20 con 0x560d2511bc40 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a1a7fc640 1 -- 172.21.15.12:0/662734426 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f5a1c005170 con 0x560d2511bc40 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.486+0000 7f5a3be65740 1 -- 172.21.15.12:0/662734426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a240041d0 con 0x560d2511bc40 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.487+0000 7f5a1a7fc640 1 -- 172.21.15.12:0/662734426 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a1c0a00d0 con 0x560d2511bc40 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.487+0000 7f5a3be65740 10 client.16218 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.487+0000 7f5a3be65740 1 -- 172.21.15.12:0/662734426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560d24e5f620 con 0x560d2511bc40 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.487+0000 7f5a1a7fc640 1 -- 172.21.15.12:0/662734426 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a240041d0 con 0x560d2511bc40 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.487+0000 7f5a1a7fc640 1 -- 172.21.15.12:0/662734426 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5a1c0ba480 con 0x560d2511bc40 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.487+0000 7f5a3be65740 10 client.16218 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.487+0000 7f5a3be65740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.487+0000 7f5a3be65740 20 client.16218 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.487+0000 7f5a3be65740 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d25130f10 0x560d25133360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.487+0000 7f5a3be65740 4 client.16218 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.487+0000 7f5a3be65740 1 -- 172.21.15.12:0/662734426 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d2508ca70 con 0x560d25130f10 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.487+0000 7f5a32d9d640 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d25130f10 0x560d25133360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.488+0000 7f5a1a7fc640 10 client.16218 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.488+0000 7f5a32d9d640 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d25130f10 0x560d25133360 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cfd174b84150fc5 server_cookie=47e8c9046828b929 in_seq=0 out_seq=0 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.488+0000 7f5a1a7fc640 1 -- 172.21.15.12:0/662734426 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d2508ca70 con 0x560d25130f10 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.488+0000 7f5a1a7fc640 10 client.16218 handle_command_reply: tid=0 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.489+0000 7f5a3be65740 1 client.16218 shutdown 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.489+0000 7f5a3be65740 1 -- 172.21.15.12:0/662734426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 msgr2=0x560d251259d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.489+0000 7f5a3be65740 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d251259d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0986f0 tx=0x7f5a1c09b920 comp rx=0 tx=0).stop 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.489+0000 7f5a3be65740 1 -- 172.21.15.12:0/662734426 shutdown_connections 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.489+0000 7f5a3be65740 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d25130f10 0x560d25133360 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.489+0000 7f5a3be65740 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d25120db0 0x560d2512ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.489+0000 7f5a3be65740 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d251259d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.489+0000 7f5a3be65740 1 --2- 172.21.15.12:0/662734426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2511b250 0x560d25125490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.489+0000 7f5a1a7fc640 0 client.16218 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.489+0000 7f5a3be65740 1 -- 172.21.15.12:0/662734426 >> 172.21.15.12:0/662734426 conn(0x560d251168c0 msgr2=0x560d25115a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.489+0000 7f5a3be65740 1 -- 172.21.15.12:0/662734426 shutdown_connections 2024-05-09T09:26:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.489+0000 7f5a3be65740 1 -- 172.21.15.12:0/662734426 wait complete. 2024-05-09T09:26:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.489+0000 7f5a3be65740 20 client.16218 trim_cache size 0 max 16384 2024-05-09T09:26:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.504+0000 7f5a3be65740 1 Processor -- start 2024-05-09T09:26:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.504+0000 7f5a3be65740 1 -- start start 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.504+0000 7f5a3be65740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2511b250 0x560d251231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.504+0000 7f5a3be65740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d2512c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.504+0000 7f5a3be65740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d25120db0 0x560d25119870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.504+0000 7f5a3be65740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341ac7b0 con 0x560d2511bc40 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.504+0000 7f5a3be65740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341ae2f0 con 0x560d2511b250 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.504+0000 7f5a3be65740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341ace80 con 0x560d25120db0 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.505+0000 7f5a3259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d2512c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.505+0000 7f5a3259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d2512c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56660/0 (socket says 172.21.15.12:56660) 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.505+0000 7f5a3259c640 1 -- 172.21.15.12:0/2322194912 learned_addr learned my addr 172.21.15.12:0/2322194912 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.505+0000 7f5a32d9d640 1 --2- 172.21.15.12:0/2322194912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2511b250 0x560d251231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.505+0000 7f5a3259c640 1 -- 172.21.15.12:0/2322194912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d25120db0 msgr2=0x560d25119870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.505+0000 7f5a3259c640 1 --2- 172.21.15.12:0/2322194912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d25120db0 0x560d25119870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.505+0000 7f5a3259c640 1 -- 172.21.15.12:0/2322194912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2511b250 msgr2=0x560d251231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.505+0000 7f5a3259c640 1 --2- 172.21.15.12:0/2322194912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2511b250 0x560d251231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.505+0000 7f5a3259c640 1 -- 172.21.15.12:0/2322194912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d25114f00 con 0x560d2511bc40 2024-05-09T09:26:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.505+0000 7f5a3259c640 1 --2- 172.21.15.12:0/2322194912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d2512c9d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00c6a0 tx=0x7f5a1c09a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ee7c3d2885dc21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.505+0000 7f5a28ff9640 1 -- 172.21.15.12:0/2322194912 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0056a0 con 0x560d2511bc40 2024-05-09T09:26:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.505+0000 7f5a28ff9640 1 -- 172.21.15.12:0/2322194912 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a1c097db0 con 0x560d2511bc40 2024-05-09T09:26:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.505+0000 7f5a28ff9640 1 -- 172.21.15.12:0/2322194912 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0a4500 con 0x560d2511bc40 2024-05-09T09:26:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.506+0000 7f5a3be65740 1 -- 172.21.15.12:0/2322194912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 msgr2=0x560d2512c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.506+0000 7f5a3be65740 1 --2- 172.21.15.12:0/2322194912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d2512c9d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00c6a0 tx=0x7f5a1c09a230 comp rx=0 tx=0).stop 2024-05-09T09:26:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.506+0000 7f5a3be65740 1 -- 172.21.15.12:0/2322194912 shutdown_connections 2024-05-09T09:26:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.506+0000 7f5a3be65740 1 --2- 172.21.15.12:0/2322194912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d25120db0 0x560d25119870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.506+0000 7f5a3be65740 1 --2- 172.21.15.12:0/2322194912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511bc40 0x560d2512c9d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.506+0000 7f5a3be65740 1 --2- 172.21.15.12:0/2322194912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2511b250 0x560d251231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.506+0000 7f5a3be65740 1 -- 172.21.15.12:0/2322194912 >> 172.21.15.12:0/2322194912 conn(0x560d251168c0 msgr2=0x560d25116ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.506+0000 7f5a3be65740 1 -- 172.21.15.12:0/2322194912 shutdown_connections 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.506+0000 7f5a3be65740 1 -- 172.21.15.12:0/2322194912 wait complete. 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.506+0000 7f5a3be65740 1 -- start start 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.506+0000 7f5a3be65740 1 Processor -- start 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a3be65740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511b250 0x560d2512a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a3be65740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d2511bc40 0x560d2512a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a3be65740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d25120db0 0x560d2512b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a3be65740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341adc20 con 0x560d2511b250 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a3be65740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341ad550 con 0x560d25120db0 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a3be65740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341aa790 con 0x560d2511bc40 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a3be65740 10 client.0 _ll_register_callbacks cb 0x7f5a1c00ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a32d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511b250 0x560d2512a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a32d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511b250 0x560d2512a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56662/0 (socket says 172.21.15.12:56662) 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a32d9d640 1 -- 172.21.15.12:0/2989704357 learned_addr learned my addr 172.21.15.12:0/2989704357 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:26:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a3359e640 1 --2- 172.21.15.12:0/2989704357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d25120db0 0x560d2512b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a3259c640 1 --2- 172.21.15.12:0/2989704357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d2511bc40 0x560d2512a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a32d9d640 1 -- 172.21.15.12:0/2989704357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d2511bc40 msgr2=0x560d2512a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a32d9d640 1 --2- 172.21.15.12:0/2989704357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d2511bc40 0x560d2512a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a32d9d640 1 -- 172.21.15.12:0/2989704357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d25120db0 msgr2=0x560d2512b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a32d9d640 1 --2- 172.21.15.12:0/2989704357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d25120db0 0x560d2512b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a32d9d640 1 -- 172.21.15.12:0/2989704357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d2512bb00 con 0x560d2511b250 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a32d9d640 1 --2- 172.21.15.12:0/2989704357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511b250 0x560d2512a010 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5a24010ca0 tx=0x7f5a2400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26f1573ec1953c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a1a7fc640 1 -- 172.21.15.12:0/2989704357 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24005450 con 0x560d2511b250 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a1a7fc640 1 -- 172.21.15.12:0/2989704357 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f5a240055f0 con 0x560d2511b250 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.507+0000 7f5a3be65740 1 -- 172.21.15.12:0/2989704357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a1c097db0 con 0x560d2511b250 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.508+0000 7f5a1a7fc640 1 -- 172.21.15.12:0/2989704357 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a24037050 con 0x560d2511b250 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.508+0000 7f5a3be65740 10 client.16224 fetch_fsmap learned FSMap version 29 2024-05-09T09:26:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.508+0000 7f5a3be65740 1 -- 172.21.15.12:0/2989704357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560d251974e0 con 0x560d2511b250 2024-05-09T09:26:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.508+0000 7f5a1a7fc640 1 -- 172.21.15.12:0/2989704357 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a1c097db0 con 0x560d2511b250 2024-05-09T09:26:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.508+0000 7f5a1a7fc640 1 -- 172.21.15.12:0/2989704357 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5a24003920 con 0x560d2511b250 2024-05-09T09:26:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.508+0000 7f5a3be65740 10 client.16224 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T09:26:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.508+0000 7f5a3be65740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:26:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.508+0000 7f5a3be65740 20 client.16224 populate_metadata read hostname 'smithi012' 2024-05-09T09:26:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.508+0000 7f5a3be65740 1 --2- 172.21.15.12:0/2989704357 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d25197b70 0x560d25199f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.508+0000 7f5a3be65740 4 client.16224 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:26:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.508+0000 7f5a3be65740 1 -- 172.21.15.12:0/2989704357 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d25086dc0 con 0x560d25197b70 2024-05-09T09:26:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.508+0000 7f5a3259c640 1 --2- 172.21.15.12:0/2989704357 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d25197b70 0x560d25199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.509+0000 7f5a1a7fc640 10 client.16224 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.509+0000 7f5a3259c640 1 --2- 172.21.15.12:0/2989704357 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d25197b70 0x560d25199f90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3980b4b77baee1d3 server_cookie=f55e0b2a8ce9c79b in_seq=0 out_seq=0 2024-05-09T09:26:58.643 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-ryos 2024-05-09T09:26:58.655 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-savu 2024-05-09T09:26:58.662 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-srws1 2024-05-09T09:26:58.666 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-wiimote 2024-05-09T09:26:58.670 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-input-axp-pek 2024-05-09T09:26:58.676 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-intel-rapid-start 2024-05-09T09:26:58.680 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-pciback 2024-05-09T09:26:58.683 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-ppi 2024-05-09T09:26:58.687 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-samsung-laptop 2024-05-09T09:26:58.691 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-sunxi-sid 2024-05-09T09:26:58.695 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-tegra-fuse 2024-05-09T09:26:58.698 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-toshiba_acpi 2024-05-09T09:26:58.702 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-wacom 2024-05-09T09:26:58.705 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkback 2024-05-09T09:26:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a1a7fc640 1 -- 172.21.15.12:0/2989704357 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d25086dc0 con 0x560d25197b70 2024-05-09T09:26:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a1a7fc640 10 client.16224 handle_command_reply: tid=0 2024-05-09T09:26:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a3be65740 1 client.16224 shutdown 2024-05-09T09:26:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a3be65740 1 -- 172.21.15.12:0/2989704357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511b250 msgr2=0x560d2512a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a3be65740 1 --2- 172.21.15.12:0/2989704357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511b250 0x560d2512a010 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5a24010ca0 tx=0x7f5a2400bf60 comp rx=0 tx=0).stop 2024-05-09T09:26:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a3be65740 1 -- 172.21.15.12:0/2989704357 shutdown_connections 2024-05-09T09:26:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a3be65740 1 --2- 172.21.15.12:0/2989704357 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d25197b70 0x560d25199f90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a3be65740 1 --2- 172.21.15.12:0/2989704357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d25120db0 0x560d2512b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a3be65740 1 --2- 172.21.15.12:0/2989704357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d2511bc40 0x560d2512a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a3be65740 1 --2- 172.21.15.12:0/2989704357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d2511b250 0x560d2512a010 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a3be65740 1 -- 172.21.15.12:0/2989704357 >> 172.21.15.12:0/2989704357 conn(0x560d251168c0 msgr2=0x560d25119870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a1a7fc640 0 client.16224 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:26:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a3be65740 1 -- 172.21.15.12:0/2989704357 shutdown_connections 2024-05-09T09:26:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a3be65740 1 -- 172.21.15.12:0/2989704357 wait complete. 2024-05-09T09:26:58.709 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkfront 2024-05-09T09:26:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.708+0000 7f5a3be65740 20 client.16224 trim_cache size 0 max 16384 2024-05-09T09:26:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:26:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:26:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:26:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:26:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.709+0000 7f5a3915a640 1 -- 172.21.15.12:0/1713814409 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5a200a2c30 msgr2=0x7f5a200a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.709+0000 7f5a3915a640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5a200a2c30 0x7f5a200a50f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x560d2511c4a0 tx=0x7f5a2c031b80 comp rx=0 tx=0).stop 2024-05-09T09:26:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.709+0000 7f5a3915a640 1 -- 172.21.15.12:0/1713814409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34104cf0 msgr2=0x7f5a34100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.709+0000 7f5a3915a640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34104cf0 0x7f5a34100370 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00e9f0 tx=0x7f5a1c00eeb0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.709+0000 7f5a3915a640 1 -- 172.21.15.12:0/1713814409 shutdown_connections 2024-05-09T09:26:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.709+0000 7f5a3915a640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5a200a2c30 0x7f5a200a50f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.709+0000 7f5a3915a640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a34107620 0x7f5a341008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.709+0000 7f5a3915a640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34104cf0 0x7f5a34100370 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.709+0000 7f5a3915a640 1 --2- 172.21.15.12:0/1713814409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a341023c0 0x7f5a34101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.709+0000 7f5a3915a640 1 -- 172.21.15.12:0/1713814409 >> 172.21.15.12:0/1713814409 conn(0x7f5a340fbdb0 msgr2=0x7f5a34102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.709+0000 7f5a3915a640 1 -- 172.21.15.12:0/1713814409 shutdown_connections 2024-05-09T09:26:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:26:58.709+0000 7f5a3915a640 1 -- 172.21.15.12:0/1713814409 wait complete. 2024-05-09T09:26:58.712 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-acpi 2024-05-09T09:26:58.714 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-dmi 2024-05-09T09:26:58.716 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi 2024-05-09T09:26:58.719 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi-runtime-map 2024-05-09T09:26:58.721 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-gsmi 2024-05-09T09:26:58.724 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-log 2024-05-09T09:26:58.726 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-memmap 2024-05-09T09:26:58.729 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-ofw 2024-05-09T09:26:58.731 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sfi 2024-05-09T09:26:58.733 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sgi_uv 2024-05-09T09:26:58.737 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-ext4 2024-05-09T09:26:58.739 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-f2fs 2024-05-09T09:26:58.741 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-nilfs2 2024-05-09T09:26:58.744 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-xfs 2024-05-09T09:26:58.746 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-gpio 2024-05-09T09:26:58.748 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-i2c-bmp085 2024-05-09T09:26:58.751 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ibft 2024-05-09T09:26:58.753 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-boot_params 2024-05-09T09:26:58.756 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-fscaps 2024-05-09T09:26:58.758 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-iommu_groups 2024-05-09T09:26:58.760 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-livepatch 2024-05-09T09:26:58.763 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm 2024-05-09T09:26:58.766 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-hugepages 2024-05-09T09:26:58.768 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-ksm 2024-05-09T09:26:58.771 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-slab 2024-05-09T09:26:58.774 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-uids 2024-05-09T09:26:58.776 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo 2024-05-09T09:26:58.778 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-memory-page-offline 2024-05-09T09:26:58.780 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-module 2024-05-09T09:26:58.782 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ocfs2 2024-05-09T09:26:58.785 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-laptop 2024-05-09T09:26:58.787 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-wmi 2024-05-09T09:26:58.790 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-at91 2024-05-09T09:26:58.792 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-brcmstb-gisb-arb 2024-05-09T09:26:58.795 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-chipidea-usb-otg 2024-05-09T09:26:58.798 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-eeepc-laptop 2024-05-09T09:26:58.800 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ideapad-laptop 2024-05-09T09:26:58.803 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-kim 2024-05-09T09:26:58.805 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-msi-laptop 2024-05-09T09:26:58.807 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-tahvo-usb 2024-05-09T09:26:58.810 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ts5500 2024-05-09T09:26:58.812 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-power 2024-05-09T09:26:58.815 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-pps 2024-05-09T09:26:58.818 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-profiling 2024-05-09T09:26:58.820 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ptp 2024-05-09T09:26:58.822 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-tty 2024-05-09T09:26:58.825 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-wusb_cbaf 2024-05-09T09:26:58.828 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/BUG-HUNTING 2024-05-09T09:26:58.831 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/Changes 2024-05-09T09:26:58.834 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/CodeOfConflict 2024-05-09T09:26:58.836 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/CodingStyle 2024-05-09T09:26:58.839 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DMA-API-HOWTO.txt 2024-05-09T09:26:58.842 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DMA-API.txt 2024-05-09T09:26:58.846 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DMA-ISA-LPC.txt 2024-05-09T09:26:58.848 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DMA-attributes.txt 2024-05-09T09:26:58.851 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/ 2024-05-09T09:26:58.853 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/.gitignore 2024-05-09T09:26:58.855 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/80211.tmpl 2024-05-09T09:26:58.858 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/Makefile 2024-05-09T09:26:58.861 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/alsa-driver-api.tmpl 2024-05-09T09:26:58.863 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/crypto-API.tmpl 2024-05-09T09:26:58.866 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/debugobjects.tmpl 2024-05-09T09:26:58.869 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/device-drivers.tmpl 2024-05-09T09:26:58.872 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/deviceiobook.tmpl 2024-05-09T09:26:58.874 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/drm.tmpl 2024-05-09T09:26:58.880 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/filesystems.tmpl 2024-05-09T09:26:58.883 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/gadget.tmpl 2024-05-09T09:26:58.886 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/genericirq.tmpl 2024-05-09T09:26:58.888 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/kernel-api.tmpl 2024-05-09T09:26:58.893 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/kernel-hacking.tmpl 2024-05-09T09:26:58.894 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/kernel-locking.tmpl 2024-05-09T09:26:58.898 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/kgdb.tmpl 2024-05-09T09:26:58.901 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/libata.tmpl 2024-05-09T09:26:58.905 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/librs.tmpl 2024-05-09T09:26:58.907 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/lsm.tmpl 2024-05-09T09:26:58.910 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/ 2024-05-09T09:26:58.912 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/Makefile 2024-05-09T09:26:58.915 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/bayer.png.b64 2024-05-09T09:26:58.917 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/constraints.png.b64 2024-05-09T09:26:58.920 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/crop.gif.b64 2024-05-09T09:26:58.922 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ 2024-05-09T09:26:58.924 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/.gitignore 2024-05-09T09:26:58.926 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/audio.xml 2024-05-09T09:26:58.929 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ca.xml 2024-05-09T09:26:58.932 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/demux.xml 2024-05-09T09:26:58.935 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbapi.xml 2024-05-09T09:26:58.937 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbproperty.xml 2024-05-09T09:26:58.940 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbstb.pdf 2024-05-09T09:26:58.943 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/examples.xml 2024-05-09T09:26:58.945 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/frontend.xml 2024-05-09T09:26:58.949 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/intro.xml 2024-05-09T09:26:58.951 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/kdapi.xml 2024-05-09T09:26:58.955 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/net.xml 2024-05-09T09:26:58.958 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/video.xml 2024-05-09T09:26:58.962 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/dvbstb.png.b64 2024-05-09T09:26:58.966 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_bt.gif.b64 2024-05-09T09:26:58.970 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_tb.gif.b64 2024-05-09T09:26:58.973 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt.gif.b64 2024-05-09T09:26:58.975 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt_example.gif.b64 2024-05-09T09:26:58.978 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/pipeline.png.b64 2024-05-09T09:26:58.981 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/selection.png.b64 2024-05-09T09:26:58.983 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/ 2024-05-09T09:26:58.985 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/.gitignore 2024-05-09T09:26:58.987 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/biblio.xml 2024-05-09T09:26:58.989 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/capture.c.xml 2024-05-09T09:26:58.992 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/common.xml 2024-05-09T09:26:58.996 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/compat.xml 2024-05-09T09:26:59.001 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/controls.xml 2024-05-09T09:26:59.007 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/crop.pdf 2024-05-09T09:26:59.010 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-capture.xml 2024-05-09T09:26:59.013 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-codec.xml 2024-05-09T09:26:59.015 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-effect.xml 2024-05-09T09:26:59.018 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-event.xml 2024-05-09T09:26:59.020 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-osd.xml 2024-05-09T09:26:59.022 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-output.xml 2024-05-09T09:26:59.025 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-overlay.xml 2024-05-09T09:26:59.027 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-radio.xml 2024-05-09T09:26:59.030 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-raw-vbi.xml 2024-05-09T09:26:59.033 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-rds.xml 2024-05-09T09:26:59.035 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sdr.xml 2024-05-09T09:26:59.037 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sliced-vbi.xml 2024-05-09T09:26:59.040 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-subdev.xml 2024-05-09T09:26:59.043 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-teletext.xml 2024-05-09T09:26:59.046 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/driver.xml 2024-05-09T09:26:59.048 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fdl-appendix.xml 2024-05-09T09:26:59.052 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_bt.pdf 2024-05-09T09:26:59.054 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_tb.pdf 2024-05-09T09:26:59.057 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-close.xml 2024-05-09T09:26:59.059 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-ioctl.xml 2024-05-09T09:26:59.061 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-mmap.xml 2024-05-09T09:26:59.063 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-munmap.xml 2024-05-09T09:26:59.066 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-open.xml 2024-05-09T09:26:59.068 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-poll.xml 2024-05-09T09:26:59.070 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-read.xml 2024-05-09T09:26:59.073 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-select.xml 2024-05-09T09:26:59.076 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-write.xml 2024-05-09T09:26:59.078 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/gen-errors.xml 2024-05-09T09:26:59.081 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/io.xml 2024-05-09T09:26:59.085 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/keytable.c.xml 2024-05-09T09:26:59.087 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/libv4l.xml 2024-05-09T09:26:59.090 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/lirc_device_interface.xml 2024-05-09T09:26:59.093 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-controller.xml 2024-05-09T09:26:59.097 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-close.xml 2024-05-09T09:26:59.098 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-ioctl.xml 2024-05-09T09:26:59.100 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-open.xml 2024-05-09T09:26:59.103 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-device-info.xml 2024-05-09T09:26:59.106 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-entities.xml 2024-05-09T09:26:59.109 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-links.xml 2024-05-09T09:26:59.111 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-setup-link.xml 2024-05-09T09:26:59.113 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pipeline.pdf 2024-05-09T09:26:59.116 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-grey.xml 2024-05-09T09:26:59.119 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-m420.xml 2024-05-09T09:26:59.122 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12.xml 2024-05-09T09:26:59.124 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12m.xml 2024-05-09T09:26:59.126 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12mt.xml 2024-05-09T09:26:59.129 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16.xml 2024-05-09T09:26:59.131 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16m.xml 2024-05-09T09:26:59.134 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv24.xml 2024-05-09T09:26:59.137 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-rgb.xml 2024-05-09T09:26:59.140 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-yuv.xml 2024-05-09T09:26:59.144 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr16.xml 2024-05-09T09:26:59.145 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr8.xml 2024-05-09T09:26:59.148 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs08.xml 2024-05-09T09:26:59.150 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs14le.xml 2024-05-09T09:26:59.153 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu08.xml 2024-05-09T09:26:59.155 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu16le.xml 2024-05-09T09:26:59.157 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-ru12le.xml 2024-05-09T09:26:59.160 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgbrg8.xml 2024-05-09T09:26:59.162 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgrbg8.xml 2024-05-09T09:26:59.165 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10.xml 2024-05-09T09:26:59.167 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10alaw8.xml 2024-05-09T09:26:59.170 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10dpcm8.xml 2024-05-09T09:26:59.172 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10p.xml 2024-05-09T09:26:59.175 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb12.xml 2024-05-09T09:26:59.177 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb8.xml 2024-05-09T09:26:59.179 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uv8.xml 2024-05-09T09:26:59.182 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uyvy.xml 2024-05-09T09:26:59.185 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-vyuy.xml 2024-05-09T09:26:59.188 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10.xml 2024-05-09T09:26:59.190 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10b.xml 2024-05-09T09:26:59.193 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y12.xml 2024-05-09T09:26:59.195 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y16.xml 2024-05-09T09:26:59.198 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y41p.xml 2024-05-09T09:26:59.200 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv410.xml 2024-05-09T09:26:59.203 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv411p.xml 2024-05-09T09:26:59.205 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420.xml 2024-05-09T09:26:59.207 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420m.xml 2024-05-09T09:26:59.210 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv422p.xml 2024-05-09T09:26:59.212 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuyv.xml 2024-05-09T09:26:59.215 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvu420m.xml 2024-05-09T09:26:59.217 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvyu.xml 2024-05-09T09:26:59.220 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt.xml 2024-05-09T09:26:59.224 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/planar-apis.xml 2024-05-09T09:26:59.227 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/remote_controllers.xml 2024-05-09T09:26:59.230 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selection-api.xml 2024-05-09T09:26:59.233 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selections-common.xml 2024-05-09T09:26:59.236 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-formats.xml 2024-05-09T09:26:59.240 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.dia 2024-05-09T09:26:59.244 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.svg 2024-05-09T09:26:59.246 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.dia 2024-05-09T09:26:59.250 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.svg 2024-05-09T09:26:59.252 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.dia 2024-05-09T09:26:59.256 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.svg 2024-05-09T09:26:59.258 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2.xml 2024-05-09T09:26:59.261 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2grab.c.xml 2024-05-09T09:26:59.265 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_525.pdf 2024-05-09T09:26:59.267 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_625.pdf 2024-05-09T09:26:59.270 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_hsync.pdf 2024-05-09T09:26:59.272 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml 2024-05-09T09:26:59.275 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-cropcap.xml 2024-05-09T09:26:59.278 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-chip-info.xml 2024-05-09T09:26:59.280 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-register.xml 2024-05-09T09:26:59.283 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-decoder-cmd.xml 2024-05-09T09:26:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dqevent.xml 2024-05-09T09:26:59.288 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml 2024-05-09T09:26:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-encoder-cmd.xml 2024-05-09T09:26:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml 2024-05-09T09:26:59.296 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-fmt.xml 2024-05-09T09:26:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-frameintervals.xml 2024-05-09T09:26:59.302 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-framesizes.xml 2024-05-09T09:26:59.305 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml 2024-05-09T09:26:59.308 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudio.xml 2024-05-09T09:26:59.310 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudioout.xml 2024-05-09T09:26:59.313 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enuminput.xml 2024-05-09T09:26:59.315 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumoutput.xml 2024-05-09T09:26:59.317 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumstd.xml 2024-05-09T09:26:59.320 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-expbuf.xml 2024-05-09T09:26:59.323 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audio.xml 2024-05-09T09:26:59.325 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audioout.xml 2024-05-09T09:26:59.328 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-crop.xml 2024-05-09T09:26:59.330 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ctrl.xml 2024-05-09T09:26:59.333 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-dv-timings.xml 2024-05-09T09:26:59.335 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-edid.xml 2024-05-09T09:26:59.338 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-enc-index.xml 2024-05-09T09:26:59.341 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ext-ctrls.xml 2024-05-09T09:26:59.343 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fbuf.xml 2024-05-09T09:26:59.346 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fmt.xml 2024-05-09T09:26:59.349 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-frequency.xml 2024-05-09T09:26:59.352 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-input.xml 2024-05-09T09:26:59.354 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-jpegcomp.xml 2024-05-09T09:26:59.357 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-modulator.xml 2024-05-09T09:26:59.359 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-output.xml 2024-05-09T09:26:59.362 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-parm.xml 2024-05-09T09:26:59.364 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-priority.xml 2024-05-09T09:26:59.367 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-selection.xml 2024-05-09T09:26:59.369 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-sliced-vbi-cap.xml 2024-05-09T09:26:59.372 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-std.xml 2024-05-09T09:26:59.374 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-tuner.xml 2024-05-09T09:26:59.377 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-log-status.xml 2024-05-09T09:26:59.380 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-overlay.xml 2024-05-09T09:26:59.382 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml 2024-05-09T09:26:59.384 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-qbuf.xml 2024-05-09T09:26:59.387 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml 2024-05-09T09:26:59.389 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querybuf.xml 2024-05-09T09:27:00.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:26:59 smithi012 ceph-mon[30140]: pgmap v410: 129 pgs: 129 active+clean; 41 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 MiB/s wr, 16 op/s 2024-05-09T09:27:00.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:26:59 smithi170 ceph-mon[33632]: pgmap v410: 129 pgs: 129 active+clean; 41 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 MiB/s wr, 16 op/s 2024-05-09T09:27:00.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:26:59 smithi016 ceph-mon[33335]: pgmap v410: 129 pgs: 129 active+clean; 41 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 MiB/s wr, 16 op/s 2024-05-09T09:27:00.468 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querycap.xml 2024-05-09T09:27:00.470 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-queryctrl.xml 2024-05-09T09:27:00.473 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querystd.xml 2024-05-09T09:27:00.476 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-reqbufs.xml 2024-05-09T09:27:00.479 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-s-hw-freq-seek.xml 2024-05-09T09:27:00.482 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-streamon.xml 2024-05-09T09:27:00.485 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml 2024-05-09T09:27:00.488 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml 2024-05-09T09:27:00.490 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml 2024-05-09T09:27:00.493 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-crop.xml 2024-05-09T09:27:00.495 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml 2024-05-09T09:27:00.498 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml 2024-05-09T09:27:00.501 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml 2024-05-09T09:27:00.504 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml 2024-05-09T09:27:00.507 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_525.gif.b64 2024-05-09T09:27:00.510 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_625.gif.b64 2024-05-09T09:27:00.512 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_hsync.gif.b64 2024-05-09T09:27:00.515 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/media_api.tmpl 2024-05-09T09:27:00.518 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/mtdnand.tmpl 2024-05-09T09:27:00.521 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/networking.tmpl 2024-05-09T09:27:00.523 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/rapidio.tmpl 2024-05-09T09:27:00.526 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/regulator.tmpl 2024-05-09T09:27:00.529 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/s390-drivers.tmpl 2024-05-09T09:27:00.531 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/scsi.tmpl 2024-05-09T09:27:00.534 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/sh.tmpl 2024-05-09T09:27:00.536 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/stylesheet.xsl 2024-05-09T09:27:00.539 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/tracepoint.tmpl 2024-05-09T09:27:00.541 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/uio-howto.tmpl 2024-05-09T09:27:00.545 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/usb.tmpl 2024-05-09T09:27:00.548 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/w1.tmpl 2024-05-09T09:27:00.551 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/writing-an-alsa-driver.tmpl 2024-05-09T09:27:00.558 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/writing_musb_glue_layer.tmpl 2024-05-09T09:27:00.561 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/writing_usb_driver.tmpl 2024-05-09T09:27:00.564 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/DocBook/z8530book.tmpl 2024-05-09T09:27:00.567 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/EDID/ 2024-05-09T09:27:00.569 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/EDID/1024x768.S 2024-05-09T09:27:00.571 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/EDID/1280x1024.S 2024-05-09T09:27:00.573 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/EDID/1600x1200.S 2024-05-09T09:27:00.575 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/EDID/1680x1050.S 2024-05-09T09:27:00.578 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/EDID/1920x1080.S 2024-05-09T09:27:00.580 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/EDID/800x600.S 2024-05-09T09:27:00.582 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/EDID/HOWTO.txt 2024-05-09T09:27:00.584 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/EDID/Makefile 2024-05-09T09:27:00.587 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/EDID/edid.S 2024-05-09T09:27:00.589 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/EDID/hex 2024-05-09T09:27:00.591 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/HOWTO 2024-05-09T09:27:00.594 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/IPMI.txt 2024-05-09T09:27:00.598 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/IRQ-affinity.txt 2024-05-09T09:27:00.600 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/IRQ-domain.txt 2024-05-09T09:27:00.603 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/IRQ.txt 2024-05-09T09:27:00.605 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/Intel-IOMMU.txt 2024-05-09T09:27:00.607 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/Makefile 2024-05-09T09:27:00.609 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ManagementStyle 2024-05-09T09:27:00.612 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/PCI/ 2024-05-09T09:27:00.614 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/PCI/00-INDEX 2024-05-09T09:27:00.616 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/PCI/MSI-HOWTO.txt 2024-05-09T09:27:00.619 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/PCI/PCIEBUS-HOWTO.txt 2024-05-09T09:27:00.621 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/PCI/pci-error-recovery.txt 2024-05-09T09:27:00.624 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/PCI/pci-iov-howto.txt 2024-05-09T09:27:00.626 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/PCI/pci.txt 2024-05-09T09:27:00.629 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/PCI/pcieaer-howto.txt 2024-05-09T09:27:00.632 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/ 2024-05-09T09:27:00.634 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/00-INDEX 2024-05-09T09:27:00.636 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/NMI-RCU.txt 2024-05-09T09:27:00.638 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/RTFP.txt 2024-05-09T09:27:00.643 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/UP.txt 2024-05-09T09:27:00.645 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/arrayRCU.txt 2024-05-09T09:27:00.648 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/checklist.txt 2024-05-09T09:27:00.651 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/listRCU.txt 2024-05-09T09:27:00.653 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/lockdep-splat.txt 2024-05-09T09:27:00.655 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/lockdep.txt 2024-05-09T09:27:00.658 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/rcu.txt 2024-05-09T09:27:00.660 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/rcu_dereference.txt 2024-05-09T09:27:00.663 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/rcubarrier.txt 2024-05-09T09:27:00.665 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/rculist_nulls.txt 2024-05-09T09:27:00.668 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/rcuref.txt 2024-05-09T09:27:00.670 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/stallwarn.txt 2024-05-09T09:27:00.673 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/torture.txt 2024-05-09T09:27:00.675 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/trace.txt 2024-05-09T09:27:00.678 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/RCU/whatisRCU.txt 2024-05-09T09:27:00.681 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/SAK.txt 2024-05-09T09:27:00.684 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/SM501.txt 2024-05-09T09:27:00.686 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/SecurityBugs 2024-05-09T09:27:00.688 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/SubmitChecklist 2024-05-09T09:27:00.691 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/SubmittingDrivers 2024-05-09T09:27:00.693 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/SubmittingPatches 2024-05-09T09:27:00.696 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/VGA-softcursor.txt 2024-05-09T09:27:00.698 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/accounting/ 2024-05-09T09:27:00.700 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/accounting/.gitignore 2024-05-09T09:27:00.702 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/accounting/Makefile 2024-05-09T09:27:00.704 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/accounting/cgroupstats.txt 2024-05-09T09:27:00.706 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/accounting/delay-accounting.txt 2024-05-09T09:27:00.708 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/accounting/getdelays.c 2024-05-09T09:27:00.711 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/accounting/taskstats-struct.txt 2024-05-09T09:27:00.713 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/accounting/taskstats.txt 2024-05-09T09:27:00.716 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/ 2024-05-09T09:27:00.718 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/apei/ 2024-05-09T09:27:00.720 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/apei/einj.txt 2024-05-09T09:27:00.723 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/apei/output_format.txt 2024-05-09T09:27:00.726 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/debug.txt 2024-05-09T09:27:00.728 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/dsdt-override.txt 2024-05-09T09:27:00.730 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/enumeration.txt 2024-05-09T09:27:00.733 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/gpio-properties.txt 2024-05-09T09:27:00.735 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/initrd_table_override.txt 2024-05-09T09:27:00.738 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/method-customizing.txt 2024-05-09T09:27:00.741 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/method-tracing.txt 2024-05-09T09:27:00.742 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/namespace.txt 2024-05-09T09:27:00.745 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/scan_handlers.txt 2024-05-09T09:27:00.747 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/acpi/video_extension.txt 2024-05-09T09:27:00.750 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/aoe/ 2024-05-09T09:27:00.752 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/aoe/aoe.txt 2024-05-09T09:27:00.754 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/aoe/autoload.sh 2024-05-09T09:27:00.757 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/aoe/status.sh 2024-05-09T09:27:00.759 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/aoe/todo.txt 2024-05-09T09:27:00.761 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/aoe/udev-install.sh 2024-05-09T09:27:00.763 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/aoe/udev.txt 2024-05-09T09:27:00.765 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/applying-patches.txt 2024-05-09T09:27:00.769 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/ 2024-05-09T09:27:00.770 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/00-INDEX 2024-05-09T09:27:00.772 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Atmel/ 2024-05-09T09:27:00.776 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Atmel/README 2024-05-09T09:27:00.778 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Booting 2024-05-09T09:27:00.781 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/CCN.txt 2024-05-09T09:27:00.783 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/IXP4xx 2024-05-09T09:27:00.785 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Interrupts 2024-05-09T09:27:00.788 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Makefile 2024-05-09T09:27:00.790 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Marvell/ 2024-05-09T09:27:00.792 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Marvell/README 2024-05-09T09:27:00.795 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Netwinder 2024-05-09T09:27:00.799 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/OMAP/ 2024-05-09T09:27:00.801 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/OMAP/DSS 2024-05-09T09:27:00.804 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/OMAP/omap_pm 2024-05-09T09:27:00.806 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Porting 2024-05-09T09:27:00.809 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/README 2024-05-09T09:27:00.812 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/ 2024-05-09T09:27:00.814 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/ADSBitsy 2024-05-09T09:27:00.816 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/Assabet 2024-05-09T09:27:00.819 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/Brutus 2024-05-09T09:27:00.821 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/CERF 2024-05-09T09:27:00.823 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/FreeBird 2024-05-09T09:27:00.826 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsClient 2024-05-09T09:27:00.828 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsMaster 2024-05-09T09:27:00.830 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/HUW_WEBPANEL 2024-05-09T09:27:00.832 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/Itsy 2024-05-09T09:27:00.834 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/LART 2024-05-09T09:27:00.837 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/PLEB 2024-05-09T09:27:00.839 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/Pangolin 2024-05-09T09:27:00.841 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/Tifon 2024-05-09T09:27:00.844 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/Victor 2024-05-09T09:27:00.846 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/Yopy 2024-05-09T09:27:00.848 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/empeg 2024-05-09T09:27:00.851 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/nanoEngine 2024-05-09T09:27:00.853 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SA1100/serial_UART 2024-05-09T09:27:00.855 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/ 2024-05-09T09:27:00.857 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/.gitignore 2024-05-09T09:27:00.859 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/Makefile 2024-05-09T09:27:00.862 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/vrl4.c 2024-05-09T09:27:00.864 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-mmcif.txt 2024-05-09T09:27:00.867 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-sdhi.txt 2024-05-09T09:27:00.869 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SPEAr/ 2024-05-09T09:27:00.871 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/SPEAr/overview.txt 2024-05-09T09:27:00.873 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/ 2024-05-09T09:27:00.876 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/CPUfreq.txt 2024-05-09T09:27:00.878 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/EB2410ITX.txt 2024-05-09T09:27:00.880 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/GPIO.txt 2024-05-09T09:27:00.882 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/H1940.txt 2024-05-09T09:27:00.885 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/NAND.txt 2024-05-09T09:27:00.887 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Overview.txt 2024-05-09T09:27:00.890 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2412.txt 2024-05-09T09:27:00.892 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2413.txt 2024-05-09T09:27:00.894 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/SMDK2440.txt 2024-05-09T09:27:00.896 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Suspend.txt 2024-05-09T09:27:00.899 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/USB-Host.txt 2024-05-09T09:27:00.901 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung/ 2024-05-09T09:27:00.904 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung/GPIO.txt 2024-05-09T09:27:00.906 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung/Overview.txt 2024-05-09T09:27:00.908 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Samsung/clksrc-change-registers.awk 2024-05-09T09:27:00.911 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/Setup 2024-05-09T09:27:00.914 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/VFP/ 2024-05-09T09:27:00.916 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/VFP/release-notes.txt 2024-05-09T09:27:00.918 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/cluster-pm-race-avoidance.txt 2024-05-09T09:27:00.921 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/firmware.txt 2024-05-09T09:27:00.924 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/kernel_mode_neon.txt 2024-05-09T09:27:00.926 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/kernel_user_helpers.txt 2024-05-09T09:27:00.928 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/mem_alignment 2024-05-09T09:27:00.930 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/memory.txt 2024-05-09T09:27:00.933 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/msm/ 2024-05-09T09:27:00.935 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/msm/gpiomux.txt 2024-05-09T09:27:00.937 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/nwfpe/ 2024-05-09T09:27:00.939 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/nwfpe/NOTES 2024-05-09T09:27:00.941 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/nwfpe/README 2024-05-09T09:27:00.945 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/nwfpe/README.FPE 2024-05-09T09:27:00.947 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/nwfpe/TODO 2024-05-09T09:27:00.949 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/pxa/ 2024-05-09T09:27:00.951 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/pxa/mfp.txt 2024-05-09T09:27:00.953 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/sti/ 2024-05-09T09:27:00.956 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/sti/overview.txt 2024-05-09T09:27:00.958 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/sti/stih407-overview.txt 2024-05-09T09:27:00.960 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/sti/stih415-overview.txt 2024-05-09T09:27:00.963 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/sti/stih416-overview.txt 2024-05-09T09:27:00.965 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/sti/stih418-overview.txt 2024-05-09T09:27:00.967 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/sunxi/ 2024-05-09T09:27:00.969 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/sunxi/README 2024-05-09T09:27:00.972 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/sunxi/clocks.txt 2024-05-09T09:27:00.974 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/swp_emulation 2024-05-09T09:27:00.977 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/tcm.txt 2024-05-09T09:27:00.979 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/uefi.txt 2024-05-09T09:27:00.981 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm/vlocks.txt 2024-05-09T09:27:00.984 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm64/ 2024-05-09T09:27:00.986 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm64/booting.txt 2024-05-09T09:27:00.988 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm64/legacy_instructions.txt 2024-05-09T09:27:00.990 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm64/memory.txt 2024-05-09T09:27:00.993 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/arm64/tagged-pointers.txt 2024-05-09T09:27:00.995 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/assoc_array.txt 2024-05-09T09:27:00.998 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/atomic_ops.txt 2024-05-09T09:27:01.001 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/auxdisplay/ 2024-05-09T09:27:01.003 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/auxdisplay/.gitignore 2024-05-09T09:27:01.005 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/auxdisplay/Makefile 2024-05-09T09:27:01.007 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b 2024-05-09T09:27:01.009 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b-example.c 2024-05-09T09:27:01.012 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/auxdisplay/ks0108 2024-05-09T09:27:01.014 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/backlight/ 2024-05-09T09:27:01.016 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/backlight/lp855x-driver.txt 2024-05-09T09:27:01.018 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/bad_memory.txt 2024-05-09T09:27:01.021 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/basic_profiling.txt 2024-05-09T09:27:01.023 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/bcache.txt 2024-05-09T09:27:01.026 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/binfmt_misc.txt 2024-05-09T09:27:01.028 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blackfin/ 2024-05-09T09:27:01.030 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blackfin/00-INDEX 2024-05-09T09:27:01.032 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blackfin/Makefile 2024-05-09T09:27:01.034 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blackfin/bfin-gpio-notes.txt 2024-05-09T09:27:01.036 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blackfin/bfin-spi-notes.txt 2024-05-09T09:27:01.039 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blackfin/gptimers-example.c 2024-05-09T09:27:01.041 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/ 2024-05-09T09:27:01.043 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/00-INDEX 2024-05-09T09:27:01.045 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/biodoc.txt 2024-05-09T09:27:01.049 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/biovecs.txt 2024-05-09T09:27:01.051 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/capability.txt 2024-05-09T09:27:01.053 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/cfq-iosched.txt 2024-05-09T09:27:01.056 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/cmdline-partition.txt 2024-05-09T09:27:01.058 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/data-integrity.txt 2024-05-09T09:27:01.060 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/deadline-iosched.txt 2024-05-09T09:27:01.063 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/ioprio.txt 2024-05-09T09:27:01.065 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/null_blk.txt 2024-05-09T09:27:01.068 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/queue-sysfs.txt 2024-05-09T09:27:01.070 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/request.txt 2024-05-09T09:27:01.073 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/stat.txt 2024-05-09T09:27:01.075 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/switching-sched.txt 2024-05-09T09:27:01.078 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/block/writeback_cache_control.txt 2024-05-09T09:27:01.080 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/ 2024-05-09T09:27:01.082 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/00-INDEX 2024-05-09T09:27:01.084 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/README.DAC960 2024-05-09T09:27:01.087 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/cciss.txt 2024-05-09T09:27:01.090 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/cpqarray.txt 2024-05-09T09:27:01.092 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/drbd/ 2024-05-09T09:27:01.094 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-8.3-data-packets.svg 2024-05-09T09:27:01.097 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-data-packets.svg 2024-05-09T09:27:01.100 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/drbd/README.txt 2024-05-09T09:27:01.102 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/drbd/conn-states-8.dot 2024-05-09T09:27:01.105 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/drbd/data-structure-v9.txt 2024-05-09T09:27:01.107 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/drbd/disk-states-8.dot 2024-05-09T09:27:01.109 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/drbd/drbd-connection-state-overview.dot 2024-05-09T09:27:01.112 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/drbd/node-states-8.dot 2024-05-09T09:27:01.114 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/floppy.txt 2024-05-09T09:27:01.117 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/mflash.txt 2024-05-09T09:27:01.119 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/nbd.txt 2024-05-09T09:27:01.121 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/paride.txt 2024-05-09T09:27:01.124 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/ramdisk.txt 2024-05-09T09:27:01.126 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/blockdev/zram.txt 2024-05-09T09:27:01.128 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/braille-console.txt 2024-05-09T09:27:01.131 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/bt8xxgpio.txt 2024-05-09T09:27:01.133 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/btmrvl.txt 2024-05-09T09:27:01.136 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/bus-devices/ 2024-05-09T09:27:01.137 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt 2024-05-09T09:27:01.140 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/bus-virt-phys-mapping.txt 2024-05-09T09:27:01.142 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cachetlb.txt 2024-05-09T09:27:01.145 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cdrom/ 2024-05-09T09:27:01.146 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cdrom/00-INDEX 2024-05-09T09:27:01.148 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cdrom/Makefile 2024-05-09T09:27:01.151 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cdrom/cdrom-standard.tex 2024-05-09T09:27:01.155 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cdrom/ide-cd 2024-05-09T09:27:01.157 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cdrom/packet-writing.txt 2024-05-09T09:27:01.160 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/ 2024-05-09T09:27:01.162 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/00-INDEX 2024-05-09T09:27:01.164 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/blkio-controller.txt 2024-05-09T09:27:01.167 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/cgroups.txt 2024-05-09T09:27:01.170 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/cpuacct.txt 2024-05-09T09:27:01.172 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/cpusets.txt 2024-05-09T09:27:01.176 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/devices.txt 2024-05-09T09:27:01.178 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/freezer-subsystem.txt 2024-05-09T09:27:01.180 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/hugetlb.txt 2024-05-09T09:27:01.183 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/memcg_test.txt 2024-05-09T09:27:01.186 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/memory.txt 2024-05-09T09:27:01.189 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/net_cls.txt 2024-05-09T09:27:01.191 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/net_prio.txt 2024-05-09T09:27:01.194 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cgroups/unified-hierarchy.txt 2024-05-09T09:27:01.197 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/circular-buffers.txt 2024-05-09T09:27:01.200 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/clk.txt 2024-05-09T09:27:01.203 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/coccinelle.txt 2024-05-09T09:27:01.205 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/connector/ 2024-05-09T09:27:01.207 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/connector/.gitignore 2024-05-09T09:27:01.209 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/connector/Makefile 2024-05-09T09:27:01.211 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/connector/cn_test.c 2024-05-09T09:27:01.214 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/connector/connector.txt 2024-05-09T09:27:01.216 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/connector/ucon.c 2024-05-09T09:27:01.219 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/console/ 2024-05-09T09:27:01.221 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/console/console.txt 2024-05-09T09:27:01.223 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-freq/ 2024-05-09T09:27:01.225 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-freq/amd-powernow.txt 2024-05-09T09:27:01.227 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-freq/boost.txt 2024-05-09T09:27:01.230 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-freq/core.txt 2024-05-09T09:27:01.232 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-freq/cpu-drivers.txt 2024-05-09T09:27:01.234 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-nforce2.txt 2024-05-09T09:27:01.237 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-stats.txt 2024-05-09T09:27:01.239 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-freq/governors.txt 2024-05-09T09:27:01.242 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-freq/index.txt 2024-05-09T09:27:01.244 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-freq/intel-pstate.txt 2024-05-09T09:27:01.247 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-freq/pcc-cpufreq.txt 2024-05-09T09:27:01.249 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-freq/user-guide.txt 2024-05-09T09:27:01.252 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-hotplug.txt 2024-05-09T09:27:01.255 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpu-load.txt 2024-05-09T09:27:01.257 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpuidle/ 2024-05-09T09:27:01.259 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpuidle/core.txt 2024-05-09T09:27:01.261 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpuidle/driver.txt 2024-05-09T09:27:01.263 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpuidle/governor.txt 2024-05-09T09:27:01.266 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cpuidle/sysfs.txt 2024-05-09T09:27:01.268 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cputopology.txt 2024-05-09T09:27:01.271 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/crc32.txt 2024-05-09T09:27:01.273 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cris/ 2024-05-09T09:27:01.275 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/cris/README 2024-05-09T09:27:01.277 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/crypto/ 2024-05-09T09:27:01.280 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/crypto/api-intro.txt 2024-05-09T09:27:01.282 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/crypto/asymmetric-keys.txt 2024-05-09T09:27:01.284 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/crypto/async-tx-api.txt 2024-05-09T09:27:01.287 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/crypto/crypto-API-userspace.txt 2024-05-09T09:27:01.290 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/crypto/descore-readme.txt 2024-05-09T09:27:01.292 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dcdbas.txt 2024-05-09T09:27:01.295 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/debugging-modules.txt 2024-05-09T09:27:01.297 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/debugging-via-ohci1394.txt 2024-05-09T09:27:01.299 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dell_rbu.txt 2024-05-09T09:27:01.302 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/development-process/ 2024-05-09T09:27:01.304 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/development-process/1.Intro 2024-05-09T09:27:01.307 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/development-process/2.Process 2024-05-09T09:27:01.310 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/development-process/3.Early-stage 2024-05-09T09:27:01.313 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/development-process/4.Coding 2024-05-09T09:27:01.316 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/development-process/5.Posting 2024-05-09T09:27:01.318 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/development-process/6.Followthrough 2024-05-09T09:27:01.321 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/development-process/7.AdvancedTopics 2024-05-09T09:27:01.323 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/development-process/8.Conclusion 2024-05-09T09:27:01.325 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/ 2024-05-09T09:27:01.327 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/cache-policies.txt 2024-05-09T09:27:01.329 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/cache.txt 2024-05-09T09:27:01.332 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/delay.txt 2024-05-09T09:27:01.335 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/dm-crypt.txt 2024-05-09T09:27:01.337 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/dm-flakey.txt 2024-05-09T09:27:01.339 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/dm-io.txt 2024-05-09T09:27:01.342 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/dm-log.txt 2024-05-09T09:27:01.344 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/dm-queue-length.txt 2024-05-09T09:27:01.347 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/dm-raid.txt 2024-05-09T09:27:01.349 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/dm-service-time.txt 2024-05-09T09:27:01.353 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/dm-uevent.txt 2024-05-09T09:27:01.354 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/era.txt 2024-05-09T09:27:01.357 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/kcopyd.txt 2024-05-09T09:27:01.359 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/linear.txt 2024-05-09T09:27:01.361 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/persistent-data.txt 2024-05-09T09:27:01.364 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/snapshot.txt 2024-05-09T09:27:01.366 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/statistics.txt 2024-05-09T09:27:01.369 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/striped.txt 2024-05-09T09:27:01.371 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/switch.txt 2024-05-09T09:27:01.373 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/thin-provisioning.txt 2024-05-09T09:27:01.376 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/verity.txt 2024-05-09T09:27:01.378 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/device-mapper/zero.txt 2024-05-09T09:27:01.380 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devices.txt 2024-05-09T09:27:01.386 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/ 2024-05-09T09:27:01.388 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/00-INDEX 2024-05-09T09:27:01.390 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ 2024-05-09T09:27:01.392 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ABI.txt 2024-05-09T09:27:01.394 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/ 2024-05-09T09:27:01.396 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/interrupts.txt 2024-05-09T09:27:01.399 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/pmu.txt 2024-05-09T09:27:01.402 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ 2024-05-09T09:27:01.405 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/adapteva.txt 2024-05-09T09:27:01.407 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/ 2024-05-09T09:27:01.410 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-clk-manager.txt 2024-05-09T09:27:01.412 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-sdram-edac.txt 2024-05-09T09:27:01.414 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-system.txt 2024-05-09T09:27:01.417 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/amlogic.txt 2024-05-09T09:27:01.420 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arch_timer.txt 2024-05-09T09:27:01.423 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arm-boards 2024-05-09T09:27:01.425 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt 2024-05-09T09:27:01.428 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp.txt 2024-05-09T09:27:01.430 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-375.txt 2024-05-09T09:27:01.433 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-380-mpcore-soc-ctrl.txt 2024-05-09T09:27:01.435 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-38x.txt 2024-05-09T09:27:01.438 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-cpu-reset.txt 2024-05-09T09:27:01.440 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armadeus.txt 2024-05-09T09:27:01.443 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-at91.txt 2024-05-09T09:27:01.445 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-pmc.txt 2024-05-09T09:27:01.448 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/axxia.txt 2024-05-09T09:27:01.451 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/ 2024-05-09T09:27:01.453 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm11351.txt 2024-05-09T09:27:01.455 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm21664.txt 2024-05-09T09:27:01.458 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm63138.txt 2024-05-09T09:27:01.460 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/brcm,bcm11351-cpu-method 2024-05-09T09:27:01.463 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/cygnus.txt 2024-05-09T09:27:01.465 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-resetmgr.txt 2024-05-09T09:27:01.468 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-timer.txt 2024-05-09T09:27:01.471 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-wdt.txt 2024-05-09T09:27:01.473 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm2835.txt 2024-05-09T09:27:01.476 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm4708.txt 2024-05-09T09:27:01.478 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/brcm-brcmstb.txt 2024-05-09T09:27:01.481 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda.txt 2024-05-09T09:27:01.483 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/ 2024-05-09T09:27:01.485 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/combophy.txt 2024-05-09T09:27:01.487 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/l2ecc.txt 2024-05-09T09:27:01.490 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/mem-ctrlr.txt 2024-05-09T09:27:01.492 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cavium-thunder.txt 2024-05-09T09:27:01.496 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cci.txt 2024-05-09T09:27:01.498 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ccn.txt 2024-05-09T09:27:01.500 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coherency-fabric.txt 2024-05-09T09:27:01.502 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coresight.txt 2024-05-09T09:27:01.505 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/ 2024-05-09T09:27:01.507 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/marvell,berlin-smp 2024-05-09T09:27:01.509 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpus.txt 2024-05-09T09:27:01.512 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci.txt 2024-05-09T09:27:01.515 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/ 2024-05-09T09:27:01.516 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/cp-intc.txt 2024-05-09T09:27:01.518 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/digicolor.txt 2024-05-09T09:27:01.521 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/ 2024-05-09T09:27:01.523 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/power_domain.txt 2024-05-09T09:27:01.525 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/smp-sysram.txt 2024-05-09T09:27:01.528 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/ 2024-05-09T09:27:01.530 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/tlm,trusted-foundations.txt 2024-05-09T09:27:01.532 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fsl.txt 2024-05-09T09:27:01.536 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fw-cfg.txt 2024-05-09T09:27:01.538 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/geniatech.txt 2024-05-09T09:27:01.540 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic-v3.txt 2024-05-09T09:27:01.543 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic.txt 2024-05-09T09:27:01.546 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/global_timer.txt 2024-05-09T09:27:01.548 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/ 2024-05-09T09:27:01.550 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt 2024-05-09T09:27:01.552 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/idle-states.txt 2024-05-09T09:27:01.557 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/insignal-boards.txt 2024-05-09T09:27:01.558 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/ 2024-05-09T09:27:01.560 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/keystone.txt 2024-05-09T09:27:01.562 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/kirkwood.txt 2024-05-09T09:27:01.565 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/l2cc.txt 2024-05-09T09:27:01.567 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx-mic.txt 2024-05-09T09:27:01.570 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx.txt 2024-05-09T09:27:01.572 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,berlin.txt 2024-05-09T09:27:01.575 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,dove.txt 2024-05-09T09:27:01.577 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,kirkwood.txt 2024-05-09T09:27:01.580 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek.txt 2024-05-09T09:27:01.582 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/ 2024-05-09T09:27:01.584 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/mediatek,sysirq.txt 2024-05-09T09:27:01.586 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/moxart.txt 2024-05-09T09:27:01.590 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/ 2024-05-09T09:27:01.592 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/feroceon.txt 2024-05-09T09:27:01.594 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/intc.txt 2024-05-09T09:27:01.597 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/mrvl.txt 2024-05-09T09:27:01.599 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/tauros2.txt 2024-05-09T09:27:01.601 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/timer.txt 2024-05-09T09:27:01.604 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ 2024-05-09T09:27:01.607 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,kpss-acc.txt 2024-05-09T09:27:01.610 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,saw2.txt 2024-05-09T09:27:01.612 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ssbi.txt 2024-05-09T09:27:01.615 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/timer.txt 2024-05-09T09:27:01.617 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mvebu-system-controller.txt 2024-05-09T09:27:01.620 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/nspire.txt 2024-05-09T09:27:01.623 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/olimex.txt 2024-05-09T09:27:01.626 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/ 2024-05-09T09:27:01.628 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/counter.txt 2024-05-09T09:27:01.630 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/crossbar.txt 2024-05-09T09:27:01.632 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dmm.txt 2024-05-09T09:27:01.635 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dsp.txt 2024-05-09T09:27:01.637 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/intc.txt 2024-05-09T09:27:01.640 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/iva.txt 2024-05-09T09:27:01.643 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/l3-noc.txt 2024-05-09T09:27:01.645 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/mpu.txt 2024-05-09T09:27:01.648 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/omap.txt 2024-05-09T09:27:01.651 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/prcm.txt 2024-05-09T09:27:01.654 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/timer.txt 2024-05-09T09:27:01.656 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/picoxcell.txt 2024-05-09T09:27:01.659 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/pmu.txt 2024-05-09T09:27:01.662 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/primecell.txt 2024-05-09T09:27:01.665 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/psci.txt 2024-05-09T09:27:01.667 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip.txt 2024-05-09T09:27:01.670 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/ 2024-05-09T09:27:01.672 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu-sram.txt 2024-05-09T09:27:01.675 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu.txt 2024-05-09T09:27:01.677 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/smp-sram.txt 2024-05-09T09:27:01.680 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rtsm-dcscb.txt 2024-05-09T09:27:01.683 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung-boards.txt 2024-05-09T09:27:01.685 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/ 2024-05-09T09:27:01.687 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-adc.txt 2024-05-09T09:27:01.689 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-chipid.txt 2024-05-09T09:27:01.692 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/interrupt-combiner.txt 2024-05-09T09:27:01.694 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/pmu.txt 2024-05-09T09:27:01.697 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/sysreg.txt 2024-05-09T09:27:01.699 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/shmobile.txt 2024-05-09T09:27:01.703 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sirf.txt 2024-05-09T09:27:01.706 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-misc.txt 2024-05-09T09:27:01.709 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-timer.txt 2024-05-09T09:27:01.711 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear.txt 2024-05-09T09:27:01.713 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/ 2024-05-09T09:27:01.715 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/shirq.txt 2024-05-09T09:27:01.718 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sprd.txt 2024-05-09T09:27:01.721 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-nomadik.txt 2024-05-09T09:27:01.723 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-u300.txt 2024-05-09T09:27:01.726 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sti.txt 2024-05-09T09:27:01.728 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sunxi.txt 2024-05-09T09:27:01.730 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra.txt 2024-05-09T09:27:01.733 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/ 2024-05-09T09:27:01.735 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-ahb.txt 2024-05-09T09:27:01.737 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-emc.txt 2024-05-09T09:27:01.740 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-flowctrl.txt 2024-05-09T09:27:01.742 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-mc.txt 2024-05-09T09:27:01.745 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-pmc.txt 2024-05-09T09:27:01.748 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra30-mc.txt 2024-05-09T09:27:01.750 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/topology.txt 2024-05-09T09:27:01.754 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/twd.txt 2024-05-09T09:27:01.756 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/ 2024-05-09T09:27:01.759 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/power_domain.txt 2024-05-09T09:27:01.761 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-fpga-irq.txt 2024-05-09T09:27:01.764 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-sysreg.txt 2024-05-09T09:27:01.766 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-scc.txt 2024-05-09T09:27:01.769 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-sysreg.txt 2024-05-09T09:27:01.771 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress.txt 2024-05-09T09:27:01.774 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vic.txt 2024-05-09T09:27:01.783 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500.txt 2024-05-09T09:27:01.841 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/ 2024-05-09T09:27:01.843 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-intc.txt 2024-05-09T09:27:02.112 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-pmc.txt 2024-05-09T09:27:02.114 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-timer.txt 2024-05-09T09:27:02.117 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xen.txt 2024-05-09T09:27:02.120 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xilinx.txt 2024-05-09T09:27:02.122 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ 2024-05-09T09:27:02.125 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-platform.txt 2024-05-09T09:27:02.128 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-st.txt 2024-05-09T09:27:02.130 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/apm-xgene.txt 2024-05-09T09:27:02.132 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/atmel-at91_cf.txt 2024-05-09T09:27:02.135 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/cavium-compact-flash.txt 2024-05-09T09:27:02.137 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/exynos-sata.txt 2024-05-09T09:27:02.139 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/fsl-sata.txt 2024-05-09T09:27:02.142 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-pata.txt 2024-05-09T09:27:02.144 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-sata.txt 2024-05-09T09:27:02.147 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/marvell.txt 2024-05-09T09:27:02.150 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/pata-arasan.txt 2024-05-09T09:27:02.152 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/qcom-sata.txt 2024-05-09T09:27:02.155 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_highbank.txt 2024-05-09T09:27:02.157 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_rcar.txt 2024-05-09T09:27:02.159 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/tegra-sata.txt 2024-05-09T09:27:02.162 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/btmrvl.txt 2024-05-09T09:27:02.165 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ 2024-05-09T09:27:02.170 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/bcma.txt 2024-05-09T09:27:02.170 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/brcm,gisb-arb.txt 2024-05-09T09:27:02.171 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/imx-weim.txt 2024-05-09T09:27:02.174 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/mvebu-mbus.txt 2024-05-09T09:27:02.176 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/omap-ocp2scp.txt 2024-05-09T09:27:02.178 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ti-gpmc.txt 2024-05-09T09:27:02.181 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/ 2024-05-09T09:27:02.184 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/clocks.txt 2024-05-09T09:27:02.186 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/dscr.txt 2024-05-09T09:27:02.189 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/emifa.txt 2024-05-09T09:27:02.192 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/interrupt.txt 2024-05-09T09:27:02.194 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/soc.txt 2024-05-09T09:27:02.197 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/timer64.txt 2024-05-09T09:27:02.199 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/chosen.txt 2024-05-09T09:27:02.203 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ 2024-05-09T09:27:02.205 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/alphascale,acc.txt 2024-05-09T09:27:02.207 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/altr_socfpga.txt 2024-05-09T09:27:02.209 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/arm-integrator.txt 2024-05-09T09:27:02.212 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/at91-clock.txt 2024-05-09T09:27:02.214 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/axi-clkgen.txt 2024-05-09T09:27:02.217 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-cygnus-clock.txt 2024-05-09T09:27:02.219 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt 2024-05-09T09:27:02.222 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/calxeda.txt 2024-05-09T09:27:02.224 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-exynos-audss.txt 2024-05-09T09:27:02.227 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-palmas-clk32kg-clocks.txt 2024-05-09T09:27:02.229 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-s5pv210-audss.txt 2024-05-09T09:27:02.232 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clock-bindings.txt 2024-05-09T09:27:02.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:01 smithi012 ceph-mon[30140]: pgmap v411: 129 pgs: 129 active+clean; 74 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.1 MiB/s wr, 73 op/s 2024-05-09T09:27:02.234 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clps711x-clock.txt 2024-05-09T09:27:02.237 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/efm32-clock.txt 2024-05-09T09:27:02.240 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/emev2-clock.txt 2024-05-09T09:27:02.242 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos3250-clock.txt 2024-05-09T09:27:02.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:01 smithi170 ceph-mon[33632]: pgmap v411: 129 pgs: 129 active+clean; 74 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.1 MiB/s wr, 73 op/s 2024-05-09T09:27:02.244 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4-clock.txt 2024-05-09T09:27:02.247 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4415-clock.txt 2024-05-09T09:27:02.249 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5250-clock.txt 2024-05-09T09:27:02.251 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5260-clock.txt 2024-05-09T09:27:02.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:01 smithi016 ceph-mon[33335]: pgmap v411: 129 pgs: 129 active+clean; 74 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.1 MiB/s wr, 73 op/s 2024-05-09T09:27:02.254 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5410-clock.txt 2024-05-09T09:27:02.257 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5420-clock.txt 2024-05-09T09:27:02.259 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5440-clock.txt 2024-05-09T09:27:02.261 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos7-clock.txt 2024-05-09T09:27:02.264 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-clock.txt 2024-05-09T09:27:02.266 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-factor-clock.txt 2024-05-09T09:27:02.269 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/gpio-gate-clock.txt 2024-05-09T09:27:02.271 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hi3620-clock.txt 2024-05-09T09:27:02.273 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hix5hd2-clock.txt 2024-05-09T09:27:02.276 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx1-clock.txt 2024-05-09T09:27:02.279 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx21-clock.txt 2024-05-09T09:27:02.281 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx23-clock.txt 2024-05-09T09:27:02.283 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx25-clock.txt 2024-05-09T09:27:02.286 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx27-clock.txt 2024-05-09T09:27:02.288 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx28-clock.txt 2024-05-09T09:27:02.290 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx31-clock.txt 2024-05-09T09:27:02.293 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx35-clock.txt 2024-05-09T09:27:02.295 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx5-clock.txt 2024-05-09T09:27:02.298 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6q-clock.txt 2024-05-09T09:27:02.300 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sl-clock.txt 2024-05-09T09:27:02.303 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sx-clock.txt 2024-05-09T09:27:02.305 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-gate.txt 2024-05-09T09:27:02.307 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-pll.txt 2024-05-09T09:27:02.310 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/lsi,axm5516-clks.txt 2024-05-09T09:27:02.313 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,mmp2.txt 2024-05-09T09:27:02.315 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa168.txt 2024-05-09T09:27:02.318 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa910.txt 2024-05-09T09:27:02.320 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77686.txt 2024-05-09T09:27:02.323 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77802.txt 2024-05-09T09:27:02.325 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/moxa,moxart-clock.txt 2024-05-09T09:27:02.327 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt 2024-05-09T09:27:02.330 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-corediv-clock.txt 2024-05-09T09:27:02.333 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt 2024-05-09T09:27:02.335 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-gated-clock.txt 2024-05-09T09:27:02.338 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nspire-clock.txt 2024-05-09T09:27:02.341 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra114-car.txt 2024-05-09T09:27:02.344 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra124-car.txt 2024-05-09T09:27:02.347 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.txt 2024-05-09T09:27:02.351 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra30-car.txt 2024-05-09T09:27:02.353 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/prima2-clock.txt 2024-05-09T09:27:02.356 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/pxa-clock.txt 2024-05-09T09:27:02.359 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,gcc.txt 2024-05-09T09:27:02.361 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,lcc.txt 2024-05-09T09:27:02.364 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,mmcc.txt 2024-05-09T09:27:02.367 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qoriq-clock.txt 2024-05-09T09:27:02.369 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-div6-clocks.txt 2024-05-09T09:27:02.374 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt 2024-05-09T09:27:02.375 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a73a4-cpg-clocks.txt 2024-05-09T09:27:02.377 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7740-cpg-clocks.txt 2024-05-09T09:27:02.380 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7779-cpg-clocks.txt 2024-05-09T09:27:02.382 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rcar-gen2-cpg-clocks.txt 2024-05-09T09:27:02.384 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rz-cpg-clocks.txt 2024-05-09T09:27:02.387 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,sh73a0-cpg-clocks.txt 2024-05-09T09:27:02.389 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt 2024-05-09T09:27:02.391 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt 2024-05-09T09:27:02.394 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip.txt 2024-05-09T09:27:02.396 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2410-clock.txt 2024-05-09T09:27:02.398 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2412-clock.txt 2024-05-09T09:27:02.401 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2443-clock.txt 2024-05-09T09:27:02.404 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c64xx-clock.txt 2024-05-09T09:27:02.406 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt 2024-05-09T09:27:02.409 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si5351.txt 2024-05-09T09:27:02.411 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si570.txt 2024-05-09T09:27:02.414 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st,nomadik.txt 2024-05-09T09:27:02.416 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/ 2024-05-09T09:27:02.418 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-divmux.txt 2024-05-09T09:27:02.420 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-mux.txt 2024-05-09T09:27:02.422 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-pll.txt 2024-05-09T09:27:02.425 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-prediv.txt 2024-05-09T09:27:02.427 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-vcc.txt 2024-05-09T09:27:02.430 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen.txt 2024-05-09T09:27:02.432 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,flexgen.txt 2024-05-09T09:27:02.435 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,quadfs.txt 2024-05-09T09:27:02.437 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ste-u300-syscon-clock.txt 2024-05-09T09:27:02.440 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/sunxi.txt 2024-05-09T09:27:02.443 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti,cdce706.txt 2024-05-09T09:27:02.445 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti-keystone-pllctrl.txt 2024-05-09T09:27:02.448 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/ 2024-05-09T09:27:02.450 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/apll.txt 2024-05-09T09:27:02.452 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/autoidle.txt 2024-05-09T09:27:02.455 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/clockdomain.txt 2024-05-09T09:27:02.458 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/composite.txt 2024-05-09T09:27:02.460 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/divider.txt 2024-05-09T09:27:02.463 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dpll.txt 2024-05-09T09:27:02.465 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dra7-atl.txt 2024-05-09T09:27:02.468 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fapll.txt 2024-05-09T09:27:02.470 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fixed-factor-clock.txt 2024-05-09T09:27:02.473 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/gate.txt 2024-05-09T09:27:02.475 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/interface.txt 2024-05-09T09:27:02.478 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/mux.txt 2024-05-09T09:27:02.481 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vf610-clock.txt 2024-05-09T09:27:02.483 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vt8500.txt 2024-05-09T09:27:02.486 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/xgene.txt 2024-05-09T09:27:02.488 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/zynq-7000.txt 2024-05-09T09:27:02.491 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/ 2024-05-09T09:27:02.494 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/arm_big_little_dt.txt 2024-05-09T09:27:02.496 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-dt.txt 2024-05-09T09:27:02.498 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-exynos5440.txt 2024-05-09T09:27:02.501 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-spear.txt 2024-05-09T09:27:02.503 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/ 2024-05-09T09:27:02.506 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/amd-ccp.txt 2024-05-09T09:27:02.508 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/atmel-crypto.txt 2024-05-09T09:27:02.511 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-dcp.txt 2024-05-09T09:27:02.513 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-imx-sahara.txt 2024-05-09T09:27:02.516 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec2.txt 2024-05-09T09:27:02.519 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec4.txt 2024-05-09T09:27:02.521 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec6.txt 2024-05-09T09:27:02.524 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/mv_cesa.txt 2024-05-09T09:27:02.527 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-aes.txt 2024-05-09T09:27:02.529 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-des.txt 2024-05-09T09:27:02.532 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-sham.txt 2024-05-09T09:27:02.534 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/picochip-spacc.txt 2024-05-09T09:27:02.537 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/qcom-qce.txt 2024-05-09T09:27:02.539 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/samsung-sss.txt 2024-05-09T09:27:02.542 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/ 2024-05-09T09:27:02.545 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/ 2024-05-09T09:27:02.546 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/exynos-ppmu.txt 2024-05-09T09:27:02.549 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ 2024-05-09T09:27:02.551 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/arm-pl330.txt 2024-05-09T09:27:02.553 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-dma.txt 2024-05-09T09:27:02.556 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-xdma.txt 2024-05-09T09:27:02.558 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/bcm2835-dma.txt 2024-05-09T09:27:02.561 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/dma.txt 2024-05-09T09:27:02.563 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-edma.txt 2024-05-09T09:27:02.566 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-dma.txt 2024-05-09T09:27:02.568 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-sdma.txt 2024-05-09T09:27:02.571 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-mxs-dma.txt 2024-05-09T09:27:02.573 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/img-mdc-dma.txt 2024-05-09T09:27:02.578 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/k3dma.txt 2024-05-09T09:27:02.578 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mmp-dma.txt 2024-05-09T09:27:02.580 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/moxa,moxart-dma.txt 2024-05-09T09:27:02.582 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mpc512x-dma.txt 2024-05-09T09:27:02.585 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mv-xor.txt 2024-05-09T09:27:02.587 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/nbpfaxi.txt 2024-05-09T09:27:02.590 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_adm.txt 2024-05-09T09:27:02.593 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt 2024-05-09T09:27:02.595 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/rcar-audmapp.txt 2024-05-09T09:27:02.597 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt 2024-05-09T09:27:02.599 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/shdma.txt 2024-05-09T09:27:02.602 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sirfsoc-dma.txt 2024-05-09T09:27:02.605 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/snps-dma.txt 2024-05-09T09:27:02.607 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-coh901318.txt 2024-05-09T09:27:02.610 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-dma40.txt 2024-05-09T09:27:02.613 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sun6i-dma.txt 2024-05-09T09:27:02.615 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/tegra20-apbdma.txt 2024-05-09T09:27:02.618 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ti-edma.txt 2024-05-09T09:27:02.620 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/ 2024-05-09T09:27:02.622 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt 2024-05-09T09:27:02.624 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt 2024-05-09T09:27:02.627 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/ 2024-05-09T09:27:02.630 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/ 2024-05-09T09:27:02.632 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/marvell,dove-lcd.txt 2024-05-09T09:27:02.634 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/ 2024-05-09T09:27:02.637 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/hlcdc-dc.txt 2024-05-09T09:27:02.639 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/ 2024-05-09T09:27:02.642 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/dw_hdmi.txt 2024-05-09T09:27:02.645 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/ 2024-05-09T09:27:02.648 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/tda998x.txt 2024-05-09T09:27:02.650 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ 2024-05-09T09:27:02.654 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/fsl-imx-drm.txt 2024-05-09T09:27:02.656 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/hdmi.txt 2024-05-09T09:27:02.660 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ldb.txt 2024-05-09T09:27:02.663 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/ 2024-05-09T09:27:02.665 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/gpu.txt 2024-05-09T09:27:02.668 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/hdmi.txt 2024-05-09T09:27:02.670 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/mdp.txt 2024-05-09T09:27:02.672 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/ 2024-05-09T09:27:02.675 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/panel.txt 2024-05-09T09:27:02.677 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/slave.txt 2024-05-09T09:27:02.680 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tfp410.txt 2024-05-09T09:27:02.682 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tilcdc.txt 2024-05-09T09:27:02.685 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/eeprom.txt 2024-05-09T09:27:02.689 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/ 2024-05-09T09:27:02.691 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-palmas.txt 2024-05-09T09:27:02.693 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-rt8973a.txt 2024-05-09T09:27:02.696 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-sm5502.txt 2024-05-09T09:27:02.698 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/ 2024-05-09T09:27:02.701 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/mxsfb.txt 2024-05-09T09:27:02.703 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/sm501fb.txt 2024-05-09T09:27:02.706 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/ 2024-05-09T09:27:02.709 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/altera-socfpga-fpga-mgr.txt 2024-05-09T09:27:02.711 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/ 2024-05-09T09:27:02.713 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.txt 2024-05-09T09:27:02.715 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/ 2024-05-09T09:27:02.718 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/8xxx_gpio.txt 2024-05-09T09:27:02.720 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/abilis,tb10x-gpio.txt 2024-05-09T09:27:02.723 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cavium-octeon-gpio.txt 2024-05-09T09:27:02.726 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cirrus,clps711x-mctrl-gpio.txt 2024-05-09T09:27:02.728 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fsl-imx-gpio.txt 2024-05-09T09:27:02.730 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fujitsu,mb86s70-gpio.txt 2024-05-09T09:27:02.733 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74x164.txt 2024-05-09T09:27:02.735 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74xx-mmio.txt 2024-05-09T09:27:02.738 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-adnp.txt 2024-05-09T09:27:02.740 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-bcm-kona.txt 2024-05-09T09:27:02.742 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-clps711x.txt 2024-05-09T09:27:02.745 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-davinci.txt 2024-05-09T09:27:02.747 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-dsp-keystone.txt 2024-05-09T09:27:02.749 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-fan.txt 2024-05-09T09:27:02.752 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-grgpio.txt 2024-05-09T09:27:02.754 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-lp3943.txt 2024-05-09T09:27:02.757 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-max732x.txt 2024-05-09T09:27:02.759 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mcp23s08.txt 2024-05-09T09:27:02.762 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mm-lantiq.txt 2024-05-09T09:27:02.764 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-msm.txt 2024-05-09T09:27:02.767 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mvebu.txt 2024-05-09T09:27:02.770 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mxs.txt 2024-05-09T09:27:02.773 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-nmk.txt 2024-05-09T09:27:02.775 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-omap.txt 2024-05-09T09:27:02.778 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-palmas.txt 2024-05-09T09:27:02.780 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pca953x.txt 2024-05-09T09:27:02.782 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pcf857x.txt 2024-05-09T09:27:02.808 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-poweroff.txt 2024-05-09T09:27:02.810 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-restart.txt 2024-05-09T09:27:02.813 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-samsung.txt 2024-05-09T09:27:02.816 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stericsson-coh901.txt 2024-05-09T09:27:02.818 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stmpe.txt 2024-05-09T09:27:02.821 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stp-xway.txt 2024-05-09T09:27:02.823 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-sx150x.txt 2024-05-09T09:27:02.826 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-twl4030.txt 2024-05-09T09:27:02.828 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090-pdc.txt 2024-05-09T09:27:02.831 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090.txt 2024-05-09T09:27:02.833 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-vf610.txt 2024-05-09T09:27:02.835 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene-sb.txt 2024-05-09T09:27:02.838 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene.txt 2024-05-09T09:27:02.840 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xilinx.txt 2024-05-09T09:27:02.843 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zevio.txt 2024-05-09T09:27:02.845 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zynq.txt 2024-05-09T09:27:02.848 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio.txt 2024-05-09T09:27:02.851 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_atmel.txt 2024-05-09T09:27:02.853 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_lpc32xx.txt 2024-05-09T09:27:02.855 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/moxa,moxart-gpio.txt 2024-05-09T09:27:02.858 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/mrvl-gpio.txt 2024-05-09T09:27:02.860 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/nvidia,tegra20-gpio.txt 2024-05-09T09:27:02.863 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/pl061-gpio.txt 2024-05-09T09:27:02.866 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt 2024-05-09T09:27:02.868 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/snps-dwapb-gpio.txt 2024-05-09T09:27:02.871 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/sodaville.txt 2024-05-09T09:27:02.873 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/spear_spics.txt 2024-05-09T09:27:02.876 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/ 2024-05-09T09:27:02.879 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,gk20a.txt 2024-05-09T09:27:02.881 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,tegra20-host1x.txt 2024-05-09T09:27:02.884 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-g2d.txt 2024-05-09T09:27:02.886 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-rotator.txt 2024-05-09T09:27:02.890 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/st,stih4xx.txt 2024-05-09T09:27:02.893 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/graph.txt 2024-05-09T09:27:02.896 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/ 2024-05-09T09:27:02.898 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/hid-over-i2c.txt 2024-05-09T09:27:02.900 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/ 2024-05-09T09:27:02.903 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/client-devices.txt 2024-05-09T09:27:02.905 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/nokia-modem.txt 2024-05-09T09:27:02.908 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/omap-ssi.txt 2024-05-09T09:27:02.910 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ 2024-05-09T09:27:02.913 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ads1015.txt 2024-05-09T09:27:02.916 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/g762.txt 2024-05-09T09:27:02.919 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ibmpowernv.txt 2024-05-09T09:27:02.922 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/lm90.txt 2024-05-09T09:27:02.925 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ltc2978.txt 2024-05-09T09:27:02.927 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ntc_thermistor.txt 2024-05-09T09:27:02.930 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/pwm-fan.txt 2024-05-09T09:27:02.932 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/vexpress.txt 2024-05-09T09:27:02.935 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/ 2024-05-09T09:27:02.937 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/atmel-trng.txt 2024-05-09T09:27:02.940 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/omap_rng.txt 2024-05-09T09:27:02.943 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/timeriomem_rng.txt 2024-05-09T09:27:02.946 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ 2024-05-09T09:27:02.949 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,bcm2835-i2c.txt 2024-05-09T09:27:02.952 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,iproc-i2c.txt 2024-05-09T09:27:02.954 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt 2024-05-09T09:27:02.956 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-at91.txt 2024-05-09T09:27:02.959 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-axxia.txt 2024-05-09T09:27:02.962 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-bcm-kona.txt 2024-05-09T09:27:02.964 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cadence.txt 2024-05-09T09:27:02.967 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cbus-gpio.txt 2024-05-09T09:27:02.969 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cros-ec-tunnel.txt 2024-05-09T09:27:02.972 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-davinci.txt 2024-05-09T09:27:02.975 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-designware.txt 2024-05-09T09:27:02.977 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-efm32.txt 2024-05-09T09:27:02.979 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt 2024-05-09T09:27:02.982 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-gpio.txt 2024-05-09T09:27:02.985 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-hix5hd2.txt 2024-05-09T09:27:02.990 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-img-scb.txt 2024-05-09T09:27:02.990 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-imx.txt 2024-05-09T09:27:02.993 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-meson.txt 2024-05-09T09:27:02.996 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mpc.txt 2024-05-09T09:27:02.998 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-gpio.txt 2024-05-09T09:27:03.001 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pca954x.txt 2024-05-09T09:27:03.004 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pinctrl.txt 2024-05-09T09:27:03.006 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux.txt 2024-05-09T09:27:03.009 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt 2024-05-09T09:27:03.011 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mxs.txt 2024-05-09T09:27:03.014 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-nomadik.txt 2024-05-09T09:27:03.016 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-ocores.txt 2024-05-09T09:27:03.019 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-octeon.txt 2024-05-09T09:27:03.021 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-omap.txt 2024-05-09T09:27:03.024 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-opal.txt 2024-05-09T09:27:03.027 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pnx.txt 2024-05-09T09:27:03.029 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa-pci-ce4100.txt 2024-05-09T09:27:03.032 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa.txt 2024-05-09T09:27:03.034 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rcar.txt 2024-05-09T09:27:03.037 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-riic.txt 2024-05-09T09:27:03.040 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rk3x.txt 2024-05-09T09:27:03.042 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-s3c2410.txt 2024-05-09T09:27:03.044 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sh_mobile.txt 2024-05-09T09:27:03.047 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sirf.txt 2024-05-09T09:27:03.051 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st-ddci2c.txt 2024-05-09T09:27:03.052 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st.txt 2024-05-09T09:27:03.055 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sunxi-p2wi.txt 2024-05-09T09:27:03.057 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-versatile.txt 2024-05-09T09:27:03.060 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-vt8500.txt 2024-05-09T09:27:03.062 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-xiic.txt 2024-05-09T09:27:03.065 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina209.txt 2024-05-09T09:27:03.067 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina2xx.txt 2024-05-09T09:27:03.069 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/max6697.txt 2024-05-09T09:27:03.072 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/nvidia,tegra20-i2c.txt 2024-05-09T09:27:03.074 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/qcom,i2c-qup.txt 2024-05-09T09:27:03.077 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ti,bq32k.txt 2024-05-09T09:27:03.080 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/trivial-devices.txt 2024-05-09T09:27:03.082 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/ 2024-05-09T09:27:03.085 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/ 2024-05-09T09:27:03.087 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/bma180.txt 2024-05-09T09:27:03.089 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/ 2024-05-09T09:27:03.092 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/at91_adc.txt 2024-05-09T09:27:03.095 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/cc10001_adc.txt 2024-05-09T09:27:03.098 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/max1027-adc.txt 2024-05-09T09:27:03.101 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/nuvoton-nau7802.txt 2024-05-09T09:27:03.104 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-iadc.txt 2024-05-09T09:27:03.107 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-vadc.txt 2024-05-09T09:27:03.110 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt 2024-05-09T09:27:03.112 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/twl4030-madc.txt 2024-05-09T09:27:03.115 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/vf610-adc.txt 2024-05-09T09:27:03.118 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/xilinx-xadc.txt 2024-05-09T09:27:03.122 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ 2024-05-09T09:27:03.125 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ad7303.txt 2024-05-09T09:27:03.128 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/max5821.txt 2024-05-09T09:27:03.131 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/ 2024-05-09T09:27:03.134 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/adf4350.txt 2024-05-09T09:27:03.136 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/ 2024-05-09T09:27:03.139 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/dht11.txt 2024-05-09T09:27:03.141 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/iio-bindings.txt 2024-05-09T09:27:03.144 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/ 2024-05-09T09:27:03.146 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/apds9300.txt 2024-05-09T09:27:03.149 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/cm36651.txt 2024-05-09T09:27:03.152 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/gp2ap020a00f.txt 2024-05-09T09:27:03.155 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/tsl2563.txt 2024-05-09T09:27:03.157 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ 2024-05-09T09:27:03.161 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt 2024-05-09T09:27:03.164 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/hmc5843.txt 2024-05-09T09:27:03.166 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/ 2024-05-09T09:27:03.169 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/as3935.txt 2024-05-09T09:27:03.171 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/sensorhub.txt 2024-05-09T09:27:03.175 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/st-sensors.txt 2024-05-09T09:27:03.177 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ 2024-05-09T09:27:03.181 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ads7846.txt 2024-05-09T09:27:03.183 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/atmel,maxtouch.txt 2024-05-09T09:27:03.186 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cap11xx.txt 2024-05-09T09:27:03.189 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/clps711x-keypad.txt 2024-05-09T09:27:03.191 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cros-ec-keyb.txt 2024-05-09T09:27:03.194 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/e3x0-button.txt 2024-05-09T09:27:03.196 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elan_i2c.txt 2024-05-09T09:27:03.199 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elants_i2c.txt 2024-05-09T09:27:03.201 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/fsl-mma8450.txt 2024-05-09T09:27:03.203 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-beeper.txt 2024-05-09T09:27:03.206 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys-polled.txt 2024-05-09T09:27:03.208 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys.txt 2024-05-09T09:27:03.211 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-matrix-keypad.txt 2024-05-09T09:27:03.214 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/imx-keypad.txt 2024-05-09T09:27:03.216 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/input-reset.txt 2024-05-09T09:27:03.218 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/lpc32xx-key.txt 2024-05-09T09:27:03.222 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/matrix-keymap.txt 2024-05-09T09:27:03.224 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/nvidia,tegra20-kbc.txt 2024-05-09T09:27:03.227 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/omap-keypad.txt 2024-05-09T09:27:03.230 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ps2keyb-mouse-apbps2.txt 2024-05-09T09:27:03.233 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pwm-beeper.txt 2024-05-09T09:27:03.234 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pxa27x-keypad.txt 2024-05-09T09:27:03.237 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-keypad.txt 2024-05-09T09:27:03.240 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-pwrkey.txt 2024-05-09T09:27:03.242 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-vib.txt 2024-05-09T09:27:03.245 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/regulator-haptic.txt 2024-05-09T09:27:03.248 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/rotary-encoder.txt 2024-05-09T09:27:03.250 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/samsung-keypad.txt 2024-05-09T09:27:03.253 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/spear-keyboard.txt 2024-05-09T09:27:03.255 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/st-keyscan.txt 2024-05-09T09:27:03.258 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/stmpe-keypad.txt 2024-05-09T09:27:03.261 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/sun4i-lradc-keys.txt 2024-05-09T09:27:03.264 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tca8418_keypad.txt 2024-05-09T09:27:03.266 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv260x.txt 2024-05-09T09:27:03.268 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv2667.txt 2024-05-09T09:27:03.271 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,nspire-keypad.txt 2024-05-09T09:27:03.273 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,palmas-pwrbutton.txt 2024-05-09T09:27:03.276 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ 2024-05-09T09:27:03.278 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/auo_pixcir_ts.txt 2024-05-09T09:27:03.280 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/bu21013.txt 2024-05-09T09:27:03.283 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/edt-ft5x06.txt 2024-05-09T09:27:03.286 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/egalax-ts.txt 2024-05-09T09:27:03.288 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/lpc32xx-tsc.txt 2024-05-09T09:27:03.291 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/mms114.txt 2024-05-09T09:27:03.293 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/pixcir_i2c_ts.txt 2024-05-09T09:27:03.296 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sitronix-st1232.txt 2024-05-09T09:27:03.298 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/stmpe.txt 2024-05-09T09:27:03.301 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sun4i.txt 2024-05-09T09:27:03.303 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ti-tsc-adc.txt 2024-05-09T09:27:03.306 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/touchscreen.txt 2024-05-09T09:27:03.308 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2005.txt 2024-05-09T09:27:03.311 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2007.txt 2024-05-09T09:27:03.313 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/zforce_ts.txt 2024-05-09T09:27:03.316 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tps65218-pwrbutton.txt 2024-05-09T09:27:03.319 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-keypad.txt 2024-05-09T09:27:03.321 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-pwrbutton.txt 2024-05-09T09:27:03.324 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ 2024-05-09T09:27:03.327 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/abilis,tb10x-ictl.txt 2024-05-09T09:27:03.329 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun4i-ic.txt 2024-05-09T09:27:03.332 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun67i-sc-nmi.txt 2024-05-09T09:27:03.334 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/atmel,aic.txt 2024-05-09T09:27:03.337 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm2835-armctrl-ic.txt 2024-05-09T09:27:03.339 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm7120-l2-intc.txt 2024-05-09T09:27:03.342 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,l2-intc.txt 2024-05-09T09:27:03.344 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/cirrus,clps711x-intc.txt 2024-05-09T09:27:03.347 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/digicolor-ic.txt 2024-05-09T09:27:03.350 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/interrupts.txt 2024-05-09T09:27:03.352 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/lsi,zevio-intc.txt 2024-05-09T09:27:03.355 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt 2024-05-09T09:27:03.357 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,orion-intc.txt 2024-05-09T09:27:03.360 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/mips-gic.txt 2024-05-09T09:27:03.362 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/opencores,or1k-pic.txt 2024-05-09T09:27:03.364 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,intc-irqpin.txt 2024-05-09T09:27:03.367 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.txt 2024-05-09T09:27:03.369 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/samsung,s3c24xx-irq.txt 2024-05-09T09:27:03.372 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt 2024-05-09T09:27:03.374 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,keystone-irq.txt 2024-05-09T09:27:03.377 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,omap-intc-irq.txt 2024-05-09T09:27:03.379 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ 2024-05-09T09:27:03.382 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/arm,smmu.txt 2024-05-09T09:27:03.384 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/iommu.txt 2024-05-09T09:27:03.387 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra20-gart.txt 2024-05-09T09:27:03.389 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra30-smmu.txt 2024-05-09T09:27:03.392 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/renesas,ipmmu-vmsa.txt 2024-05-09T09:27:03.394 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/rockchip,iommu.txt 2024-05-09T09:27:03.397 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt 2024-05-09T09:27:03.399 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ti,omap-iommu.txt 2024-05-09T09:27:03.402 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/ 2024-05-09T09:27:03.404 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/common.txt 2024-05-09T09:27:03.406 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-gpio.txt 2024-05-09T09:27:03.409 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp55xx.txt 2024-05-09T09:27:03.411 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp8860.txt 2024-05-09T09:27:03.414 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-ns2.txt 2024-05-09T09:27:03.683 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-pwm.txt 2024-05-09T09:27:03.685 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/pca963x.txt 2024-05-09T09:27:03.688 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/register-bit-led.txt 2024-05-09T09:27:03.690 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/tca6507.txt 2024-05-09T09:27:03.694 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/ 2024-05-09T09:27:03.696 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2-timings.txt 2024-05-09T09:27:03.699 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2.txt 2024-05-09T09:27:03.701 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/ 2024-05-09T09:27:03.704 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/altera-mailbox.txt 2024-05-09T09:27:03.707 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/mailbox.txt 2024-05-09T09:27:03.710 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt 2024-05-09T09:27:03.713 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/marvell.txt 2024-05-09T09:27:03.716 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ 2024-05-09T09:27:03.718 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/atmel-isi.txt 2024-05-09T09:27:03.720 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/coda.txt 2024-05-09T09:27:03.723 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-fimc-lite.txt 2024-05-09T09:27:03.725 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-jpeg-codec.txt 2024-05-09T09:27:03.728 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos4-fimc-is.txt 2024-05-09T09:27:03.730 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos5-gsc.txt 2024-05-09T09:27:03.733 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/gpio-ir-receiver.txt 2024-05-09T09:27:03.735 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/hix5hd2-ir.txt 2024-05-09T09:27:03.737 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ 2024-05-09T09:27:03.739 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7343.txt 2024-05-09T09:27:03.741 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7604.txt 2024-05-09T09:27:03.744 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9m111.txt 2024-05-09T09:27:03.746 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9p031.txt 2024-05-09T09:27:03.749 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/nokia,smia.txt 2024-05-09T09:27:03.752 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ths8200.txt 2024-05-09T09:27:03.755 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp514x.txt 2024-05-09T09:27:03.757 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp7002.txt 2024-05-09T09:27:03.760 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/img-ir-rev1.txt 2024-05-09T09:27:03.763 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/meson-ir.txt 2024-05-09T09:27:03.765 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/pxa-camera.txt 2024-05-09T09:27:03.768 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/rcar_vin.txt 2024-05-09T09:27:03.771 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/renesas,vsp1.txt 2024-05-09T09:27:03.773 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/s5p-mfc.txt 2024-05-09T09:27:03.776 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-fimc.txt 2024-05-09T09:27:03.778 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-mipi-csis.txt 2024-05-09T09:27:03.781 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5c73m3.txt 2024-05-09T09:27:03.783 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k5baf.txt 2024-05-09T09:27:03.786 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k6a3.txt 2024-05-09T09:27:03.788 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sh_mobile_ceu.txt 2024-05-09T09:27:03.796 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/si4713.txt 2024-05-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/st-rc.txt 2024-05-09T09:27:03.815 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sunxi-ir.txt 2024-05-09T09:27:03.818 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ti-am437x-vpfe.txt 2024-05-09T09:27:03.820 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/video-interfaces.txt 2024-05-09T09:27:03.823 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ 2024-05-09T09:27:03.826 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ 2024-05-09T09:27:03.827 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt 2024-05-09T09:27:03.829 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-devbus.txt 2024-05-09T09:27:03.834 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-sdram-controller.txt 2024-05-09T09:27:03.836 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra-mc.txt 2024-05-09T09:27:03.839 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/renesas-memory-controllers.txt 2024-05-09T09:27:03.842 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/synopsys.txt 2024-05-09T09:27:03.844 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti-aemif.txt 2024-05-09T09:27:03.847 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/ 2024-05-09T09:27:03.849 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/emif.txt 2024-05-09T09:27:03.851 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/ 2024-05-09T09:27:03.854 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta-intc.txt 2024-05-09T09:27:03.856 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta.txt 2024-05-09T09:27:03.859 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/pdc-intc.txt 2024-05-09T09:27:03.862 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ 2024-05-09T09:27:03.864 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/88pm860x.txt 2024-05-09T09:27:03.866 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ab8500.txt 2024-05-09T09:27:03.869 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/arizona.txt 2024-05-09T09:27:03.872 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3711.txt 2024-05-09T09:27:03.874 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3722.txt 2024-05-09T09:27:03.877 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-gpbr.txt 2024-05-09T09:27:03.880 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-hlcdc.txt 2024-05-09T09:27:03.882 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-matrix.txt 2024-05-09T09:27:03.885 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-smc.txt 2024-05-09T09:27:03.887 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bcm590xx.txt 2024-05-09T09:27:03.890 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bfticu.txt 2024-05-09T09:27:03.893 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/cros-ec.txt 2024-05-09T09:27:03.895 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9052-i2c.txt 2024-05-09T09:27:03.897 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9055.txt 2024-05-09T09:27:03.900 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9063.txt 2024-05-09T09:27:03.902 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/hi6421.txt 2024-05-09T09:27:03.905 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/lp3943.txt 2024-05-09T09:27:03.908 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max14577.txt 2024-05-09T09:27:03.911 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77686.txt 2024-05-09T09:27:03.914 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77693.txt 2024-05-09T09:27:03.916 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8925.txt 2024-05-09T09:27:03.919 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8998.txt 2024-05-09T09:27:03.922 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/mc13xxx.txt 2024-05-09T09:27:03.924 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-host.txt 2024-05-09T09:27:03.927 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-tll.txt 2024-05-09T09:27:03.929 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/palmas.txt 2024-05-09T09:27:03.932 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom,spmi-pmic.txt 2024-05-09T09:27:03.934 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-pm8xxx.txt 2024-05-09T09:27:03.937 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-rpm.txt 2024-05-09T09:27:03.940 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qriox.txt 2024-05-09T09:27:03.943 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rk808.txt 2024-05-09T09:27:03.946 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rn5t618.txt 2024-05-09T09:27:03.948 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mpa01.txt 2024-05-09T09:27:03.950 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mps11.txt 2024-05-09T09:27:03.953 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/stmpe.txt 2024-05-09T09:27:03.955 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/sun6i-prcm.txt 2024-05-09T09:27:03.958 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/syscon.txt 2024-05-09T09:27:03.961 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tc3589x.txt 2024-05-09T09:27:03.964 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ti-keystone-devctrl.txt 2024-05-09T09:27:03.967 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps6507x.txt 2024-05-09T09:27:03.970 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps65910.txt 2024-05-09T09:27:03.973 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl-familly.txt 2024-05-09T09:27:03.976 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-audio.txt 2024-05-09T09:27:03.978 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-power.txt 2024-05-09T09:27:03.981 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl6040.txt 2024-05-09T09:27:03.984 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/ 2024-05-09T09:27:03.986 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/ 2024-05-09T09:27:03.988 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/mipi-dsi-bus.txt 2024-05-09T09:27:03.990 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/nvidia,tegra114-mipi.txt 2024-05-09T09:27:03.993 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ 2024-05-09T09:27:03.996 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/ 2024-05-09T09:27:03.997 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bcm3384-intc.txt 2024-05-09T09:27:04.000 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bmips.txt 2024-05-09T09:27:04.002 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/cm-dsl.txt 2024-05-09T09:27:04.005 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/usb.txt 2024-05-09T09:27:04.007 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ 2024-05-09T09:27:04.009 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/bootbus.txt 2024-05-09T09:27:04.012 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/cib.txt 2024-05-09T09:27:04.014 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu.txt 2024-05-09T09:27:04.017 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu2.txt 2024-05-09T09:27:04.020 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/dma-engine.txt 2024-05-09T09:27:04.022 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/uctl.txt 2024-05-09T09:27:04.025 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cpu_irq.txt 2024-05-09T09:27:04.028 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ralink.txt 2024-05-09T09:27:04.030 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ 2024-05-09T09:27:04.033 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/allwinner,sunxi-sid.txt 2024-05-09T09:27:04.035 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/arm-charlcd.txt 2024-05-09T09:27:04.037 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/at25.txt 2024-05-09T09:27:04.040 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/atmel-ssc.txt 2024-05-09T09:27:04.042 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/bmp085.txt 2024-05-09T09:27:04.045 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt 2024-05-09T09:27:04.048 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ifm-csi.txt 2024-05-09T09:27:04.050 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/lis302.txt 2024-05-09T09:27:04.053 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.txt 2024-05-09T09:27:04.056 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/smc.txt 2024-05-09T09:27:04.059 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/sram.txt 2024-05-09T09:27:04.062 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ti,dac7512.txt 2024-05-09T09:27:04.064 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ 2024-05-09T09:27:04.067 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/arasan,sdhci.txt 2024-05-09T09:27:04.070 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/atmel-hsmci.txt 2024-05-09T09:27:04.073 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/brcm,bcm2835-sdhci.txt 2024-05-09T09:27:04.075 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/davinci_mmc.txt 2024-05-09T09:27:04.078 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/exynos-dw-mshc.txt 2024-05-09T09:27:04.081 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-esdhc.txt 2024-05-09T09:27:04.084 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt 2024-05-09T09:27:04.086 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-mmc.txt 2024-05-09T09:27:04.089 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/img-dw-mshc.txt 2024-05-09T09:27:04.091 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/k3-dw-mshc.txt 2024-05-09T09:27:04.095 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/kona-sdhci.txt 2024-05-09T09:27:04.097 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt 2024-05-09T09:27:04.100 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-simple.txt 2024-05-09T09:27:04.102 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-spi-slot.txt 2024-05-09T09:27:04.105 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc.txt 2024-05-09T09:27:04.108 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmci.txt 2024-05-09T09:27:04.110 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/moxa,moxart-mmc.txt 2024-05-09T09:27:04.113 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mxs-mmc.txt 2024-05-09T09:27:04.115 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.txt 2024-05-09T09:27:04.117 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/orion-sdio.txt 2024-05-09T09:27:04.119 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/pxa-mmc.txt 2024-05-09T09:27:04.123 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/renesas,mmcif.txt 2024-05-09T09:27:04.125 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.txt 2024-05-09T09:27:04.127 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/samsung-sdhci.txt 2024-05-09T09:27:04.130 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-dove.txt 2024-05-09T09:27:04.132 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-fujitsu.txt 2024-05-09T09:27:04.134 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-msm.txt 2024-05-09T09:27:04.137 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-pxa.txt 2024-05-09T09:27:04.139 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-sirf.txt 2024-05-09T09:27:04.142 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-spear.txt 2024-05-09T09:27:04.144 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-st.txt 2024-05-09T09:27:04.147 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/socfpga-dw-mshc.txt 2024-05-09T09:27:04.149 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sunxi-mmc.txt 2024-05-09T09:27:04.152 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt 2024-05-09T09:27:04.155 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap-hsmmc.txt 2024-05-09T09:27:04.157 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap.txt 2024-05-09T09:27:04.160 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/tmio_mmc.txt 2024-05-09T09:27:04.163 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/usdhi6rol0.txt 2024-05-09T09:27:04.166 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/vt8500-sdmmc.txt 2024-05-09T09:27:04.169 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/ 2024-05-09T09:27:04.171 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/arm-versatile.txt 2024-05-09T09:27:04.173 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-dataflash.txt 2024-05-09T09:27:04.176 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-nand.txt 2024-05-09T09:27:04.179 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/davinci-nand.txt 2024-05-09T09:27:04.181 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/denali-nand.txt 2024-05-09T09:27:04.183 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/diskonchip.txt 2024-05-09T09:27:04.186 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/elm.txt 2024-05-09T09:27:04.188 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/flctl-nand.txt 2024-05-09T09:27:04.190 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-quadspi.txt 2024-05-09T09:27:04.193 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-upm-nand.txt 2024-05-09T09:27:04.196 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsmc-nand.txt 2024-05-09T09:27:04.199 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpio-control-nand.txt 2024-05-09T09:27:04.201 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nand.txt 2024-05-09T09:27:04.205 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nor.txt 2024-05-09T09:27:04.207 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-onenand.txt 2024-05-09T09:27:04.210 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmi-nand.txt 2024-05-09T09:27:04.213 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/hisi504-nand.txt 2024-05-09T09:27:04.215 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-mlc.txt 2024-05-09T09:27:04.218 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-slc.txt 2024-05-09T09:27:04.220 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/m25p80.txt 2024-05-09T09:27:04.223 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mtd-physmap.txt 2024-05-09T09:27:04.226 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mxc-nand.txt 2024-05-09T09:27:04.228 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/nand.txt 2024-05-09T09:27:04.231 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/orion-nand.txt 2024-05-09T09:27:04.233 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/partition.txt 2024-05-09T09:27:04.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:03 smithi012 ceph-mon[30140]: pgmap v412: 129 pgs: 129 active+clean; 83 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.9 MiB/s wr, 104 op/s 2024-05-09T09:27:04.236 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt 2024-05-09T09:27:04.239 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/spear_smi.txt 2024-05-09T09:27:04.241 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/st-fsm.txt 2024-05-09T09:27:04.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:03 smithi170 ceph-mon[33632]: pgmap v412: 129 pgs: 129 active+clean; 83 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.9 MiB/s wr, 104 op/s 2024-05-09T09:27:04.244 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/sunxi-nand.txt 2024-05-09T09:27:04.247 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ 2024-05-09T09:27:04.250 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-emac.txt 2024-05-09T09:27:04.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:03 smithi016 ceph-mon[33335]: pgmap v412: 129 pgs: 129 active+clean; 83 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.9 MiB/s wr, 104 op/s 2024-05-09T09:27:04.253 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-mdio.txt 2024-05-09T09:27:04.256 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.txt 2024-05-09T09:27:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/altera_tse.txt 2024-05-09T09:27:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe-phy.txt 2024-05-09T09:27:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe.txt 2024-05-09T09:27:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/apm-xgene-enet.txt 2024-05-09T09:27:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/arc_emac.txt 2024-05-09T09:27:04.272 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcm87xx.txt 2024-05-09T09:27:04.274 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcmgenet.txt 2024-05-09T09:27:04.277 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-mdio-unimac.txt 2024-05-09T09:27:04.280 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-sf2.txt 2024-05-09T09:27:04.283 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-systemport.txt 2024-05-09T09:27:04.285 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/calxeda-xgmac.txt 2024-05-09T09:27:04.288 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/ 2024-05-09T09:27:04.290 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/atmel-can.txt 2024-05-09T09:27:04.292 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/c_can.txt 2024-05-09T09:27:04.294 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/cc770.txt 2024-05-09T09:27:04.297 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/fsl-flexcan.txt 2024-05-09T09:27:04.299 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/grcan.txt 2024-05-09T09:27:04.302 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/m_can.txt 2024-05-09T09:27:04.305 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt 2024-05-09T09:27:04.307 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/mpc5xxx-mscan.txt 2024-05-09T09:27:04.310 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/rcar_can.txt 2024-05-09T09:27:04.313 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/sja1000.txt 2024-05-09T09:27:04.315 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/xilinx_can.txt 2024-05-09T09:27:04.318 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mdio.txt 2024-05-09T09:27:04.321 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mix.txt 2024-05-09T09:27:04.324 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-pip.txt 2024-05-09T09:27:04.327 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cdns-emac.txt 2024-05-09T09:27:04.330 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw-phy-sel.txt 2024-05-09T09:27:04.332 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw.txt 2024-05-09T09:27:04.335 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davicom-dm9000.txt 2024-05-09T09:27:04.338 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci-mdio.txt 2024-05-09T09:27:04.341 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci_emac.txt 2024-05-09T09:27:04.343 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/ 2024-05-09T09:27:04.345 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/dsa.txt 2024-05-09T09:27:04.348 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/emac_rockchip.txt 2024-05-09T09:27:04.352 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ethernet.txt 2024-05-09T09:27:04.356 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fixed-link.txt 2024-05-09T09:27:04.358 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-fec.txt 2024-05-09T09:27:04.360 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-tsec-phy.txt 2024-05-09T09:27:04.363 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/gpmc-eth.txt 2024-05-09T09:27:04.366 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hip04-net.txt 2024-05-09T09:27:04.369 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hix5hd2-gmac.txt 2024-05-09T09:27:04.372 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/ 2024-05-09T09:27:04.374 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/at86rf230.txt 2024-05-09T09:27:04.376 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/cc2520.txt 2024-05-09T09:27:04.379 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/keystone-netcp.txt 2024-05-09T09:27:04.382 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/lpc-eth.txt 2024-05-09T09:27:04.385 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/macb.txt 2024-05-09T09:27:04.388 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt 2024-05-09T09:27:04.391 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-mdio.txt 2024-05-09T09:27:04.394 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-net.txt 2024-05-09T09:27:04.397 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pp2.txt 2024-05-09T09:27:04.400 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pxa168.txt 2024-05-09T09:27:04.403 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-gpio.txt 2024-05-09T09:27:04.405 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-gpio.txt 2024-05-09T09:27:04.408 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-mmioreg.txt 2024-05-09T09:27:04.411 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux.txt 2024-05-09T09:27:04.414 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/meson-dwmac.txt 2024-05-09T09:27:04.418 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ks8851.txt 2024-05-09T09:27:04.419 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ksz90x1.txt 2024-05-09T09:27:04.422 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel.txt 2024-05-09T09:27:04.425 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/moxa,moxart-mac.txt 2024-05-09T09:27:04.427 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/ 2024-05-09T09:27:04.429 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/pn544.txt 2024-05-09T09:27:04.432 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfca.txt 2024-05-09T09:27:04.435 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfcb.txt 2024-05-09T09:27:04.438 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/trf7970a.txt 2024-05-09T09:27:04.441 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/opencores-ethoc.txt 2024-05-09T09:27:04.444 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/phy.txt 2024-05-09T09:27:04.447 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/qca-qca7000-spi.txt 2024-05-09T09:27:04.449 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/rockchip-dwmac.txt 2024-05-09T09:27:04.452 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/samsung-sxgbe.txt 2024-05-09T09:27:04.455 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sh_eth.txt 2024-05-09T09:27:04.457 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc-lan91c111.txt 2024-05-09T09:27:04.460 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc911x.txt 2024-05-09T09:27:04.463 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/socfpga-dwmac.txt 2024-05-09T09:27:04.466 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sti-dwmac.txt 2024-05-09T09:27:04.469 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/stmmac.txt 2024-05-09T09:27:04.473 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-rhine.txt 2024-05-09T09:27:04.475 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-velocity.txt 2024-05-09T09:27:04.478 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ 2024-05-09T09:27:04.480 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/brcm,bcm43xx-fmac.txt 2024-05-09T09:27:04.484 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt 2024-05-09T09:27:04.487 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ti,wl1251.txt 2024-05-09T09:27:04.490 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/ 2024-05-09T09:27:04.494 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/nios2.txt 2024-05-09T09:27:04.497 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/timer.txt 2024-05-09T09:27:04.499 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/ 2024-05-09T09:27:04.502 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/nvidia,nvec.txt 2024-05-09T09:27:04.505 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/open-pic.txt 2024-05-09T09:27:04.508 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/ 2024-05-09T09:27:04.511 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101aw03.txt 2024-05-09T09:27:04.514 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101xtn01.txt 2024-05-09T09:27:04.517 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b116xw03.txt 2024-05-09T09:27:04.520 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133htn01.txt 2024-05-09T09:27:04.524 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133xtn01.txt 2024-05-09T09:27:04.527 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/avic,tm070ddh03.txt 2024-05-09T09:27:04.529 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wa01a.txt 2024-05-09T09:27:04.532 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wb03.txt 2024-05-09T09:27:04.535 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et057090dhu.txt 2024-05-09T09:27:04.537 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et070080dh6.txt 2024-05-09T09:27:04.540 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,etm0700g0dh6.txt 2024-05-09T09:27:04.543 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/foxlink,fl500wvr00-a0t.txt 2024-05-09T09:27:04.546 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/giantplus,gpg482739qs5.txt 2024-05-09T09:27:04.549 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hannstar,hsd070pww1.txt 2024-05-09T09:27:04.552 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hit,tx23d38vm0caa.txt 2024-05-09T09:27:04.555 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,g121i1-l01.txt 2024-05-09T09:27:04.558 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n116bge.txt 2024-05-09T09:27:04.561 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n156bge-l21.txt 2024-05-09T09:27:04.564 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,ld070wx3-sl01.txt 2024-05-09T09:27:04.567 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lh500wx1-sd03.txt 2024-05-09T09:27:04.570 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lp129qe.txt 2024-05-09T09:27:04.573 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/panasonic,vvx10f004b00.txt 2024-05-09T09:27:04.576 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ld9040.txt 2024-05-09T09:27:04.579 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ltn101nt05.txt 2024-05-09T09:27:04.582 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,s6e8aa0.txt 2024-05-09T09:27:04.585 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/sharp,lq101r1sx01.txt 2024-05-09T09:27:04.588 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/simple-panel.txt 2024-05-09T09:27:04.591 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ 2024-05-09T09:27:04.594 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/83xx-512x-pci.txt 2024-05-09T09:27:04.597 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/designware-pcie.txt 2024-05-09T09:27:04.599 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt 2024-05-09T09:27:04.601 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,pci.txt 2024-05-09T09:27:04.604 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/host-generic-pci.txt 2024-05-09T09:27:04.606 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/layerscape-pci.txt 2024-05-09T09:27:04.608 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/mvebu-pci.txt 2024-05-09T09:27:04.611 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt 2024-05-09T09:27:04.613 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-keystone.txt 2024-05-09T09:27:04.616 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-rcar-gen2.txt 2024-05-09T09:27:04.619 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci.txt 2024-05-09T09:27:04.621 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ralink,rt3883-pci.txt 2024-05-09T09:27:04.624 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/rcar-pci.txt 2024-05-09T09:27:04.626 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/samsung,exynos5440-pcie.txt 2024-05-09T09:27:04.630 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/spear13xx-pcie.txt 2024-05-09T09:27:04.631 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ti-pci.txt 2024-05-09T09:27:04.633 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/v3-v360epc-pci.txt 2024-05-09T09:27:04.636 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/versatile.txt 2024-05-09T09:27:04.638 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xgene-pci.txt 2024-05-09T09:27:04.641 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xilinx-pcie.txt 2024-05-09T09:27:04.643 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ 2024-05-09T09:27:04.646 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/apm-xgene-phy.txt 2024-05-09T09:27:04.648 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/bcm-phy.txt 2024-05-09T09:27:04.651 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-sata-phy.txt 2024-05-09T09:27:04.653 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-usb-phy.txt 2024-05-09T09:27:04.655 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/hix5hd2-phy.txt 2024-05-09T09:27:04.658 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-bindings.txt 2024-05-09T09:27:04.660 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy28lp.txt 2024-05-09T09:27:04.663 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy365x.txt 2024-05-09T09:27:04.665 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-mvebu.txt 2024-05-09T09:27:04.668 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih407-usb.txt 2024-05-09T09:27:04.671 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt 2024-05-09T09:27:04.674 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-apq8064-sata-phy.txt 2024-05-09T09:27:04.676 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-dwc3-usb-phy.txt 2024-05-09T09:27:04.679 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-ipq806x-sata-phy.txt 2024-05-09T09:27:04.682 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt 2024-05-09T09:27:04.684 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rockchip-usb-phy.txt 2024-05-09T09:27:04.687 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/samsung-phy.txt 2024-05-09T09:27:04.690 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/st-spear-miphy.txt 2024-05-09T09:27:04.692 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/sun4i-usb-phy.txt 2024-05-09T09:27:04.695 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ti-phy.txt 2024-05-09T09:27:04.698 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ 2024-05-09T09:27:04.701 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/abilis,tb10x-iomux.txt 2024-05-09T09:27:04.704 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt 2024-05-09T09:27:04.707 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/atmel,at91-pinctrl.txt 2024-05-09T09:27:04.709 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm11351-pinctrl.txt 2024-05-09T09:27:04.712 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm2835-gpio.txt 2024-05-09T09:27:04.714 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx-pinctrl.txt 2024-05-09T09:27:04.717 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx25-pinctrl.txt 2024-05-09T09:27:04.719 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx27-pinctrl.txt 2024-05-09T09:27:04.722 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx35-pinctrl.txt 2024-05-09T09:27:04.724 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx51-pinctrl.txt 2024-05-09T09:27:04.726 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx53-pinctrl.txt 2024-05-09T09:27:04.729 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6dl-pinctrl.txt 2024-05-09T09:27:04.731 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6q-pinctrl.txt 2024-05-09T09:27:04.734 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sl-pinctrl.txt 2024-05-09T09:27:04.736 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sx-pinctrl.txt 2024-05-09T09:27:04.739 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,mxs-pinctrl.txt 2024-05-09T09:27:04.741 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,vf610-pinctrl.txt 2024-05-09T09:27:04.744 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pdc-pinctrl.txt 2024-05-09T09:27:04.746 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pinctrl.txt 2024-05-09T09:27:04.749 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,falcon-pinumx.txt 2024-05-09T09:27:04.752 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,xway-pinumx.txt 2024-05-09T09:27:04.755 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-370-pinctrl.txt 2024-05-09T09:27:04.757 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-375-pinctrl.txt 2024-05-09T09:27:04.760 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-38x-pinctrl.txt 2024-05-09T09:27:04.762 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-xp-pinctrl.txt 2024-05-09T09:27:04.765 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,dove-pinctrl.txt 2024-05-09T09:27:04.767 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,kirkwood-pinctrl.txt 2024-05-09T09:27:04.770 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,mvebu-pinctrl.txt 2024-05-09T09:27:04.773 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,orion-pinctrl.txt 2024-05-09T09:27:04.775 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/meson,pinctrl.txt 2024-05-09T09:27:04.777 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra114-pinmux.txt 2024-05-09T09:27:04.780 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-pinmux.txt 2024-05-09T09:27:04.783 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt 2024-05-09T09:27:04.785 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra20-pinmux.txt 2024-05-09T09:27:04.788 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra30-pinmux.txt 2024-05-09T09:27:04.791 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt 2024-05-09T09:27:04.793 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt 2024-05-09T09:27:04.810 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-single.txt 2024-05-09T09:27:04.920 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-sirf.txt 2024-05-09T09:27:04.923 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-st.txt 2024-05-09T09:27:04.925 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-vt8500.txt 2024-05-09T09:27:04.928 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl_spear.txt 2024-05-09T09:27:04.931 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8064-pinctrl.txt 2024-05-09T09:27:04.933 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8084-pinctrl.txt 2024-05-09T09:27:04.936 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,ipq8064-pinctrl.txt 2024-05-09T09:27:04.938 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8916-pinctrl.txt 2024-05-09T09:27:04.941 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8960-pinctrl.txt 2024-05-09T09:27:04.943 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8974-pinctrl.txt 2024-05-09T09:27:04.946 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt 2024-05-09T09:27:04.948 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt 2024-05-09T09:27:04.951 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt 2024-05-09T09:27:04.954 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt 2024-05-09T09:27:04.956 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt 2024-05-09T09:27:04.959 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,abx500.txt 2024-05-09T09:27:04.962 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt 2024-05-09T09:27:04.964 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ti,omap-pinctrl.txt 2024-05-09T09:27:04.967 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/xlnx,zynq-pinctrl.txt 2024-05-09T09:27:04.969 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ 2024-05-09T09:27:04.972 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/bq2415x.txt 2024-05-09T09:27:04.974 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/isp1704.txt 2024-05-09T09:27:04.977 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ltc2941.txt 2024-05-09T09:27:04.979 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/opp.txt 2024-05-09T09:27:04.982 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power-controller.txt 2024-05-09T09:27:04.985 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power_domain.txt 2024-05-09T09:27:04.987 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/renesas,sysc-rmobile.txt 2024-05-09T09:27:04.990 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ 2024-05-09T09:27:04.992 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/keystone-reset.txt 2024-05-09T09:27:04.994 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ltc2952-poweroff.txt 2024-05-09T09:27:04.996 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/st-reset.txt 2024-05-09T09:27:04.999 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/syscon-reboot.txt 2024-05-09T09:27:05.001 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rockchip-io-domain.txt 2024-05-09T09:27:05.005 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rx51-battery.txt 2024-05-09T09:27:05.007 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/twl-charger.txt 2024-05-09T09:27:05.010 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ 2024-05-09T09:27:05.013 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/ 2024-05-09T09:27:05.014 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/btemp.txt 2024-05-09T09:27:05.016 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/chargalg.txt 2024-05-09T09:27:05.019 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/charger.txt 2024-05-09T09:27:05.022 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/fg.txt 2024-05-09T09:27:05.024 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/axxia-reset.txt 2024-05-09T09:27:05.028 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/charger-manager.txt 2024-05-09T09:27:05.031 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/gpio-charger.txt 2024-05-09T09:27:05.034 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/imx-snvs-poweroff.txt 2024-05-09T09:27:05.036 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/lp8727_charger.txt 2024-05-09T09:27:05.038 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max17042_battery.txt 2024-05-09T09:27:05.041 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max8925_batter.txt 2024-05-09T09:27:05.044 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/msm-poweroff.txt 2024-05-09T09:27:05.047 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/olpc_battery.txt 2024-05-09T09:27:05.049 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/power_supply.txt 2024-05-09T09:27:05.052 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/qnap-poweroff.txt 2024-05-09T09:27:05.054 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/restart-poweroff.txt 2024-05-09T09:27:05.056 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/sbs_sbs-battery.txt 2024-05-09T09:27:05.059 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ti,bq24735.txt 2024-05-09T09:27:05.061 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/tps65090.txt 2024-05-09T09:27:05.064 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/ 2024-05-09T09:27:05.067 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ 2024-05-09T09:27:05.069 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/akebono.txt 2024-05-09T09:27:05.071 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/cpm.txt 2024-05-09T09:27:05.074 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/emac.txt 2024-05-09T09:27:05.077 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/hsta.txt 2024-05-09T09:27:05.080 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ndfc.txt 2024-05-09T09:27:05.083 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ppc440spe-adma.txt 2024-05-09T09:27:05.087 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/reboot.txt 2024-05-09T09:27:05.090 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ 2024-05-09T09:27:05.092 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/board.txt 2024-05-09T09:27:05.095 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cache_sram.txt 2024-05-09T09:27:05.098 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ccf.txt 2024-05-09T09:27:05.101 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/ 2024-05-09T09:27:05.103 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm.txt 2024-05-09T09:27:05.105 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/ 2024-05-09T09:27:05.108 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/brg.txt 2024-05-09T09:27:05.110 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/i2c.txt 2024-05-09T09:27:05.113 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/pic.txt 2024-05-09T09:27:05.116 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/usb.txt 2024-05-09T09:27:05.118 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/gpio.txt 2024-05-09T09:27:05.122 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/network.txt 2024-05-09T09:27:05.124 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe.txt 2024-05-09T09:27:05.127 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ 2024-05-09T09:27:05.129 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/firmware.txt 2024-05-09T09:27:05.132 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/par_io.txt 2024-05-09T09:27:05.134 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/pincfg.txt 2024-05-09T09:27:05.137 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ucc.txt 2024-05-09T09:27:05.140 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/usb.txt 2024-05-09T09:27:05.143 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/serial.txt 2024-05-09T09:27:05.147 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt 2024-05-09T09:27:05.150 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dcsr.txt 2024-05-09T09:27:05.153 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/diu.txt 2024-05-09T09:27:05.155 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dma.txt 2024-05-09T09:27:05.159 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ecm.txt 2024-05-09T09:27:05.161 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/fman.txt 2024-05-09T09:27:05.164 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/gtm.txt 2024-05-09T09:27:05.167 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/guts.txt 2024-05-09T09:27:05.169 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/interlaken-lac.txt 2024-05-09T09:27:05.172 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/l2cache.txt 2024-05-09T09:27:05.174 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/lbc.txt 2024-05-09T09:27:05.177 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcm.txt 2024-05-09T09:27:05.180 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcu-mpc8349emitx.txt 2024-05-09T09:27:05.182 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mem-ctrlr.txt 2024-05-09T09:27:05.185 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5121-psc.txt 2024-05-09T09:27:05.188 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5200.txt 2024-05-09T09:27:05.191 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt 2024-05-09T09:27:05.194 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-timer.txt 2024-05-09T09:27:05.196 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic.txt 2024-05-09T09:27:05.199 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/msi-pic.txt 2024-05-09T09:27:05.202 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt 2024-05-09T09:27:05.205 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pmc.txt 2024-05-09T09:27:05.208 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/raideng.txt 2024-05-09T09:27:05.211 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio-rmu.txt 2024-05-09T09:27:05.214 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio.txt 2024-05-09T09:27:05.216 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/ 2024-05-09T09:27:05.219 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/gamecube.txt 2024-05-09T09:27:05.222 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/wii.txt 2024-05-09T09:27:05.224 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/ 2024-05-09T09:27:05.228 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/pps-gpio.txt 2024-05-09T09:27:05.230 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ 2024-05-09T09:27:05.233 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-hlcdc-pwm.txt 2024-05-09T09:27:05.235 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-pwm.txt 2024-05-09T09:27:05.237 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-tcb-pwm.txt 2024-05-09T09:27:05.240 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/bcm-kona-pwm.txt 2024-05-09T09:27:05.246 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/cirrus,clps711x-pwm.txt 2024-05-09T09:27:05.246 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/img-pwm.txt 2024-05-09T09:27:05.247 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/imx-pwm.txt 2024-05-09T09:27:05.250 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/lpc32xx-pwm.txt 2024-05-09T09:27:05.252 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/mxs-pwm.txt 2024-05-09T09:27:05.255 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nvidia,tegra20-pwm.txt 2024-05-09T09:27:05.257 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nxp,pca9685-pwm.txt 2024-05-09T09:27:05.259 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-bcm2835.txt 2024-05-09T09:27:05.262 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-fsl-ftm.txt 2024-05-09T09:27:05.265 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-lp3943.txt 2024-05-09T09:27:05.267 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-rockchip.txt 2024-05-09T09:27:05.270 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-samsung.txt 2024-05-09T09:27:05.272 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-st.txt 2024-05-09T09:27:05.274 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-sun4i.txt 2024-05-09T09:27:05.277 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiecap.txt 2024-05-09T09:27:05.280 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiehrpwm.txt 2024-05-09T09:27:05.283 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tipwmss.txt 2024-05-09T09:27:05.286 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm.txt 2024-05-09T09:27:05.288 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pxa-pwm.txt 2024-05-09T09:27:05.290 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/renesas,tpu-pwm.txt 2024-05-09T09:27:05.293 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/spear-pwm.txt 2024-05-09T09:27:05.295 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwm.txt 2024-05-09T09:27:05.298 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwmled.txt 2024-05-09T09:27:05.300 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/vt8500-pwm.txt 2024-05-09T09:27:05.302 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/ 2024-05-09T09:27:05.305 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/regmap.txt 2024-05-09T09:27:05.307 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ 2024-05-09T09:27:05.310 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm800.txt 2024-05-09T09:27:05.312 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm860x.txt 2024-05-09T09:27:05.315 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/act8865-regulator.txt 2024-05-09T09:27:05.317 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/anatop-regulator.txt 2024-05-09T09:27:05.322 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/as3722-regulator.txt 2024-05-09T09:27:05.327 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9210.txt 2024-05-09T09:27:05.330 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9211.txt 2024-05-09T09:27:05.333 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fan53555.txt 2024-05-09T09:27:05.335 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fixed-regulator.txt 2024-05-09T09:27:05.338 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/gpio-regulator.txt 2024-05-09T09:27:05.341 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/isl9305.txt 2024-05-09T09:27:05.344 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/lp872x.txt 2024-05-09T09:27:05.346 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ltc3589.txt 2024-05-09T09:27:05.349 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max1586-regulator.txt 2024-05-09T09:27:05.352 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max77802.txt 2024-05-09T09:27:05.355 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8660.txt 2024-05-09T09:27:05.359 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8907.txt 2024-05-09T09:27:05.362 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8925-regulator.txt 2024-05-09T09:27:05.365 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8952.txt 2024-05-09T09:27:05.368 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8973-regulator.txt 2024-05-09T09:27:05.371 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8997-regulator.txt 2024-05-09T09:27:05.374 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/mt6397-regulator.txt 2024-05-09T09:27:05.377 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/palmas-pmic.txt 2024-05-09T09:27:05.380 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pbias-regulator.txt 2024-05-09T09:27:05.382 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pfuze100.txt 2024-05-09T09:27:05.385 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pwm-regulator.txt 2024-05-09T09:27:05.388 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/regulator.txt 2024-05-09T09:27:05.391 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/s5m8767-regulator.txt 2024-05-09T09:27:05.393 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/sky81452-regulator.txt 2024-05-09T09:27:05.396 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ti-abb-regulator.txt 2024-05-09T09:27:05.399 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps51632-regulator.txt 2024-05-09T09:27:05.402 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps62360-regulator.txt 2024-05-09T09:27:05.404 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65090.txt 2024-05-09T09:27:05.407 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65217.txt 2024-05-09T09:27:05.410 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65218.txt 2024-05-09T09:27:05.413 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps6586x.txt 2024-05-09T09:27:05.416 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/twl-regulator.txt 2024-05-09T09:27:05.418 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/vexpress.txt 2024-05-09T09:27:05.421 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/ 2024-05-09T09:27:05.424 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt 2024-05-09T09:27:05.427 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/ 2024-05-09T09:27:05.430 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/allwinner,sunxi-clock-reset.txt 2024-05-09T09:27:05.433 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/fsl,imx-src.txt 2024-05-09T09:27:05.436 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/reset.txt 2024-05-09T09:27:05.438 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/sirf,rstc.txt 2024-05-09T09:27:05.441 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/socfpga-reset.txt 2024-05-09T09:27:05.443 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-picophyreset.txt 2024-05-09T09:27:05.447 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-powerdown.txt 2024-05-09T09:27:05.450 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-softreset.txt 2024-05-09T09:27:05.455 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/resource-names.txt 2024-05-09T09:27:05.459 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/ 2024-05-09T09:27:05.461 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/apm,rng.txt 2024-05-09T09:27:05.464 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/brcm,bcm2835.txt 2024-05-09T09:27:05.467 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/qcom,prng.txt 2024-05-09T09:27:05.469 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/ 2024-05-09T09:27:05.472 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/armada-380-rtc.txt 2024-05-09T09:27:05.476 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91rm9200-rtc.txt 2024-05-09T09:27:05.478 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91sam9-rtc.txt 2024-05-09T09:27:05.482 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dallas,ds1339.txt 2024-05-09T09:27:05.485 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dw-apb.txt 2024-05-09T09:27:05.488 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt 2024-05-09T09:27:05.491 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/imxdi-rtc.txt 2024-05-09T09:27:05.494 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/isil,isl12057.txt 2024-05-09T09:27:05.497 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/lpc32xx-rtc.txt 2024-05-09T09:27:05.500 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/maxim,ds1742.txt 2024-05-09T09:27:05.503 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/moxa,moxart-rtc.txt 2024-05-09T09:27:05.507 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.txt 2024-05-09T09:27:05.510 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nxp,rtc-2123.txt 2024-05-09T09:27:05.512 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/olpc-xo1-rtc.txt 2024-05-09T09:27:05.515 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/orion-rtc.txt 2024-05-09T09:27:05.517 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/pxa-rtc.txt 2024-05-09T09:27:05.520 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-cmos.txt 2024-05-09T09:27:05.523 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-omap.txt 2024-05-09T09:27:05.525 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-opal.txt 2024-05-09T09:27:05.528 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-palmas.txt 2024-05-09T09:27:05.531 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/s3c-rtc.txt 2024-05-09T09:27:05.533 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt 2024-05-09T09:27:05.536 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/snvs-rtc.txt 2024-05-09T09:27:05.539 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/spear-rtc.txt 2024-05-09T09:27:05.541 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/stmp3xxx-rtc.txt 2024-05-09T09:27:05.544 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sun6i-rtc.txt 2024-05-09T09:27:05.547 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sunxi-rtc.txt 2024-05-09T09:27:05.550 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/twl-rtc.txt 2024-05-09T09:27:05.554 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/via,vt8500-rtc.txt 2024-05-09T09:27:05.556 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/xgene-rtc.txt 2024-05-09T09:27:05.559 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/ 2024-05-09T09:27:05.562 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/ 2024-05-09T09:27:05.565 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/st33zp24-i2c.txt 2024-05-09T09:27:05.568 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/ 2024-05-09T09:27:05.572 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/8250.txt 2024-05-09T09:27:05.575 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_jtaguart.txt 2024-05-09T09:27:05.578 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_uart.txt 2024-05-09T09:27:05.581 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/arc-uart.txt 2024-05-09T09:27:05.584 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/atmel-usart.txt 2024-05-09T09:27:05.587 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/axis,etraxfs-uart.txt 2024-05-09T09:27:05.590 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/bcm63xx-uart.txt 2024-05-09T09:27:05.593 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cavium-uart.txt 2024-05-09T09:27:05.596 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cdns,uart.txt 2024-05-09T09:27:05.599 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cirrus,clps711x-uart.txt 2024-05-09T09:27:05.601 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/digicolor-usart.txt 2024-05-09T09:27:05.605 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/efm32-uart.txt 2024-05-09T09:27:05.608 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt 2024-05-09T09:27:05.610 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-lpuart.txt 2024-05-09T09:27:05.613 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt 2024-05-09T09:27:05.616 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/lantiq_asc.txt 2024-05-09T09:27:05.619 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/maxim,max310x.txt 2024-05-09T09:27:05.623 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl,pxa-ssp.txt 2024-05-09T09:27:05.626 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl-serial.txt 2024-05-09T09:27:05.628 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mtk-uart.txt 2024-05-09T09:27:05.632 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nvidia,tegra20-hsuart.txt 2024-05-09T09:27:05.635 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp,sc16is7xx.txt 2024-05-09T09:27:05.637 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp-lpc32xx-hsuart.txt 2024-05-09T09:27:05.640 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/omap_serial.txt 2024-05-09T09:27:05.643 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/pl011.txt 2024-05-09T09:27:05.646 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qca,ar9330-uart.txt 2024-05-09T09:27:05.648 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uart.txt 2024-05-09T09:27:05.651 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uartdm.txt 2024-05-09T09:27:05.654 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/renesas,sci-serial.txt 2024-05-09T09:27:05.656 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/rs485.txt 2024-05-09T09:27:05.659 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/samsung_uart.txt 2024-05-09T09:27:05.662 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sirf-uart.txt 2024-05-09T09:27:05.665 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.txt 2024-05-09T09:27:05.668 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sprd-uart.txt 2024-05-09T09:27:05.671 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/st-asc.txt 2024-05-09T09:27:05.674 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/vt8500-uart.txt 2024-05-09T09:27:05.676 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/ 2024-05-09T09:27:05.679 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/allwinner,sun4i-ps2.txt 2024-05-09T09:27:05.683 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/altera_ps2.txt 2024-05-09T09:27:05.686 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/olpc,ap-sp.txt 2024-05-09T09:27:05.690 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/snps-arc_ps2.txt 2024-05-09T09:27:05.693 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ 2024-05-09T09:27:05.696 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/ 2024-05-09T09:27:05.699 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman-portals.txt 2024-05-09T09:27:05.702 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman.txt 2024-05-09T09:27:05.705 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman-portals.txt 2024-05-09T09:27:05.708 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman.txt 2024-05-09T09:27:05.711 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/ 2024-05-09T09:27:05.714 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/qcom,gsbi.txt 2024-05-09T09:27:05.717 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/ 2024-05-09T09:27:05.721 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-dma.txt 2024-05-09T09:27:05.725 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt 2024-05-09T09:27:05.729 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ 2024-05-09T09:27:05.733 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,adau1701.txt 2024-05-09T09:27:05.736 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-i2s.txt 2024-05-09T09:27:05.739 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-spdif-tx.txt 2024-05-09T09:27:05.741 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,ssm2602.txt 2024-05-09T09:27:05.744 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4104.txt 2024-05-09T09:27:05.746 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4554.c 2024-05-09T09:27:05.749 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4642.txt 2024-05-09T09:27:05.753 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak5386.txt 2024-05-09T09:27:05.756 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5623.txt 2024-05-09T09:27:05.759 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5632.txt 2024-05-09T09:27:05.762 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/armada-370db-audio.txt 2024-05-09T09:27:05.765 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/arndale.txt 2024-05-09T09:27:05.768 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-at91sam9g20ek-wm8731-audio.txt 2024-05-09T09:27:05.771 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-sam9x5-wm8731-audio.txt 2024-05-09T09:27:05.774 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-wm8904.txt 2024-05-09T09:27:05.778 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel_ac97c.txt 2024-05-09T09:27:05.781 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/bcm2835-i2s.txt 2024-05-09T09:27:05.784 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cdns,xtfpga-i2s.txt 2024-05-09T09:27:05.787 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs35l32.txt 2024-05-09T09:27:05.791 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4265.txt 2024-05-09T09:27:05.794 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4270.txt 2024-05-09T09:27:05.798 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4271.txt 2024-05-09T09:27:05.800 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l52.txt 2024-05-09T09:27:05.803 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l56.txt 2024-05-09T09:27:05.806 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l73.txt 2024-05-09T09:27:05.809 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42xx8.txt 2024-05-09T09:27:05.821 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/da9055.txt 2024-05-09T09:27:05.870 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-evm-audio.txt 2024-05-09T09:27:05.872 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-mcasp-audio.txt 2024-05-09T09:27:05.875 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/designware-i2s.txt 2024-05-09T09:27:05.877 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/es8328.txt 2024-05-09T09:27:05.880 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/eukrea-tlv320.txt 2024-05-09T09:27:05.882 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,asrc.txt 2024-05-09T09:27:05.884 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,esai.txt 2024-05-09T09:27:05.887 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,spdif.txt 2024-05-09T09:27:05.890 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,ssi.txt 2024-05-09T09:27:05.893 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-asoc-card.txt 2024-05-09T09:27:05.895 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-sai.txt 2024-05-09T09:27:05.897 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/hdmi.txt 2024-05-09T09:27:05.899 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-es8328.txt 2024-05-09T09:27:05.902 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-sgtl5000.txt 2024-05-09T09:27:05.904 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-spdif.txt 2024-05-09T09:27:05.907 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-wm8962.txt 2024-05-09T09:27:05.909 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audmux.txt 2024-05-09T09:27:05.911 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ingenic,jz4740-i2s.txt 2024-05-09T09:27:05.914 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98090.txt 2024-05-09T09:27:05.916 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98095.txt 2024-05-09T09:27:05.919 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98357a.txt 2024-05-09T09:27:05.921 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa-ssp.txt 2024-05-09T09:27:05.924 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa2xx-pcm.txt 2024-05-09T09:27:05.927 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mvebu-audio.txt 2024-05-09T09:27:05.929 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-audio-sgtl5000.txt 2024-05-09T09:27:05.931 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-saif.txt 2024-05-09T09:27:05.934 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nokia,rx51.txt 2024-05-09T09:27:05.936 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-alc5632.txt 2024-05-09T09:27:05.939 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-max98090.txt 2024-05-09T09:27:05.941 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5640.txt 2024-05-09T09:27:05.944 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5677.txt 2024-05-09T09:27:05.946 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-trimslice.txt 2024-05-09T09:27:05.949 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8753.txt 2024-05-09T09:27:05.951 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8903.txt 2024-05-09T09:27:05.954 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm9712.txt 2024-05-09T09:27:05.956 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-ac97.txt 2024-05-09T09:27:05.959 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-das.txt 2024-05-09T09:27:05.962 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-i2s.txt 2024-05-09T09:27:05.965 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-ahub.txt 2024-05-09T09:27:05.967 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt 2024-05-09T09:27:05.970 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-i2s.txt 2024-05-09T09:27:05.972 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-abe-twl6040.txt 2024-05-09T09:27:05.975 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-dmic.txt 2024-05-09T09:27:05.978 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcbsp.txt 2024-05-09T09:27:05.980 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcpdm.txt 2024-05-09T09:27:05.982 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-twl4030.txt 2024-05-09T09:27:05.985 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm1792a.txt 2024-05-09T09:27:05.987 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm512x.txt 2024-05-09T09:27:05.990 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,fsi.txt 2024-05-09T09:27:05.992 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,rsnd.txt 2024-05-09T09:27:05.995 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rockchip-i2s.txt 2024-05-09T09:27:05.997 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5631.txt 2024-05-09T09:27:06.000 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5640.txt 2024-05-09T09:27:06.002 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5677.txt 2024-05-09T09:27:06.005 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,odroidx2-max98090.txt 2024-05-09T09:27:06.007 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,smdk-wm8994.txt 2024-05-09T09:27:06.010 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung-i2s.txt 2024-05-09T09:27:06.012 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sgtl5000.txt 2024-05-09T09:27:06.015 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/simple-card.txt 2024-05-09T09:27:06.018 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-codec.txt 2024-05-09T09:27:06.020 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-port.txt 2024-05-09T09:27:06.023 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio.txt 2024-05-09T09:27:06.026 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-usp.txt 2024-05-09T09:27:06.028 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/snow.txt 2024-05-09T09:27:06.031 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/soc-ac97link.txt 2024-05-09T09:27:06.033 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-receiver.txt 2024-05-09T09:27:06.036 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-transmitter.txt 2024-05-09T09:27:06.038 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm2518.txt 2024-05-09T09:27:06.041 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm4567.txt 2024-05-09T09:27:06.044 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta32x.txt 2024-05-09T09:27:06.046 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta350.txt 2024-05-09T09:27:06.049 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tas2552.txt 2024-05-09T09:27:06.051 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tdm-slot.txt 2024-05-09T09:27:06.054 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,pcm1681.txt 2024-05-09T09:27:06.057 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,tas5086.txt 2024-05-09T09:27:06.059 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic31xx.txt 2024-05-09T09:27:06.062 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic32x4.txt 2024-05-09T09:27:06.065 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic3x.txt 2024-05-09T09:27:06.068 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tpa6130a2.txt 2024-05-09T09:27:06.070 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ts3a227e.txt 2024-05-09T09:27:06.073 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-mop500.txt 2024-05-09T09:27:06.075 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-msp.txt 2024-05-09T09:27:06.078 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/widgets.txt 2024-05-09T09:27:06.081 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8510.txt 2024-05-09T09:27:06.084 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8523.txt 2024-05-09T09:27:06.086 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8580.txt 2024-05-09T09:27:06.089 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8711.txt 2024-05-09T09:27:06.091 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8728.txt 2024-05-09T09:27:06.094 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8731.txt 2024-05-09T09:27:06.096 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8737.txt 2024-05-09T09:27:06.099 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8741.txt 2024-05-09T09:27:06.101 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8750.txt 2024-05-09T09:27:06.104 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8753.txt 2024-05-09T09:27:06.106 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8770.txt 2024-05-09T09:27:06.108 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8776.txt 2024-05-09T09:27:06.111 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8804.txt 2024-05-09T09:27:06.113 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8903.txt 2024-05-09T09:27:06.115 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8904.txt 2024-05-09T09:27:06.118 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8960.txt 2024-05-09T09:27:06.121 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8962.txt 2024-05-09T09:27:06.124 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8994.txt 2024-05-09T09:27:06.126 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ 2024-05-09T09:27:06.129 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/brcm,bcm2835-spi.txt 2024-05-09T09:27:06.131 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/efm32-spi.txt 2024-05-09T09:27:06.134 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-imx-cspi.txt 2024-05-09T09:27:06.136 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-spi.txt 2024-05-09T09:27:06.138 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/mxs-spi.txt 2024-05-09T09:27:06.141 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra114-spi.txt 2024-05-09T09:27:06.144 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-sflash.txt 2024-05-09T09:27:06.146 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-slink.txt 2024-05-09T09:27:06.148 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/omap-spi.txt 2024-05-09T09:27:06.151 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/qcom,spi-qup.txt 2024-05-09T09:27:06.154 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-hspi.txt 2024-05-09T09:27:06.156 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-msiof.txt 2024-05-09T09:27:06.159 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/snps,dw-apb-ssi.txt 2024-05-09T09:27:06.162 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-bus.txt 2024-05-09T09:27:06.165 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-cadence.txt 2024-05-09T09:27:06.167 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-davinci.txt 2024-05-09T09:27:06.171 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-dw.txt 2024-05-09T09:27:06.173 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt 2024-05-09T09:27:06.175 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-gpio.txt 2024-05-09T09:27:06.178 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-img-spfi.txt 2024-05-09T09:27:06.181 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-meson.txt 2024-05-09T09:27:06.183 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-octeon.txt 2024-05-09T09:27:06.186 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-orion.txt 2024-05-09T09:27:06.188 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rockchip.txt 2024-05-09T09:27:06.191 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rspi.txt 2024-05-09T09:27:06.193 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-samsung.txt 2024-05-09T09:27:06.196 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sc18is602.txt 2024-05-09T09:27:06.198 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sirf.txt 2024-05-09T09:27:06.201 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-st-ssc.txt 2024-05-09T09:27:06.204 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun4i.txt 2024-05-09T09:27:06.206 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun6i.txt 2024-05-09T09:27:06.209 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-xtensa-xtfpga.txt 2024-05-09T09:27:06.211 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_altera.txt 2024-05-09T09:27:06.213 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_atmel.txt 2024-05-09T09:27:06.216 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_oc_tiny.txt 2024-05-09T09:27:06.218 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_pl022.txt 2024-05-09T09:27:06.221 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ti_qspi.txt 2024-05-09T09:27:06.224 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/ 2024-05-09T09:27:06.226 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/qcom,spmi-pmic-arb.txt 2024-05-09T09:27:06.229 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/spmi.txt 2024-05-09T09:27:06.231 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/ 2024-05-09T09:27:06.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:05 smithi012 ceph-mon[30140]: pgmap v413: 129 pgs: 129 active+clean; 88 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 7.4 MiB/s wr, 160 op/s 2024-05-09T09:27:06.234 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/ 2024-05-09T09:27:06.235 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/ 2024-05-09T09:27:06.237 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/lpc32xx-adc.txt 2024-05-09T09:27:06.240 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/mxs-lradc.txt 2024-05-09T09:27:06.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:05 smithi170 ceph-mon[33632]: pgmap v413: 129 pgs: 129 active+clean; 88 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 7.4 MiB/s wr, 160 op/s 2024-05-09T09:27:06.243 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/spear-adc.txt 2024-05-09T09:27:06.246 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/submitting-patches.txt 2024-05-09T09:27:06.250 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ 2024-05-09T09:27:06.252 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/armada-thermal.txt 2024-05-09T09:27:06.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:05 smithi016 ceph-mon[33335]: pgmap v413: 129 pgs: 129 active+clean; 88 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 7.4 MiB/s wr, 160 op/s 2024-05-09T09:27:06.255 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/db8500-thermal.txt 2024-05-09T09:27:06.257 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/dove-thermal.txt 2024-05-09T09:27:06.260 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/exynos-thermal.txt 2024-05-09T09:27:06.262 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/imx-thermal.txt 2024-05-09T09:27:06.266 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/kirkwood-thermal.txt 2024-05-09T09:27:06.267 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rcar-thermal.txt 2024-05-09T09:27:06.270 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rockchip-thermal.txt 2024-05-09T09:27:06.272 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/spear-thermal.txt 2024-05-09T09:27:06.274 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/st-thermal.txt 2024-05-09T09:27:06.277 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/tegra-soctherm.txt 2024-05-09T09:27:06.280 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/thermal.txt 2024-05-09T09:27:06.283 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ti_soc_thermal.txt 2024-05-09T09:27:06.286 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ 2024-05-09T09:27:06.288 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun4i-timer.txt 2024-05-09T09:27:06.291 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun5i-a13-hstimer.txt 2024-05-09T09:27:06.294 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/amlogic,meson6-timer.txt 2024-05-09T09:27:06.297 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/arm,sp804.txt 2024-05-09T09:27:06.300 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/brcm,bcm2835-system-timer.txt 2024-05-09T09:27:06.302 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cadence,ttc-timer.txt 2024-05-09T09:27:06.305 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cirrus,clps711x-timer.txt 2024-05-09T09:27:06.307 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/digicolor-timer.txt 2024-05-09T09:27:06.309 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/energymicro,efm32-timer.txt 2024-05-09T09:27:06.312 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,ftm-timer.txt 2024-05-09T09:27:06.314 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt 2024-05-09T09:27:06.317 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/lsi,zevio-timer.txt 2024-05-09T09:27:06.319 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,armada-370-xp-timer.txt 2024-05-09T09:27:06.321 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,orion-timer.txt 2024-05-09T09:27:06.324 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/mediatek,mtk-timer.txt 2024-05-09T09:27:06.326 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/moxa,moxart-timer.txt 2024-05-09T09:27:06.329 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra20-timer.txt 2024-05-09T09:27:06.331 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra30-timer.txt 2024-05-09T09:27:06.334 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,cmt.txt 2024-05-09T09:27:06.336 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,mtu2.txt 2024-05-09T09:27:06.339 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,tmu.txt 2024-05-09T09:27:06.341 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/rockchip,rk3288-timer.txt 2024-05-09T09:27:06.344 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/samsung,exynos4210-mct.txt 2024-05-09T09:27:06.346 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/stericsson-u300-apptimer.txt 2024-05-09T09:27:06.349 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ti,keystone-timer.txt 2024-05-09T09:27:06.351 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ 2024-05-09T09:27:06.354 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ufshcd-pltfrm.txt 2024-05-09T09:27:06.357 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/unittest.txt 2024-05-09T09:27:06.360 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ 2024-05-09T09:27:06.362 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/am33xx-usb.txt 2024-05-09T09:27:06.365 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/atmel-usb.txt 2024-05-09T09:27:06.367 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-imx.txt 2024-05-09T09:27:06.370 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-qcom.txt 2024-05-09T09:27:06.373 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-usb2.txt 2024-05-09T09:27:06.376 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-zevio.txt 2024-05-09T09:27:06.379 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc2.txt 2024-05-09T09:27:06.381 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3-st.txt 2024-05-09T09:27:06.384 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3.txt 2024-05-09T09:27:06.386 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-omap.txt 2024-05-09T09:27:06.389 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-orion.txt 2024-05-09T09:27:06.391 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-st.txt 2024-05-09T09:27:06.394 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/exynos-usb.txt 2024-05-09T09:27:06.397 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/fsl-usb.txt 2024-05-09T09:27:06.399 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/generic.txt 2024-05-09T09:27:06.402 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/gr-udc.txt 2024-05-09T09:27:06.404 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/isp1301.txt 2024-05-09T09:27:06.407 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-phy.txt 2024-05-09T09:27:06.409 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-usb.txt 2024-05-09T09:27:06.412 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/lpc32xx-udc.txt 2024-05-09T09:27:06.415 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/msm-hsusb.txt 2024-05-09T09:27:06.418 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/mxs-phy.txt 2024-05-09T09:27:06.421 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-ehci.txt 2024-05-09T09:27:06.424 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-usb-phy.txt 2024-05-09T09:27:06.426 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-nxp.txt 2024-05-09T09:27:06.429 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-omap3.txt 2024-05-09T09:27:06.434 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-st.txt 2024-05-09T09:27:06.434 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/omap-usb.txt 2024-05-09T09:27:06.436 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/pxa-usb.txt 2024-05-09T09:27:06.439 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/qcom,dwc3.txt 2024-05-09T09:27:06.441 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/renesas_usbhs.txt 2024-05-09T09:27:06.444 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-hsotg.txt 2024-05-09T09:27:06.446 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-usbphy.txt 2024-05-09T09:27:06.448 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/spear-usb.txt 2024-05-09T09:27:06.451 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/twlxxxx-usb.txt 2024-05-09T09:27:06.453 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/udc-xilinx.txt 2024-05-09T09:27:06.456 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ehci.txt 2024-05-09T09:27:06.459 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-nop-xceiv.txt 2024-05-09T09:27:06.461 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ohci.txt 2024-05-09T09:27:06.464 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-uhci.txt 2024-05-09T09:27:06.466 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-xhci.txt 2024-05-09T09:27:06.469 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb3503.txt 2024-05-09T09:27:06.471 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usbmisc-imx.txt 2024-05-09T09:27:06.474 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ux500-usb.txt 2024-05-09T09:27:06.476 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/vendor-prefixes.txt 2024-05-09T09:27:06.480 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ 2024-05-09T09:27:06.482 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7123.txt 2024-05-09T09:27:06.484 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7511.txt 2024-05-09T09:27:06.487 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/analog-tv-connector.txt 2024-05-09T09:27:06.489 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/arm,pl11x.txt 2024-05-09T09:27:06.491 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/atmel,lcdc.txt 2024-05-09T09:27:06.494 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/ 2024-05-09T09:27:06.496 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/88pm860x.txt 2024-05-09T09:27:06.498 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/gpio-backlight.txt 2024-05-09T09:27:06.500 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/lp855x.txt 2024-05-09T09:27:06.503 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/max8925-backlight.txt 2024-05-09T09:27:06.506 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/pwm-backlight.txt 2024-05-09T09:27:06.508 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/tps65217-backlight.txt 2024-05-09T09:27:06.511 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ 2024-05-09T09:27:06.514 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ps8622.txt 2024-05-09T09:27:06.516 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ptn3460.txt 2024-05-09T09:27:06.518 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/cirrus,clps711x-fb.txt 2024-05-09T09:27:06.522 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/display-timing.txt 2024-05-09T09:27:06.524 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dvi-connector.txt 2024-05-09T09:27:06.527 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dw_hdmi-rockchip.txt 2024-05-09T09:27:06.529 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos7-decon.txt 2024-05-09T09:27:06.532 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dp.txt 2024-05-09T09:27:06.535 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dsim.txt 2024-05-09T09:27:06.538 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmi.txt 2024-05-09T09:27:06.541 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiddc.txt 2024-05-09T09:27:06.543 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiphy.txt 2024-05-09T09:27:06.546 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_mixer.txt 2024-05-09T09:27:06.548 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/fsl,imx-fb.txt 2024-05-09T09:27:06.551 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/hdmi-connector.txt 2024-05-09T09:27:06.553 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/lgphilips,lb035q02.txt 2024-05-09T09:27:06.556 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dpi.txt 2024-05-09T09:27:06.559 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dsi-cm.txt 2024-05-09T09:27:06.561 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/renesas,du.txt 2024-05-09T09:27:06.564 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-drm.txt 2024-05-09T09:27:06.566 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-vop.txt 2024-05-09T09:27:06.569 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/samsung-fimd.txt 2024-05-09T09:27:06.571 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sharp,ls037v7dw01.txt 2024-05-09T09:27:06.574 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer-sunxi.txt 2024-05-09T09:27:06.576 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer.txt 2024-05-09T09:27:06.579 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sony,acx565akm.txt 2024-05-09T09:27:06.582 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1289fb.txt 2024-05-09T09:27:06.584 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1307fb.txt 2024-05-09T09:27:06.586 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/thine,thc63lvdm83d 2024-05-09T09:27:06.589 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,dra7-dss.txt 2024-05-09T09:27:06.591 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap-dss.txt 2024-05-09T09:27:06.594 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap2-dss.txt 2024-05-09T09:27:06.596 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap3-dss.txt 2024-05-09T09:27:06.598 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap4-dss.txt 2024-05-09T09:27:06.601 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap5-dss.txt 2024-05-09T09:27:06.604 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,opa362.txt 2024-05-09T09:27:06.606 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tfp410.txt 2024-05-09T09:27:06.608 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tpd12s015.txt 2024-05-09T09:27:06.611 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/toppoly,td028ttec1.txt 2024-05-09T09:27:06.613 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/tpo,td043mtea1.txt 2024-05-09T09:27:06.616 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/vga-connector.txt 2024-05-09T09:27:06.618 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/via,vt8500-fb.txt 2024-05-09T09:27:06.621 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,prizm-ge-rops.txt 2024-05-09T09:27:06.623 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,wm8505-fb.txt 2024-05-09T09:27:06.626 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/ 2024-05-09T09:27:06.629 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/mmio.txt 2024-05-09T09:27:06.632 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/ 2024-05-09T09:27:06.635 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/fsl-imx-owire.txt 2024-05-09T09:27:06.638 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/omap-hdq.txt 2024-05-09T09:27:06.641 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/w1-gpio.txt 2024-05-09T09:27:06.643 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ 2024-05-09T09:27:06.646 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-at91rm9200-wdt.txt 2024-05-09T09:27:06.648 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-wdt.txt 2024-05-09T09:27:06.651 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/brcm,bcm2835-pm-wdog.txt 2024-05-09T09:27:06.653 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/cadence-wdt.txt 2024-05-09T09:27:06.655 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/davinci-wdt.txt 2024-05-09T09:27:06.658 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/dw_wdt.txt 2024-05-09T09:27:06.661 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/fsl-imx-wdt.txt 2024-05-09T09:27:06.664 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/gpio-wdt.txt 2024-05-09T09:27:06.666 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/imgpdc-wdt.txt 2024-05-09T09:27:06.669 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ingenic,jz4740-wdt.txt 2024-05-09T09:27:06.671 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/marvel.txt 2024-05-09T09:27:06.674 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/men-a021-wdt.txt 2024-05-09T09:27:06.677 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/meson6-wdt.txt 2024-05-09T09:27:06.680 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/moxa,moxart-watchdog.txt 2024-05-09T09:27:06.682 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/mtk-wdt.txt 2024-05-09T09:27:06.685 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/of-xilinx-wdt.txt 2024-05-09T09:27:06.687 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/omap-wdt.txt 2024-05-09T09:27:06.690 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/pnx4008-wdt.txt 2024-05-09T09:27:06.692 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qca-ar7130-wdt.txt 2024-05-09T09:27:06.695 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qcom-wdt.txt 2024-05-09T09:27:06.697 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/rt2880-wdt.txt 2024-05-09T09:27:06.699 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/samsung-wdt.txt 2024-05-09T09:27:06.702 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sirfsoc_wdt.txt 2024-05-09T09:27:06.704 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/stericsson-coh901327.txt 2024-05-09T09:27:06.707 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sunxi-wdt.txt 2024-05-09T09:27:06.710 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/twl4030-wdt.txt 2024-05-09T09:27:06.712 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ 2024-05-09T09:27:06.715 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ce4100.txt 2024-05-09T09:27:06.717 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/interrupt.txt 2024-05-09T09:27:06.720 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/timer.txt 2024-05-09T09:27:06.722 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/xilinx.txt 2024-05-09T09:27:06.725 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/ 2024-05-09T09:27:06.727 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/xillybus.txt 2024-05-09T09:27:06.729 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/booting-without-of.txt 2024-05-09T09:27:06.734 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/changesets.txt 2024-05-09T09:27:06.737 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/dynamic-resolution-notes.txt 2024-05-09T09:27:06.739 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/of_selftest.txt 2024-05-09T09:27:06.742 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/overlay-notes.txt 2024-05-09T09:27:06.744 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/todo.txt 2024-05-09T09:27:06.747 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/devicetree/usage-model.txt 2024-05-09T09:27:06.750 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/digsig.txt 2024-05-09T09:27:06.752 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dma-buf-sharing.txt 2024-05-09T09:27:06.755 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dmaengine/ 2024-05-09T09:27:06.757 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dmaengine/00-INDEX 2024-05-09T09:27:06.759 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dmaengine/client.txt 2024-05-09T09:27:06.762 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dmaengine/dmatest.txt 2024-05-09T09:27:06.764 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dmaengine/provider.txt 2024-05-09T09:27:06.767 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dontdiff 2024-05-09T09:27:06.769 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/driver-model/ 2024-05-09T09:27:06.771 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/driver-model/binding.txt 2024-05-09T09:27:06.773 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/driver-model/bus.txt 2024-05-09T09:27:06.776 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/driver-model/class.txt 2024-05-09T09:27:06.778 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/driver-model/design-patterns.txt 2024-05-09T09:27:06.780 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/driver-model/device.txt 2024-05-09T09:27:06.783 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/driver-model/devres.txt 2024-05-09T09:27:06.785 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/driver-model/driver.txt 2024-05-09T09:27:06.788 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/driver-model/overview.txt 2024-05-09T09:27:06.790 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/driver-model/platform.txt 2024-05-09T09:27:06.793 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/driver-model/porting.txt 2024-05-09T09:27:06.796 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/ 2024-05-09T09:27:06.798 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/README.dvb-usb 2024-05-09T09:27:06.800 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/avermedia.txt 2024-05-09T09:27:06.803 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/bt8xx.txt 2024-05-09T09:27:06.805 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/cards.txt 2024-05-09T09:27:06.808 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/ci.txt 2024-05-09T09:27:06.810 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/contributors.txt 2024-05-09T09:27:06.813 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/faq.txt 2024-05-09T09:27:06.815 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/get_dvb_firmware 2024-05-09T09:27:06.818 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/lmedm04.txt 2024-05-09T09:27:06.843 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/opera-firmware.txt 2024-05-09T09:27:07.000 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/readme.txt 2024-05-09T09:27:07.002 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/technisat.txt 2024-05-09T09:27:07.005 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/ttusb-dec.txt 2024-05-09T09:27:07.007 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dvb/udev.txt 2024-05-09T09:27:07.010 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/dynamic-debug-howto.txt 2024-05-09T09:27:07.013 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/early-userspace/ 2024-05-09T09:27:07.015 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/early-userspace/README 2024-05-09T09:27:07.017 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/early-userspace/buffer-format.txt 2024-05-09T09:27:07.020 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/edac.txt 2024-05-09T09:27:07.023 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/efi-stub.txt 2024-05-09T09:27:07.025 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/eisa.txt 2024-05-09T09:27:07.027 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/email-clients.txt 2024-05-09T09:27:07.029 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/extcon/ 2024-05-09T09:27:07.032 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/extcon/porting-android-switch-class 2024-05-09T09:27:07.034 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fault-injection/ 2024-05-09T09:27:07.036 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fault-injection/fault-injection.txt 2024-05-09T09:27:07.038 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fault-injection/notifier-error-inject.txt 2024-05-09T09:27:07.041 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fault-injection/provoke-crashes.txt 2024-05-09T09:27:07.043 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/ 2024-05-09T09:27:07.046 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/00-INDEX 2024-05-09T09:27:07.048 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/api.txt 2024-05-09T09:27:07.050 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/arkfb.txt 2024-05-09T09:27:07.053 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/aty128fb.txt 2024-05-09T09:27:07.055 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/cirrusfb.txt 2024-05-09T09:27:07.058 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/cmap_xfbdev.txt 2024-05-09T09:27:07.060 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/deferred_io.txt 2024-05-09T09:27:07.062 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/efifb.txt 2024-05-09T09:27:07.065 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/ep93xx-fb.txt 2024-05-09T09:27:07.067 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/fbcon.txt 2024-05-09T09:27:07.070 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/framebuffer.txt 2024-05-09T09:27:07.073 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/gxfb.txt 2024-05-09T09:27:07.075 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/intel810.txt 2024-05-09T09:27:07.078 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/intelfb.txt 2024-05-09T09:27:07.081 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/internals.txt 2024-05-09T09:27:07.083 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/lxfb.txt 2024-05-09T09:27:07.085 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/matroxfb.txt 2024-05-09T09:27:07.088 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/metronomefb.txt 2024-05-09T09:27:07.090 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/modedb.txt 2024-05-09T09:27:07.093 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/pvr2fb.txt 2024-05-09T09:27:07.095 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/pxafb.txt 2024-05-09T09:27:07.097 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/s3fb.txt 2024-05-09T09:27:07.100 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/sa1100fb.txt 2024-05-09T09:27:07.102 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/sh7760fb.txt 2024-05-09T09:27:07.105 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/sisfb.txt 2024-05-09T09:27:07.107 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/sm501.txt 2024-05-09T09:27:07.110 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/sstfb.txt 2024-05-09T09:27:07.113 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/tgafb.txt 2024-05-09T09:27:07.115 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/tridentfb.txt 2024-05-09T09:27:07.117 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/udlfb.txt 2024-05-09T09:27:07.119 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/uvesafb.txt 2024-05-09T09:27:07.122 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/vesafb.txt 2024-05-09T09:27:07.125 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/viafb.modes 2024-05-09T09:27:07.128 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/viafb.txt 2024-05-09T09:27:07.130 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fb/vt8623fb.txt 2024-05-09T09:27:07.133 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/ 2024-05-09T09:27:07.135 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/.gitignore 2024-05-09T09:27:07.138 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/00-INDEX 2024-05-09T09:27:07.141 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/9p.txt 2024-05-09T09:27:07.143 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/Locking 2024-05-09T09:27:07.146 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/Makefile 2024-05-09T09:27:07.148 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/adfs.txt 2024-05-09T09:27:07.151 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/affs.txt 2024-05-09T09:27:07.153 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/afs.txt 2024-05-09T09:27:07.156 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/autofs4-mount-control.txt 2024-05-09T09:27:07.159 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/autofs4.txt 2024-05-09T09:27:07.162 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/automount-support.txt 2024-05-09T09:27:07.165 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/befs.txt 2024-05-09T09:27:07.167 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/bfs.txt 2024-05-09T09:27:07.169 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/btrfs.txt 2024-05-09T09:27:07.172 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/caching/ 2024-05-09T09:27:07.174 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/caching/backend-api.txt 2024-05-09T09:27:07.178 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/caching/cachefiles.txt 2024-05-09T09:27:07.181 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/caching/fscache.txt 2024-05-09T09:27:07.184 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/caching/netfs-api.txt 2024-05-09T09:27:07.187 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/caching/object.txt 2024-05-09T09:27:07.190 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/caching/operations.txt 2024-05-09T09:27:07.193 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/ceph.txt 2024-05-09T09:27:07.196 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/cifs/ 2024-05-09T09:27:07.198 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/cifs/AUTHORS 2024-05-09T09:27:07.200 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/cifs/CHANGES 2024-05-09T09:27:07.203 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/cifs/README 2024-05-09T09:27:07.207 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/cifs/TODO 2024-05-09T09:27:07.209 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/cifs/cifs.txt 2024-05-09T09:27:07.212 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/cifs/winucase_convert.pl 2024-05-09T09:27:07.214 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/coda.txt 2024-05-09T09:27:07.218 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/configfs/ 2024-05-09T09:27:07.220 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/configfs/Makefile 2024-05-09T09:27:07.222 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs.txt 2024-05-09T09:27:07.225 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_explicit.c 2024-05-09T09:27:07.228 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_macros.c 2024-05-09T09:27:07.231 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/cramfs.txt 2024-05-09T09:27:07.234 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/dax.txt 2024-05-09T09:27:07.237 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/debugfs.txt 2024-05-09T09:27:07.240 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/devpts.txt 2024-05-09T09:27:07.243 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/directory-locking 2024-05-09T09:27:07.246 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/dlmfs.txt 2024-05-09T09:27:07.248 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/dnotify.txt 2024-05-09T09:27:07.250 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/dnotify_test.c 2024-05-09T09:27:07.252 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/ecryptfs.txt 2024-05-09T09:27:07.254 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/efivarfs.txt 2024-05-09T09:27:07.257 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/exofs.txt 2024-05-09T09:27:07.260 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/ext2.txt 2024-05-09T09:27:07.262 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/ext3.txt 2024-05-09T09:27:07.265 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/ext4.txt 2024-05-09T09:27:07.267 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/f2fs.txt 2024-05-09T09:27:07.270 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/fiemap.txt 2024-05-09T09:27:07.272 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/files.txt 2024-05-09T09:27:07.275 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/fuse.txt 2024-05-09T09:27:07.278 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/gfs2-glocks.txt 2024-05-09T09:27:07.280 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/gfs2-uevents.txt 2024-05-09T09:27:07.282 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/gfs2.txt 2024-05-09T09:27:07.286 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/hfs.txt 2024-05-09T09:27:07.287 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/hfsplus.txt 2024-05-09T09:27:07.290 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/hpfs.txt 2024-05-09T09:27:07.292 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/inotify.txt 2024-05-09T09:27:07.295 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/isofs.txt 2024-05-09T09:27:07.297 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/jfs.txt 2024-05-09T09:27:07.299 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/locks.txt 2024-05-09T09:27:07.301 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/logfs.txt 2024-05-09T09:27:07.307 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/mandatory-locking.txt 2024-05-09T09:27:07.309 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/ncpfs.txt 2024-05-09T09:27:07.312 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/ 2024-05-09T09:27:07.314 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/00-INDEX 2024-05-09T09:27:07.316 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/Exporting 2024-05-09T09:27:07.318 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/fault_injection.txt 2024-05-09T09:27:07.320 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/idmapper.txt 2024-05-09T09:27:07.323 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/knfsd-stats.txt 2024-05-09T09:27:07.325 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs-rdma.txt 2024-05-09T09:27:07.327 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs.txt 2024-05-09T09:27:07.330 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs41-server.txt 2024-05-09T09:27:07.333 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsd-admin-interfaces.txt 2024-05-09T09:27:07.336 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsroot.txt 2024-05-09T09:27:07.338 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs-block-server.txt 2024-05-09T09:27:07.340 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs.txt 2024-05-09T09:27:07.343 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-cache.txt 2024-05-09T09:27:07.345 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-server-gss.txt 2024-05-09T09:27:07.348 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/nilfs2.txt 2024-05-09T09:27:07.351 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/ntfs.txt 2024-05-09T09:27:07.353 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/ocfs2.txt 2024-05-09T09:27:07.355 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/omfs.txt 2024-05-09T09:27:07.358 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/overlayfs.txt 2024-05-09T09:27:07.361 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/path-lookup.txt 2024-05-09T09:27:07.363 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/ 2024-05-09T09:27:07.365 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/design_notes.txt 2024-05-09T09:27:07.368 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/info.txt 2024-05-09T09:27:07.370 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/network_protocol.txt 2024-05-09T09:27:07.373 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/porting 2024-05-09T09:27:07.376 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/proc.txt 2024-05-09T09:27:07.381 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/qnx6.txt 2024-05-09T09:27:07.384 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/quota.txt 2024-05-09T09:27:07.386 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/ramfs-rootfs-initramfs.txt 2024-05-09T09:27:07.389 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/relay.txt 2024-05-09T09:27:07.392 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/romfs.txt 2024-05-09T09:27:07.394 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/seq_file.txt 2024-05-09T09:27:07.396 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/sharedsubtree.txt 2024-05-09T09:27:07.400 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/spufs.txt 2024-05-09T09:27:07.402 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/squashfs.txt 2024-05-09T09:27:07.405 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/sysfs-pci.txt 2024-05-09T09:27:07.408 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/sysfs-tagging.txt 2024-05-09T09:27:07.410 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/sysfs.txt 2024-05-09T09:27:07.413 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/sysv-fs.txt 2024-05-09T09:27:07.416 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/tmpfs.txt 2024-05-09T09:27:07.418 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/ubifs.txt 2024-05-09T09:27:07.420 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/udf.txt 2024-05-09T09:27:07.422 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/ufs.txt 2024-05-09T09:27:07.425 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/vfat.txt 2024-05-09T09:27:07.427 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/vfs.txt 2024-05-09T09:27:07.431 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/xfs-delayed-logging-design.txt 2024-05-09T09:27:07.434 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/xfs-self-describing-metadata.txt 2024-05-09T09:27:07.437 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/filesystems/xfs.txt 2024-05-09T09:27:07.439 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/firmware_class/ 2024-05-09T09:27:07.442 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/firmware_class/README 2024-05-09T09:27:07.444 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/firmware_class/hotplug-script 2024-05-09T09:27:07.447 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/flexible-arrays.txt 2024-05-09T09:27:07.449 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fmc/ 2024-05-09T09:27:07.451 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fmc/00-INDEX 2024-05-09T09:27:07.454 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fmc/API.txt 2024-05-09T09:27:07.456 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fmc/FMC-and-SDB.txt 2024-05-09T09:27:07.458 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fmc/carrier.txt 2024-05-09T09:27:07.460 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fmc/fmc-chardev.txt 2024-05-09T09:27:07.462 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fmc/fmc-fakedev.txt 2024-05-09T09:27:07.465 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fmc/fmc-trivial.txt 2024-05-09T09:27:07.467 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fmc/fmc-write-eeprom.txt 2024-05-09T09:27:07.470 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fmc/identifiers.txt 2024-05-09T09:27:07.472 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fmc/mezzanine.txt 2024-05-09T09:27:07.475 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/fmc/parameters.txt 2024-05-09T09:27:07.478 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/frv/ 2024-05-09T09:27:07.480 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/frv/README.txt 2024-05-09T09:27:07.483 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/frv/atomic-ops.txt 2024-05-09T09:27:07.486 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/frv/booting.txt 2024-05-09T09:27:07.490 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/frv/clock.txt 2024-05-09T09:27:07.491 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/frv/configuring.txt 2024-05-09T09:27:07.494 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/frv/features.txt 2024-05-09T09:27:07.496 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/frv/gdbinit 2024-05-09T09:27:07.499 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/frv/gdbstub.txt 2024-05-09T09:27:07.501 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/frv/kernel-ABI.txt 2024-05-09T09:27:07.504 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/frv/mmu-layout.txt 2024-05-09T09:27:07.506 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/futex-requeue-pi.txt 2024-05-09T09:27:07.509 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/gcov.txt 2024-05-09T09:27:07.512 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/gdb-kernel-debugging.txt 2024-05-09T09:27:07.514 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/gpio/ 2024-05-09T09:27:07.516 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/gpio/00-INDEX 2024-05-09T09:27:07.518 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/gpio/board.txt 2024-05-09T09:27:07.520 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/gpio/consumer.txt 2024-05-09T09:27:07.523 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/gpio/driver.txt 2024-05-09T09:27:07.526 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/gpio/gpio-legacy.txt 2024-05-09T09:27:07.529 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/gpio/gpio.txt 2024-05-09T09:27:07.531 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/gpio/sysfs.txt 2024-05-09T09:27:07.534 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hid/ 2024-05-09T09:27:07.536 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hid/hid-sensor.txt 2024-05-09T09:27:07.538 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hid/hid-transport.txt 2024-05-09T09:27:07.541 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hid/hiddev.txt 2024-05-09T09:27:07.544 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hid/hidraw.txt 2024-05-09T09:27:07.547 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hid/uhid.txt 2024-05-09T09:27:07.549 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/highuid.txt 2024-05-09T09:27:07.552 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hsi.txt 2024-05-09T09:27:07.554 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hw_random.txt 2024-05-09T09:27:07.557 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ 2024-05-09T09:27:07.559 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ab8500 2024-05-09T09:27:07.561 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/abituguru 2024-05-09T09:27:07.563 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/abituguru-datasheet 2024-05-09T09:27:07.566 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/abituguru3 2024-05-09T09:27:07.568 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/abx500 2024-05-09T09:27:07.570 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/acpi_power_meter 2024-05-09T09:27:07.573 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ad7314 2024-05-09T09:27:07.575 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/adc128d818 2024-05-09T09:27:07.577 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/adm1021 2024-05-09T09:27:07.579 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/adm1025 2024-05-09T09:27:07.582 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/adm1026 2024-05-09T09:27:07.584 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/adm1031 2024-05-09T09:27:07.586 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/adm1275 2024-05-09T09:27:07.589 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/adm9240 2024-05-09T09:27:07.591 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ads1015 2024-05-09T09:27:07.593 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ads7828 2024-05-09T09:27:07.596 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/adt7410 2024-05-09T09:27:07.598 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/adt7411 2024-05-09T09:27:07.601 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/adt7462 2024-05-09T09:27:07.603 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/adt7470 2024-05-09T09:27:07.606 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/adt7475 2024-05-09T09:27:07.608 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/amc6821 2024-05-09T09:27:07.610 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/asb100 2024-05-09T09:27:07.613 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/asc7621 2024-05-09T09:27:07.615 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/coretemp 2024-05-09T09:27:07.618 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/da9052 2024-05-09T09:27:07.620 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/da9055 2024-05-09T09:27:07.623 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/dme1737 2024-05-09T09:27:07.626 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ds1621 2024-05-09T09:27:07.629 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ds620 2024-05-09T09:27:07.631 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/emc1403 2024-05-09T09:27:07.633 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/emc2103 2024-05-09T09:27:07.635 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/emc6w201 2024-05-09T09:27:07.637 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/f71805f 2024-05-09T09:27:07.640 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/f71882fg 2024-05-09T09:27:07.643 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/fam15h_power 2024-05-09T09:27:07.645 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/g760a 2024-05-09T09:27:07.648 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/g762 2024-05-09T09:27:07.650 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/gl518sm 2024-05-09T09:27:07.653 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/hih6130 2024-05-09T09:27:07.655 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/htu21 2024-05-09T09:27:07.658 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/hwmon-kernel-api.txt 2024-05-09T09:27:07.661 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ibmaem 2024-05-09T09:27:07.663 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ibmpowernv 2024-05-09T09:27:07.666 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ina209 2024-05-09T09:27:07.668 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ina2xx 2024-05-09T09:27:07.670 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/it87 2024-05-09T09:27:07.673 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/jc42 2024-05-09T09:27:07.675 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/k10temp 2024-05-09T09:27:07.678 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/k8temp 2024-05-09T09:27:07.680 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lineage-pem 2024-05-09T09:27:07.683 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm25066 2024-05-09T09:27:07.685 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm63 2024-05-09T09:27:07.687 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm70 2024-05-09T09:27:07.689 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm73 2024-05-09T09:27:07.692 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm75 2024-05-09T09:27:07.694 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm77 2024-05-09T09:27:07.698 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm78 2024-05-09T09:27:07.699 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm80 2024-05-09T09:27:07.701 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm83 2024-05-09T09:27:07.703 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm85 2024-05-09T09:27:07.706 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm87 2024-05-09T09:27:07.708 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm90 2024-05-09T09:27:07.711 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm92 2024-05-09T09:27:07.714 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm93 2024-05-09T09:27:07.716 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm95234 2024-05-09T09:27:07.718 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/lm95245 2024-05-09T09:27:07.721 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ltc2945 2024-05-09T09:27:07.723 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ltc2978 2024-05-09T09:27:07.726 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ltc4151 2024-05-09T09:27:07.728 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ltc4215 2024-05-09T09:27:07.731 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ltc4245 2024-05-09T09:27:07.733 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ltc4260 2024-05-09T09:27:07.736 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ltc4261 2024-05-09T09:27:07.739 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/max16064 2024-05-09T09:27:07.741 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/max16065 2024-05-09T09:27:07.744 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/max1619 2024-05-09T09:27:07.746 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/max1668 2024-05-09T09:27:07.748 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/max197 2024-05-09T09:27:07.751 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/max34440 2024-05-09T09:27:07.753 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/max6639 2024-05-09T09:27:07.755 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/max6642 2024-05-09T09:27:07.758 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/max6650 2024-05-09T09:27:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/max6697 2024-05-09T09:27:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/max8688 2024-05-09T09:27:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/mc13783-adc 2024-05-09T09:27:07.767 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/mcp3021 2024-05-09T09:27:07.770 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/menf21bmc 2024-05-09T09:27:07.772 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/nct6683 2024-05-09T09:27:07.775 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/nct6775 2024-05-09T09:27:07.778 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/nct7802 2024-05-09T09:27:07.780 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ntc_thermistor 2024-05-09T09:27:07.783 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/pc87360 2024-05-09T09:27:07.785 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/pc87427 2024-05-09T09:27:07.788 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/pcf8591 2024-05-09T09:27:07.790 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/pmbus 2024-05-09T09:27:07.793 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/pmbus-core 2024-05-09T09:27:07.795 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/powr1220 2024-05-09T09:27:07.797 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/pwm-fan 2024-05-09T09:27:07.800 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/sch5627 2024-05-09T09:27:07.802 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/sch5636 2024-05-09T09:27:07.804 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/sht15 2024-05-09T09:27:07.807 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/sht21 2024-05-09T09:27:07.810 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/shtc1 2024-05-09T09:27:07.812 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/sis5595 2024-05-09T09:27:07.815 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/smm665 2024-05-09T09:27:07.817 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/smsc47b397 2024-05-09T09:27:07.819 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/smsc47m1 2024-05-09T09:27:07.822 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/smsc47m192 2024-05-09T09:27:07.824 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/submitting-patches 2024-05-09T09:27:07.827 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/sysfs-interface 2024-05-09T09:27:07.830 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/thmc50 2024-05-09T09:27:07.832 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/tmp102 2024-05-09T09:27:07.834 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/tmp103 2024-05-09T09:27:07.837 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/tmp401 2024-05-09T09:27:07.839 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/tmp421 2024-05-09T09:27:07.841 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/tps40422 2024-05-09T09:27:07.857 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/twl4030-madc-hwmon 2024-05-09T09:27:08.096 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ucd9000 2024-05-09T09:27:08.099 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/ucd9200 2024-05-09T09:27:08.102 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/userspace-tools 2024-05-09T09:27:08.106 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/vexpress 2024-05-09T09:27:08.107 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/via686a 2024-05-09T09:27:08.110 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/vt1211 2024-05-09T09:27:08.113 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/w83627ehf 2024-05-09T09:27:08.116 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/w83627hf 2024-05-09T09:27:08.118 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/w83781d 2024-05-09T09:27:08.121 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/w83791d 2024-05-09T09:27:08.125 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/w83792d 2024-05-09T09:27:08.127 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/w83793 2024-05-09T09:27:08.130 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/w83795 2024-05-09T09:27:08.133 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/w83l785ts 2024-05-09T09:27:08.136 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/w83l786ng 2024-05-09T09:27:08.138 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/wm831x 2024-05-09T09:27:08.141 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/wm8350 2024-05-09T09:27:08.144 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwmon/zl6100 2024-05-09T09:27:08.148 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/hwspinlock.txt 2024-05-09T09:27:08.152 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/ 2024-05-09T09:27:08.154 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/ 2024-05-09T09:27:08.157 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1535 2024-05-09T09:27:08.160 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1563 2024-05-09T09:27:08.163 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali15x3 2024-05-09T09:27:08.166 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd756 2024-05-09T09:27:08.169 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd8111 2024-05-09T09:27:08.171 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-diolan-u2c 2024-05-09T09:27:08.174 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-i801 2024-05-09T09:27:08.177 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ismt 2024-05-09T09:27:08.179 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-nforce2 2024-05-09T09:27:08.182 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ocores 2024-05-09T09:27:08.185 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport 2024-05-09T09:27:08.188 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport-light 2024-05-09T09:27:08.191 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-pca-isa 2024-05-09T09:27:08.194 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-piix4 2024-05-09T09:27:08.197 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis5595 2024-05-09T09:27:08.199 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis630 2024-05-09T09:27:08.202 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis96x 2024-05-09T09:27:08.205 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-taos-evm 2024-05-09T09:27:08.208 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-via 2024-05-09T09:27:08.210 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-viapro 2024-05-09T09:27:08.213 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/busses/scx200_acb 2024-05-09T09:27:08.216 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/dev-interface 2024-05-09T09:27:08.219 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/fault-codes 2024-05-09T09:27:08.223 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/functionality 2024-05-09T09:27:08.225 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/i2c-protocol 2024-05-09T09:27:08.228 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/i2c-stub 2024-05-09T09:27:08.231 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/instantiating-devices 2024-05-09T09:27:08.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:07 smithi012 ceph-mon[30140]: pgmap v414: 129 pgs: 129 active+clean; 90 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 6.9 MiB/s wr, 209 op/s 2024-05-09T09:27:08.234 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/muxes/ 2024-05-09T09:27:08.236 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/muxes/i2c-mux-gpio 2024-05-09T09:27:08.240 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/old-module-parameters 2024-05-09T09:27:08.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:07 smithi170 ceph-mon[33632]: pgmap v414: 129 pgs: 129 active+clean; 90 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 6.9 MiB/s wr, 209 op/s 2024-05-09T09:27:08.243 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/smbus-protocol 2024-05-09T09:27:08.245 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/summary 2024-05-09T09:27:08.248 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/ten-bit-addresses 2024-05-09T09:27:08.252 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/upgrading-clients 2024-05-09T09:27:08.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:07 smithi016 ceph-mon[33335]: pgmap v414: 129 pgs: 129 active+clean; 90 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 6.9 MiB/s wr, 209 op/s 2024-05-09T09:27:08.255 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2c/writing-clients 2024-05-09T09:27:08.258 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2o/ 2024-05-09T09:27:08.261 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2o/README 2024-05-09T09:27:08.264 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/i2o/ioctl 2024-05-09T09:27:08.267 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ia64/ 2024-05-09T09:27:08.270 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ia64/.gitignore 2024-05-09T09:27:08.273 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ia64/IRQ-redir.txt 2024-05-09T09:27:08.277 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ia64/Makefile 2024-05-09T09:27:08.280 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ia64/README 2024-05-09T09:27:08.283 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ia64/aliasing-test.c 2024-05-09T09:27:08.691 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ia64/aliasing.txt 2024-05-09T09:27:08.693 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ia64/efirtc.txt 2024-05-09T09:27:08.696 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ia64/err_inject.txt 2024-05-09T09:27:08.699 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ia64/fsys.txt 2024-05-09T09:27:08.702 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ia64/mca.txt 2024-05-09T09:27:08.704 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ia64/serial.txt 2024-05-09T09:27:08.706 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ia64/xen.txt 2024-05-09T09:27:08.709 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ide/ 2024-05-09T09:27:08.711 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ide/00-INDEX 2024-05-09T09:27:08.713 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-cd.1994-2004 2024-05-09T09:27:08.717 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-floppy.1996-2002 2024-05-09T09:27:08.718 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-tape.1995-2002 2024-05-09T09:27:08.720 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ide/ide-tape.txt 2024-05-09T09:27:08.722 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ide/ide.txt 2024-05-09T09:27:08.725 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ide/warm-plug-howto.txt 2024-05-09T09:27:08.728 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/infiniband/ 2024-05-09T09:27:08.730 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/infiniband/core_locking.txt 2024-05-09T09:27:08.733 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/infiniband/ipoib.txt 2024-05-09T09:27:08.736 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/infiniband/sysfs.txt 2024-05-09T09:27:08.738 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/infiniband/user_mad.txt 2024-05-09T09:27:08.740 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/infiniband/user_verbs.txt 2024-05-09T09:27:08.743 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/init.txt 2024-05-09T09:27:08.745 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/initrd.txt 2024-05-09T09:27:08.748 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/ 2024-05-09T09:27:08.750 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/alps.txt 2024-05-09T09:27:08.752 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/amijoy.txt 2024-05-09T09:27:08.754 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/appletouch.txt 2024-05-09T09:27:08.757 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/atarikbd.txt 2024-05-09T09:27:08.760 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/bcm5974.txt 2024-05-09T09:27:08.762 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/cd32.txt 2024-05-09T09:27:08.764 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/cma3000_d0x.txt 2024-05-09T09:27:08.767 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/cs461x.txt 2024-05-09T09:27:08.769 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/edt-ft5x06.txt 2024-05-09T09:27:08.771 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/elantech.txt 2024-05-09T09:27:08.774 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/event-codes.txt 2024-05-09T09:27:08.776 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/ff.txt 2024-05-09T09:27:08.779 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/gamepad.txt 2024-05-09T09:27:08.781 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/gameport-programming.txt 2024-05-09T09:27:08.783 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/gpio-tilt.txt 2024-05-09T09:27:08.786 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/iforce-protocol.txt 2024-05-09T09:27:08.789 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/input-programming.txt 2024-05-09T09:27:08.791 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/input.txt 2024-05-09T09:27:08.794 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/interactive.fig 2024-05-09T09:27:08.796 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/joystick-api.txt 2024-05-09T09:27:08.799 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/joystick-parport.txt 2024-05-09T09:27:08.801 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/joystick.txt 2024-05-09T09:27:08.804 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/multi-touch-protocol.txt 2024-05-09T09:27:08.808 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/notifier.txt 2024-05-09T09:27:08.809 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/ntrig.txt 2024-05-09T09:27:08.811 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/rotary-encoder.txt 2024-05-09T09:27:08.814 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/sentelic.txt 2024-05-09T09:27:08.817 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/shape.fig 2024-05-09T09:27:08.819 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/walkera0701.txt 2024-05-09T09:27:08.821 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/xpad.txt 2024-05-09T09:27:08.824 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/input/yealink.txt 2024-05-09T09:27:08.826 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/intel_txt.txt 2024-05-09T09:27:08.829 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/io-mapping.txt 2024-05-09T09:27:08.831 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/io_ordering.txt 2024-05-09T09:27:08.834 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ioctl/ 2024-05-09T09:27:08.835 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ioctl/00-INDEX 2024-05-09T09:27:08.837 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ioctl/botching-up-ioctls.txt 2024-05-09T09:27:08.840 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ioctl/cdrom.txt 2024-05-09T09:27:08.843 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ioctl/hdio.txt 2024-05-09T09:27:08.846 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ioctl/ioctl-decoding.txt 2024-05-09T09:27:08.848 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ioctl/ioctl-number.txt 2024-05-09T09:27:08.851 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/iostats.txt 2024-05-09T09:27:08.853 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/irqflags-tracing.txt 2024-05-09T09:27:08.856 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isapnp.txt 2024-05-09T09:27:08.878 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/ 2024-05-09T09:27:08.975 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/00-INDEX 2024-05-09T09:27:08.977 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/CREDITS 2024-05-09T09:27:08.979 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/HiSax.cert 2024-05-09T09:27:08.981 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/INTERFACE 2024-05-09T09:27:08.985 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.CAPI 2024-05-09T09:27:08.987 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.fax 2024-05-09T09:27:08.990 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README 2024-05-09T09:27:08.993 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.FAQ 2024-05-09T09:27:08.995 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.HiSax 2024-05-09T09:27:08.998 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.act2000 2024-05-09T09:27:09.000 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.audio 2024-05-09T09:27:09.002 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.avmb1 2024-05-09T09:27:09.005 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.concap 2024-05-09T09:27:09.007 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.diversion 2024-05-09T09:27:09.010 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.fax 2024-05-09T09:27:09.012 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.gigaset 2024-05-09T09:27:09.015 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.hfc-pci 2024-05-09T09:27:09.018 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.hysdn 2024-05-09T09:27:09.021 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.icn 2024-05-09T09:27:09.023 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.mISDN 2024-05-09T09:27:09.025 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.pcbit 2024-05-09T09:27:09.027 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.sc 2024-05-09T09:27:09.030 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.syncppp 2024-05-09T09:27:09.033 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/README.x25 2024-05-09T09:27:09.035 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/isdn/syncPPP.FAQ 2024-05-09T09:27:09.038 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ja_JP/ 2024-05-09T09:27:09.040 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ja_JP/HOWTO 2024-05-09T09:27:09.043 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ja_JP/SubmitChecklist 2024-05-09T09:27:09.045 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ja_JP/SubmittingPatches 2024-05-09T09:27:09.048 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ja_JP/stable_api_nonsense.txt 2024-05-09T09:27:09.051 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ja_JP/stable_kernel_rules.txt 2024-05-09T09:27:09.053 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/java.txt 2024-05-09T09:27:09.056 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kasan.txt 2024-05-09T09:27:09.058 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kbuild/ 2024-05-09T09:27:09.060 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kbuild/00-INDEX 2024-05-09T09:27:09.062 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kbuild/headers_install.txt 2024-05-09T09:27:09.064 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kbuild/kbuild.txt 2024-05-09T09:27:09.067 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kbuild/kconfig-language.txt 2024-05-09T09:27:09.069 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kbuild/kconfig.txt 2024-05-09T09:27:09.072 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kbuild/makefiles.txt 2024-05-09T09:27:09.076 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kbuild/modules.txt 2024-05-09T09:27:09.079 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kdump/ 2024-05-09T09:27:09.081 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kdump/gdbmacros.txt 2024-05-09T09:27:09.083 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kdump/kdump.txt 2024-05-09T09:27:09.086 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kernel-doc-nano-HOWTO.txt 2024-05-09T09:27:09.089 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kernel-docs.txt 2024-05-09T09:27:09.092 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kernel-parameters.txt 2024-05-09T09:27:09.098 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kernel-per-CPU-kthreads.txt 2024-05-09T09:27:09.100 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kmemcheck.txt 2024-05-09T09:27:09.103 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kmemleak.txt 2024-05-09T09:27:09.106 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ko_KR/ 2024-05-09T09:27:09.107 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ko_KR/HOWTO 2024-05-09T09:27:09.110 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ko_KR/stable_api_nonsense.txt 2024-05-09T09:27:09.113 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kobject.txt 2024-05-09T09:27:09.116 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kprobes.txt 2024-05-09T09:27:09.119 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kref.txt 2024-05-09T09:27:09.122 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/kselftest.txt 2024-05-09T09:27:09.125 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/laptops/ 2024-05-09T09:27:09.126 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/laptops/.gitignore 2024-05-09T09:27:09.128 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/laptops/00-INDEX 2024-05-09T09:27:09.130 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/laptops/Makefile 2024-05-09T09:27:09.133 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/laptops/asus-laptop.txt 2024-05-09T09:27:09.135 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/laptops/disk-shock-protection.txt 2024-05-09T09:27:09.138 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/laptops/dslm.c 2024-05-09T09:27:09.140 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/laptops/freefall.c 2024-05-09T09:27:09.143 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/laptops/laptop-mode.txt 2024-05-09T09:27:09.146 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/laptops/sony-laptop.txt 2024-05-09T09:27:09.148 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/laptops/sonypi.txt 2024-05-09T09:27:09.151 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/laptops/thinkpad-acpi.txt 2024-05-09T09:27:09.155 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/laptops/toshiba_haps.txt 2024-05-09T09:27:09.157 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ldm.txt 2024-05-09T09:27:09.162 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/leds/ 2024-05-09T09:27:09.164 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/leds/00-INDEX 2024-05-09T09:27:09.166 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/leds/leds-blinkm.txt 2024-05-09T09:27:09.168 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/leds/leds-class.txt 2024-05-09T09:27:09.171 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/leds/leds-lm3556.txt 2024-05-09T09:27:09.173 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/leds/leds-lp3944.txt 2024-05-09T09:27:09.175 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/leds/leds-lp5521.txt 2024-05-09T09:27:09.177 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/leds/leds-lp5523.txt 2024-05-09T09:27:09.179 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/leds/leds-lp5562.txt 2024-05-09T09:27:09.182 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/leds/leds-lp55xx.txt 2024-05-09T09:27:09.185 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/leds/ledtrig-oneshot.txt 2024-05-09T09:27:09.187 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/leds/ledtrig-transient.txt 2024-05-09T09:27:09.190 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/local_ops.txt 2024-05-09T09:27:09.193 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/locking/ 2024-05-09T09:27:09.195 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/locking/00-INDEX 2024-05-09T09:27:09.196 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/locking/lglock.txt 2024-05-09T09:27:09.199 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/locking/lockdep-design.txt 2024-05-09T09:27:09.201 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/locking/lockstat.txt 2024-05-09T09:27:09.204 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/locking/locktorture.txt 2024-05-09T09:27:09.207 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/locking/mutex-design.txt 2024-05-09T09:27:09.209 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/locking/rt-mutex-design.txt 2024-05-09T09:27:09.212 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/locking/rt-mutex.txt 2024-05-09T09:27:09.215 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/locking/spinlocks.txt 2024-05-09T09:27:09.217 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/locking/ww-mutex-design.txt 2024-05-09T09:27:09.220 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/lockup-watchdogs.txt 2024-05-09T09:27:09.223 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/logo.gif 2024-05-09T09:27:09.226 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/logo.txt 2024-05-09T09:27:09.229 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/lzo.txt 2024-05-09T09:27:09.232 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/m68k/ 2024-05-09T09:27:09.233 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/m68k/00-INDEX 2024-05-09T09:27:09.235 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/m68k/README.buddha 2024-05-09T09:27:09.238 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/m68k/kernel-options.txt 2024-05-09T09:27:09.241 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/magic-number.txt 2024-05-09T09:27:09.244 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mailbox.txt 2024-05-09T09:27:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/md.txt 2024-05-09T09:27:09.250 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/media-framework.txt 2024-05-09T09:27:09.252 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/memory-barriers.txt 2024-05-09T09:27:09.419 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/memory-devices/ 2024-05-09T09:27:09.421 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/memory-devices/ti-emif.txt 2024-05-09T09:27:09.423 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/memory-hotplug.txt 2024-05-09T09:27:09.426 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/metag/ 2024-05-09T09:27:09.428 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/metag/00-INDEX 2024-05-09T09:27:09.431 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/metag/kernel-ABI.txt 2024-05-09T09:27:09.433 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mic/ 2024-05-09T09:27:09.435 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mic/Makefile 2024-05-09T09:27:09.437 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mic/mic_overview.txt 2024-05-09T09:27:09.440 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mic/mpssd/ 2024-05-09T09:27:09.442 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mic/mpssd/.gitignore 2024-05-09T09:27:09.444 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mic/mpssd/Makefile 2024-05-09T09:27:09.446 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mic/mpssd/micctrl 2024-05-09T09:27:09.448 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mic/mpssd/mpss 2024-05-09T09:27:09.451 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.c 2024-05-09T09:27:09.454 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.h 2024-05-09T09:27:09.457 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mic/mpssd/sysfs.c 2024-05-09T09:27:09.459 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mips/ 2024-05-09T09:27:09.462 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mips/00-INDEX 2024-05-09T09:27:09.464 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mips/AU1xxx_IDE.README 2024-05-09T09:27:09.466 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/ 2024-05-09T09:27:09.468 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/Makefile 2024-05-09T09:27:09.470 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/ad525x_dpot.txt 2024-05-09T09:27:09.472 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/apds990x.txt 2024-05-09T09:27:09.474 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/bh1770glc.txt 2024-05-09T09:27:09.477 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/c2port.txt 2024-05-09T09:27:09.479 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/eeprom 2024-05-09T09:27:09.481 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/ics932s401 2024-05-09T09:27:09.484 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/isl29003 2024-05-09T09:27:09.486 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/lis3lv02d 2024-05-09T09:27:09.488 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/max6875 2024-05-09T09:27:09.491 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/mei/ 2024-05-09T09:27:09.492 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/mei/.gitignore 2024-05-09T09:27:09.495 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/mei/Makefile 2024-05-09T09:27:09.497 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/mei/TODO 2024-05-09T09:27:09.500 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-amt-version.c 2024-05-09T09:27:09.502 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-client-bus.txt 2024-05-09T09:27:09.504 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei.txt 2024-05-09T09:27:09.507 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/misc-devices/spear-pcie-gadget.txt 2024-05-09T09:27:09.511 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mmc/ 2024-05-09T09:27:09.512 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mmc/00-INDEX 2024-05-09T09:27:09.514 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mmc/mmc-async-req.txt 2024-05-09T09:27:09.516 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-attrs.txt 2024-05-09T09:27:09.518 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-parts.txt 2024-05-09T09:27:09.521 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mn10300/ 2024-05-09T09:27:09.523 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mn10300/ABI.txt 2024-05-09T09:27:09.525 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mn10300/compartmentalisation.txt 2024-05-09T09:27:09.527 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/module-signing.txt 2024-05-09T09:27:09.530 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mono.txt 2024-05-09T09:27:09.534 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mtd/ 2024-05-09T09:27:09.535 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mtd/nand/ 2024-05-09T09:27:09.536 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt 2024-05-09T09:27:09.538 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mtd/nand_ecc.txt 2024-05-09T09:27:09.542 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/mtd/spi-nor.txt 2024-05-09T09:27:09.544 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/namespaces/ 2024-05-09T09:27:09.546 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/namespaces/compatibility-list.txt 2024-05-09T09:27:09.548 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/namespaces/resource-control.txt 2024-05-09T09:27:09.550 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/netlabel/ 2024-05-09T09:27:09.552 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/netlabel/00-INDEX 2024-05-09T09:27:09.554 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/netlabel/cipso_ipv4.txt 2024-05-09T09:27:09.557 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/netlabel/draft-ietf-cipso-ipsecurity-01.txt 2024-05-09T09:27:09.560 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/netlabel/introduction.txt 2024-05-09T09:27:09.562 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/netlabel/lsm_interface.txt 2024-05-09T09:27:09.564 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ 2024-05-09T09:27:09.566 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/00-INDEX 2024-05-09T09:27:09.568 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/3c509.txt 2024-05-09T09:27:09.571 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/6pack.txt 2024-05-09T09:27:09.574 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/LICENSE.qla3xxx 2024-05-09T09:27:09.576 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlcnic 2024-05-09T09:27:09.578 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlge 2024-05-09T09:27:09.581 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/Makefile 2024-05-09T09:27:09.583 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/PLIP.txt 2024-05-09T09:27:09.585 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/README.ipw2100 2024-05-09T09:27:09.588 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/README.ipw2200 2024-05-09T09:27:09.590 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/README.sb1000 2024-05-09T09:27:09.593 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/alias.txt 2024-05-09T09:27:09.595 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/altera_tse.txt 2024-05-09T09:27:09.598 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/arcnet-hardware.txt 2024-05-09T09:27:09.602 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/arcnet.txt 2024-05-09T09:27:09.606 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/atm.txt 2024-05-09T09:27:09.607 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ax25.txt 2024-05-09T09:27:09.610 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/batman-adv.txt 2024-05-09T09:27:09.612 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/baycom.txt 2024-05-09T09:27:09.615 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/bonding.txt 2024-05-09T09:27:09.619 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/bridge.txt 2024-05-09T09:27:09.622 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/caif/ 2024-05-09T09:27:09.624 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/caif/Linux-CAIF.txt 2024-05-09T09:27:09.626 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/caif/README 2024-05-09T09:27:09.628 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/caif/spi_porting.txt 2024-05-09T09:27:09.631 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/can.txt 2024-05-09T09:27:09.635 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/cdc_mbim.txt 2024-05-09T09:27:09.638 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/cops.txt 2024-05-09T09:27:09.641 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/cs89x0.txt 2024-05-09T09:27:09.643 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/cxacru-cf.py 2024-05-09T09:27:09.645 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/cxacru.txt 2024-05-09T09:27:09.649 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/cxgb.txt 2024-05-09T09:27:09.652 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/dccp.txt 2024-05-09T09:27:09.653 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/dctcp.txt 2024-05-09T09:27:09.655 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/de4x5.txt 2024-05-09T09:27:09.657 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/decnet.txt 2024-05-09T09:27:09.663 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/dl2k.txt 2024-05-09T09:27:09.664 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/dm9000.txt 2024-05-09T09:27:09.665 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/dmfe.txt 2024-05-09T09:27:09.667 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/dns_resolver.txt 2024-05-09T09:27:09.669 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/driver.txt 2024-05-09T09:27:09.671 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/e100.txt 2024-05-09T09:27:09.673 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/e1000.txt 2024-05-09T09:27:09.676 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/e1000e.txt 2024-05-09T09:27:09.680 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/eql.txt 2024-05-09T09:27:09.682 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/fib_trie.txt 2024-05-09T09:27:09.684 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/filter.txt 2024-05-09T09:27:09.687 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/fore200e.txt 2024-05-09T09:27:09.689 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/framerelay.txt 2024-05-09T09:27:09.692 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/gen_stats.txt 2024-05-09T09:27:09.694 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/generic-hdlc.txt 2024-05-09T09:27:09.697 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/generic_netlink.txt 2024-05-09T09:27:09.699 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/gianfar.txt 2024-05-09T09:27:09.701 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/i40e.txt 2024-05-09T09:27:09.704 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/i40evf.txt 2024-05-09T09:27:09.706 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ieee802154.txt 2024-05-09T09:27:09.708 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/igb.txt 2024-05-09T09:27:09.711 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/igbvf.txt 2024-05-09T09:27:09.713 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ip-sysctl.txt 2024-05-09T09:27:09.718 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ip_dynaddr.txt 2024-05-09T09:27:09.719 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ipddp.txt 2024-05-09T09:27:09.722 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/iphase.txt 2024-05-09T09:27:09.724 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ipsec.txt 2024-05-09T09:27:09.727 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ipv6.txt 2024-05-09T09:27:09.729 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ipvlan.txt 2024-05-09T09:27:09.732 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ipvs-sysctl.txt 2024-05-09T09:27:09.737 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/irda.txt 2024-05-09T09:27:09.739 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ixgb.txt 2024-05-09T09:27:09.741 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ixgbe.txt 2024-05-09T09:27:09.742 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ixgbevf.txt 2024-05-09T09:27:09.745 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/l2tp.txt 2024-05-09T09:27:09.748 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/lapb-module.txt 2024-05-09T09:27:09.750 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ltpc.txt 2024-05-09T09:27:09.753 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/mac80211-auth-assoc-deauth.txt 2024-05-09T09:27:09.755 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/mac80211-injection.txt 2024-05-09T09:27:09.757 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/ 2024-05-09T09:27:09.759 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/README 2024-05-09T09:27:09.762 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/hostapd.conf 2024-05-09T09:27:09.764 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/wpa_supplicant.conf 2024-05-09T09:27:09.766 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/multiqueue.txt 2024-05-09T09:27:09.769 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/netconsole.txt 2024-05-09T09:27:09.772 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/netdev-FAQ.txt 2024-05-09T09:27:09.774 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/netdev-features.txt 2024-05-09T09:27:09.777 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/netdevices.txt 2024-05-09T09:27:09.779 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/netif-msg.txt 2024-05-09T09:27:09.782 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/netlink_mmap.txt 2024-05-09T09:27:09.784 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/nf_conntrack-sysctl.txt 2024-05-09T09:27:09.786 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/nfc.txt 2024-05-09T09:27:09.789 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/openvswitch.txt 2024-05-09T09:27:09.791 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/operstates.txt 2024-05-09T09:27:09.793 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/packet_mmap.txt 2024-05-09T09:27:09.796 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/phonet.txt 2024-05-09T09:27:09.799 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/phy.txt 2024-05-09T09:27:09.802 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/pktgen.txt 2024-05-09T09:27:09.804 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/policy-routing.txt 2024-05-09T09:27:09.806 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ppp_generic.txt 2024-05-09T09:27:09.809 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/proc_net_tcp.txt 2024-05-09T09:27:09.811 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/radiotap-headers.txt 2024-05-09T09:27:09.813 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/ray_cs.txt 2024-05-09T09:27:09.816 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/rds.txt 2024-05-09T09:27:09.818 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/regulatory.txt 2024-05-09T09:27:09.821 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/rxrpc.txt 2024-05-09T09:27:09.824 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/s2io.txt 2024-05-09T09:27:09.827 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/scaling.txt 2024-05-09T09:27:09.829 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/sctp.txt 2024-05-09T09:27:09.832 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/secid.txt 2024-05-09T09:27:09.834 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/skfp.txt 2024-05-09T09:27:09.836 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/smc9.txt 2024-05-09T09:27:09.838 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/spider_net.txt 2024-05-09T09:27:09.841 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/stmmac.txt 2024-05-09T09:27:09.844 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/switchdev.txt 2024-05-09T09:27:09.846 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/tc-actions-env-rules.txt 2024-05-09T09:27:09.848 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/tcp-thin.txt 2024-05-09T09:27:09.851 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/tcp.txt 2024-05-09T09:27:09.853 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/team.txt 2024-05-09T09:27:09.856 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/timestamping.txt 2024-05-09T09:27:09.858 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/timestamping/ 2024-05-09T09:27:09.860 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/timestamping/.gitignore 2024-05-09T09:27:09.862 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/timestamping/Makefile 2024-05-09T09:27:09.864 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c 2024-05-09T09:27:09.866 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/timestamping/timestamping.c 2024-05-09T09:27:09.869 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c 2024-05-09T09:27:09.871 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/tlan.txt 2024-05-09T09:27:09.875 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/tproxy.txt 2024-05-09T09:27:09.877 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/tuntap.txt 2024-05-09T09:27:09.892 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/udplite.txt 2024-05-09T09:27:10.027 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/vortex.txt 2024-05-09T09:27:10.030 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/vxge.txt 2024-05-09T09:27:10.033 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/vxlan.txt 2024-05-09T09:27:10.035 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/x25-iface.txt 2024-05-09T09:27:10.037 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/x25.txt 2024-05-09T09:27:10.040 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/xfrm_proc.txt 2024-05-09T09:27:10.042 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/xfrm_sync.txt 2024-05-09T09:27:10.044 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/xfrm_sysctl.txt 2024-05-09T09:27:10.047 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/networking/z8530drv.txt 2024-05-09T09:27:10.050 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/nfc/ 2024-05-09T09:27:10.052 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/nfc/nfc-hci.txt 2024-05-09T09:27:10.054 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/nfc/nfc-pn544.txt 2024-05-09T09:27:10.056 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/nios2/ 2024-05-09T09:27:10.059 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/nios2/README 2024-05-09T09:27:10.061 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/nommu-mmap.txt 2024-05-09T09:27:10.064 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/numastat.txt 2024-05-09T09:27:10.066 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/oops-tracing.txt 2024-05-09T09:27:10.069 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/padata.txt 2024-05-09T09:27:10.071 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/parisc/ 2024-05-09T09:27:10.073 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/parisc/00-INDEX 2024-05-09T09:27:10.075 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/parisc/debugging 2024-05-09T09:27:10.077 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/parisc/registers 2024-05-09T09:27:10.079 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/parport-lowlevel.txt 2024-05-09T09:27:10.083 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/parport.txt 2024-05-09T09:27:10.085 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pcmcia/ 2024-05-09T09:27:10.087 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pcmcia/.gitignore 2024-05-09T09:27:10.089 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pcmcia/Makefile 2024-05-09T09:27:10.092 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pcmcia/crc32hash.c 2024-05-09T09:27:10.094 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pcmcia/devicetable.txt 2024-05-09T09:27:10.096 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pcmcia/driver-changes.txt 2024-05-09T09:27:10.099 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pcmcia/driver.txt 2024-05-09T09:27:10.101 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pcmcia/locking.txt 2024-05-09T09:27:10.103 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/percpu-rw-semaphore.txt 2024-05-09T09:27:10.106 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/phy.txt 2024-05-09T09:27:10.108 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/phy/ 2024-05-09T09:27:10.111 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/phy/samsung-usb2.txt 2024-05-09T09:27:10.113 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pi-futex.txt 2024-05-09T09:27:10.115 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pinctrl.txt 2024-05-09T09:27:10.119 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/platform/ 2024-05-09T09:27:10.120 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/platform/x86-laptop-drivers.txt 2024-05-09T09:27:10.123 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pnp.txt 2024-05-09T09:27:10.125 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/ 2024-05-09T09:27:10.127 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/00-INDEX 2024-05-09T09:27:10.129 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/apm-acpi.txt 2024-05-09T09:27:10.132 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/basic-pm-debugging.txt 2024-05-09T09:27:10.134 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/charger-manager.txt 2024-05-09T09:27:10.137 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/devices.txt 2024-05-09T09:27:10.140 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/drivers-testing.txt 2024-05-09T09:27:10.142 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/freezing-of-tasks.txt 2024-05-09T09:27:10.145 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/interface.txt 2024-05-09T09:27:10.147 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/notifiers.txt 2024-05-09T09:27:10.150 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/opp.txt 2024-05-09T09:27:10.152 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/pci.txt 2024-05-09T09:27:10.156 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/pm_qos_interface.txt 2024-05-09T09:27:10.158 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/power_supply_class.txt 2024-05-09T09:27:10.161 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/powercap/ 2024-05-09T09:27:10.162 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/powercap/powercap.txt 2024-05-09T09:27:10.165 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/regulator/ 2024-05-09T09:27:10.167 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/regulator/consumer.txt 2024-05-09T09:27:10.169 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/regulator/design.txt 2024-05-09T09:27:10.172 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/regulator/machine.txt 2024-05-09T09:27:10.174 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/regulator/overview.txt 2024-05-09T09:27:10.177 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/regulator/regulator.txt 2024-05-09T09:27:10.179 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/runtime_pm.txt 2024-05-09T09:27:10.182 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/s2ram.txt 2024-05-09T09:27:10.185 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/states.txt 2024-05-09T09:27:10.187 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/suspend-and-cpuhotplug.txt 2024-05-09T09:27:10.189 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/suspend-and-interrupts.txt 2024-05-09T09:27:10.192 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/swsusp-and-swap-files.txt 2024-05-09T09:27:10.195 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/swsusp-dmcrypt.txt 2024-05-09T09:27:10.196 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/swsusp.txt 2024-05-09T09:27:10.199 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/tricks.txt 2024-05-09T09:27:10.201 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/userland-swsusp.txt 2024-05-09T09:27:10.204 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/power/video.txt 2024-05-09T09:27:10.206 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/ 2024-05-09T09:27:10.208 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/00-INDEX 2024-05-09T09:27:10.210 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/bootwrapper.txt 2024-05-09T09:27:10.212 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/cpu_families.txt 2024-05-09T09:27:10.215 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/cpu_features.txt 2024-05-09T09:27:10.217 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/cxl.txt 2024-05-09T09:27:10.220 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/eeh-pci-error-recovery.txt 2024-05-09T09:27:10.223 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/firmware-assisted-dump.txt 2024-05-09T09:27:10.226 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/hvcs.txt 2024-05-09T09:27:10.228 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/mpc52xx.txt 2024-05-09T09:27:10.230 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/pmu-ebb.txt 2024-05-09T09:27:10.233 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/ptrace.txt 2024-05-09T09:27:10.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:09 smithi012 ceph-mon[30140]: pgmap v415: 129 pgs: 129 active+clean; 91 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 6.0 MiB/s wr, 237 op/s 2024-05-09T09:27:10.235 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/qe_firmware.txt 2024-05-09T09:27:10.238 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/powerpc/transactional_memory.txt 2024-05-09T09:27:10.240 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pps/ 2024-05-09T09:27:10.242 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:09 smithi170 ceph-mon[33632]: pgmap v415: 129 pgs: 129 active+clean; 91 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 6.0 MiB/s wr, 237 op/s 2024-05-09T09:27:10.243 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pps/pps.txt 2024-05-09T09:27:10.245 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/prctl/ 2024-05-09T09:27:10.248 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/prctl/.gitignore 2024-05-09T09:27:10.250 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/prctl/Makefile 2024-05-09T09:27:10.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:09 smithi016 ceph-mon[33335]: pgmap v415: 129 pgs: 129 active+clean; 91 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 6.0 MiB/s wr, 237 op/s 2024-05-09T09:27:10.253 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-ctxt-sw-stress-test.c 2024-05-09T09:27:10.255 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-on-off-stress-test.c 2024-05-09T09:27:10.258 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-test.c 2024-05-09T09:27:10.260 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/prctl/no_new_privs.txt 2024-05-09T09:27:10.262 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/prctl/seccomp_filter.txt 2024-05-09T09:27:10.265 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/preempt-locking.txt 2024-05-09T09:27:10.267 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/printk-formats.txt 2024-05-09T09:27:10.269 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pti/ 2024-05-09T09:27:10.271 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pti/pti_intel_mid.txt 2024-05-09T09:27:10.273 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ptp/ 2024-05-09T09:27:10.275 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ptp/.gitignore 2024-05-09T09:27:10.277 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ptp/Makefile 2024-05-09T09:27:10.279 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ptp/ptp.txt 2024-05-09T09:27:10.282 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ptp/testptp.c 2024-05-09T09:27:10.284 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ptp/testptp.mk 2024-05-09T09:27:10.287 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/pwm.txt 2024-05-09T09:27:10.290 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/ramoops.txt 2024-05-09T09:27:10.293 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/rapidio/ 2024-05-09T09:27:10.295 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/rapidio/rapidio.txt 2024-05-09T09:27:10.298 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/rapidio/sysfs.txt 2024-05-09T09:27:10.300 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/rapidio/tsi721.txt 2024-05-09T09:27:10.303 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/rbtree.txt 2024-05-09T09:27:10.306 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/remoteproc.txt 2024-05-09T09:27:10.309 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/rfkill.txt 2024-05-09T09:27:10.312 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/robust-futex-ABI.txt 2024-05-09T09:27:10.315 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/robust-futexes.txt 2024-05-09T09:27:10.318 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/rpmsg.txt 2024-05-09T09:27:10.321 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/rtc.txt 2024-05-09T09:27:10.330 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/ 2024-05-09T09:27:10.333 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/00-INDEX 2024-05-09T09:27:10.337 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/3270.ChangeLog 2024-05-09T09:27:10.340 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/3270.txt 2024-05-09T09:27:10.343 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/CommonIO 2024-05-09T09:27:10.347 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/DASD 2024-05-09T09:27:10.349 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/Debugging390.txt 2024-05-09T09:27:10.354 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/cds.txt 2024-05-09T09:27:10.358 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/config3270.sh 2024-05-09T09:27:10.362 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/driver-model.txt 2024-05-09T09:27:10.365 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/kvm.txt 2024-05-09T09:27:10.367 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/monreader.txt 2024-05-09T09:27:10.370 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/qeth.txt 2024-05-09T09:27:10.373 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/s390dbf.txt 2024-05-09T09:27:10.376 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/s390/zfcpdump.txt 2024-05-09T09:27:10.378 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scheduler/ 2024-05-09T09:27:10.381 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scheduler/00-INDEX 2024-05-09T09:27:10.384 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scheduler/completion.txt 2024-05-09T09:27:10.387 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scheduler/sched-arch.txt 2024-05-09T09:27:10.390 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scheduler/sched-bwc.txt 2024-05-09T09:27:10.393 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scheduler/sched-deadline.txt 2024-05-09T09:27:10.396 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scheduler/sched-design-CFS.txt 2024-05-09T09:27:10.398 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scheduler/sched-domains.txt 2024-05-09T09:27:10.401 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scheduler/sched-nice-design.txt 2024-05-09T09:27:10.404 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scheduler/sched-rt-group.txt 2024-05-09T09:27:10.406 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scheduler/sched-stats.txt 2024-05-09T09:27:10.409 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/ 2024-05-09T09:27:10.413 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/00-INDEX 2024-05-09T09:27:10.416 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/53c700.txt 2024-05-09T09:27:10.419 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/BusLogic.txt 2024-05-09T09:27:10.422 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.1992-1997 2024-05-09T09:27:10.427 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.arcmsr 2024-05-09T09:27:10.429 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ips 2024-05-09T09:27:10.432 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.lpfc 2024-05-09T09:27:10.437 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid 2024-05-09T09:27:10.441 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid_sas 2024-05-09T09:27:10.444 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ncr53c8xx 2024-05-09T09:27:10.447 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx 2024-05-09T09:27:10.450 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx_2 2024-05-09T09:27:10.453 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/FlashPoint.txt 2024-05-09T09:27:10.456 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/LICENSE.FlashPoint 2024-05-09T09:27:10.458 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla2xxx 2024-05-09T09:27:10.461 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla4xxx 2024-05-09T09:27:10.465 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/Mylex.txt 2024-05-09T09:27:10.468 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/NinjaSCSI.txt 2024-05-09T09:27:10.471 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/aacraid.txt 2024-05-09T09:27:10.473 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/advansys.txt 2024-05-09T09:27:10.476 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/aha152x.txt 2024-05-09T09:27:10.480 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/aic79xx.txt 2024-05-09T09:27:10.483 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/aic7xxx.txt 2024-05-09T09:27:10.486 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/arcmsr_spec.txt 2024-05-09T09:27:10.489 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/bfa.txt 2024-05-09T09:27:10.492 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/bnx2fc.txt 2024-05-09T09:27:10.494 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/cxgb3i.txt 2024-05-09T09:27:10.496 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/dc395x.txt 2024-05-09T09:27:10.499 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/dpti.txt 2024-05-09T09:27:10.502 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/dtc3x80.txt 2024-05-09T09:27:10.505 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/g_NCR5380.txt 2024-05-09T09:27:10.507 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/hpsa.txt 2024-05-09T09:27:10.510 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/hptiop.txt 2024-05-09T09:27:10.513 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/in2000.txt 2024-05-09T09:27:10.516 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/libsas.txt 2024-05-09T09:27:10.519 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/link_power_management_policy.txt 2024-05-09T09:27:10.522 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/lpfc.txt 2024-05-09T09:27:10.524 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/megaraid.txt 2024-05-09T09:27:10.527 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/ncr53c8xx.txt 2024-05-09T09:27:10.532 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/osd.txt 2024-05-09T09:27:10.535 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/osst.txt 2024-05-09T09:27:10.538 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/ppa.txt 2024-05-09T09:27:10.540 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/qlogicfas.txt 2024-05-09T09:27:10.543 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/scsi-changer.txt 2024-05-09T09:27:10.545 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/scsi-generic.txt 2024-05-09T09:27:10.548 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/scsi-parameters.txt 2024-05-09T09:27:10.551 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/scsi.txt 2024-05-09T09:27:10.553 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/scsi_eh.txt 2024-05-09T09:27:10.556 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/scsi_fc_transport.txt 2024-05-09T09:27:10.559 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/scsi_mid_low_api.txt 2024-05-09T09:27:10.563 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/ 2024-05-09T09:27:10.566 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/Makefile 2024-05-09T09:27:10.569 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/rport_state_diagram.dot 2024-05-09T09:27:10.571 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/st.txt 2024-05-09T09:27:10.575 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/sym53c500_cs.txt 2024-05-09T09:27:10.577 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/sym53c8xx_2.txt 2024-05-09T09:27:10.581 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/tmscsim.txt 2024-05-09T09:27:10.585 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/ufs.txt 2024-05-09T09:27:10.587 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/scsi/wd719x.txt 2024-05-09T09:27:10.590 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/ 2024-05-09T09:27:10.593 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/00-INDEX 2024-05-09T09:27:10.596 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/IMA-templates.txt 2024-05-09T09:27:10.599 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/LSM.txt 2024-05-09T09:27:10.601 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/SELinux.txt 2024-05-09T09:27:10.604 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/Smack.txt 2024-05-09T09:27:10.608 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/Yama.txt 2024-05-09T09:27:10.612 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/apparmor.txt 2024-05-09T09:27:10.615 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/credentials.txt 2024-05-09T09:27:10.618 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/keys-ecryptfs.txt 2024-05-09T09:27:10.621 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/keys-request-key.txt 2024-05-09T09:27:10.624 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/keys-trusted-encrypted.txt 2024-05-09T09:27:10.627 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/keys.txt 2024-05-09T09:27:10.631 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/security/tomoyo.txt 2024-05-09T09:27:10.634 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/serial-console.txt 2024-05-09T09:27:10.637 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/serial/ 2024-05-09T09:27:10.639 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/serial/00-INDEX 2024-05-09T09:27:10.643 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/serial/README.cycladesZ 2024-05-09T09:27:10.646 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/serial/driver 2024-05-09T09:27:10.649 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/serial/moxa-smartio 2024-05-09T09:27:10.652 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/serial/n_gsm.txt 2024-05-09T09:27:10.655 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/serial/rocket.txt 2024-05-09T09:27:10.657 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/serial/serial-rs485.txt 2024-05-09T09:27:10.660 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/serial/tty.txt 2024-05-09T09:27:10.664 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sgi-ioc4.txt 2024-05-09T09:27:10.667 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sh/ 2024-05-09T09:27:10.669 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sh/new-machine.txt 2024-05-09T09:27:10.672 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sh/register-banks.txt 2024-05-09T09:27:10.675 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/smsc_ece1099.txt 2024-05-09T09:27:10.678 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/ 2024-05-09T09:27:10.681 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/ 2024-05-09T09:27:10.684 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/ALSA-Configuration.txt 2024-05-09T09:27:10.689 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/Audigy-mixer.txt 2024-05-09T09:27:10.692 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/Audiophile-Usb.txt 2024-05-09T09:27:10.696 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/Bt87x.txt 2024-05-09T09:27:10.699 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/CMIPCI.txt 2024-05-09T09:27:10.702 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/Channel-Mapping-API.txt 2024-05-09T09:27:10.705 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/ControlNames.txt 2024-05-09T09:27:10.708 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Controls.txt 2024-05-09T09:27:10.710 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Models.txt 2024-05-09T09:27:10.713 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio.txt 2024-05-09T09:27:10.717 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/Joystick.txt 2024-05-09T09:27:10.720 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/MIXART.txt 2024-05-09T09:27:10.723 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/OSS-Emulation.txt 2024-05-09T09:27:10.726 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/Procfile.txt 2024-05-09T09:27:10.730 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/README.maya44 2024-05-09T09:27:10.733 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/SB-Live-mixer.txt 2024-05-09T09:27:10.736 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/VIA82xx-mixer.txt 2024-05-09T09:27:10.739 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/alsa-parameters.txt 2024-05-09T09:27:10.742 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/compress_offload.txt 2024-05-09T09:27:10.746 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/emu10k1-jack.txt 2024-05-09T09:27:10.749 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/hda_codec.txt 2024-05-09T09:27:10.752 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/hdspm.txt 2024-05-09T09:27:10.757 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/powersave.txt 2024-05-09T09:27:10.758 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/seq_oss.html 2024-05-09T09:27:10.762 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/serial-u16550.txt 2024-05-09T09:27:10.765 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/soc/ 2024-05-09T09:27:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt 2024-05-09T09:27:10.770 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt 2024-05-09T09:27:10.773 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt 2024-05-09T09:27:10.775 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/soc/codec.txt 2024-05-09T09:27:10.778 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt 2024-05-09T09:27:10.781 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/soc/jack.txt 2024-05-09T09:27:10.785 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/soc/machine.txt 2024-05-09T09:27:10.789 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/soc/overview.txt 2024-05-09T09:27:10.792 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/soc/platform.txt 2024-05-09T09:27:10.795 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt 2024-05-09T09:27:10.798 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/ 2024-05-09T09:27:10.802 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/ALS 2024-05-09T09:27:10.805 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16 2024-05-09T09:27:10.808 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/CMI8330 2024-05-09T09:27:10.811 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/ESS 2024-05-09T09:27:10.813 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/ESS1868 2024-05-09T09:27:10.816 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/Introduction 2024-05-09T09:27:10.819 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/MultiSound 2024-05-09T09:27:10.823 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/OPL3 2024-05-09T09:27:10.826 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/Opti 2024-05-09T09:27:10.829 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/PAS16 2024-05-09T09:27:10.832 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/PSS 2024-05-09T09:27:10.835 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/PSS-updates 2024-05-09T09:27:10.838 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/README.OSS 2024-05-09T09:27:10.841 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/README.modules 2024-05-09T09:27:10.844 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/README.ymfsb 2024-05-09T09:27:10.846 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/SoundPro 2024-05-09T09:27:10.849 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/Soundblaster 2024-05-09T09:27:10.851 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/Tropez+ 2024-05-09T09:27:10.853 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/VIBRA16 2024-05-09T09:27:10.856 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/WaveArtist 2024-05-09T09:27:10.859 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/btaudio 2024-05-09T09:27:10.862 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/mwave 2024-05-09T09:27:10.864 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/oss-parameters.txt 2024-05-09T09:27:10.866 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sound/oss/ultrasound 2024-05-09T09:27:10.868 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sparse.txt 2024-05-09T09:27:10.872 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/spi/ 2024-05-09T09:27:10.874 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/spi/.gitignore 2024-05-09T09:27:10.877 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/spi/00-INDEX 2024-05-09T09:27:10.879 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/spi/Makefile 2024-05-09T09:27:10.882 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/spi/butterfly 2024-05-09T09:27:10.884 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/spi/ep93xx_spi 2024-05-09T09:27:10.886 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/spi/pxa2xx 2024-05-09T09:27:10.888 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/spi/spi-lm70llp 2024-05-09T09:27:10.890 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/spi/spi-sc18is602 2024-05-09T09:27:10.908 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/spi/spi-summary 2024-05-09T09:27:11.036 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/spi/spidev 2024-05-09T09:27:11.039 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/spi/spidev_fdx.c 2024-05-09T09:27:11.042 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/spi/spidev_test.c 2024-05-09T09:27:11.044 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/stable_api_nonsense.txt 2024-05-09T09:27:11.047 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/stable_kernel_rules.txt 2024-05-09T09:27:11.050 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/static-keys.txt 2024-05-09T09:27:11.053 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/svga.txt 2024-05-09T09:27:11.056 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sysctl/ 2024-05-09T09:27:11.060 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sysctl/00-INDEX 2024-05-09T09:27:11.063 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sysctl/README 2024-05-09T09:27:11.066 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sysctl/abi.txt 2024-05-09T09:27:11.069 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sysctl/fs.txt 2024-05-09T09:27:11.072 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sysctl/kernel.txt 2024-05-09T09:27:11.075 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sysctl/net.txt 2024-05-09T09:27:11.078 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sysctl/sunrpc.txt 2024-05-09T09:27:11.080 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sysctl/vm.txt 2024-05-09T09:27:11.084 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sysfs-rules.txt 2024-05-09T09:27:11.087 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/sysrq.txt 2024-05-09T09:27:11.090 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/target/ 2024-05-09T09:27:11.092 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.py 2024-05-09T09:27:11.096 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.txt 2024-05-09T09:27:11.099 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/target/tcmu-design.txt 2024-05-09T09:27:11.102 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/thermal/ 2024-05-09T09:27:11.105 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/thermal/cpu-cooling-api.txt 2024-05-09T09:27:11.108 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal 2024-05-09T09:27:11.111 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal_emulation 2024-05-09T09:27:11.114 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/thermal/intel_powerclamp.txt 2024-05-09T09:27:11.116 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/thermal/nouveau_thermal 2024-05-09T09:27:11.119 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/thermal/sysfs-api.txt 2024-05-09T09:27:11.123 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/thermal/x86_pkg_temperature_thermal 2024-05-09T09:27:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/this_cpu_ops.txt 2024-05-09T09:27:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/timers/ 2024-05-09T09:27:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/timers/.gitignore 2024-05-09T09:27:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/timers/00-INDEX 2024-05-09T09:27:11.136 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/timers/Makefile 2024-05-09T09:27:11.138 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/timers/NO_HZ.txt 2024-05-09T09:27:11.141 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/timers/highres.txt 2024-05-09T09:27:11.144 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/timers/hpet.txt 2024-05-09T09:27:11.147 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/timers/hpet_example.c 2024-05-09T09:27:11.149 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/timers/hrtimers.txt 2024-05-09T09:27:11.152 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/timers/timekeeping.txt 2024-05-09T09:27:11.155 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/timers/timer_stats.txt 2024-05-09T09:27:11.158 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/timers/timers-howto.txt 2024-05-09T09:27:11.160 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/tpm/ 2024-05-09T09:27:11.166 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/tpm/xen-tpmfront.txt 2024-05-09T09:27:11.166 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/ 2024-05-09T09:27:11.168 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/coresight.txt 2024-05-09T09:27:11.172 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/events-kmem.txt 2024-05-09T09:27:11.174 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/events-nmi.txt 2024-05-09T09:27:11.177 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/events-power.txt 2024-05-09T09:27:11.179 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/events.txt 2024-05-09T09:27:11.182 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/ftrace-design.txt 2024-05-09T09:27:11.186 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/ftrace.txt 2024-05-09T09:27:11.190 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/function-graph-fold.vim 2024-05-09T09:27:11.193 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/kprobetrace.txt 2024-05-09T09:27:11.195 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/mmiotrace.txt 2024-05-09T09:27:11.198 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/postprocess/ 2024-05-09T09:27:11.200 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-pagealloc-postprocess.pl 2024-05-09T09:27:11.211 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-vmscan-postprocess.pl 2024-05-09T09:27:11.217 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/ring-buffer-design.txt 2024-05-09T09:27:11.222 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/tracepoint-analysis.txt 2024-05-09T09:27:11.226 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/tracepoints.txt 2024-05-09T09:27:11.228 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/trace/uprobetracer.txt 2024-05-09T09:27:11.231 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/unaligned-memory-access.txt 2024-05-09T09:27:11.235 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/unicode.txt 2024-05-09T09:27:11.237 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/unshare.txt 2024-05-09T09:27:11.240 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/ 2024-05-09T09:27:11.242 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/CREDITS 2024-05-09T09:27:11.245 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/URB.txt 2024-05-09T09:27:11.248 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/WUSB-Design-overview.txt 2024-05-09T09:27:11.252 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/acm.txt 2024-05-09T09:27:11.256 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/anchors.txt 2024-05-09T09:27:11.272 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/authorization.txt 2024-05-09T09:27:11.304 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/bulk-streams.txt 2024-05-09T09:27:11.309 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/callbacks.txt 2024-05-09T09:27:11.311 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/chipidea.txt 2024-05-09T09:27:11.314 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/dma.txt 2024-05-09T09:27:11.316 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/dwc3.txt 2024-05-09T09:27:11.319 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/ehci.txt 2024-05-09T09:27:11.322 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/error-codes.txt 2024-05-09T09:27:11.325 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/functionfs.txt 2024-05-09T09:27:11.330 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/gadget-testing.txt 2024-05-09T09:27:11.333 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/gadget_configfs.txt 2024-05-09T09:27:11.335 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/gadget_hid.txt 2024-05-09T09:27:11.338 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/gadget_multi.txt 2024-05-09T09:27:11.341 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/gadget_printer.txt 2024-05-09T09:27:11.344 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/gadget_serial.txt 2024-05-09T09:27:11.362 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/hotplug.txt 2024-05-09T09:27:11.365 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/iuu_phoenix.txt 2024-05-09T09:27:11.368 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/linux-cdc-acm.inf 2024-05-09T09:27:11.370 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/linux.inf 2024-05-09T09:27:11.372 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/mass-storage.txt 2024-05-09T09:27:11.375 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/misc_usbsevseg.txt 2024-05-09T09:27:11.377 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/mtouchusb.txt 2024-05-09T09:27:11.379 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/ohci.txt 2024-05-09T09:27:11.382 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/persist.txt 2024-05-09T09:27:11.384 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/power-management.txt 2024-05-09T09:27:11.387 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/proc_usb_info.txt 2024-05-09T09:27:11.390 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/rio.txt 2024-05-09T09:27:11.393 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/usb-help.txt 2024-05-09T09:27:11.395 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/usb-serial.txt 2024-05-09T09:27:11.398 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/usbmon.txt 2024-05-09T09:27:11.401 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/usb/wusb-cbaf 2024-05-09T09:27:11.403 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vDSO/ 2024-05-09T09:27:11.405 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vDSO/.gitignore 2024-05-09T09:27:11.407 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vDSO/Makefile 2024-05-09T09:27:11.409 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vDSO/parse_vdso.c 2024-05-09T09:27:11.412 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vDSO/vdso_standalone_test_x86.c 2024-05-09T09:27:11.414 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vDSO/vdso_test.c 2024-05-09T09:27:11.416 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vfio.txt 2024-05-09T09:27:11.421 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vgaarbiter.txt 2024-05-09T09:27:11.423 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video-output.txt 2024-05-09T09:27:11.426 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/ 2024-05-09T09:27:11.427 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/4CCs.txt 2024-05-09T09:27:11.429 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/API.html 2024-05-09T09:27:11.432 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.au0828 2024-05-09T09:27:11.434 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.bttv 2024-05-09T09:27:11.437 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx23885 2024-05-09T09:27:11.439 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx88 2024-05-09T09:27:11.442 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.em28xx 2024-05-09T09:27:11.445 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.ivtv 2024-05-09T09:27:11.447 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7134 2024-05-09T09:27:11.449 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7164 2024-05-09T09:27:11.451 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tm6000 2024-05-09T09:27:11.454 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tuner 2024-05-09T09:27:11.456 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.usbvision 2024-05-09T09:27:11.459 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/Makefile 2024-05-09T09:27:11.461 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/README.cpia2 2024-05-09T09:27:11.463 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/README.cx88 2024-05-09T09:27:11.465 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/README.davinci-vpbe 2024-05-09T09:27:11.467 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/README.ir 2024-05-09T09:27:11.470 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/README.ivtv 2024-05-09T09:27:11.472 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/README.pvrusb2 2024-05-09T09:27:11.475 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/README.saa7134 2024-05-09T09:27:11.477 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/Zoran 2024-05-09T09:27:11.480 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/ 2024-05-09T09:27:11.482 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/CONTRIBUTORS 2024-05-09T09:27:11.484 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/Cards 2024-05-09T09:27:11.487 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/ICs 2024-05-09T09:27:11.490 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/Insmod-options 2024-05-09T09:27:11.492 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/MAKEDEV 2024-05-09T09:27:11.495 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modprobe.conf 2024-05-09T09:27:11.498 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modules.conf 2024-05-09T09:27:11.500 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/PROBLEMS 2024-05-09T09:27:11.503 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/README 2024-05-09T09:27:11.505 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.WINVIEW 2024-05-09T09:27:11.508 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.freeze 2024-05-09T09:27:11.510 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.quirks 2024-05-09T09:27:11.513 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/Sound-FAQ 2024-05-09T09:27:11.516 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/Specs 2024-05-09T09:27:11.518 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/THANKS 2024-05-09T09:27:11.520 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/bttv/Tuners 2024-05-09T09:27:11.522 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cafe_ccic 2024-05-09T09:27:11.526 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cpia2_overview.txt 2024-05-09T09:27:11.528 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx18.txt 2024-05-09T09:27:11.530 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/ 2024-05-09T09:27:11.532 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.hm12 2024-05-09T09:27:11.534 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.vbi 2024-05-09T09:27:11.537 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-calling.txt 2024-05-09T09:27:11.539 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-api.txt 2024-05-09T09:27:11.542 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-regs.txt 2024-05-09T09:27:11.544 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-dma.txt 2024-05-09T09:27:11.547 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-encoder-api.txt 2024-05-09T09:27:11.550 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-memory.txt 2024-05-09T09:27:11.553 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-osd-api.txt 2024-05-09T09:27:11.555 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-upload.txt 2024-05-09T09:27:11.558 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx88/ 2024-05-09T09:27:11.560 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/cx88/hauppauge-wintv-cx88-ir.txt 2024-05-09T09:27:11.562 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/extract_xc3028.pl 2024-05-09T09:27:11.566 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/fimc.txt 2024-05-09T09:27:11.568 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/gspca.txt 2024-05-09T09:27:11.571 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/hauppauge-wintv-cx88-ir.txt 2024-05-09T09:27:11.574 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/lifeview.txt 2024-05-09T09:27:11.576 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/meye.txt 2024-05-09T09:27:11.579 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/not-in-cx2388x-datasheet.txt 2024-05-09T09:27:11.581 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/omap3isp.txt 2024-05-09T09:27:11.584 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/omap4_camera.txt 2024-05-09T09:27:11.586 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/pxa_camera.txt 2024-05-09T09:27:11.589 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/radiotrack.txt 2024-05-09T09:27:11.591 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/sh_mobile_ceu_camera.txt 2024-05-09T09:27:11.593 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/si470x.txt 2024-05-09T09:27:11.596 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/si4713.txt 2024-05-09T09:27:11.598 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/si476x.txt 2024-05-09T09:27:11.601 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/soc-camera.txt 2024-05-09T09:27:11.603 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/uvcvideo.txt 2024-05-09T09:27:11.606 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/v4l2-controls.txt 2024-05-09T09:27:11.609 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/v4l2-framework.txt 2024-05-09T09:27:11.612 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/v4l2-pci-skeleton.c 2024-05-09T09:27:11.615 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/videobuf 2024-05-09T09:27:11.618 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/vivid.txt 2024-05-09T09:27:11.621 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/video4linux/zr364xx.txt 2024-05-09T09:27:11.624 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/ 2024-05-09T09:27:11.626 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/00-INDEX 2024-05-09T09:27:11.628 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/ 2024-05-09T09:27:11.629 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/00-INDEX 2024-05-09T09:27:11.632 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/api.txt 2024-05-09T09:27:11.637 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/cpuid.txt 2024-05-09T09:27:11.639 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/ 2024-05-09T09:27:11.641 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/README 2024-05-09T09:27:11.643 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt 2024-05-09T09:27:11.646 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/mpic.txt 2024-05-09T09:27:11.649 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/s390_flic.txt 2024-05-09T09:27:11.652 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vfio.txt 2024-05-09T09:27:11.654 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vm.txt 2024-05-09T09:27:11.657 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/xics.txt 2024-05-09T09:27:11.660 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/hypercalls.txt 2024-05-09T09:27:11.663 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/locking.txt 2024-05-09T09:27:11.665 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/mmu.txt 2024-05-09T09:27:11.668 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/msr.txt 2024-05-09T09:27:11.670 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/nested-vmx.txt 2024-05-09T09:27:11.673 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/ppc-pv.txt 2024-05-09T09:27:11.676 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/review-checklist.txt 2024-05-09T09:27:11.678 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/s390-diag.txt 2024-05-09T09:27:11.681 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/kvm/timekeeping.txt 2024-05-09T09:27:11.684 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/paravirt_ops.txt 2024-05-09T09:27:11.687 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/uml/ 2024-05-09T09:27:11.689 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt 2024-05-09T09:27:11.694 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/ 2024-05-09T09:27:11.697 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/.gitignore 2024-05-09T09:27:11.699 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/00-INDEX 2024-05-09T09:27:11.701 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/active_mm.txt 2024-05-09T09:27:11.704 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/balance 2024-05-09T09:27:11.706 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/cleancache.txt 2024-05-09T09:27:11.709 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/frontswap.txt 2024-05-09T09:27:11.711 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/highmem.txt 2024-05-09T09:27:11.714 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/hugetlbpage.txt 2024-05-09T09:27:11.717 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/hwpoison.txt 2024-05-09T09:27:11.719 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/ksm.txt 2024-05-09T09:27:11.722 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/numa 2024-05-09T09:27:11.724 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/numa_memory_policy.txt 2024-05-09T09:27:11.727 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/overcommit-accounting 2024-05-09T09:27:11.730 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/page_migration 2024-05-09T09:27:11.732 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/page_owner.txt 2024-05-09T09:27:11.735 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/pagemap.txt 2024-05-09T09:27:11.737 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/remap_file_pages.txt 2024-05-09T09:27:11.740 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/slub.txt 2024-05-09T09:27:11.743 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/soft-dirty.txt 2024-05-09T09:27:11.745 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/split_page_table_lock 2024-05-09T09:27:11.747 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/transhuge.txt 2024-05-09T09:27:11.750 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/unevictable-lru.txt 2024-05-09T09:27:11.754 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vm/zswap.txt 2024-05-09T09:27:11.756 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/vme_api.txt 2024-05-09T09:27:11.760 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/volatile-considered-harmful.txt 2024-05-09T09:27:11.762 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/ 2024-05-09T09:27:11.765 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/00-INDEX 2024-05-09T09:27:11.767 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/masters/ 2024-05-09T09:27:11.768 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/masters/00-INDEX 2024-05-09T09:27:11.771 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/masters/ds2482 2024-05-09T09:27:11.774 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/masters/ds2490 2024-05-09T09:27:11.778 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/masters/mxc-w1 2024-05-09T09:27:11.780 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/masters/omap-hdq 2024-05-09T09:27:11.784 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/masters/w1-gpio 2024-05-09T09:27:11.786 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/slaves/ 2024-05-09T09:27:11.790 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/slaves/00-INDEX 2024-05-09T09:27:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2406 2024-05-09T09:27:11.795 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2423 2024-05-09T09:27:11.798 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds28e04 2024-05-09T09:27:11.800 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/slaves/w1_therm 2024-05-09T09:27:11.804 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/w1.generic 2024-05-09T09:27:11.807 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/w1/w1.netlink 2024-05-09T09:27:11.810 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/ 2024-05-09T09:27:11.812 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/Makefile 2024-05-09T09:27:11.815 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/convert_drivers_to_kernel_api.txt 2024-05-09T09:27:11.817 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/hpwdt.txt 2024-05-09T09:27:11.821 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/pcwd-watchdog.txt 2024-05-09T09:27:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.823+0000 7f7580570640 1 -- 172.21.15.12:0/1570321101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7578109760 msgr2=0x7f757810bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.823+0000 7f7580570640 1 --2- 172.21.15.12:0/1570321101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7578109760 0x7f757810bbf0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7574009900 tx=0x7f757402a410 comp rx=0 tx=0).stop 2024-05-09T09:27:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.823+0000 7f7580570640 1 -- 172.21.15.12:0/1570321101 shutdown_connections 2024-05-09T09:27:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.823+0000 7f7580570640 1 --2- 172.21.15.12:0/1570321101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7578109760 0x7f757810bbf0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.823+0000 7f7580570640 1 --2- 172.21.15.12:0/1570321101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7578074290 0x7f7578074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.823+0000 7f7580570640 1 --2- 172.21.15.12:0/1570321101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578075870 0x7f7578073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.823+0000 7f7580570640 1 -- 172.21.15.12:0/1570321101 >> 172.21.15.12:0/1570321101 conn(0x7f75780fcdf0 msgr2=0x7f75780ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.823+0000 7f7580570640 1 -- 172.21.15.12:0/1570321101 shutdown_connections 2024-05-09T09:27:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.823+0000 7f7580570640 1 -- 172.21.15.12:0/1570321101 wait complete. 2024-05-09T09:27:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.824+0000 7f7580570640 1 Processor -- start 2024-05-09T09:27:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.824+0000 7f7580570640 1 -- start start 2024-05-09T09:27:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.824+0000 7f7580570640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578074290 0x7f7578199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.824+0000 7f7580570640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7578075870 0x7f757819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.824+0000 7f7580570640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7578109760 0x7f75781a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.824+0000 7f7580570640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f757810f740 con 0x7f7578075870 2024-05-09T09:27:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.824+0000 7f7580570640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f757810f5c0 con 0x7f7578109760 2024-05-09T09:27:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.824+0000 7f7580570640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f757810f8c0 con 0x7f7578074290 2024-05-09T09:27:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.824+0000 7f757db0b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7578075870 0x7f757819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.824+0000 7f757e30c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578074290 0x7f7578199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.824+0000 7f757db0b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7578075870 0x7f757819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38556/0 (socket says 172.21.15.12:38556) 2024-05-09T09:27:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f757db0b640 1 -- 172.21.15.12:0/796855247 learned_addr learned my addr 172.21.15.12:0/796855247 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.824+0000 7f757e30c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578074290 0x7f7578199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:52874/0 (socket says 172.21.15.12:52874) 2024-05-09T09:27:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f757eb0d640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7578109760 0x7f75781a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.827 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/src/ 2024-05-09T09:27:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f757db0b640 1 -- 172.21.15.12:0/796855247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578074290 msgr2=0x7f7578199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f757db0b640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578074290 0x7f7578199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f757db0b640 1 -- 172.21.15.12:0/796855247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7578109760 msgr2=0x7f75781a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f757db0b640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7578109760 0x7f75781a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f757db0b640 1 -- 172.21.15.12:0/796855247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75781a18b0 con 0x7f7578075870 2024-05-09T09:27:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f757eb0d640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7578109760 0x7f75781a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f757db0b640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7578075870 0x7f757819a1b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f756c00e9f0 tx=0x7f756c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f756b7fe640 1 -- 172.21.15.12:0/796855247 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c00cd80 con 0x7f7578075870 2024-05-09T09:27:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f7580570640 1 -- 172.21.15.12:0/796855247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75781a1ba0 con 0x7f7578075870 2024-05-09T09:27:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f756b7fe640 1 -- 172.21.15.12:0/796855247 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f756c010900 con 0x7f7578075870 2024-05-09T09:27:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f756b7fe640 1 -- 172.21.15.12:0/796855247 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0189e0 con 0x7f7578075870 2024-05-09T09:27:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.825+0000 7f7580570640 1 -- 172.21.15.12:0/796855247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75781a2140 con 0x7f7578075870 2024-05-09T09:27:11.829 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/src/.gitignore 2024-05-09T09:27:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.829+0000 7f7583280740 1 Processor -- start 2024-05-09T09:27:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.829+0000 7f756b7fe640 1 -- 172.21.15.12:0/796855247 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f756c010ab0 con 0x7f7578075870 2024-05-09T09:27:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.829+0000 7f7583280740 1 -- start start 2024-05-09T09:27:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.829+0000 7f7583280740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed81250 0x5634eed81670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.829+0000 7f7583280740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81c40 0x5634eed86740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.829+0000 7f7583280740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed86db0 0x5634eed89240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.829+0000 7f7583280740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5634eece5340 con 0x5634eed81c40 2024-05-09T09:27:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.829+0000 7f7583280740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634eec8d290 con 0x5634eed86db0 2024-05-09T09:27:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.829+0000 7f7583280740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5634eece1300 con 0x5634eed81250 2024-05-09T09:27:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f757e30c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81c40 0x5634eed86740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f757eb0d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed81250 0x5634eed81670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f757db0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed86db0 0x5634eed89240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f757e30c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81c40 0x5634eed86740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38558/0 (socket says 172.21.15.12:38558) 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f757eb0d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed81250 0x5634eed81670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:52878/0 (socket says 172.21.15.12:52878) 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f757e30c640 1 -- 172.21.15.12:0/1358858347 learned_addr learned my addr 172.21.15.12:0/1358858347 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f757e30c640 1 -- 172.21.15.12:0/1358858347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed81250 msgr2=0x5634eed81670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f757e30c640 1 --2- 172.21.15.12:0/1358858347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed81250 0x5634eed81670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f757e30c640 1 -- 172.21.15.12:0/1358858347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed86db0 msgr2=0x5634eed89240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f757e30c640 1 --2- 172.21.15.12:0/1358858347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed86db0 0x5634eed89240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f757e30c640 1 -- 172.21.15.12:0/1358858347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634eecb18e0 con 0x5634eed81c40 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f756b7fe640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f75640a2cb0 0x7f75640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f757e30c640 1 --2- 172.21.15.12:0/1358858347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81c40 0x5634eed86740 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7570009fd0 tx=0x7f7570003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293425e19876053f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f757eb0d640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f75640a2cb0 0x7f75640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f756b7fe640 1 -- 172.21.15.12:0/796855247 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f756c0cf6b0 con 0x7f7578075870 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f7568bfe640 1 -- 172.21.15.12:0/1358858347 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570012450 con 0x5634eed81c40 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f7568bfe640 1 -- 172.21.15.12:0/1358858347 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7570012ac0 con 0x5634eed81c40 2024-05-09T09:27:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.830+0000 7f7568bfe640 1 -- 172.21.15.12:0/1358858347 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757001ab90 con 0x5634eed81c40 2024-05-09T09:27:11.834 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/src/Makefile 2024-05-09T09:27:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.831+0000 7f7583280740 1 -- 172.21.15.12:0/1358858347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81c40 msgr2=0x5634eed86740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.831+0000 7f757eb0d640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f75640a2cb0 0x7f75640a5170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7574031570 tx=0x7f7574005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.831+0000 7f7583280740 1 --2- 172.21.15.12:0/1358858347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81c40 0x5634eed86740 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7570009fd0 tx=0x7f7570003e80 comp rx=0 tx=0).stop 2024-05-09T09:27:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.832+0000 7f7583280740 1 -- 172.21.15.12:0/1358858347 shutdown_connections 2024-05-09T09:27:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.832+0000 7f7583280740 1 --2- 172.21.15.12:0/1358858347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed86db0 0x5634eed89240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.832+0000 7f7583280740 1 --2- 172.21.15.12:0/1358858347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81c40 0x5634eed86740 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.832+0000 7f7583280740 1 --2- 172.21.15.12:0/1358858347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed81250 0x5634eed81670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.832+0000 7f7583280740 1 -- 172.21.15.12:0/1358858347 >> 172.21.15.12:0/1358858347 conn(0x5634eed7c8c0 msgr2=0x5634eed7ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.832+0000 7f7583280740 1 -- 172.21.15.12:0/1358858347 shutdown_connections 2024-05-09T09:27:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.832+0000 7f7583280740 1 -- 172.21.15.12:0/1358858347 wait complete. 2024-05-09T09:27:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.832+0000 7f7583280740 1 -- start start 2024-05-09T09:27:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.832+0000 7f7583280740 1 Processor -- start 2024-05-09T09:27:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f7583280740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81250 0x5634eed8b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f7583280740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed81c40 0x5634eed8b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f7583280740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed86db0 0x5634eed929f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f7583280740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5634eed89d60 con 0x5634eed81250 2024-05-09T09:27:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f7583280740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634eeca0790 con 0x5634eed86db0 2024-05-09T09:27:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f7583280740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5634eed89ee0 con 0x5634eed81c40 2024-05-09T09:27:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f7583280740 10 client.0 _ll_register_callbacks cb 0x5634eecdba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f757db0b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed81c40 0x5634eed8b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f757eb0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed86db0 0x5634eed929f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f757db0b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed81c40 0x5634eed8b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:52886/0 (socket says 172.21.15.12:52886) 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f757e30c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81250 0x5634eed8b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f757db0b640 1 -- 172.21.15.12:0/1697986800 learned_addr learned my addr 172.21.15.12:0/1697986800 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f757db0b640 1 -- 172.21.15.12:0/1697986800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed86db0 msgr2=0x5634eed929f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f757db0b640 1 --2- 172.21.15.12:0/1697986800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed86db0 0x5634eed929f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f757db0b640 1 -- 172.21.15.12:0/1697986800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81250 msgr2=0x5634eed8b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f757db0b640 1 --2- 172.21.15.12:0/1697986800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81250 0x5634eed8b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.833+0000 7f757db0b640 1 -- 172.21.15.12:0/1697986800 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634ee9f9210 con 0x5634eed81c40 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.834+0000 7f757db0b640 1 --2- 172.21.15.12:0/1697986800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed81c40 0x5634eed8b9b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f756c098050 tx=0x7f756c010f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a736444c9aa59c52 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.834+0000 7f7561ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.834+0000 7f7561ffb640 1 -- 172.21.15.12:0/1697986800 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c09b550 con 0x5634eed81c40 2024-05-09T09:27:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.834+0000 7f7583280740 1 -- 172.21.15.12:0/1697986800 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7570012ac0 con 0x5634eed81c40 2024-05-09T09:27:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.834+0000 7f7561ffb640 1 -- 172.21.15.12:0/1697986800 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f756c018b80 con 0x5634eed81c40 2024-05-09T09:27:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.834+0000 7f7561ffb640 1 -- 172.21.15.12:0/1697986800 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f756c09bbf0 con 0x5634eed81c40 2024-05-09T09:27:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.834+0000 7f7583280740 10 client.25831 fetch_fsmap learned FSMap version 30 2024-05-09T09:27:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.834+0000 7f7583280740 1 -- 172.21.15.12:0/1697986800 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5634eeac5620 con 0x5634eed81c40 2024-05-09T09:27:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.835+0000 7f7561ffb640 1 -- 172.21.15.12:0/1697986800 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7570012ac0 con 0x5634eed81c40 2024-05-09T09:27:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.835+0000 7f7561ffb640 1 -- 172.21.15.12:0/1697986800 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f756c010430 con 0x5634eed81c40 2024-05-09T09:27:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.835+0000 7f7583280740 10 client.25831 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:27:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.835+0000 7f7583280740 10 client.25831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.835+0000 7f7583280740 20 client.25831 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.835+0000 7f7583280740 1 --2- 172.21.15.12:0/1697986800 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5634eed96f60 0x5634eed99380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.835+0000 7f7583280740 4 client.25831 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.835+0000 7f7583280740 1 -- 172.21.15.12:0/1697986800 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634eecf2a70 con 0x5634eed96f60 2024-05-09T09:27:11.840 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-simple.c 2024-05-09T09:27:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.835+0000 7f757e30c640 1 --2- 172.21.15.12:0/1697986800 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5634eed96f60 0x5634eed99380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.836+0000 7f7561ffb640 10 client.25831 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.836+0000 7f757e30c640 1 --2- 172.21.15.12:0/1697986800 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5634eed96f60 0x5634eed99380 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a64368b08e193f6 server_cookie=24f25bdeb3121303 in_seq=0 out_seq=0 2024-05-09T09:27:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7561ffb640 1 -- 172.21.15.12:0/1697986800 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634eecf2a70 con 0x5634eed96f60 2024-05-09T09:27:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7561ffb640 10 client.25831 handle_command_reply: tid=0 2024-05-09T09:27:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7583280740 1 client.25831 shutdown 2024-05-09T09:27:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7583280740 1 -- 172.21.15.12:0/1697986800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed81c40 msgr2=0x5634eed8b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7583280740 1 --2- 172.21.15.12:0/1697986800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed81c40 0x5634eed8b9b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f756c098050 tx=0x7f756c010f30 comp rx=0 tx=0).stop 2024-05-09T09:27:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7583280740 1 -- 172.21.15.12:0/1697986800 shutdown_connections 2024-05-09T09:27:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7583280740 1 --2- 172.21.15.12:0/1697986800 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5634eed96f60 0x5634eed99380 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7583280740 1 --2- 172.21.15.12:0/1697986800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed86db0 0x5634eed929f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7583280740 1 --2- 172.21.15.12:0/1697986800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed81c40 0x5634eed8b9b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7583280740 1 --2- 172.21.15.12:0/1697986800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81250 0x5634eed8b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7561ffb640 0 client.25831 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7583280740 1 -- 172.21.15.12:0/1697986800 >> 172.21.15.12:0/1697986800 conn(0x5634eed7c8c0 msgr2=0x5634eed7ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7583280740 1 -- 172.21.15.12:0/1697986800 shutdown_connections 2024-05-09T09:27:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7583280740 1 -- 172.21.15.12:0/1697986800 wait complete. 2024-05-09T09:27:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.837+0000 7f7583280740 20 client.25831 trim_cache size 0 max 16384 2024-05-09T09:27:11.843 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-test.c 2024-05-09T09:27:11.843 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/watchdog-api.txt 2024-05-09T09:27:11.843 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/watchdog-kernel-api.txt 2024-05-09T09:27:11.846 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/watchdog-parameters.txt 2024-05-09T09:27:11.849 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/watchdog/wdt.txt 2024-05-09T09:27:11.852 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/wimax/ 2024-05-09T09:27:11.854 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/wimax/README.i2400m 2024-05-09T09:27:11.857 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/wimax/README.wimax 2024-05-09T09:27:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.859+0000 7f7583280740 1 Processor -- start 2024-05-09T09:27:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.859+0000 7f7583280740 1 -- start start 2024-05-09T09:27:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.859+0000 7f7583280740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed81250 0x5634eed82e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.859+0000 7f7583280740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81c40 0x5634eed929b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.859+0000 7f7583280740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed86db0 0x5634eed833c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.859+0000 7f7583280740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f75781ac750 con 0x5634eed81c40 2024-05-09T09:27:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.859+0000 7f7583280740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75781ae2f0 con 0x5634eed81250 2024-05-09T09:27:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.859+0000 7f7583280740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75781ace20 con 0x5634eed86db0 2024-05-09T09:27:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.860+0000 7f757e30c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed81250 0x5634eed82e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.860+0000 7f757e30c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed81250 0x5634eed82e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:43034/0 (socket says 172.21.15.12:43034) 2024-05-09T09:27:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.860+0000 7f757db0b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81c40 0x5634eed929b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.860+0000 7f757e30c640 1 -- 172.21.15.12:0/3405270836 learned_addr learned my addr 172.21.15.12:0/3405270836 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.860+0000 7f757eb0d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed86db0 0x5634eed833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.860+0000 7f757e30c640 1 -- 172.21.15.12:0/3405270836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed86db0 msgr2=0x5634eed833c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.860+0000 7f757e30c640 1 --2- 172.21.15.12:0/3405270836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed86db0 0x5634eed833c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.860+0000 7f757e30c640 1 -- 172.21.15.12:0/3405270836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81c40 msgr2=0x5634eed929b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.860+0000 7f757e30c640 1 --2- 172.21.15.12:0/3405270836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81c40 0x5634eed929b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.860+0000 7f757e30c640 1 -- 172.21.15.12:0/3405270836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634eed7d630 con 0x5634eed81250 2024-05-09T09:27:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.860+0000 7f757e30c640 1 --2- 172.21.15.12:0/3405270836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed81250 0x5634eed82e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7570009fd0 tx=0x7f7570005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe4e149d77226504 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.861+0000 7f7563fff640 1 -- 172.21.15.12:0/3405270836 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570029070 con 0x5634eed81250 2024-05-09T09:27:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.861+0000 7f7563fff640 1 -- 172.21.15.12:0/3405270836 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7570012450 con 0x5634eed81250 2024-05-09T09:27:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.861+0000 7f7563fff640 1 -- 172.21.15.12:0/3405270836 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570029470 con 0x5634eed81250 2024-05-09T09:27:11.862 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/workqueue.txt 2024-05-09T09:27:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.862+0000 7f7583280740 1 -- 172.21.15.12:0/3405270836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed81250 msgr2=0x5634eed82e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.862+0000 7f7583280740 1 --2- 172.21.15.12:0/3405270836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed81250 0x5634eed82e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7570009fd0 tx=0x7f7570005bd0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.862+0000 7f7583280740 1 -- 172.21.15.12:0/3405270836 shutdown_connections 2024-05-09T09:27:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.862+0000 7f7583280740 1 --2- 172.21.15.12:0/3405270836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed86db0 0x5634eed833c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.862+0000 7f7583280740 1 --2- 172.21.15.12:0/3405270836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81c40 0x5634eed929b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.862+0000 7f7583280740 1 --2- 172.21.15.12:0/3405270836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed81250 0x5634eed82e80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.862+0000 7f7583280740 1 -- 172.21.15.12:0/3405270836 >> 172.21.15.12:0/3405270836 conn(0x5634eed7c8c0 msgr2=0x5634eed7cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.862+0000 7f7583280740 1 -- 172.21.15.12:0/3405270836 shutdown_connections 2024-05-09T09:27:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.862+0000 7f7583280740 1 -- 172.21.15.12:0/3405270836 wait complete. 2024-05-09T09:27:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.862+0000 7f7583280740 1 -- start start 2024-05-09T09:27:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.862+0000 7f7583280740 1 Processor -- start 2024-05-09T09:27:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f7583280740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81250 0x5634eed903a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f7583280740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed81c40 0x5634eed908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f7583280740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed86db0 0x5634eed918f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f7583280740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f75781adbc0 con 0x5634eed81250 2024-05-09T09:27:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f7583280740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75781ad4f0 con 0x5634eed81c40 2024-05-09T09:27:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f7583280740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75781aa730 con 0x5634eed86db0 2024-05-09T09:27:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f757eb0d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed86db0 0x5634eed918f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f757eb0d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed86db0 0x5634eed918f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:52906/0 (socket says 172.21.15.12:52906) 2024-05-09T09:27:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f757eb0d640 1 -- 172.21.15.12:0/1473000933 learned_addr learned my addr 172.21.15.12:0/1473000933 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f757db0b640 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed81c40 0x5634eed908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f757e30c640 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81250 0x5634eed903a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f7583280740 10 client.0 _ll_register_callbacks cb 0x7f756c010da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f757db0b640 1 -- 172.21.15.12:0/1473000933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed86db0 msgr2=0x5634eed918f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f757db0b640 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed86db0 0x5634eed918f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f757db0b640 1 -- 172.21.15.12:0/1473000933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81250 msgr2=0x5634eed903a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f757db0b640 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81250 0x5634eed903a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f757db0b640 1 -- 172.21.15.12:0/1473000933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634eed91ee0 con 0x5634eed81c40 2024-05-09T09:27:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.863+0000 7f757e30c640 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81250 0x5634eed903a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.864+0000 7f757db0b640 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed81c40 0x5634eed908e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f756c098570 tx=0x7f756c09b550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cebe4d352487d4e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.864+0000 7f7561ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:27:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.864+0000 7f7561ffb640 1 -- 172.21.15.12:0/1473000933 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0b8020 con 0x5634eed81c40 2024-05-09T09:27:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.864+0000 7f7583280740 1 -- 172.21.15.12:0/1473000933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7570012450 con 0x5634eed81c40 2024-05-09T09:27:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.864+0000 7f7561ffb640 1 -- 172.21.15.12:0/1473000933 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f756c018d80 con 0x5634eed81c40 2024-05-09T09:27:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.864+0000 7f7561ffb640 1 -- 172.21.15.12:0/1473000933 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f756c0a47b0 con 0x5634eed81c40 2024-05-09T09:27:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.864+0000 7f7583280740 10 client.25724 fetch_fsmap learned FSMap version 30 2024-05-09T09:27:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.864+0000 7f7583280740 1 -- 172.21.15.12:0/1473000933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5634eedfd800 con 0x5634eed81c40 2024-05-09T09:27:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.864+0000 7f7561ffb640 1 -- 172.21.15.12:0/1473000933 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7570012450 con 0x5634eed81c40 2024-05-09T09:27:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.865+0000 7f7561ffb640 1 -- 172.21.15.12:0/1473000933 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f756c097940 con 0x5634eed81c40 2024-05-09T09:27:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.865+0000 7f7583280740 10 client.25724 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:27:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.865+0000 7f7583280740 10 client.25724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.865+0000 7f7583280740 20 client.25724 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.865+0000 7f7583280740 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5634eedfe150 0x5634eee00570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.865+0000 7f7583280740 4 client.25724 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:27:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.865+0000 7f7583280740 1 -- 172.21.15.12:0/1473000933 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634eececdc0 con 0x5634eedfe150 2024-05-09T09:27:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.865+0000 7f757e30c640 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5634eedfe150 0x5634eee00570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:11.869 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/ 2024-05-09T09:27:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.865+0000 7f757e30c640 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5634eedfe150 0x5634eee00570 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a4b61c80d106d1 server_cookie=4a36068a98f81641 in_seq=0 out_seq=0 2024-05-09T09:27:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:11.865+0000 7f7561ffb640 10 client.25724 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:11.870 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/00-INDEX 2024-05-09T09:27:11.871 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/boot.txt 2024-05-09T09:27:11.874 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/early-microcode.txt 2024-05-09T09:27:11.877 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/earlyprintk.txt 2024-05-09T09:27:11.880 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/entry_64.txt 2024-05-09T09:27:11.883 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/exception-tables.txt 2024-05-09T09:27:11.886 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/i386/ 2024-05-09T09:27:11.888 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/i386/IO-APIC.txt 2024-05-09T09:27:11.891 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/intel_mpx.txt 2024-05-09T09:27:11.894 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/mtrr.txt 2024-05-09T09:27:11.897 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/pat.txt 2024-05-09T09:27:11.900 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/tlb.txt 2024-05-09T09:27:11.902 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/usb-legacy-support.txt 2024-05-09T09:27:11.905 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/x86_64/ 2024-05-09T09:27:11.907 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/x86_64/00-INDEX 2024-05-09T09:27:12.065 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/x86_64/boot-options.txt 2024-05-09T09:27:12.068 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/x86_64/cpu-hotplug-spec 2024-05-09T09:27:12.071 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/x86_64/fake-numa-for-cpusets 2024-05-09T09:27:12.073 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/x86_64/kernel-stacks 2024-05-09T09:27:12.076 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/x86_64/machinecheck 2024-05-09T09:27:12.079 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/x86_64/mm.txt 2024-05-09T09:27:12.081 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/x86_64/uefi.txt 2024-05-09T09:27:12.084 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/x86/zero-page.txt 2024-05-09T09:27:12.086 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/xillybus.txt 2024-05-09T09:27:12.090 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/xtensa/ 2024-05-09T09:27:12.091 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/xtensa/atomctl.txt 2024-05-09T09:27:12.093 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/xtensa/mmu.txt 2024-05-09T09:27:12.096 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/xz.txt 2024-05-09T09:27:12.099 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/ 2024-05-09T09:27:12.101 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/CodingStyle 2024-05-09T09:27:12.103 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/HOWTO 2024-05-09T09:27:12.107 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/IRQ.txt 2024-05-09T09:27:12.109 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/SecurityBugs 2024-05-09T09:27:12.112 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingDrivers 2024-05-09T09:27:12.114 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingPatches 2024-05-09T09:27:12.117 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/arm/ 2024-05-09T09:27:12.119 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/arm/Booting 2024-05-09T09:27:12.122 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/arm/kernel_user_helpers.txt 2024-05-09T09:27:12.124 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/arm64/ 2024-05-09T09:27:12.127 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/arm64/booting.txt 2024-05-09T09:27:12.129 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/arm64/memory.txt 2024-05-09T09:27:12.131 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/arm64/tagged-pointers.txt 2024-05-09T09:27:12.133 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/basic_profiling.txt 2024-05-09T09:27:12.136 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/email-clients.txt 2024-05-09T09:27:12.138 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/ 2024-05-09T09:27:12.140 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/sysfs.txt 2024-05-09T09:27:12.143 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/gpio.txt 2024-05-09T09:27:12.146 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/io_ordering.txt 2024-05-09T09:27:12.148 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/magic-number.txt 2024-05-09T09:27:12.150 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/oops-tracing.txt 2024-05-09T09:27:12.153 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/sparse.txt 2024-05-09T09:27:12.156 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/stable_api_nonsense.txt 2024-05-09T09:27:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/stable_kernel_rules.txt 2024-05-09T09:27:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/ 2024-05-09T09:27:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/omap3isp.txt 2024-05-09T09:27:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/v4l2-framework.txt 2024-05-09T09:27:12.167 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zh_CN/volatile-considered-harmful.txt 2024-05-09T09:27:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:linux-4.0.5/Documentation/zorro.txt 2024-05-09T09:27:12.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:11 smithi012 ceph-mon[30140]: pgmap v416: 129 pgs: 129 active+clean; 99 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 5.0 MiB/s wr, 306 op/s 2024-05-09T09:27:12.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:11 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:27:12.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:11 smithi170 ceph-mon[33632]: pgmap v416: 129 pgs: 129 active+clean; 99 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 5.0 MiB/s wr, 306 op/s 2024-05-09T09:27:12.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:11 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:27:12.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:11 smithi016 ceph-mon[33335]: pgmap v416: 129 pgs: 129 active+clean; 99 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 5.0 MiB/s wr, 306 op/s 2024-05-09T09:27:12.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:11 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:27:14.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:13 smithi012 ceph-mon[30140]: pgmap v417: 129 pgs: 129 active+clean; 105 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-05-09T09:27:14.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:13 smithi170 ceph-mon[33632]: pgmap v417: 129 pgs: 129 active+clean; 105 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-05-09T09:27:14.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:13 smithi016 ceph-mon[33335]: pgmap v417: 129 pgs: 129 active+clean; 105 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-05-09T09:27:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.304+0000 7f7561ffb640 1 -- 172.21.15.12:0/1473000933 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634eececdc0 con 0x5634eedfe150 2024-05-09T09:27:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.304+0000 7f7561ffb640 10 client.25724 handle_command_reply: tid=0 2024-05-09T09:27:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.304+0000 7f7583280740 1 client.25724 shutdown 2024-05-09T09:27:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7583280740 1 -- 172.21.15.12:0/1473000933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed81c40 msgr2=0x5634eed908e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7583280740 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed81c40 0x5634eed908e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f756c098570 tx=0x7f756c09b550 comp rx=0 tx=0).stop 2024-05-09T09:27:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7583280740 1 -- 172.21.15.12:0/1473000933 shutdown_connections 2024-05-09T09:27:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7583280740 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5634eedfe150 0x5634eee00570 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7583280740 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5634eed86db0 0x5634eed918f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7561ffb640 0 client.25724 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7583280740 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634eed81c40 0x5634eed908e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7583280740 1 --2- 172.21.15.12:0/1473000933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634eed81250 0x5634eed903a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7583280740 1 -- 172.21.15.12:0/1473000933 >> 172.21.15.12:0/1473000933 conn(0x5634eed7c8c0 msgr2=0x5634eed82e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7583280740 1 -- 172.21.15.12:0/1473000933 shutdown_connections 2024-05-09T09:27:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7583280740 1 -- 172.21.15.12:0/1473000933 wait complete. 2024-05-09T09:27:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7583280740 20 client.25724 trim_cache size 0 max 16384 2024-05-09T09:27:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:27:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:27:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:27:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:27:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7580570640 1 -- 172.21.15.12:0/796855247 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f75640a2cb0 msgr2=0x7f75640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7580570640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f75640a2cb0 0x7f75640a5170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7574031570 tx=0x7f7574005a20 comp rx=0 tx=0).stop 2024-05-09T09:27:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7580570640 1 -- 172.21.15.12:0/796855247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7578075870 msgr2=0x7f757819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.305+0000 7f7580570640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7578075870 0x7f757819a1b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f756c00e9f0 tx=0x7f756c00eeb0 comp rx=0 tx=0).stop 2024-05-09T09:27:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.306+0000 7f7580570640 1 -- 172.21.15.12:0/796855247 shutdown_connections 2024-05-09T09:27:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.306+0000 7f7580570640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f75640a2cb0 0x7f75640a5170 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.306+0000 7f7580570640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7578109760 0x7f75781a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.306+0000 7f7580570640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7578075870 0x7f757819a1b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.306+0000 7f7580570640 1 --2- 172.21.15.12:0/796855247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578074290 0x7f7578199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.306+0000 7f7580570640 1 -- 172.21.15.12:0/796855247 >> 172.21.15.12:0/796855247 conn(0x7f75780fcdf0 msgr2=0x7f7578105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.306+0000 7f7580570640 1 -- 172.21.15.12:0/796855247 shutdown_connections 2024-05-09T09:27:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:14.306+0000 7f7580570640 1 -- 172.21.15.12:0/796855247 wait complete. 2024-05-09T09:27:16.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:15 smithi012 ceph-mon[30140]: pgmap v418: 129 pgs: 129 active+clean; 105 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 2.4 MiB/s wr, 603 op/s 2024-05-09T09:27:16.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:15 smithi170 ceph-mon[33632]: pgmap v418: 129 pgs: 129 active+clean; 105 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 2.4 MiB/s wr, 603 op/s 2024-05-09T09:27:16.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:15 smithi016 ceph-mon[33335]: pgmap v418: 129 pgs: 129 active+clean; 105 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 2.4 MiB/s wr, 603 op/s 2024-05-09T09:27:17.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:16 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:27:17.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:16 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:27:17.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:16 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:27:18.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:27:18.142 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:27:18.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:17 smithi170 ceph-mon[33632]: pgmap v419: 129 pgs: 129 active+clean; 108 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 2.2 MiB/s wr, 940 op/s 2024-05-09T09:27:18.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:17 smithi016 ceph-mon[33335]: pgmap v419: 129 pgs: 129 active+clean; 108 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 2.2 MiB/s wr, 940 op/s 2024-05-09T09:27:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.264+0000 7ff94ed90640 1 -- 172.21.15.12:0/3429782106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff948100770 msgr2=0x7ff948100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.264+0000 7ff94ed90640 1 --2- 172.21.15.12:0/3429782106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff948100770 0x7ff948100b90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff938009900 tx=0x7ff93802a410 comp rx=0 tx=0).stop 2024-05-09T09:27:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.264+0000 7ff94ed90640 1 -- 172.21.15.12:0/3429782106 shutdown_connections 2024-05-09T09:27:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.264+0000 7ff94ed90640 1 --2- 172.21.15.12:0/3429782106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948102320 0x7ff948108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.264+0000 7ff94ed90640 1 --2- 172.21.15.12:0/3429782106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff948101960 0x7ff948101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.264+0000 7ff94ed90640 1 --2- 172.21.15.12:0/3429782106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff948100770 0x7ff948100b90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.264+0000 7ff94ed90640 1 -- 172.21.15.12:0/3429782106 >> 172.21.15.12:0/3429782106 conn(0x7ff9480fbce0 msgr2=0x7ff9480fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.264+0000 7ff94ed90640 1 -- 172.21.15.12:0/3429782106 shutdown_connections 2024-05-09T09:27:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.264+0000 7ff94ed90640 1 -- 172.21.15.12:0/3429782106 wait complete. 2024-05-09T09:27:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.265+0000 7ff94ed90640 1 Processor -- start 2024-05-09T09:27:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.265+0000 7ff94ed90640 1 -- start start 2024-05-09T09:27:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.265+0000 7ff94ed90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100770 0x7ff948197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.265+0000 7ff94ed90640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff948101960 0x7ff948198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.265+0000 7ff94ed90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff948102320 0x7ff94819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.265+0000 7ff94ed90640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff94810c3d0 con 0x7ff948102320 2024-05-09T09:27:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.265+0000 7ff94ed90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff94810c250 con 0x7ff948100770 2024-05-09T09:27:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.265+0000 7ff94ed90640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff94810c550 con 0x7ff948101960 2024-05-09T09:27:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.265+0000 7ff94d32d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff948102320 0x7ff94819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.265+0000 7ff94d32d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff948102320 0x7ff94819f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38604/0 (socket says 172.21.15.12:38604) 2024-05-09T09:27:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.265+0000 7ff94d32d640 1 -- 172.21.15.12:0/2090902500 learned_addr learned my addr 172.21.15.12:0/2090902500 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.265+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100770 0x7ff948197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.266+0000 7ff947fff640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff948101960 0x7ff948198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.266+0000 7ff94d32d640 1 -- 172.21.15.12:0/2090902500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff948101960 msgr2=0x7ff948198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.266+0000 7ff94d32d640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff948101960 0x7ff948198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.266+0000 7ff94d32d640 1 -- 172.21.15.12:0/2090902500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100770 msgr2=0x7ff948197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.266+0000 7ff94d32d640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100770 0x7ff948197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.266+0000 7ff94d32d640 1 -- 172.21.15.12:0/2090902500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94819f810 con 0x7ff948102320 2024-05-09T09:27:18.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.266+0000 7ff94d32d640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff948102320 0x7ff94819f1a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff94000e970 tx=0x7ff94000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.266+0000 7ff945ffb640 1 -- 172.21.15.12:0/2090902500 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94000ccc0 con 0x7ff948102320 2024-05-09T09:27:18.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.266+0000 7ff945ffb640 1 -- 172.21.15.12:0/2090902500 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff94000ce60 con 0x7ff948102320 2024-05-09T09:27:18.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.266+0000 7ff94ed90640 1 -- 172.21.15.12:0/2090902500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94819fb00 con 0x7ff948102320 2024-05-09T09:27:18.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.266+0000 7ff945ffb640 1 -- 172.21.15.12:0/2090902500 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940010430 con 0x7ff948102320 2024-05-09T09:27:18.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.266+0000 7ff94ed90640 1 -- 172.21.15.12:0/2090902500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9481a0000 con 0x7ff948102320 2024-05-09T09:27:18.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.269+0000 7ff951b7b740 1 Processor -- start 2024-05-09T09:27:18.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.269+0000 7ff945ffb640 1 -- 172.21.15.12:0/2090902500 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff940005000 con 0x7ff948102320 2024-05-09T09:27:18.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.269+0000 7ff951b7b740 1 -- start start 2024-05-09T09:27:18.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.269+0000 7ff951b7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d19b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.269+0000 7ff951b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d19b920 0x55fa2d1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.269+0000 7ff951b7b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d1a0a90 0x55fa2d1a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.269+0000 7ff951b7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fa2d0ffd00 con 0x55fa2d19af30 2024-05-09T09:27:18.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.269+0000 7ff951b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fa2d0a74c0 con 0x55fa2d19b920 2024-05-09T09:27:18.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.269+0000 7ff951b7b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fa2d0fbcc0 con 0x55fa2d1a0a90 2024-05-09T09:27:18.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.269+0000 7ff947fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.269+0000 7ff947fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38620/0 (socket says 172.21.15.12:38620) 2024-05-09T09:27:18.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.269+0000 7ff947fff640 1 -- 172.21.15.12:0/23699921 learned_addr learned my addr 172.21.15.12:0/23699921 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.269+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/23699921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d19b920 0x55fa2d1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff947fff640 1 --2- 172.21.15.12:0/23699921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d1a0a90 0x55fa2d1a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff945ffb640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff9340a2b90 0x7ff9340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff94cb2c640 1 -- 172.21.15.12:0/23699921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d1a0a90 msgr2=0x55fa2d1a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/23699921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d1a0a90 0x55fa2d1a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff94cb2c640 1 -- 172.21.15.12:0/23699921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 msgr2=0x55fa2d19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/23699921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff94cb2c640 1 -- 172.21.15.12:0/23699921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa2cedd7d0 con 0x55fa2d19b920 2024-05-09T09:27:18.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff947fff640 1 --2- 172.21.15.12:0/23699921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:18.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff94d32d640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff9340a2b90 0x7ff9340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff945ffb640 1 -- 172.21.15.12:0/2090902500 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7ff9400cf870 con 0x7ff948102320 2024-05-09T09:27:18.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/23699921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d19b920 0x55fa2d1a0420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff938009230 tx=0x7ff938005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a378cee812f322c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff93effd640 1 -- 172.21.15.12:0/23699921 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938040070 con 0x55fa2d19b920 2024-05-09T09:27:18.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff93effd640 1 -- 172.21.15.12:0/23699921 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff938004020 con 0x55fa2d19b920 2024-05-09T09:27:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.270+0000 7ff93effd640 1 -- 172.21.15.12:0/23699921 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938035040 con 0x55fa2d19b920 2024-05-09T09:27:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.271+0000 7ff951b7b740 1 -- 172.21.15.12:0/23699921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d19b920 msgr2=0x55fa2d1a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.271+0000 7ff94d32d640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff9340a2b90 0x7ff9340a5050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff94009bc90 tx=0x7ff94006ac80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.271+0000 7ff951b7b740 1 --2- 172.21.15.12:0/23699921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d19b920 0x55fa2d1a0420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff938009230 tx=0x7ff938005dd0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.271+0000 7ff951b7b740 1 -- 172.21.15.12:0/23699921 shutdown_connections 2024-05-09T09:27:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.271+0000 7ff951b7b740 1 --2- 172.21.15.12:0/23699921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d1a0a90 0x55fa2d1a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.271+0000 7ff951b7b740 1 --2- 172.21.15.12:0/23699921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d19b920 0x55fa2d1a0420 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.271+0000 7ff951b7b740 1 --2- 172.21.15.12:0/23699921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.271+0000 7ff951b7b740 1 -- 172.21.15.12:0/23699921 >> 172.21.15.12:0/23699921 conn(0x55fa2d1965a0 msgr2=0x55fa2d1969a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.271+0000 7ff951b7b740 1 -- 172.21.15.12:0/23699921 shutdown_connections 2024-05-09T09:27:18.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.271+0000 7ff951b7b740 1 -- 172.21.15.12:0/23699921 wait complete. 2024-05-09T09:27:18.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.271+0000 7ff951b7b740 1 -- start start 2024-05-09T09:27:18.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff951b7b740 1 Processor -- start 2024-05-09T09:27:18.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff951b7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff951b7b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d19b920 0x55fa2d1a5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff951b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d1a0a90 0x55fa2d1ac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff951b7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fa2d1a39e0 con 0x55fa2d19af30 2024-05-09T09:27:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff951b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fa2d0bab90 con 0x55fa2d1a0a90 2024-05-09T09:27:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff951b7b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fa2d1a3b60 con 0x55fa2d19b920 2024-05-09T09:27:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff951b7b740 10 client.0 _ll_register_callbacks cb 0x55fa2d0f6da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff947fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff947fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1a52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38630/0 (socket says 172.21.15.12:38630) 2024-05-09T09:27:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff947fff640 1 -- 172.21.15.12:0/2103063666 learned_addr learned my addr 172.21.15.12:0/2103063666 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d19b920 0x55fa2d1a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff947fff640 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d1a0a90 0x55fa2d1ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.272+0000 7ff947fff640 1 -- 172.21.15.12:0/2103063666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d19b920 msgr2=0x55fa2d1a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff947fff640 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d19b920 0x55fa2d1a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff947fff640 1 -- 172.21.15.12:0/2103063666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d1a0a90 msgr2=0x55fa2d1ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff947fff640 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d1a0a90 0x55fa2d1ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff947fff640 1 -- 172.21.15.12:0/2103063666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa2ce133f0 con 0x55fa2d19af30 2024-05-09T09:27:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d19b920 0x55fa2d1a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff947fff640 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1a52d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff930004480 tx=0x7ff930009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d962844c404077b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff927fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff927fff640 1 -- 172.21.15.12:0/2103063666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930012bc0 con 0x55fa2d19af30 2024-05-09T09:27:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff927fff640 1 -- 172.21.15.12:0/2103063666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7ff930012d60 con 0x55fa2d19af30 2024-05-09T09:27:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff951b7b740 1 -- 172.21.15.12:0/2103063666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff938004020 con 0x55fa2d19af30 2024-05-09T09:27:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff927fff640 1 -- 172.21.15.12:0/2103063666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff930004b20 con 0x55fa2d19af30 2024-05-09T09:27:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff951b7b740 10 client.16254 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff951b7b740 1 -- 172.21.15.12:0/2103063666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fa2cdd30f0 con 0x55fa2d19af30 2024-05-09T09:27:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.273+0000 7ff927fff640 1 -- 172.21.15.12:0/2103063666 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff938004020 con 0x55fa2d19af30 2024-05-09T09:27:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.274+0000 7ff927fff640 1 -- 172.21.15.12:0/2103063666 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7ff930020500 con 0x55fa2d19af30 2024-05-09T09:27:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.274+0000 7ff951b7b740 10 client.16254 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.274+0000 7ff951b7b740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.274+0000 7ff951b7b740 20 client.16254 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.274+0000 7ff951b7b740 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fa2d1b1180 0x55fa2d1b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.274+0000 7ff951b7b740 4 client.16254 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:18.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.274+0000 7ff951b7b740 1 -- 172.21.15.12:0/2103063666 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa2d10d430 con 0x55fa2d1b1180 2024-05-09T09:27:18.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.274+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fa2d1b1180 0x55fa2d1b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.274+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fa2d1b1180 0x55fa2d1b3620 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0dc9a12029e81cf server_cookie=8ab4757170456121 in_seq=0 out_seq=0 2024-05-09T09:27:18.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.274+0000 7ff927fff640 10 client.16254 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:18.290 INFO:tasks.workunit.client.0.smithi012.stderr:+ tar cf doc.tar linux-4.0.5 2024-05-09T09:27:18.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff927fff640 1 -- 172.21.15.12:0/2103063666 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa2d10d430 con 0x55fa2d1b1180 2024-05-09T09:27:18.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff927fff640 10 client.16254 handle_command_reply: tid=0 2024-05-09T09:27:18.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff951b7b740 1 client.16254 shutdown 2024-05-09T09:27:18.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff951b7b740 1 -- 172.21.15.12:0/2103063666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 msgr2=0x55fa2d1a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff951b7b740 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1a52d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff930004480 tx=0x7ff930009500 comp rx=0 tx=0).stop 2024-05-09T09:27:18.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff951b7b740 1 -- 172.21.15.12:0/2103063666 shutdown_connections 2024-05-09T09:27:18.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff951b7b740 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fa2d1b1180 0x55fa2d1b3620 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff951b7b740 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d1a0a90 0x55fa2d1ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff951b7b740 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d19b920 0x55fa2d1a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff927fff640 0 client.16254 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:18.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff951b7b740 1 --2- 172.21.15.12:0/2103063666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1a52d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff951b7b740 1 -- 172.21.15.12:0/2103063666 >> 172.21.15.12:0/2103063666 conn(0x55fa2d1965a0 msgr2=0x55fa2d1957e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff951b7b740 1 -- 172.21.15.12:0/2103063666 shutdown_connections 2024-05-09T09:27:18.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff951b7b740 1 -- 172.21.15.12:0/2103063666 wait complete. 2024-05-09T09:27:18.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.276+0000 7ff951b7b740 20 client.16254 trim_cache size 0 max 16384 2024-05-09T09:27:18.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.295+0000 7ff951b7b740 1 Processor -- start 2024-05-09T09:27:18.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.295+0000 7ff951b7b740 1 -- start start 2024-05-09T09:27:18.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.295+0000 7ff951b7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.295+0000 7ff951b7b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d19b920 0x55fa2d1ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.295+0000 7ff951b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d1a0a90 0x55fa2d199590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.295+0000 7ff951b7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff9481aa620 con 0x55fa2d19af30 2024-05-09T09:27:18.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.295+0000 7ff951b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9481ac1f0 con 0x55fa2d1a0a90 2024-05-09T09:27:18.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.295+0000 7ff951b7b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff9481aacf0 con 0x55fa2d19b920 2024-05-09T09:27:18.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.295+0000 7ff947fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.295+0000 7ff947fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38636/0 (socket says 172.21.15.12:38636) 2024-05-09T09:27:18.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.295+0000 7ff947fff640 1 -- 172.21.15.12:0/3666524023 learned_addr learned my addr 172.21.15.12:0/3666524023 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.295+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/3666524023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d19b920 0x55fa2d1ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.296+0000 7ff947fff640 1 --2- 172.21.15.12:0/3666524023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d1a0a90 0x55fa2d199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.296+0000 7ff947fff640 1 -- 172.21.15.12:0/3666524023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d19b920 msgr2=0x55fa2d1ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.296+0000 7ff947fff640 1 --2- 172.21.15.12:0/3666524023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d19b920 0x55fa2d1ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.296+0000 7ff947fff640 1 -- 172.21.15.12:0/3666524023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d1a0a90 msgr2=0x55fa2d199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.296+0000 7ff947fff640 1 --2- 172.21.15.12:0/3666524023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d1a0a90 0x55fa2d199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.296+0000 7ff947fff640 1 -- 172.21.15.12:0/3666524023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa2d19aa90 con 0x55fa2d19af30 2024-05-09T09:27:18.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.296+0000 7ff947fff640 1 --2- 172.21.15.12:0/3666524023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1a4610 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff930002230 tx=0x7ff930009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743f80871921fb82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.296+0000 7ff93e7fc640 1 -- 172.21.15.12:0/3666524023 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930012bc0 con 0x55fa2d19af30 2024-05-09T09:27:18.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.296+0000 7ff93e7fc640 1 -- 172.21.15.12:0/3666524023 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff930012d60 con 0x55fa2d19af30 2024-05-09T09:27:18.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.296+0000 7ff93e7fc640 1 -- 172.21.15.12:0/3666524023 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93002ba50 con 0x55fa2d19af30 2024-05-09T09:27:18.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.297+0000 7ff951b7b740 1 -- 172.21.15.12:0/3666524023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 msgr2=0x55fa2d1a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.297+0000 7ff951b7b740 1 --2- 172.21.15.12:0/3666524023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1a4610 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff930002230 tx=0x7ff930009610 comp rx=0 tx=0).stop 2024-05-09T09:27:18.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.297+0000 7ff951b7b740 1 -- 172.21.15.12:0/3666524023 shutdown_connections 2024-05-09T09:27:18.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.297+0000 7ff951b7b740 1 --2- 172.21.15.12:0/3666524023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d1a0a90 0x55fa2d199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.297+0000 7ff951b7b740 1 --2- 172.21.15.12:0/3666524023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d19b920 0x55fa2d1ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.297+0000 7ff951b7b740 1 --2- 172.21.15.12:0/3666524023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1a4610 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.297+0000 7ff951b7b740 1 -- 172.21.15.12:0/3666524023 >> 172.21.15.12:0/3666524023 conn(0x55fa2d1965a0 msgr2=0x55fa2d1a3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.297+0000 7ff951b7b740 1 -- 172.21.15.12:0/3666524023 shutdown_connections 2024-05-09T09:27:18.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.297+0000 7ff951b7b740 1 -- 172.21.15.12:0/3666524023 wait complete. 2024-05-09T09:27:18.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.297+0000 7ff951b7b740 1 -- start start 2024-05-09T09:27:18.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.297+0000 7ff951b7b740 1 Processor -- start 2024-05-09T09:27:18.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.297+0000 7ff951b7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.297+0000 7ff951b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d19b920 0x55fa2d1aa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff951b7b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d1a0a90 0x55fa2d1ab680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff951b7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff9481aba90 con 0x55fa2d19af30 2024-05-09T09:27:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff951b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9481ab3c0 con 0x55fa2d19b920 2024-05-09T09:27:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff951b7b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff9481a8600 con 0x55fa2d1a0a90 2024-05-09T09:27:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff951b7b740 10 client.0 _ll_register_callbacks cb 0x7ff930005c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff947fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff947fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1aa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38638/0 (socket says 172.21.15.12:38638) 2024-05-09T09:27:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff947fff640 1 -- 172.21.15.12:0/908569999 learned_addr learned my addr 172.21.15.12:0/908569999 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff947fff640 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d1a0a90 0x55fa2d1ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d19b920 0x55fa2d1aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff947fff640 1 -- 172.21.15.12:0/908569999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d1a0a90 msgr2=0x55fa2d1ab680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff947fff640 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d1a0a90 0x55fa2d1ab680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff947fff640 1 -- 172.21.15.12:0/908569999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d19b920 msgr2=0x55fa2d1aa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff947fff640 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d19b920 0x55fa2d1aa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff947fff640 1 -- 172.21.15.12:0/908569999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa2d1abc70 con 0x55fa2d19af30 2024-05-09T09:27:18.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d19b920 0x55fa2d1aa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:27:18.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff947fff640 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1aa130 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff930022010 tx=0x7ff930009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbea0357115fdaa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff927fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:18.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff927fff640 1 -- 172.21.15.12:0/908569999 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930020da0 con 0x55fa2d19af30 2024-05-09T09:27:18.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff927fff640 1 -- 172.21.15.12:0/908569999 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7ff9300203e0 con 0x55fa2d19af30 2024-05-09T09:27:18.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff951b7b740 1 -- 172.21.15.12:0/908569999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff930012d60 con 0x55fa2d19af30 2024-05-09T09:27:18.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff927fff640 1 -- 172.21.15.12:0/908569999 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff9300205e0 con 0x55fa2d19af30 2024-05-09T09:27:18.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff951b7b740 10 client.16260 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:18.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.298+0000 7ff951b7b740 1 -- 172.21.15.12:0/908569999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fa2d217980 con 0x55fa2d19af30 2024-05-09T09:27:18.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.299+0000 7ff927fff640 1 -- 172.21.15.12:0/908569999 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff930012d60 con 0x55fa2d19af30 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.299+0000 7ff927fff640 1 -- 172.21.15.12:0/908569999 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7ff93003b040 con 0x55fa2d19af30 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.299+0000 7ff951b7b740 10 client.16260 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.299+0000 7ff951b7b740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.299+0000 7ff951b7b740 20 client.16260 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.299+0000 7ff951b7b740 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fa2d2180e0 0x55fa2d21a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.299+0000 7ff951b7b740 4 client.16260 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.299+0000 7ff951b7b740 1 -- 172.21.15.12:0/908569999 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa2d107780 con 0x55fa2d2180e0 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.299+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fa2d2180e0 0x55fa2d21a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.299+0000 7ff94cb2c640 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fa2d2180e0 0x55fa2d21a500 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae0a57d099efd37b server_cookie=11538d9243993746 in_seq=0 out_seq=0 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.299+0000 7ff927fff640 10 client.16260 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff927fff640 1 -- 172.21.15.12:0/908569999 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fa2d107780 con 0x55fa2d2180e0 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff927fff640 10 client.16260 handle_command_reply: tid=0 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff951b7b740 1 client.16260 shutdown 2024-05-09T09:27:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff951b7b740 1 -- 172.21.15.12:0/908569999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 msgr2=0x55fa2d1aa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff951b7b740 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1aa130 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff930022010 tx=0x7ff930009d10 comp rx=0 tx=0).stop 2024-05-09T09:27:18.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff951b7b740 1 -- 172.21.15.12:0/908569999 shutdown_connections 2024-05-09T09:27:18.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff951b7b740 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fa2d2180e0 0x55fa2d21a500 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff951b7b740 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa2d1a0a90 0x55fa2d1ab680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff951b7b740 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa2d19b920 0x55fa2d1aa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff927fff640 0 client.16260 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:18.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff951b7b740 1 --2- 172.21.15.12:0/908569999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa2d19af30 0x55fa2d1aa130 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff951b7b740 1 -- 172.21.15.12:0/908569999 >> 172.21.15.12:0/908569999 conn(0x55fa2d1965a0 msgr2=0x55fa2d1a4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff951b7b740 1 -- 172.21.15.12:0/908569999 shutdown_connections 2024-05-09T09:27:18.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff951b7b740 1 -- 172.21.15.12:0/908569999 wait complete. 2024-05-09T09:27:18.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff951b7b740 20 client.16260 trim_cache size 0 max 16384 2024-05-09T09:27:18.309 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:27:18.310 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T09:27:18.310 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T09:27:18.310 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:27:18.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff94ed90640 1 -- 172.21.15.12:0/2090902500 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff9340a2b90 msgr2=0x7ff9340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff94ed90640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff9340a2b90 0x7ff9340a5050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff94009bc90 tx=0x7ff94006ac80 comp rx=0 tx=0).stop 2024-05-09T09:27:18.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff94ed90640 1 -- 172.21.15.12:0/2090902500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff948102320 msgr2=0x7ff94819f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.300+0000 7ff94ed90640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff948102320 0x7ff94819f1a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff94000e970 tx=0x7ff94000ed70 comp rx=0 tx=0).stop 2024-05-09T09:27:18.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.301+0000 7ff94ed90640 1 -- 172.21.15.12:0/2090902500 shutdown_connections 2024-05-09T09:27:18.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.301+0000 7ff94ed90640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff9340a2b90 0x7ff9340a5050 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.301+0000 7ff94ed90640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff948102320 0x7ff94819f1a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.301+0000 7ff94ed90640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff948101960 0x7ff948198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.301+0000 7ff94ed90640 1 --2- 172.21.15.12:0/2090902500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100770 0x7ff948197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.301+0000 7ff94ed90640 1 -- 172.21.15.12:0/2090902500 >> 172.21.15.12:0/2090902500 conn(0x7ff9480fbce0 msgr2=0x7ff9480689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.301+0000 7ff94ed90640 1 -- 172.21.15.12:0/2090902500 shutdown_connections 2024-05-09T09:27:18.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.301+0000 7ff94ed90640 1 -- 172.21.15.12:0/2090902500 wait complete. 2024-05-09T09:27:18.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:17 smithi012 ceph-mon[30140]: pgmap v419: 129 pgs: 129 active+clean; 108 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 2.2 MiB/s wr, 940 op/s 2024-05-09T09:27:18.313 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:27:18.313 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:27:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.422+0000 7fa455cf5640 1 -- 172.21.15.12:0/1411254810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa450107570 msgr2=0x7fa4501099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.422+0000 7fa455cf5640 1 --2- 172.21.15.12:0/1411254810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa450107570 0x7fa4501099c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa448009ca0 tx=0x7fa44802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.422+0000 7fa455cf5640 1 -- 172.21.15.12:0/1411254810 shutdown_connections 2024-05-09T09:27:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.422+0000 7fa455cf5640 1 --2- 172.21.15.12:0/1411254810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa450107570 0x7fa4501099c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.422+0000 7fa455cf5640 1 --2- 172.21.15.12:0/1411254810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450104c40 0x7fa450107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.422+0000 7fa455cf5640 1 --2- 172.21.15.12:0/1411254810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa450102310 0x7fa450104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.422+0000 7fa455cf5640 1 -- 172.21.15.12:0/1411254810 >> 172.21.15.12:0/1411254810 conn(0x7fa4500fbd20 msgr2=0x7fa4500fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.422+0000 7fa455cf5640 1 -- 172.21.15.12:0/1411254810 shutdown_connections 2024-05-09T09:27:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.422+0000 7fa455cf5640 1 -- 172.21.15.12:0/1411254810 wait complete. 2024-05-09T09:27:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa455cf5640 1 Processor -- start 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa455cf5640 1 -- start start 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa455cf5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa450102310 0x7fa450197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa455cf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450104c40 0x7fa4501980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa455cf5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa450107570 0x7fa45019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa455cf5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa45010d510 con 0x7fa450102310 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa455cf5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa45010d390 con 0x7fa450104c40 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa455cf5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa45010d690 con 0x7fa450107570 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa44f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa450102310 0x7fa450197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa44f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa450102310 0x7fa450197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38652/0 (socket says 172.21.15.12:38652) 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa44f7fe640 1 -- 172.21.15.12:0/1880569615 learned_addr learned my addr 172.21.15.12:0/1880569615 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa44effd640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450104c40 0x7fa4501980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa44ffff640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa450107570 0x7fa45019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa44f7fe640 1 -- 172.21.15.12:0/1880569615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa450107570 msgr2=0x7fa45019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa44f7fe640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa450107570 0x7fa45019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa44f7fe640 1 -- 172.21.15.12:0/1880569615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450104c40 msgr2=0x7fa4501980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa44f7fe640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450104c40 0x7fa4501980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa44f7fe640 1 -- 172.21.15.12:0/1880569615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45019f7a0 con 0x7fa450102310 2024-05-09T09:27:18.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa44effd640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450104c40 0x7fa4501980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:18.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.423+0000 7fa44ffff640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa450107570 0x7fa45019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:27:18.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.424+0000 7fa44f7fe640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa450102310 0x7fa450197b80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa44000d770 tx=0x7fa44000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.424+0000 7fa44cff9640 1 -- 172.21.15.12:0/1880569615 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440017070 con 0x7fa450102310 2024-05-09T09:27:18.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.424+0000 7fa455cf5640 1 -- 172.21.15.12:0/1880569615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45019fa90 con 0x7fa450102310 2024-05-09T09:27:18.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.424+0000 7fa44cff9640 1 -- 172.21.15.12:0/1880569615 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa4400054f0 con 0x7fa450102310 2024-05-09T09:27:18.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.424+0000 7fa44cff9640 1 -- 172.21.15.12:0/1880569615 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440004d60 con 0x7fa450102310 2024-05-09T09:27:18.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.424+0000 7fa455cf5640 1 -- 172.21.15.12:0/1880569615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45019ffe0 con 0x7fa450102310 2024-05-09T09:27:18.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.425+0000 7fa455cf5640 1 -- 172.21.15.12:0/1880569615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4500fd810 con 0x7fa450102310 2024-05-09T09:27:18.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.426+0000 7fa44cff9640 1 -- 172.21.15.12:0/1880569615 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa440004f00 con 0x7fa450102310 2024-05-09T09:27:18.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.427+0000 7fa44cff9640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa4380a2b90 0x7fa4380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.427+0000 7fa44cff9640 1 -- 172.21.15.12:0/1880569615 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fa4400cef70 con 0x7fa450102310 2024-05-09T09:27:18.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.427+0000 7fa44effd640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa4380a2b90 0x7fa4380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.428+0000 7fa44effd640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa4380a2b90 0x7fa4380a5050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa4440058c0 tx=0x7fa444005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.432+0000 7fa44cff9640 1 -- 172.21.15.12:0/1880569615 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa44009bb00 con 0x7fa450102310 2024-05-09T09:27:18.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.659+0000 7fa455cf5640 1 -- 172.21.15.12:0/1880569615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa45002d1c0 con 0x7fa450102310 2024-05-09T09:27:18.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.659+0000 7fa44cff9640 1 -- 172.21.15.12:0/1880569615 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11936 (secure 0 0 0) 0x7fa45002d1c0 con 0x7fa450102310 2024-05-09T09:27:18.659 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:27:18.660 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:27:15.324580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3413591353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3413591353},{"type":"v1","addr":"172.21.15.16:6837","nonce":3413591353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[24484,14628,24515,24502,24523]},"id":1}]} 2024-05-09T09:27:18.660 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 31 2024-05-09T09:27:18.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.661+0000 7fa455cf5640 1 -- 172.21.15.12:0/1880569615 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa4380a2b90 msgr2=0x7fa4380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.661+0000 7fa455cf5640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa4380a2b90 0x7fa4380a5050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa4440058c0 tx=0x7fa444005850 comp rx=0 tx=0).stop 2024-05-09T09:27:18.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.661+0000 7fa455cf5640 1 -- 172.21.15.12:0/1880569615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa450102310 msgr2=0x7fa450197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.661+0000 7fa455cf5640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa450102310 0x7fa450197b80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa44000d770 tx=0x7fa44000dc30 comp rx=0 tx=0).stop 2024-05-09T09:27:18.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.661+0000 7fa455cf5640 1 -- 172.21.15.12:0/1880569615 shutdown_connections 2024-05-09T09:27:18.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.661+0000 7fa455cf5640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa4380a2b90 0x7fa4380a5050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.661+0000 7fa455cf5640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa450107570 0x7fa45019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.661+0000 7fa455cf5640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450104c40 0x7fa4501980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.661+0000 7fa455cf5640 1 --2- 172.21.15.12:0/1880569615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa450102310 0x7fa450197b80 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.661+0000 7fa455cf5640 1 -- 172.21.15.12:0/1880569615 >> 172.21.15.12:0/1880569615 conn(0x7fa4500fbd20 msgr2=0x7fa450102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.662+0000 7fa455cf5640 1 -- 172.21.15.12:0/1880569615 shutdown_connections 2024-05-09T09:27:18.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.662+0000 7fa455cf5640 1 -- 172.21.15.12:0/1880569615 wait complete. 2024-05-09T09:27:18.672 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-09T09:27:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.781+0000 7f8c73fe9640 1 -- 172.21.15.12:0/3610099599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c6c101960 msgr2=0x7f8c6c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.781+0000 7f8c73fe9640 1 --2- 172.21.15.12:0/3610099599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c6c101960 0x7f8c6c101de0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009900 tx=0x7f8c6002a410 comp rx=0 tx=0).stop 2024-05-09T09:27:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.781+0000 7f8c73fe9640 1 -- 172.21.15.12:0/3610099599 shutdown_connections 2024-05-09T09:27:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.781+0000 7f8c73fe9640 1 --2- 172.21.15.12:0/3610099599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c102320 0x7f8c6c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.781+0000 7f8c73fe9640 1 --2- 172.21.15.12:0/3610099599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c6c101960 0x7f8c6c101de0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.781+0000 7f8c73fe9640 1 --2- 172.21.15.12:0/3610099599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c6c100770 0x7f8c6c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.781+0000 7f8c73fe9640 1 -- 172.21.15.12:0/3610099599 >> 172.21.15.12:0/3610099599 conn(0x7f8c6c0fbce0 msgr2=0x7f8c6c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.781+0000 7f8c73fe9640 1 -- 172.21.15.12:0/3610099599 shutdown_connections 2024-05-09T09:27:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.781+0000 7f8c73fe9640 1 -- 172.21.15.12:0/3610099599 wait complete. 2024-05-09T09:27:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.781+0000 7f8c73fe9640 1 Processor -- start 2024-05-09T09:27:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.781+0000 7f8c73fe9640 1 -- start start 2024-05-09T09:27:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c73fe9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c6c100770 0x7f8c6c069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c73fe9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c101960 0x7f8c6c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c73fe9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c6c102320 0x7f8c6c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c73fe9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c10c3d0 con 0x7f8c6c102320 2024-05-09T09:27:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c73fe9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c10c250 con 0x7f8c6c101960 2024-05-09T09:27:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c73fe9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c10c550 con 0x7f8c6c100770 2024-05-09T09:27:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c72586640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c6c102320 0x7f8c6c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c72586640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c6c102320 0x7f8c6c19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38674/0 (socket says 172.21.15.12:38674) 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c72586640 1 -- 172.21.15.12:0/2988737365 learned_addr learned my addr 172.21.15.12:0/2988737365 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c71d85640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c6c100770 0x7f8c6c069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c71584640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c101960 0x7f8c6c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c72586640 1 -- 172.21.15.12:0/2988737365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c6c100770 msgr2=0x7f8c6c069550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c72586640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c6c100770 0x7f8c6c069550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c72586640 1 -- 172.21.15.12:0/2988737365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c101960 msgr2=0x7f8c6c19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c72586640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c101960 0x7f8c6c19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c72586640 1 -- 172.21.15.12:0/2988737365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6c19f7e0 con 0x7f8c6c102320 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c72586640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c6c102320 0x7f8c6c19ef20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400e970 tx=0x7f8c6400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c6affd640 1 -- 172.21.15.12:0/2988737365 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6400ccc0 con 0x7f8c6c102320 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c6affd640 1 -- 172.21.15.12:0/2988737365 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8c6400ce60 con 0x7f8c6c102320 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c73fe9640 1 -- 172.21.15.12:0/2988737365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6c19fad0 con 0x7f8c6c102320 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c6affd640 1 -- 172.21.15.12:0/2988737365 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64010430 con 0x7f8c6c102320 2024-05-09T09:27:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.782+0000 7f8c73fe9640 1 -- 172.21.15.12:0/2988737365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c6c19ffd0 con 0x7f8c6c102320 2024-05-09T09:27:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.785+0000 7f8c6affd640 1 -- 172.21.15.12:0/2988737365 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8c64005000 con 0x7f8c6c102320 2024-05-09T09:27:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.785+0000 7f8c76dcb740 1 Processor -- start 2024-05-09T09:27:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.785+0000 7f8c76dcb740 1 -- start start 2024-05-09T09:27:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.785+0000 7f8c76dcb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b369ff40 0x5653b36a0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.785+0000 7f8c76dcb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b36a0930 0x5653b36a5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.785+0000 7f8c76dcb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a5aa0 0x5653b36a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.785+0000 7f8c76dcb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5653b3604c40 con 0x5653b369ff40 2024-05-09T09:27:18.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.785+0000 7f8c76dcb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5653b35ac580 con 0x5653b36a5aa0 2024-05-09T09:27:18.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.785+0000 7f8c76dcb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5653b3600c00 con 0x5653b36a0930 2024-05-09T09:27:18.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.785+0000 7f8c71584640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b369ff40 0x5653b36a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.785+0000 7f8c71584640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b369ff40 0x5653b36a0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38676/0 (socket says 172.21.15.12:38676) 2024-05-09T09:27:18.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.785+0000 7f8c71584640 1 -- 172.21.15.12:0/3227296945 learned_addr learned my addr 172.21.15.12:0/3227296945 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c6affd640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8c540a2b90 0x7f8c540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c71584640 1 --2- 172.21.15.12:0/3227296945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a5aa0 0x5653b36a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c71d85640 1 --2- 172.21.15.12:0/3227296945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b36a0930 0x5653b36a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c71584640 1 -- 172.21.15.12:0/3227296945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b36a0930 msgr2=0x5653b36a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c71584640 1 --2- 172.21.15.12:0/3227296945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b36a0930 0x5653b36a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c71584640 1 -- 172.21.15.12:0/3227296945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a5aa0 msgr2=0x5653b36a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c71584640 1 --2- 172.21.15.12:0/3227296945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a5aa0 0x5653b36a7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c71584640 1 -- 172.21.15.12:0/3227296945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653b33183f0 con 0x5653b369ff40 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c72586640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8c540a2b90 0x7f8c540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c6affd640 1 -- 172.21.15.12:0/2988737365 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f8c640cf980 con 0x7f8c6c102320 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c71584640 1 --2- 172.21.15.12:0/3227296945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b369ff40 0x5653b36a0360 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8c60005ec0 tx=0x7f8c60009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53394e11e185232b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c53fff640 1 -- 172.21.15.12:0/3227296945 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60035070 con 0x5653b369ff40 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c72586640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8c540a2b90 0x7f8c540a5050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8c640a4550 tx=0x7f8c640bc040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c53fff640 1 -- 172.21.15.12:0/3227296945 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8c60031810 con 0x5653b369ff40 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.786+0000 7f8c53fff640 1 -- 172.21.15.12:0/3227296945 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c600046c0 con 0x5653b369ff40 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.787+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3227296945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b369ff40 msgr2=0x5653b36a0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.787+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3227296945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b369ff40 0x5653b36a0360 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8c60005ec0 tx=0x7f8c60009620 comp rx=0 tx=0).stop 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.787+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3227296945 shutdown_connections 2024-05-09T09:27:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.787+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3227296945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a5aa0 0x5653b36a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.787+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3227296945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b36a0930 0x5653b36a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.787+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3227296945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b369ff40 0x5653b36a0360 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.787+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3227296945 >> 172.21.15.12:0/3227296945 conn(0x5653b369b5b0 msgr2=0x5653b369b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.787+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3227296945 shutdown_connections 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.787+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3227296945 wait complete. 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.787+0000 7f8c76dcb740 1 -- start start 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.787+0000 7f8c76dcb740 1 Processor -- start 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c76dcb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b369ff40 0x5653b36aa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c76dcb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a0930 0x5653b36aa850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c76dcb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a5aa0 0x5653b36b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c76dcb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5653b36a8990 con 0x5653b36a5aa0 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c76dcb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5653b35bfc40 con 0x5653b36a0930 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c76dcb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5653b36a8b10 con 0x5653b369ff40 2024-05-09T09:27:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c76dcb740 10 client.0 _ll_register_callbacks cb 0x5653b3317be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:18.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c71584640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a5aa0 0x5653b36b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c71584640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a5aa0 0x5653b36b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38678/0 (socket says 172.21.15.12:38678) 2024-05-09T09:27:18.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c71584640 1 -- 172.21.15.12:0/1361747064 learned_addr learned my addr 172.21.15.12:0/1361747064 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c71584640 1 --2- 172.21.15.12:0/1361747064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b369ff40 0x5653b36aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c71d85640 1 --2- 172.21.15.12:0/1361747064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a0930 0x5653b36aa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c71584640 1 -- 172.21.15.12:0/1361747064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b369ff40 msgr2=0x5653b36aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c71584640 1 --2- 172.21.15.12:0/1361747064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b369ff40 0x5653b36aa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c71584640 1 -- 172.21.15.12:0/1361747064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a0930 msgr2=0x5653b36aa850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c71584640 1 --2- 172.21.15.12:0/1361747064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a0930 0x5653b36aa850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.788+0000 7f8c71584640 1 -- 172.21.15.12:0/1361747064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653b33e27d0 con 0x5653b36a5aa0 2024-05-09T09:27:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c71584640 1 --2- 172.21.15.12:0/1361747064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a5aa0 0x5653b36b1820 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8c60031b00 tx=0x7f8c60042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad9b931acb688d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c517fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c517fa640 1 -- 172.21.15.12:0/1361747064 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60054060 con 0x5653b36a5aa0 2024-05-09T09:27:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c76dcb740 1 -- 172.21.15.12:0/1361747064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c60031810 con 0x5653b36a5aa0 2024-05-09T09:27:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c517fa640 1 -- 172.21.15.12:0/1361747064 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f8c6003e840 con 0x5653b36a5aa0 2024-05-09T09:27:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c517fa640 1 -- 172.21.15.12:0/1361747064 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8c60035070 con 0x5653b36a5aa0 2024-05-09T09:27:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c76dcb740 10 client.16278 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c76dcb740 1 -- 172.21.15.12:0/1361747064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5653b36b4b90 con 0x5653b36a5aa0 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c517fa640 1 -- 172.21.15.12:0/1361747064 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8c60031810 con 0x5653b36a5aa0 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c517fa640 1 -- 172.21.15.12:0/1361747064 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f8c6004f9a0 con 0x5653b36a5aa0 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c76dcb740 10 client.16278 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c76dcb740 10 client.16278 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c76dcb740 20 client.16278 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/1361747064 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5653b36b6310 0x5653b36b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c76dcb740 4 client.16278 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.789+0000 7f8c76dcb740 1 -- 172.21.15.12:0/1361747064 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653b3612370 con 0x5653b36b6310 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.790+0000 7f8c71d85640 1 --2- 172.21.15.12:0/1361747064 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5653b36b6310 0x5653b36b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.790+0000 7f8c71d85640 1 --2- 172.21.15.12:0/1361747064 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5653b36b6310 0x5653b36b8730 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=78e9badb0e2fd893 server_cookie=95396d53058d779a in_seq=0 out_seq=0 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.790+0000 7f8c517fa640 10 client.16278 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.790+0000 7f8c517fa640 1 -- 172.21.15.12:0/1361747064 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653b3612370 con 0x5653b36b6310 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.790+0000 7f8c517fa640 10 client.16278 handle_command_reply: tid=0 2024-05-09T09:27:18.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.790+0000 7f8c76dcb740 1 client.16278 shutdown 2024-05-09T09:27:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.791+0000 7f8c76dcb740 1 -- 172.21.15.12:0/1361747064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a5aa0 msgr2=0x5653b36b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.791+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/1361747064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a5aa0 0x5653b36b1820 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8c60031b00 tx=0x7f8c60042040 comp rx=0 tx=0).stop 2024-05-09T09:27:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.791+0000 7f8c76dcb740 1 -- 172.21.15.12:0/1361747064 shutdown_connections 2024-05-09T09:27:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.791+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/1361747064 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5653b36b6310 0x5653b36b8730 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.791+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/1361747064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a5aa0 0x5653b36b1820 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.791+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/1361747064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a0930 0x5653b36aa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.791+0000 7f8c517fa640 0 client.16278 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:27:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.791+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/1361747064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b369ff40 0x5653b36aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.791+0000 7f8c76dcb740 1 -- 172.21.15.12:0/1361747064 >> 172.21.15.12:0/1361747064 conn(0x5653b369b5b0 msgr2=0x5653b369a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.791+0000 7f8c76dcb740 1 -- 172.21.15.12:0/1361747064 shutdown_connections 2024-05-09T09:27:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.791+0000 7f8c76dcb740 1 -- 172.21.15.12:0/1361747064 wait complete. 2024-05-09T09:27:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.791+0000 7f8c76dcb740 20 client.16278 trim_cache size 0 max 16384 2024-05-09T09:27:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.804+0000 7f8c76dcb740 1 Processor -- start 2024-05-09T09:27:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.804+0000 7f8c76dcb740 1 -- start start 2024-05-09T09:27:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.804+0000 7f8c76dcb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b369ff40 0x5653b36a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.804+0000 7f8c76dcb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 0x5653b36a2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.804+0000 7f8c76dcb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a5aa0 0x5653b36b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.804+0000 7f8c76dcb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c1aa530 con 0x5653b36a0930 2024-05-09T09:27:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.804+0000 7f8c76dcb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c1ac100 con 0x5653b36a5aa0 2024-05-09T09:27:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.804+0000 7f8c76dcb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c1aac00 con 0x5653b369ff40 2024-05-09T09:27:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.805+0000 7f8c71d85640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 0x5653b36a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.805+0000 7f8c71d85640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 0x5653b36a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38690/0 (socket says 172.21.15.12:38690) 2024-05-09T09:27:18.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.805+0000 7f8c71d85640 1 -- 172.21.15.12:0/3667431140 learned_addr learned my addr 172.21.15.12:0/3667431140 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.805+0000 7f8c71d85640 1 -- 172.21.15.12:0/3667431140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b369ff40 msgr2=0x5653b36a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.805+0000 7f8c71d85640 1 --2- 172.21.15.12:0/3667431140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b369ff40 0x5653b36a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.805+0000 7f8c71d85640 1 -- 172.21.15.12:0/3667431140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a5aa0 msgr2=0x5653b36b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.805+0000 7f8c71d85640 1 --2- 172.21.15.12:0/3667431140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a5aa0 0x5653b36b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.805+0000 7f8c71d85640 1 -- 172.21.15.12:0/3667431140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653b369c320 con 0x5653b36a0930 2024-05-09T09:27:18.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.805+0000 7f8c71d85640 1 --2- 172.21.15.12:0/3667431140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 0x5653b36a2330 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c003a80 tx=0x7f8c5c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75653688ba2b9f44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.805+0000 7f8c537fe640 1 -- 172.21.15.12:0/3667431140 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c013110 con 0x5653b36a0930 2024-05-09T09:27:18.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.805+0000 7f8c537fe640 1 -- 172.21.15.12:0/3667431140 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8c5c0132b0 con 0x5653b36a0930 2024-05-09T09:27:18.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.805+0000 7f8c537fe640 1 -- 172.21.15.12:0/3667431140 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c011440 con 0x5653b36a0930 2024-05-09T09:27:18.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.806+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3667431140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 msgr2=0x5653b36a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.806+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3667431140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 0x5653b36a2330 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c003a80 tx=0x7f8c5c009f90 comp rx=0 tx=0).stop 2024-05-09T09:27:18.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.806+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3667431140 shutdown_connections 2024-05-09T09:27:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.806+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3667431140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b36a5aa0 0x5653b36b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.806+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3667431140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 0x5653b36a2330 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.806+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3667431140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b369ff40 0x5653b36a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.806+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3667431140 >> 172.21.15.12:0/3667431140 conn(0x5653b369b5b0 msgr2=0x5653b369b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.806+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3667431140 shutdown_connections 2024-05-09T09:27:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.806+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3667431140 wait complete. 2024-05-09T09:27:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.806+0000 7f8c76dcb740 1 -- start start 2024-05-09T09:27:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.806+0000 7f8c76dcb740 1 Processor -- start 2024-05-09T09:27:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c76dcb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b369ff40 0x5653b36ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c76dcb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 0x5653b36aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c76dcb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b36a5aa0 0x5653b371a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c76dcb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c1ab9a0 con 0x5653b36a0930 2024-05-09T09:27:18.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c76dcb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c1ab2d0 con 0x5653b369ff40 2024-05-09T09:27:18.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c76dcb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c1a8510 con 0x5653b36a5aa0 2024-05-09T09:27:18.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c76dcb740 10 client.0 _ll_register_callbacks cb 0x7f8c5c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:18.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c71d85640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 0x5653b36aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c71d85640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 0x5653b36aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38698/0 (socket says 172.21.15.12:38698) 2024-05-09T09:27:18.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c71d85640 1 -- 172.21.15.12:0/3937879803 learned_addr learned my addr 172.21.15.12:0/3937879803 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c71584640 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b36a5aa0 0x5653b371a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c71584640 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b369ff40 0x5653b36ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c71d85640 1 -- 172.21.15.12:0/3937879803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b36a5aa0 msgr2=0x5653b371a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c71d85640 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b36a5aa0 0x5653b371a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c71d85640 1 -- 172.21.15.12:0/3937879803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b369ff40 msgr2=0x5653b36ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c71d85640 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b369ff40 0x5653b36ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c71d85640 1 -- 172.21.15.12:0/3937879803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653b36af1e0 con 0x5653b36a0930 2024-05-09T09:27:18.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c71584640 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b36a5aa0 0x5653b371a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:18.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c71d85640 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 0x5653b36aebd0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c020040 tx=0x7f8c5c010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d296ca800a7f20 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c517fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:18.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c517fa640 1 -- 172.21.15.12:0/3937879803 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c02c030 con 0x5653b36a0930 2024-05-09T09:27:18.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c517fa640 1 -- 172.21.15.12:0/3937879803 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f8c5c004c50 con 0x5653b36a0930 2024-05-09T09:27:18.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.807+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3937879803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c5c0132b0 con 0x5653b36a0930 2024-05-09T09:27:18.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c517fa640 1 -- 172.21.15.12:0/3937879803 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8c5c036050 con 0x5653b36a0930 2024-05-09T09:27:18.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c76dcb740 10 client.16284 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:18.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3937879803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5653b371bea0 con 0x5653b36a0930 2024-05-09T09:27:18.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c517fa640 1 -- 172.21.15.12:0/3937879803 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8c5c0132b0 con 0x5653b36a0930 2024-05-09T09:27:18.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c517fa640 1 -- 172.21.15.12:0/3937879803 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f8c5c005160 con 0x5653b36a0930 2024-05-09T09:27:18.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c76dcb740 10 client.16284 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:18.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c76dcb740 10 client.16284 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:27:18.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c76dcb740 20 client.16284 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:18.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5653b371c880 0x5653b371eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c76dcb740 4 client.16284 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:27:18.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3937879803 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653b360c6c0 con 0x5653b371c880 2024-05-09T09:27:18.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c71584640 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5653b371c880 0x5653b371eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c517fa640 10 client.16284 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:27:18.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.808+0000 7f8c71584640 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5653b371c880 0x5653b371eca0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b53def9f1588680f server_cookie=9cba7febbd612a87 in_seq=0 out_seq=0 2024-05-09T09:27:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c517fa640 1 -- 172.21.15.12:0/3937879803 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653b360c6c0 con 0x5653b371c880 2024-05-09T09:27:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c517fa640 10 client.16284 handle_command_reply: tid=0 2024-05-09T09:27:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c76dcb740 1 client.16284 shutdown 2024-05-09T09:27:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3937879803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 msgr2=0x5653b36aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 0x5653b36aebd0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c020040 tx=0x7f8c5c010de0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3937879803 shutdown_connections 2024-05-09T09:27:18.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5653b371c880 0x5653b371eca0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653b36a5aa0 0x5653b371a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b36a0930 0x5653b36aebd0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c76dcb740 1 --2- 172.21.15.12:0/3937879803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653b369ff40 0x5653b36ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c517fa640 0 client.16284 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:27:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3937879803 >> 172.21.15.12:0/3937879803 conn(0x5653b369b5b0 msgr2=0x5653b36aa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3937879803 shutdown_connections 2024-05-09T09:27:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c76dcb740 1 -- 172.21.15.12:0/3937879803 wait complete. 2024-05-09T09:27:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c76dcb740 20 client.16284 trim_cache size 0 max 16384 2024-05-09T09:27:18.821 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:27:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c73fe9640 1 -- 172.21.15.12:0/2988737365 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8c540a2b90 msgr2=0x7f8c540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c73fe9640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8c540a2b90 0x7f8c540a5050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8c640a4550 tx=0x7f8c640bc040 comp rx=0 tx=0).stop 2024-05-09T09:27:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c73fe9640 1 -- 172.21.15.12:0/2988737365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c6c102320 msgr2=0x7f8c6c19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.809+0000 7f8c73fe9640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c6c102320 0x7f8c6c19ef20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400e970 tx=0x7f8c6400ed70 comp rx=0 tx=0).stop 2024-05-09T09:27:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.810+0000 7f8c73fe9640 1 -- 172.21.15.12:0/2988737365 shutdown_connections 2024-05-09T09:27:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.810+0000 7f8c73fe9640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8c540a2b90 0x7f8c540a5050 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.810+0000 7f8c73fe9640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c6c102320 0x7f8c6c19ef20 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.810+0000 7f8c73fe9640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c101960 0x7f8c6c19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.810+0000 7f8c73fe9640 1 --2- 172.21.15.12:0/2988737365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c6c100770 0x7f8c6c069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.810+0000 7f8c73fe9640 1 -- 172.21.15.12:0/2988737365 >> 172.21.15.12:0/2988737365 conn(0x7f8c6c0fbce0 msgr2=0x7f8c6c104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.810+0000 7f8c73fe9640 1 -- 172.21.15.12:0/2988737365 shutdown_connections 2024-05-09T09:27:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.810+0000 7f8c73fe9640 1 -- 172.21.15.12:0/2988737365 wait complete. 2024-05-09T09:27:18.824 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.932+0000 7f5a7a6fc640 1 -- 172.21.15.12:0/2729686562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a74101980 msgr2=0x7f5a74101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.932+0000 7f5a7a6fc640 1 --2- 172.21.15.12:0/2729686562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a74101980 0x7f5a74101e00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009900 tx=0x7f5a6802a410 comp rx=0 tx=0).stop 2024-05-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.932+0000 7f5a7a6fc640 1 -- 172.21.15.12:0/2729686562 shutdown_connections 2024-05-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.932+0000 7f5a7a6fc640 1 --2- 172.21.15.12:0/2729686562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a74102340 0x7f5a74108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.932+0000 7f5a7a6fc640 1 --2- 172.21.15.12:0/2729686562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a74101980 0x7f5a74101e00 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.932+0000 7f5a7a6fc640 1 --2- 172.21.15.12:0/2729686562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a74100790 0x7f5a74100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.932+0000 7f5a7a6fc640 1 -- 172.21.15.12:0/2729686562 >> 172.21.15.12:0/2729686562 conn(0x7f5a740fbd20 msgr2=0x7f5a740fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.932+0000 7f5a7a6fc640 1 -- 172.21.15.12:0/2729686562 shutdown_connections 2024-05-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.932+0000 7f5a7a6fc640 1 -- 172.21.15.12:0/2729686562 wait complete. 2024-05-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.932+0000 7f5a7a6fc640 1 Processor -- start 2024-05-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.932+0000 7f5a7a6fc640 1 -- start start 2024-05-09T09:27:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a7a6fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a74100790 0x7f5a74196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a7a6fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a74101980 0x7f5a74196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a7a6fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a74102340 0x7f5a7419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a7a6fc640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7410c3f0 con 0x7f5a74101980 2024-05-09T09:27:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a7a6fc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7410c270 con 0x7f5a74100790 2024-05-09T09:27:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a7a6fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7410c570 con 0x7f5a74102340 2024-05-09T09:27:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a74101980 0x7f5a74196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a74101980 0x7f5a74196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38724/0 (socket says 172.21.15.12:38724) 2024-05-09T09:27:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a737fe640 1 -- 172.21.15.12:0/1290725410 learned_addr learned my addr 172.21.15.12:0/1290725410 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a73fff640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a74100790 0x7f5a74196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a78c99640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a74102340 0x7f5a7419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a737fe640 1 -- 172.21.15.12:0/1290725410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a74102340 msgr2=0x7f5a7419dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a737fe640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a74102340 0x7f5a7419dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a737fe640 1 -- 172.21.15.12:0/1290725410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a74100790 msgr2=0x7f5a74196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a737fe640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a74100790 0x7f5a74196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a737fe640 1 -- 172.21.15.12:0/1290725410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a74069f30 con 0x7f5a74101980 2024-05-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.933+0000 7f5a737fe640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a74101980 0x7f5a74196f80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009a20 tx=0x7f5a68031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.934+0000 7f5a717fa640 1 -- 172.21.15.12:0/1290725410 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6803f030 con 0x7f5a74101980 2024-05-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.934+0000 7f5a717fa640 1 -- 172.21.15.12:0/1290725410 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a68003ac0 con 0x7f5a74101980 2024-05-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.934+0000 7f5a717fa640 1 -- 172.21.15.12:0/1290725410 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68035070 con 0x7f5a74101980 2024-05-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.934+0000 7f5a7a6fc640 1 -- 172.21.15.12:0/1290725410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a7406a1c0 con 0x7f5a74101980 2024-05-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.934+0000 7f5a7a6fc640 1 -- 172.21.15.12:0/1290725410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a7406a700 con 0x7f5a74101980 2024-05-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.936+0000 7f5a717fa640 1 -- 172.21.15.12:0/1290725410 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5a68042030 con 0x7f5a74101980 2024-05-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.936+0000 7f5a7d409740 1 Processor -- start 2024-05-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.936+0000 7f5a7d409740 1 -- start start 2024-05-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a7d409740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4f9f40 0x55677d4fa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a7d409740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4fa930 0x55677d4ff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a7d409740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4ffaa0 0x55677d501f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a7d409740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55677d45ec40 con 0x55677d4f9f40 2024-05-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a7d409740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55677d406580 con 0x55677d4fa930 2024-05-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a7d409740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55677d45ac00 con 0x55677d4ffaa0 2024-05-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a78c99640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4f9f40 0x55677d4fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a717fa640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5a5c0a2c10 0x7f5a5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a78c99640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4f9f40 0x55677d4fa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38732/0 (socket says 172.21.15.12:38732) 2024-05-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a78c99640 1 -- 172.21.15.12:0/2305245218 learned_addr learned my addr 172.21.15.12:0/2305245218 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a73fff640 1 --2- 172.21.15.12:0/2305245218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4fa930 0x55677d4ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a78c99640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5a5c0a2c10 0x7f5a5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a717fa640 1 -- 172.21.15.12:0/1290725410 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f5a680ef970 con 0x7f5a74101980 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a78c99640 1 -- 172.21.15.12:0/2305245218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4ffaa0 msgr2=0x55677d501f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a737fe640 1 --2- 172.21.15.12:0/2305245218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4ffaa0 0x55677d501f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a78c99640 1 --2- 172.21.15.12:0/2305245218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4ffaa0 0x55677d501f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a78c99640 1 -- 172.21.15.12:0/2305245218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4fa930 msgr2=0x55677d4ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a78c99640 1 --2- 172.21.15.12:0/2305245218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4fa930 0x55677d4ff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a78c99640 1 -- 172.21.15.12:0/2305245218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55677d1723f0 con 0x55677d4f9f40 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a737fe640 1 --2- 172.21.15.12:0/2305245218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4ffaa0 0x55677d501f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a73fff640 1 --2- 172.21.15.12:0/2305245218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4fa930 0x55677d4ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.937+0000 7f5a78c99640 1 --2- 172.21.15.12:0/2305245218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4f9f40 0x55677d4fa360 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0139d0 tx=0x7f5a6c013e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1872a6adbc97529 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.938+0000 7f5a5a7fc640 1 -- 172.21.15.12:0/2305245218 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c013050 con 0x55677d4f9f40 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.938+0000 7f5a5a7fc640 1 -- 172.21.15.12:0/2305245218 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a6c022070 con 0x55677d4f9f40 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.938+0000 7f5a5a7fc640 1 -- 172.21.15.12:0/2305245218 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0126b0 con 0x55677d4f9f40 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.938+0000 7f5a7d409740 1 -- 172.21.15.12:0/2305245218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4f9f40 msgr2=0x55677d4fa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.938+0000 7f5a7d409740 1 --2- 172.21.15.12:0/2305245218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4f9f40 0x55677d4fa360 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0139d0 tx=0x7f5a6c013e90 comp rx=0 tx=0).stop 2024-05-09T09:27:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.938+0000 7f5a78c99640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5a5c0a2c10 0x7f5a5c0a50d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c021000 tx=0x7f5a6c01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.938+0000 7f5a7d409740 1 -- 172.21.15.12:0/2305245218 shutdown_connections 2024-05-09T09:27:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.938+0000 7f5a7d409740 1 --2- 172.21.15.12:0/2305245218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4ffaa0 0x55677d501f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.938+0000 7f5a7d409740 1 --2- 172.21.15.12:0/2305245218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4fa930 0x55677d4ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.938+0000 7f5a7d409740 1 --2- 172.21.15.12:0/2305245218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4f9f40 0x55677d4fa360 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.938+0000 7f5a7d409740 1 -- 172.21.15.12:0/2305245218 >> 172.21.15.12:0/2305245218 conn(0x55677d4f55b0 msgr2=0x55677d4f59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.938+0000 7f5a7d409740 1 -- 172.21.15.12:0/2305245218 shutdown_connections 2024-05-09T09:27:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.938+0000 7f5a7d409740 1 -- 172.21.15.12:0/2305245218 wait complete. 2024-05-09T09:27:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.939+0000 7f5a7d409740 1 -- start start 2024-05-09T09:27:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.939+0000 7f5a7d409740 1 Processor -- start 2024-05-09T09:27:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.939+0000 7f5a7d409740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4f9f40 0x55677d504390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.939+0000 7f5a7d409740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4fa930 0x55677d5048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.939+0000 7f5a7d409740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4ffaa0 0x55677d50b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.939+0000 7f5a7d409740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55677d502a50 con 0x55677d4ffaa0 2024-05-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.939+0000 7f5a7d409740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55677d419c40 con 0x55677d4f9f40 2024-05-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.939+0000 7f5a7d409740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55677d502bd0 con 0x55677d4fa930 2024-05-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.939+0000 7f5a7d409740 10 client.0 _ll_register_callbacks cb 0x55677d171be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.939+0000 7f5a78c99640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4ffaa0 0x55677d50b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.939+0000 7f5a78c99640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4ffaa0 0x55677d50b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38744/0 (socket says 172.21.15.12:38744) 2024-05-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.939+0000 7f5a73fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4f9f40 0x55677d504390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.939+0000 7f5a78c99640 1 -- 172.21.15.12:0/326025985 learned_addr learned my addr 172.21.15.12:0/326025985 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a737fe640 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4fa930 0x55677d5048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a78c99640 1 -- 172.21.15.12:0/326025985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4fa930 msgr2=0x55677d5048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a78c99640 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4fa930 0x55677d5048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a78c99640 1 -- 172.21.15.12:0/326025985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4f9f40 msgr2=0x55677d504390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a78c99640 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4f9f40 0x55677d504390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a78c99640 1 -- 172.21.15.12:0/326025985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55677d23c7d0 con 0x55677d4ffaa0 2024-05-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a73fff640 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4f9f40 0x55677d504390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a737fe640 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4fa930 0x55677d5048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a78c99640 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4ffaa0 0x55677d50b910 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0077e0 tx=0x7f5a6c007f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5962145906f9a2a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a537fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a537fe640 1 -- 172.21.15.12:0/326025985 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c007630 con 0x55677d4ffaa0 2024-05-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a537fe640 1 -- 172.21.15.12:0/326025985 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f5a6c013050 con 0x55677d4ffaa0 2024-05-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a7d409740 1 -- 172.21.15.12:0/326025985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a6c022070 con 0x55677d4ffaa0 2024-05-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a537fe640 1 -- 172.21.15.12:0/326025985 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5a6c01d810 con 0x55677d4ffaa0 2024-05-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a7d409740 10 client.16296 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a7d409740 1 -- 172.21.15.12:0/326025985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55677d50ec60 con 0x55677d4ffaa0 2024-05-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.940+0000 7f5a537fe640 1 -- 172.21.15.12:0/326025985 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a6c022070 con 0x55677d4ffaa0 2024-05-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.941+0000 7f5a537fe640 1 -- 172.21.15.12:0/326025985 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f5a6c003b50 con 0x55677d4ffaa0 2024-05-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.941+0000 7f5a7d409740 10 client.16296 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.941+0000 7f5a7d409740 10 client.16296 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.941+0000 7f5a7d409740 20 client.16296 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.941+0000 7f5a7d409740 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55677d510110 0x55677d512500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.941+0000 7f5a7d409740 4 client.16296 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.941+0000 7f5a7d409740 1 -- 172.21.15.12:0/326025985 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55677d46c370 con 0x55677d510110 2024-05-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.941+0000 7f5a73fff640 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55677d510110 0x55677d512500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.941+0000 7f5a537fe640 10 client.16296 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.941+0000 7f5a73fff640 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55677d510110 0x55677d512500 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b9df87cc9d1df41c server_cookie=922c224a56506a84 in_seq=0 out_seq=0 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a537fe640 1 -- 172.21.15.12:0/326025985 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55677d46c370 con 0x55677d510110 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a537fe640 10 client.16296 handle_command_reply: tid=0 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a7d409740 1 client.16296 shutdown 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a7d409740 1 -- 172.21.15.12:0/326025985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4ffaa0 msgr2=0x55677d50b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a7d409740 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4ffaa0 0x55677d50b910 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0077e0 tx=0x7f5a6c007f80 comp rx=0 tx=0).stop 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a7d409740 1 -- 172.21.15.12:0/326025985 shutdown_connections 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a7d409740 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55677d510110 0x55677d512500 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a7d409740 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4ffaa0 0x55677d50b910 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a7d409740 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4fa930 0x55677d5048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a537fe640 0 client.16296 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a7d409740 1 --2- 172.21.15.12:0/326025985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4f9f40 0x55677d504390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a7d409740 1 -- 172.21.15.12:0/326025985 >> 172.21.15.12:0/326025985 conn(0x55677d4f55b0 msgr2=0x55677d4f47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a7d409740 1 -- 172.21.15.12:0/326025985 shutdown_connections 2024-05-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a7d409740 1 -- 172.21.15.12:0/326025985 wait complete. 2024-05-09T09:27:18.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.942+0000 7f5a7d409740 20 client.16296 trim_cache size 0 max 16384 2024-05-09T09:27:18.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a7d409740 1 Processor -- start 2024-05-09T09:27:18.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a7d409740 1 -- start start 2024-05-09T09:27:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a7d409740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4f9f40 0x55677d4fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a7d409740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 0x55677d4fc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a7d409740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4ffaa0 0x55677d50b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a7d409740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741aa550 con 0x55677d4fa930 2024-05-09T09:27:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a7d409740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741ac120 con 0x55677d4ffaa0 2024-05-09T09:27:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a7d409740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741aac20 con 0x55677d4f9f40 2024-05-09T09:27:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 0x55677d4fc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 0x55677d4fc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38750/0 (socket says 172.21.15.12:38750) 2024-05-09T09:27:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a737fe640 1 -- 172.21.15.12:0/1946148769 learned_addr learned my addr 172.21.15.12:0/1946148769 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a78c99640 1 --2- 172.21.15.12:0/1946148769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4ffaa0 0x55677d50b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a73fff640 1 --2- 172.21.15.12:0/1946148769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4f9f40 0x55677d4fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a737fe640 1 -- 172.21.15.12:0/1946148769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4f9f40 msgr2=0x55677d4fbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a737fe640 1 --2- 172.21.15.12:0/1946148769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4f9f40 0x55677d4fbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a737fe640 1 -- 172.21.15.12:0/1946148769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4ffaa0 msgr2=0x55677d50b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a737fe640 1 --2- 172.21.15.12:0/1946148769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4ffaa0 0x55677d50b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a737fe640 1 -- 172.21.15.12:0/1946148769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55677d502ad0 con 0x55677d4fa930 2024-05-09T09:27:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.956+0000 7f5a73fff640 1 --2- 172.21.15.12:0/1946148769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4f9f40 0x55677d4fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:27:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.957+0000 7f5a737fe640 1 --2- 172.21.15.12:0/1946148769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 0x55677d4fc2d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5a680b8880 tx=0x7f5a680bb810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43383819332a1ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.957+0000 7f5a59ffb640 1 -- 172.21.15.12:0/1946148769 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68049050 con 0x55677d4fa930 2024-05-09T09:27:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.957+0000 7f5a59ffb640 1 -- 172.21.15.12:0/1946148769 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a680c01b0 con 0x55677d4fa930 2024-05-09T09:27:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.957+0000 7f5a59ffb640 1 -- 172.21.15.12:0/1946148769 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68003c70 con 0x55677d4fa930 2024-05-09T09:27:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.957+0000 7f5a7d409740 1 -- 172.21.15.12:0/1946148769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 msgr2=0x55677d4fc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.957+0000 7f5a7d409740 1 --2- 172.21.15.12:0/1946148769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 0x55677d4fc2d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5a680b8880 tx=0x7f5a680bb810 comp rx=0 tx=0).stop 2024-05-09T09:27:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.957+0000 7f5a7d409740 1 -- 172.21.15.12:0/1946148769 shutdown_connections 2024-05-09T09:27:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.957+0000 7f5a7d409740 1 --2- 172.21.15.12:0/1946148769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4ffaa0 0x55677d50b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.957+0000 7f5a7d409740 1 --2- 172.21.15.12:0/1946148769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 0x55677d4fc2d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.957+0000 7f5a7d409740 1 --2- 172.21.15.12:0/1946148769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4f9f40 0x55677d4fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.957+0000 7f5a7d409740 1 -- 172.21.15.12:0/1946148769 >> 172.21.15.12:0/1946148769 conn(0x55677d4f55b0 msgr2=0x55677d504390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.957+0000 7f5a7d409740 1 -- 172.21.15.12:0/1946148769 shutdown_connections 2024-05-09T09:27:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.957+0000 7f5a7d409740 1 -- 172.21.15.12:0/1946148769 wait complete. 2024-05-09T09:27:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.958+0000 7f5a7d409740 1 -- start start 2024-05-09T09:27:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.958+0000 7f5a7d409740 1 Processor -- start 2024-05-09T09:27:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.958+0000 7f5a7d409740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4f9f40 0x55677d508d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.958+0000 7f5a7d409740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 0x55677d5092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.958+0000 7f5a7d409740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4ffaa0 0x55677d50a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.958+0000 7f5a7d409740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741ab9c0 con 0x55677d4fa930 2024-05-09T09:27:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.958+0000 7f5a7d409740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741ab2f0 con 0x55677d4f9f40 2024-05-09T09:27:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.958+0000 7f5a7d409740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741a8530 con 0x55677d4ffaa0 2024-05-09T09:27:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.958+0000 7f5a7d409740 10 client.0 _ll_register_callbacks cb 0x7f5a6c0137d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.958+0000 7f5a737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 0x55677d5092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.958+0000 7f5a737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 0x55677d5092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38760/0 (socket says 172.21.15.12:38760) 2024-05-09T09:27:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.958+0000 7f5a737fe640 1 -- 172.21.15.12:0/179656500 learned_addr learned my addr 172.21.15.12:0/179656500 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.958+0000 7f5a73fff640 1 --2- 172.21.15.12:0/179656500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4f9f40 0x55677d508d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a78c99640 1 --2- 172.21.15.12:0/179656500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4ffaa0 0x55677d50a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a737fe640 1 -- 172.21.15.12:0/179656500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4ffaa0 msgr2=0x55677d50a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a737fe640 1 --2- 172.21.15.12:0/179656500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4ffaa0 0x55677d50a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a737fe640 1 -- 172.21.15.12:0/179656500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4f9f40 msgr2=0x55677d508d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a737fe640 1 --2- 172.21.15.12:0/179656500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4f9f40 0x55677d508d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a737fe640 1 -- 172.21.15.12:0/179656500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55677d50a8a0 con 0x55677d4fa930 2024-05-09T09:27:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a737fe640 1 --2- 172.21.15.12:0/179656500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 0x55677d5092a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5a680ba400 tx=0x7f5a680bb940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5c109d4f1dc4c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a537fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a537fe640 1 -- 172.21.15.12:0/179656500 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a680da050 con 0x55677d4fa930 2024-05-09T09:27:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a537fe640 1 -- 172.21.15.12:0/179656500 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f5a68004750 con 0x55677d4fa930 2024-05-09T09:27:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a7d409740 1 -- 172.21.15.12:0/179656500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a680c01b0 con 0x55677d4fa930 2024-05-09T09:27:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a537fe640 1 -- 172.21.15.12:0/179656500 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5a68049020 con 0x55677d4fa930 2024-05-09T09:27:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a7d409740 10 client.16302 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a7d409740 1 -- 172.21.15.12:0/179656500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55677d50bb80 con 0x55677d4fa930 2024-05-09T09:27:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a537fe640 1 -- 172.21.15.12:0/179656500 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a680c01b0 con 0x55677d4fa930 2024-05-09T09:27:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a537fe640 1 -- 172.21.15.12:0/179656500 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f5a680a2030 con 0x55677d4fa930 2024-05-09T09:27:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a7d409740 10 client.16302 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a7d409740 10 client.16302 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:27:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a7d409740 20 client.16302 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a7d409740 1 --2- 172.21.15.12:0/179656500 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55677d576d80 0x55677d5791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a7d409740 4 client.16302 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.959+0000 7f5a7d409740 1 -- 172.21.15.12:0/179656500 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55677d4666c0 con 0x55677d576d80 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a73fff640 1 --2- 172.21.15.12:0/179656500 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55677d576d80 0x55677d5791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a73fff640 1 --2- 172.21.15.12:0/179656500 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55677d576d80 0x55677d5791a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=43e2ef293ffc6915 server_cookie=832ea934cc1518c7 in_seq=0 out_seq=0 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a537fe640 10 client.16302 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a537fe640 1 -- 172.21.15.12:0/179656500 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55677d4666c0 con 0x55677d576d80 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a537fe640 10 client.16302 handle_command_reply: tid=0 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a7d409740 1 client.16302 shutdown 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a7d409740 1 -- 172.21.15.12:0/179656500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 msgr2=0x55677d5092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a7d409740 1 --2- 172.21.15.12:0/179656500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 0x55677d5092a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5a680ba400 tx=0x7f5a680bb940 comp rx=0 tx=0).stop 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a7d409740 1 -- 172.21.15.12:0/179656500 shutdown_connections 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a7d409740 1 --2- 172.21.15.12:0/179656500 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55677d576d80 0x55677d5791a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a7d409740 1 --2- 172.21.15.12:0/179656500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55677d4ffaa0 0x55677d50a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a7d409740 1 --2- 172.21.15.12:0/179656500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55677d4fa930 0x55677d5092a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a537fe640 0 client.16302 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:27:18.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a7d409740 1 --2- 172.21.15.12:0/179656500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55677d4f9f40 0x55677d508d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.960+0000 7f5a7d409740 1 -- 172.21.15.12:0/179656500 >> 172.21.15.12:0/179656500 conn(0x55677d4f55b0 msgr2=0x55677d504390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7d409740 1 -- 172.21.15.12:0/179656500 shutdown_connections 2024-05-09T09:27:18.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7d409740 1 -- 172.21.15.12:0/179656500 wait complete. 2024-05-09T09:27:18.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7d409740 20 client.16302 trim_cache size 0 max 16384 2024-05-09T09:27:18.971 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:27:18.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7a6fc640 1 -- 172.21.15.12:0/1290725410 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5a5c0a2c10 msgr2=0x7f5a5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7a6fc640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5a5c0a2c10 0x7f5a5c0a50d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c021000 tx=0x7f5a6c01f000 comp rx=0 tx=0).stop 2024-05-09T09:27:18.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7a6fc640 1 -- 172.21.15.12:0/1290725410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a74101980 msgr2=0x7f5a74196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:18.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7a6fc640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a74101980 0x7f5a74196f80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009a20 tx=0x7f5a68031ac0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7a6fc640 1 -- 172.21.15.12:0/1290725410 shutdown_connections 2024-05-09T09:27:18.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7a6fc640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5a5c0a2c10 0x7f5a5c0a50d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7a6fc640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a74102340 0x7f5a7419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7a6fc640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a74101980 0x7f5a74196f80 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7a6fc640 1 --2- 172.21.15.12:0/1290725410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a74100790 0x7f5a74196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:18.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7a6fc640 1 -- 172.21.15.12:0/1290725410 >> 172.21.15.12:0/1290725410 conn(0x7f5a740fbd20 msgr2=0x7f5a74104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:18.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7a6fc640 1 -- 172.21.15.12:0/1290725410 shutdown_connections 2024-05-09T09:27:18.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:18.961+0000 7f5a7a6fc640 1 -- 172.21.15.12:0/1290725410 wait complete. 2024-05-09T09:27:18.974 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:27:19.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.084+0000 7fbc04c53640 1 -- 172.21.15.12:0/1459021470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc00104cc0 msgr2=0x7fbc001070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.084+0000 7fbc04c53640 1 --2- 172.21.15.12:0/1459021470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc00104cc0 0x7fbc001070b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009900 tx=0x7fbbf402a410 comp rx=0 tx=0).stop 2024-05-09T09:27:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.084+0000 7fbc04c53640 1 -- 172.21.15.12:0/1459021470 shutdown_connections 2024-05-09T09:27:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.084+0000 7fbc04c53640 1 --2- 172.21.15.12:0/1459021470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc001075f0 0x7fbc001099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.084+0000 7fbc04c53640 1 --2- 172.21.15.12:0/1459021470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc00104cc0 0x7fbc001070b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.084+0000 7fbc04c53640 1 --2- 172.21.15.12:0/1459021470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc00102390 0x7fbc00104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.084+0000 7fbc04c53640 1 -- 172.21.15.12:0/1459021470 >> 172.21.15.12:0/1459021470 conn(0x7fbc000fbd20 msgr2=0x7fbc000fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.084+0000 7fbc04c53640 1 -- 172.21.15.12:0/1459021470 shutdown_connections 2024-05-09T09:27:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.084+0000 7fbc04c53640 1 -- 172.21.15.12:0/1459021470 wait complete. 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbc04c53640 1 Processor -- start 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbc04c53640 1 -- start start 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbc04c53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc00102390 0x7fbc00197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbc04c53640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc00104cc0 0x7fbc001980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbc04c53640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc001075f0 0x7fbc0019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbc04c53640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0010d520 con 0x7fbc00104cc0 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbc04c53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0010d3a0 con 0x7fbc00102390 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbc04c53640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0010d6a0 con 0x7fbc001075f0 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbbfdd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc00104cc0 0x7fbc001980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbbfdd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc00104cc0 0x7fbc001980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38778/0 (socket says 172.21.15.12:38778) 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbbfdd9b640 1 -- 172.21.15.12:0/2899847026 learned_addr learned my addr 172.21.15.12:0/2899847026 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbbfe59c640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc00102390 0x7fbc00197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbbfed9d640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc001075f0 0x7fbc0019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbbfdd9b640 1 -- 172.21.15.12:0/2899847026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc001075f0 msgr2=0x7fbc0019f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc001075f0 0x7fbc0019f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbbfdd9b640 1 -- 172.21.15.12:0/2899847026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc00102390 msgr2=0x7fbc00197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc00102390 0x7fbc00197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.085+0000 7fbbfdd9b640 1 -- 172.21.15.12:0/2899847026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0019f790 con 0x7fbc00104cc0 2024-05-09T09:27:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.086+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc00104cc0 0x7fbc001980b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009230 tx=0x7fbbf4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.086+0000 7fbbef7fe640 1 -- 172.21.15.12:0/2899847026 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf403f070 con 0x7fbc00104cc0 2024-05-09T09:27:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.086+0000 7fbc04c53640 1 -- 172.21.15.12:0/2899847026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0019fa20 con 0x7fbc00104cc0 2024-05-09T09:27:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.086+0000 7fbbef7fe640 1 -- 172.21.15.12:0/2899847026 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbbf4005620 con 0x7fbc00104cc0 2024-05-09T09:27:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.086+0000 7fbbef7fe640 1 -- 172.21.15.12:0/2899847026 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4035030 con 0x7fbc00104cc0 2024-05-09T09:27:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.086+0000 7fbc04c53640 1 -- 172.21.15.12:0/2899847026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0019ff60 con 0x7fbc00104cc0 2024-05-09T09:27:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.088+0000 7fbbef7fe640 1 -- 172.21.15.12:0/2899847026 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbbf4042070 con 0x7fbc00104cc0 2024-05-09T09:27:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.088+0000 7fbc07a34740 1 Processor -- start 2024-05-09T09:27:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.088+0000 7fbc07a34740 1 -- start start 2024-05-09T09:27:19.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.088+0000 7fbc07a34740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f979f40 0x564f4f97a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbc07a34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f97f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbc07a34740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 0x564f4f981f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbc07a34740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564f4f8dec40 con 0x564f4f97faa0 2024-05-09T09:27:19.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbc07a34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564f4f886580 con 0x564f4f97a930 2024-05-09T09:27:19.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbc07a34740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564f4f8dac00 con 0x564f4f979f40 2024-05-09T09:27:19.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbfdd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 0x564f4f981f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbfdd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 0x564f4f981f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38794/0 (socket says 172.21.15.12:38794) 2024-05-09T09:27:19.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbfdd9b640 1 -- 172.21.15.12:0/864458188 learned_addr learned my addr 172.21.15.12:0/864458188 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbfe59c640 1 --2- 172.21.15.12:0/864458188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f97f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbfed9d640 1 --2- 172.21.15.12:0/864458188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f979f40 0x564f4f97a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbfdd9b640 1 -- 172.21.15.12:0/864458188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f979f40 msgr2=0x564f4f97a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/864458188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f979f40 0x564f4f97a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbfdd9b640 1 -- 172.21.15.12:0/864458188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 msgr2=0x564f4f97f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/864458188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f97f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbfdd9b640 1 -- 172.21.15.12:0/864458188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4f5f23f0 con 0x564f4f97faa0 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbef7fe640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbbe80a2b90 0x7fbbe80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbfed9d640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbbe80a2b90 0x7fbbe80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/864458188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 0x564f4f981f30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40f3ef0 tx=0x7fbbf40f6450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827a06fa28da43a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.089+0000 7fbbef7fe640 1 -- 172.21.15.12:0/2899847026 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbbf40efdd0 con 0x7fbc00104cc0 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.090+0000 7fbbecff9640 1 -- 172.21.15.12:0/864458188 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf406ee60 con 0x564f4f97faa0 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.090+0000 7fbbecff9640 1 -- 172.21.15.12:0/864458188 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbbf40035f0 con 0x564f4f97faa0 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.090+0000 7fbbecff9640 1 -- 172.21.15.12:0/864458188 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40bc0e0 con 0x564f4f97faa0 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.090+0000 7fbc07a34740 1 -- 172.21.15.12:0/864458188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 msgr2=0x564f4f981f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.090+0000 7fbc07a34740 1 --2- 172.21.15.12:0/864458188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 0x564f4f981f30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40f3ef0 tx=0x7fbbf40f6450 comp rx=0 tx=0).stop 2024-05-09T09:27:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.090+0000 7fbbfed9d640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbbe80a2b90 0x7fbbe80a5050 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x564f4f97b190 tx=0x7fbbf8007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.090+0000 7fbc07a34740 1 -- 172.21.15.12:0/864458188 shutdown_connections 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.090+0000 7fbc07a34740 1 --2- 172.21.15.12:0/864458188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 0x564f4f981f30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.090+0000 7fbc07a34740 1 --2- 172.21.15.12:0/864458188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f97f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.090+0000 7fbc07a34740 1 --2- 172.21.15.12:0/864458188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f979f40 0x564f4f97a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.090+0000 7fbc07a34740 1 -- 172.21.15.12:0/864458188 >> 172.21.15.12:0/864458188 conn(0x564f4f9755b0 msgr2=0x564f4f9759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.090+0000 7fbc07a34740 1 -- 172.21.15.12:0/864458188 shutdown_connections 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.090+0000 7fbc07a34740 1 -- 172.21.15.12:0/864458188 wait complete. 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbc07a34740 1 -- start start 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbc07a34740 1 Processor -- start 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbc07a34740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 0x564f4f984090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbc07a34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f9845d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbc07a34740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f97faa0 0x564f4f98b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbc07a34740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564f4f982810 con 0x564f4f979f40 2024-05-09T09:27:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbc07a34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564f4f899c40 con 0x564f4f97a930 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbc07a34740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564f4f982990 con 0x564f4f97faa0 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbc07a34740 10 client.0 _ll_register_callbacks cb 0x564f4f5f1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbbfe59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 0x564f4f984090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbbfe59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 0x564f4f984090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38796/0 (socket says 172.21.15.12:38796) 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbbfe59c640 1 -- 172.21.15.12:0/288283959 learned_addr learned my addr 172.21.15.12:0/288283959 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f9845d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbbfed9d640 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f97faa0 0x564f4f98b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbbfe59c640 1 -- 172.21.15.12:0/288283959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f97faa0 msgr2=0x564f4f98b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbbfe59c640 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f97faa0 0x564f4f98b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbbfe59c640 1 -- 172.21.15.12:0/288283959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 msgr2=0x564f4f9845d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbbfe59c640 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f9845d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbbfe59c640 1 -- 172.21.15.12:0/288283959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4f6bc7d0 con 0x564f4f979f40 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbbfed9d640 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f97faa0 0x564f4f98b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:27:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.091+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f9845d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbbfe59c640 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 0x564f4f984090 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0010720 tx=0x7fbbf0010be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee8f2352349d283 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbbe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbbe5ffb640 1 -- 172.21.15.12:0/288283959 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf000fb30 con 0x564f4f979f40 2024-05-09T09:27:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbbe5ffb640 1 -- 172.21.15.12:0/288283959 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbbf000fcd0 con 0x564f4f979f40 2024-05-09T09:27:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbc07a34740 1 -- 172.21.15.12:0/288283959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564f4f8ec370 con 0x564f4f979f40 2024-05-09T09:27:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbbe5ffb640 1 -- 172.21.15.12:0/288283959 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbbf0024050 con 0x564f4f979f40 2024-05-09T09:27:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbc07a34740 10 client.16314 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbc07a34740 1 -- 172.21.15.12:0/288283959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564f4f98e9e0 con 0x564f4f979f40 2024-05-09T09:27:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbbe5ffb640 1 -- 172.21.15.12:0/288283959 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x564f4f8ec370 con 0x564f4f979f40 2024-05-09T09:27:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbbe5ffb640 1 -- 172.21.15.12:0/288283959 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fbbf000e4d0 con 0x564f4f979f40 2024-05-09T09:27:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbc07a34740 10 client.16314 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbc07a34740 10 client.16314 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:27:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbc07a34740 20 client.16314 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbc07a34740 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x564f4f990120 0x564f4f9925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbc07a34740 4 client.16314 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.092+0000 7fbc07a34740 1 -- 172.21.15.12:0/288283959 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f4f8e66c0 con 0x564f4f990120 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.093+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x564f4f990120 0x564f4f9925e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.093+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x564f4f990120 0x564f4f9925e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0ed65927f9c061c server_cookie=87e2bf684f7fe2fd in_seq=0 out_seq=0 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.093+0000 7fbbe5ffb640 10 client.16314 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbbe5ffb640 1 -- 172.21.15.12:0/288283959 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fbbf40a1ae0 con 0x564f4f990120 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbbe5ffb640 10 client.16314 handle_command_reply: tid=0 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbc07a34740 1 client.16314 shutdown 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbc07a34740 1 -- 172.21.15.12:0/288283959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 msgr2=0x564f4f984090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbc07a34740 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 0x564f4f984090 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0010720 tx=0x7fbbf0010be0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbc07a34740 1 -- 172.21.15.12:0/288283959 shutdown_connections 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbc07a34740 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x564f4f990120 0x564f4f9925e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbc07a34740 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f97faa0 0x564f4f98b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbbe5ffb640 0 client.16314 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:27:19.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbc07a34740 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f9845d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbc07a34740 1 --2- 172.21.15.12:0/288283959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 0x564f4f984090 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbc07a34740 1 -- 172.21.15.12:0/288283959 >> 172.21.15.12:0/288283959 conn(0x564f4f9755b0 msgr2=0x564f4f9747b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbc07a34740 1 -- 172.21.15.12:0/288283959 shutdown_connections 2024-05-09T09:27:19.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbc07a34740 1 -- 172.21.15.12:0/288283959 wait complete. 2024-05-09T09:27:19.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.094+0000 7fbc07a34740 20 client.16314 trim_cache size 0 max 16384 2024-05-09T09:27:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbc07a34740 1 Processor -- start 2024-05-09T09:27:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbc07a34740 1 -- start start 2024-05-09T09:27:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbc07a34740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f979f40 0x564f4f984390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbc07a34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f98b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbc07a34740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 0x564f4f978c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbc07a34740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001aa5e0 con 0x564f4f97faa0 2024-05-09T09:27:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbc07a34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001ac1b0 con 0x564f4f97a930 2024-05-09T09:27:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbc07a34740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001aacb0 con 0x564f4f979f40 2024-05-09T09:27:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbbfed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 0x564f4f978c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbbfed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 0x564f4f978c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38808/0 (socket says 172.21.15.12:38808) 2024-05-09T09:27:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbbfed9d640 1 -- 172.21.15.12:0/2078617576 learned_addr learned my addr 172.21.15.12:0/2078617576 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbbfe59c640 1 --2- 172.21.15.12:0/2078617576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f979f40 0x564f4f984390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/2078617576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f98b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbbfed9d640 1 -- 172.21.15.12:0/2078617576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f979f40 msgr2=0x564f4f984390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbbfed9d640 1 --2- 172.21.15.12:0/2078617576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f979f40 0x564f4f984390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbbfed9d640 1 -- 172.21.15.12:0/2078617576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 msgr2=0x564f4f98b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbbfed9d640 1 --2- 172.21.15.12:0/2078617576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f98b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbbfed9d640 1 -- 172.21.15.12:0/2078617576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4f97efc0 con 0x564f4f97faa0 2024-05-09T09:27:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbbfe59c640 1 --2- 172.21.15.12:0/2078617576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f979f40 0x564f4f984390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.108+0000 7fbbfed9d640 1 --2- 172.21.15.12:0/2078617576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 0x564f4f978c40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8003c10 tx=0x7fbbf8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ed83c0bf17898 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.109+0000 7fbbe7fff640 1 -- 172.21.15.12:0/2078617576 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8027670 con 0x564f4f97faa0 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.109+0000 7fbbe7fff640 1 -- 172.21.15.12:0/2078617576 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbbf8022070 con 0x564f4f97faa0 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.109+0000 7fbbe7fff640 1 -- 172.21.15.12:0/2078617576 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf801ec50 con 0x564f4f97faa0 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.109+0000 7fbc07a34740 1 -- 172.21.15.12:0/2078617576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 msgr2=0x564f4f978c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.109+0000 7fbc07a34740 1 --2- 172.21.15.12:0/2078617576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 0x564f4f978c40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8003c10 tx=0x7fbbf8014000 comp rx=0 tx=0).stop 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.109+0000 7fbc07a34740 1 -- 172.21.15.12:0/2078617576 shutdown_connections 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.109+0000 7fbc07a34740 1 --2- 172.21.15.12:0/2078617576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f97faa0 0x564f4f978c40 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.109+0000 7fbc07a34740 1 --2- 172.21.15.12:0/2078617576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f98b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.109+0000 7fbc07a34740 1 --2- 172.21.15.12:0/2078617576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f979f40 0x564f4f984390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.109+0000 7fbc07a34740 1 -- 172.21.15.12:0/2078617576 >> 172.21.15.12:0/2078617576 conn(0x564f4f9755b0 msgr2=0x564f4f978530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.109+0000 7fbc07a34740 1 -- 172.21.15.12:0/2078617576 shutdown_connections 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.109+0000 7fbc07a34740 1 -- 172.21.15.12:0/2078617576 wait complete. 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbc07a34740 1 -- start start 2024-05-09T09:27:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbc07a34740 1 Processor -- start 2024-05-09T09:27:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbc07a34740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 0x564f4f988c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbc07a34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f989190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbc07a34740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f97faa0 0x564f4f98a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbc07a34740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001aba50 con 0x564f4f979f40 2024-05-09T09:27:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbc07a34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001ab380 con 0x564f4f97a930 2024-05-09T09:27:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbc07a34740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001a85c0 con 0x564f4f97faa0 2024-05-09T09:27:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbc07a34740 10 client.0 _ll_register_callbacks cb 0x7fbbf0005b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbbfe59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 0x564f4f988c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbbfe59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 0x564f4f988c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38818/0 (socket says 172.21.15.12:38818) 2024-05-09T09:27:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbbfe59c640 1 -- 172.21.15.12:0/1503468411 learned_addr learned my addr 172.21.15.12:0/1503468411 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/1503468411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f989190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbbfe59c640 1 -- 172.21.15.12:0/1503468411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f97faa0 msgr2=0x564f4f98a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbbfe59c640 1 --2- 172.21.15.12:0/1503468411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f97faa0 0x564f4f98a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbbfe59c640 1 -- 172.21.15.12:0/1503468411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 msgr2=0x564f4f989190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbbfe59c640 1 --2- 172.21.15.12:0/1503468411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f989190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbbfe59c640 1 -- 172.21.15.12:0/1503468411 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4f98a790 con 0x564f4f979f40 2024-05-09T09:27:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbbfe59c640 1 --2- 172.21.15.12:0/1503468411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 0x564f4f988c50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0006010 tx=0x7fbbf000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50633e0a8570a797 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.110+0000 7fbbe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbbe5ffb640 1 -- 172.21.15.12:0/1503468411 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf002c070 con 0x564f4f979f40 2024-05-09T09:27:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbbe5ffb640 1 -- 172.21.15.12:0/1503468411 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbbf000fd40 con 0x564f4f979f40 2024-05-09T09:27:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbc07a34740 1 -- 172.21.15.12:0/1503468411 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbf8022070 con 0x564f4f979f40 2024-05-09T09:27:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbbe5ffb640 1 -- 172.21.15.12:0/1503468411 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbbf0038050 con 0x564f4f979f40 2024-05-09T09:27:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbc07a34740 10 client.16320 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbc07a34740 1 -- 172.21.15.12:0/1503468411 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564f4f9f6790 con 0x564f4f979f40 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbbe5ffb640 1 -- 172.21.15.12:0/1503468411 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbbf8022070 con 0x564f4f979f40 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbbe5ffb640 1 -- 172.21.15.12:0/1503468411 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fbbf0031a50 con 0x564f4f979f40 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbc07a34740 10 client.16320 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbc07a34740 10 client.16320 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbc07a34740 20 client.16320 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbc07a34740 1 --2- 172.21.15.12:0/1503468411 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x564f4f9f7160 0x564f4f9f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbc07a34740 4 client.16320 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbc07a34740 1 -- 172.21.15.12:0/1503468411 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f4f863820 con 0x564f4f9f7160 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.111+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/1503468411 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x564f4f9f7160 0x564f4f9f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbbe5ffb640 10 client.16320 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbbfdd9b640 1 --2- 172.21.15.12:0/1503468411 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x564f4f9f7160 0x564f4f9f9580 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1aab3743530058e server_cookie=5456095ae7a9a269 in_seq=0 out_seq=0 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbbe5ffb640 1 -- 172.21.15.12:0/1503468411 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f4f863820 con 0x564f4f9f7160 2024-05-09T09:27:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbbe5ffb640 10 client.16320 handle_command_reply: tid=0 2024-05-09T09:27:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbc07a34740 1 client.16320 shutdown 2024-05-09T09:27:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbc07a34740 1 -- 172.21.15.12:0/1503468411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 msgr2=0x564f4f988c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbc07a34740 1 --2- 172.21.15.12:0/1503468411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 0x564f4f988c50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0006010 tx=0x7fbbf000ec00 comp rx=0 tx=0).stop 2024-05-09T09:27:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbc07a34740 1 -- 172.21.15.12:0/1503468411 shutdown_connections 2024-05-09T09:27:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbc07a34740 1 --2- 172.21.15.12:0/1503468411 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x564f4f9f7160 0x564f4f9f9580 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbc07a34740 1 --2- 172.21.15.12:0/1503468411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4f97faa0 0x564f4f98a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbc07a34740 1 --2- 172.21.15.12:0/1503468411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f4f97a930 0x564f4f989190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbbe5ffb640 0 client.16320 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:27:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbc07a34740 1 --2- 172.21.15.12:0/1503468411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f4f979f40 0x564f4f988c50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbc07a34740 1 -- 172.21.15.12:0/1503468411 >> 172.21.15.12:0/1503468411 conn(0x564f4f9755b0 msgr2=0x564f4f978dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbc07a34740 1 -- 172.21.15.12:0/1503468411 shutdown_connections 2024-05-09T09:27:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.112+0000 7fbc07a34740 1 -- 172.21.15.12:0/1503468411 wait complete. 2024-05-09T09:27:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.113+0000 7fbc07a34740 20 client.16320 trim_cache size 0 max 16384 2024-05-09T09:27:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:27:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.113+0000 7fbc04c53640 1 -- 172.21.15.12:0/2899847026 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbbe80a2b90 msgr2=0x7fbbe80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.113+0000 7fbc04c53640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbbe80a2b90 0x7fbbe80a5050 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x564f4f97b190 tx=0x7fbbf8007520 comp rx=0 tx=0).stop 2024-05-09T09:27:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.113+0000 7fbc04c53640 1 -- 172.21.15.12:0/2899847026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc00104cc0 msgr2=0x7fbc001980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.113+0000 7fbc04c53640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc00104cc0 0x7fbc001980b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009230 tx=0x7fbbf4005020 comp rx=0 tx=0).stop 2024-05-09T09:27:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.113+0000 7fbc04c53640 1 -- 172.21.15.12:0/2899847026 shutdown_connections 2024-05-09T09:27:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.113+0000 7fbc04c53640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbbe80a2b90 0x7fbbe80a5050 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.113+0000 7fbc04c53640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc001075f0 0x7fbc0019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.113+0000 7fbc04c53640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc00104cc0 0x7fbc001980b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.113+0000 7fbc04c53640 1 --2- 172.21.15.12:0/2899847026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc00102390 0x7fbc00197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.113+0000 7fbc04c53640 1 -- 172.21.15.12:0/2899847026 >> 172.21.15.12:0/2899847026 conn(0x7fbc000fbd20 msgr2=0x7fbc00102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.113+0000 7fbc04c53640 1 -- 172.21.15.12:0/2899847026 shutdown_connections 2024-05-09T09:27:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.113+0000 7fbc04c53640 1 -- 172.21.15.12:0/2899847026 wait complete. 2024-05-09T09:27:19.124 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:27:19.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.232+0000 7fb38313b640 1 -- 172.21.15.12:0/2533273584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb37c104c80 msgr2=0x7fb37c107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.232+0000 7fb38313b640 1 --2- 172.21.15.12:0/2533273584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb37c104c80 0x7fb37c107070 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb368009900 tx=0x7fb36802a410 comp rx=0 tx=0).stop 2024-05-09T09:27:19.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.232+0000 7fb38313b640 1 -- 172.21.15.12:0/2533273584 shutdown_connections 2024-05-09T09:27:19.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.232+0000 7fb38313b640 1 --2- 172.21.15.12:0/2533273584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37c1075b0 0x7fb37c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.232+0000 7fb38313b640 1 --2- 172.21.15.12:0/2533273584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb37c104c80 0x7fb37c107070 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.232+0000 7fb38313b640 1 --2- 172.21.15.12:0/2533273584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb37c102350 0x7fb37c104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.232+0000 7fb38313b640 1 -- 172.21.15.12:0/2533273584 >> 172.21.15.12:0/2533273584 conn(0x7fb37c0fbce0 msgr2=0x7fb37c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.232+0000 7fb38313b640 1 -- 172.21.15.12:0/2533273584 shutdown_connections 2024-05-09T09:27:19.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.232+0000 7fb38313b640 1 -- 172.21.15.12:0/2533273584 wait complete. 2024-05-09T09:27:19.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb38313b640 1 Processor -- start 2024-05-09T09:27:19.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb38313b640 1 -- start start 2024-05-09T09:27:19.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb38313b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37c102350 0x7fb37c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb38313b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb37c104c80 0x7fb37c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb38313b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb37c1075b0 0x7fb37c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb38313b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c10d4b0 con 0x7fb37c104c80 2024-05-09T09:27:19.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb38313b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c10d330 con 0x7fb37c102350 2024-05-09T09:27:19.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb38313b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c10d630 con 0x7fb37c1075b0 2024-05-09T09:27:19.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb37bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb37c104c80 0x7fb37c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb37bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb37c104c80 0x7fb37c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38838/0 (socket says 172.21.15.12:38838) 2024-05-09T09:27:19.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb37bfff640 1 -- 172.21.15.12:0/2866521361 learned_addr learned my addr 172.21.15.12:0/2866521361 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb380ed7640 1 --2- 172.21.15.12:0/2866521361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37c102350 0x7fb37c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb37bfff640 1 -- 172.21.15.12:0/2866521361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb37c1075b0 msgr2=0x7fb37c19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.233+0000 7fb37bfff640 1 --2- 172.21.15.12:0/2866521361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb37c1075b0 0x7fb37c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.234+0000 7fb37bfff640 1 -- 172.21.15.12:0/2866521361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37c102350 msgr2=0x7fb37c197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.234+0000 7fb37bfff640 1 --2- 172.21.15.12:0/2866521361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37c102350 0x7fb37c197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.234+0000 7fb37bfff640 1 -- 172.21.15.12:0/2866521361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c19f800 con 0x7fb37c104c80 2024-05-09T09:27:19.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.234+0000 7fb37bfff640 1 --2- 172.21.15.12:0/2866521361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb37c104c80 0x7fb37c198120 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb368005ab0 tx=0x7fb3680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.234+0000 7fb379ffb640 1 -- 172.21.15.12:0/2866521361 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36803f030 con 0x7fb37c104c80 2024-05-09T09:27:19.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.234+0000 7fb379ffb640 1 -- 172.21.15.12:0/2866521361 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb368003ac0 con 0x7fb37c104c80 2024-05-09T09:27:19.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.234+0000 7fb38313b640 1 -- 172.21.15.12:0/2866521361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c19fa90 con 0x7fb37c104c80 2024-05-09T09:27:19.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.234+0000 7fb379ffb640 1 -- 172.21.15.12:0/2866521361 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368035070 con 0x7fb37c104c80 2024-05-09T09:27:19.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.234+0000 7fb38313b640 1 -- 172.21.15.12:0/2866521361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c19ffd0 con 0x7fb37c104c80 2024-05-09T09:27:19.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:18 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1880569615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:27:19.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb379ffb640 1 -- 172.21.15.12:0/2866521361 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb368042030 con 0x7fb37c104c80 2024-05-09T09:27:19.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb385e46740 1 Processor -- start 2024-05-09T09:27:19.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb385e46740 1 -- start start 2024-05-09T09:27:19.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb385e46740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 0x555ed778d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb385e46740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed7792430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb385e46740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed7792aa0 0x555ed7794f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb385e46740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555ed76f1c40 con 0x555ed778d930 2024-05-09T09:27:19.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb385e46740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555ed7699580 con 0x555ed7792aa0 2024-05-09T09:27:19.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb385e46740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555ed76edc00 con 0x555ed778cf40 2024-05-09T09:27:19.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb380ed7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed7792430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb380ed7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed7792430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38840/0 (socket says 172.21.15.12:38840) 2024-05-09T09:27:19.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb380ed7640 1 -- 172.21.15.12:0/1529572793 learned_addr learned my addr 172.21.15.12:0/1529572793 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb3816d8640 1 --2- 172.21.15.12:0/1529572793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 0x555ed778d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb37bfff640 1 --2- 172.21.15.12:0/1529572793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed7792aa0 0x555ed7794f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb379ffb640 1 --2- 172.21.15.12:0/2866521361 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb36c0a2c10 0x7fb36c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb380ed7640 1 -- 172.21.15.12:0/1529572793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 msgr2=0x555ed778d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.237+0000 7fb380ed7640 1 --2- 172.21.15.12:0/1529572793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 0x555ed778d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.238+0000 7fb380ed7640 1 -- 172.21.15.12:0/1529572793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed7792aa0 msgr2=0x555ed7794f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.238+0000 7fb380ed7640 1 --2- 172.21.15.12:0/1529572793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed7792aa0 0x555ed7794f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.238+0000 7fb380ed7640 1 -- 172.21.15.12:0/1529572793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ed74053f0 con 0x555ed778d930 2024-05-09T09:27:19.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.238+0000 7fb3816d8640 1 --2- 172.21.15.12:0/2866521361 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb36c0a2c10 0x7fb36c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.238+0000 7fb379ffb640 1 -- 172.21.15.12:0/2866521361 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fb3680efa50 con 0x7fb37c104c80 2024-05-09T09:27:19.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.238+0000 7fb380ed7640 1 --2- 172.21.15.12:0/1529572793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed7792430 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb37000bd10 tx=0x7fb37000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa2a41f2d04ab2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.238+0000 7fb366ffd640 1 -- 172.21.15.12:0/1529572793 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370015070 con 0x555ed778d930 2024-05-09T09:27:19.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.238+0000 7fb366ffd640 1 -- 172.21.15.12:0/1529572793 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb37001a930 con 0x555ed778d930 2024-05-09T09:27:19.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.238+0000 7fb366ffd640 1 -- 172.21.15.12:0/1529572793 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370011a00 con 0x555ed778d930 2024-05-09T09:27:19.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 -- 172.21.15.12:0/1529572793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 msgr2=0x555ed7792430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb3816d8640 1 --2- 172.21.15.12:0/2866521361 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb36c0a2c10 0x7fb36c0a50d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x555ed778e190 tx=0x7fb374004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 --2- 172.21.15.12:0/1529572793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed7792430 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb37000bd10 tx=0x7fb37000f730 comp rx=0 tx=0).stop 2024-05-09T09:27:19.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 -- 172.21.15.12:0/1529572793 shutdown_connections 2024-05-09T09:27:19.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 --2- 172.21.15.12:0/1529572793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed7792aa0 0x555ed7794f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 --2- 172.21.15.12:0/1529572793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed7792430 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 --2- 172.21.15.12:0/1529572793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 0x555ed778d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 -- 172.21.15.12:0/1529572793 >> 172.21.15.12:0/1529572793 conn(0x555ed77885b0 msgr2=0x555ed77889b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 -- 172.21.15.12:0/1529572793 shutdown_connections 2024-05-09T09:27:19.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 -- 172.21.15.12:0/1529572793 wait complete. 2024-05-09T09:27:19.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 -- start start 2024-05-09T09:27:19.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 Processor -- start 2024-05-09T09:27:19.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 0x555ed7797390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed778d930 0x555ed77978d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed7792aa0 0x555ed779e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555ed7795a50 con 0x555ed7792aa0 2024-05-09T09:27:19.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555ed76acc40 con 0x555ed778d930 2024-05-09T09:27:19.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555ed7795bd0 con 0x555ed778cf40 2024-05-09T09:27:19.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.239+0000 7fb385e46740 10 client.0 _ll_register_callbacks cb 0x555ed7404be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:19.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb3816d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed7792aa0 0x555ed779e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb3816d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed7792aa0 0x555ed779e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38848/0 (socket says 172.21.15.12:38848) 2024-05-09T09:27:19.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb3816d8640 1 -- 172.21.15.12:0/3052544255 learned_addr learned my addr 172.21.15.12:0/3052544255 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb37bfff640 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed778d930 0x555ed77978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb380ed7640 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 0x555ed7797390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb3816d8640 1 -- 172.21.15.12:0/3052544255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 msgr2=0x555ed7797390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb3816d8640 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 0x555ed7797390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb3816d8640 1 -- 172.21.15.12:0/3052544255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed778d930 msgr2=0x555ed77978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb3816d8640 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed778d930 0x555ed77978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb3816d8640 1 -- 172.21.15.12:0/3052544255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ed74cf7d0 con 0x555ed7792aa0 2024-05-09T09:27:19.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb380ed7640 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 0x555ed7797390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:19.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb3816d8640 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed7792aa0 0x555ed779e910 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb374013c50 tx=0x7fb374009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774f207c7a80dc1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb35bfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:19.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb35bfff640 1 -- 172.21.15.12:0/3052544255 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37400fbc0 con 0x555ed7792aa0 2024-05-09T09:27:19.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb35bfff640 1 -- 172.21.15.12:0/3052544255 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fb37400fd60 con 0x555ed7792aa0 2024-05-09T09:27:19.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb385e46740 1 -- 172.21.15.12:0/3052544255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb37001a930 con 0x555ed7792aa0 2024-05-09T09:27:19.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb35bfff640 1 -- 172.21.15.12:0/3052544255 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb37402f050 con 0x555ed7792aa0 2024-05-09T09:27:19.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb385e46740 10 client.16332 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:19.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.240+0000 7fb385e46740 1 -- 172.21.15.12:0/3052544255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555ed77a1d50 con 0x555ed7792aa0 2024-05-09T09:27:19.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.241+0000 7fb35bfff640 1 -- 172.21.15.12:0/3052544255 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb37001a930 con 0x555ed7792aa0 2024-05-09T09:27:19.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.241+0000 7fb35bfff640 1 -- 172.21.15.12:0/3052544255 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fb37401d540 con 0x555ed7792aa0 2024-05-09T09:27:19.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.241+0000 7fb385e46740 10 client.16332 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:19.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.241+0000 7fb385e46740 10 client.16332 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:27:19.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.241+0000 7fb385e46740 20 client.16332 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:19.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.241+0000 7fb385e46740 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555ed77a34a0 0x555ed77a5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.241+0000 7fb385e46740 4 client.16332 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:19.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.241+0000 7fb385e46740 1 -- 172.21.15.12:0/3052544255 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ed76ff370 con 0x555ed77a34a0 2024-05-09T09:27:19.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.241+0000 7fb380ed7640 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555ed77a34a0 0x555ed77a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.242+0000 7fb380ed7640 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555ed77a34a0 0x555ed77a5960 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25fc5d181e96db33 server_cookie=b9540c34948d2ff7 in_seq=0 out_seq=0 2024-05-09T09:27:19.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.242+0000 7fb35bfff640 10 client.16332 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:27:19.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.242+0000 7fb35bfff640 1 -- 172.21.15.12:0/3052544255 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ed76ff370 con 0x555ed77a34a0 2024-05-09T09:27:19.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.242+0000 7fb35bfff640 10 client.16332 handle_command_reply: tid=0 2024-05-09T09:27:19.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.242+0000 7fb385e46740 1 client.16332 shutdown 2024-05-09T09:27:19.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.243+0000 7fb385e46740 1 -- 172.21.15.12:0/3052544255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed7792aa0 msgr2=0x555ed779e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.243+0000 7fb385e46740 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed7792aa0 0x555ed779e910 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb374013c50 tx=0x7fb374009500 comp rx=0 tx=0).stop 2024-05-09T09:27:19.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.243+0000 7fb385e46740 1 -- 172.21.15.12:0/3052544255 shutdown_connections 2024-05-09T09:27:19.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.243+0000 7fb385e46740 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555ed77a34a0 0x555ed77a5960 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.243+0000 7fb385e46740 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed7792aa0 0x555ed779e910 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.243+0000 7fb385e46740 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed778d930 0x555ed77978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.243+0000 7fb35bfff640 0 client.16332 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:27:19.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.243+0000 7fb385e46740 1 --2- 172.21.15.12:0/3052544255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 0x555ed7797390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.243+0000 7fb385e46740 1 -- 172.21.15.12:0/3052544255 >> 172.21.15.12:0/3052544255 conn(0x555ed77885b0 msgr2=0x555ed77877b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.243+0000 7fb385e46740 1 -- 172.21.15.12:0/3052544255 shutdown_connections 2024-05-09T09:27:19.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.243+0000 7fb385e46740 1 -- 172.21.15.12:0/3052544255 wait complete. 2024-05-09T09:27:19.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.243+0000 7fb385e46740 20 client.16332 trim_cache size 0 max 16384 2024-05-09T09:27:19.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:19 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1880569615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:27:19.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.256+0000 7fb385e46740 1 Processor -- start 2024-05-09T09:27:19.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.256+0000 7fb385e46740 1 -- start start 2024-05-09T09:27:19.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.256+0000 7fb385e46740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed778cf40 0x555ed7787630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.256+0000 7fb385e46740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed7787b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.256+0000 7fb385e46740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed7792aa0 0x555ed779e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.256+0000 7fb385e46740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1aa6f0 con 0x555ed778d930 2024-05-09T09:27:19.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.256+0000 7fb385e46740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1ac2c0 con 0x555ed778cf40 2024-05-09T09:27:19.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.256+0000 7fb385e46740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1aadc0 con 0x555ed7792aa0 2024-05-09T09:27:19.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.256+0000 7fb37bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed7787b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.257+0000 7fb37bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed7787b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38850/0 (socket says 172.21.15.12:38850) 2024-05-09T09:27:19.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.257+0000 7fb37bfff640 1 -- 172.21.15.12:0/2628922674 learned_addr learned my addr 172.21.15.12:0/2628922674 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.257+0000 7fb380ed7640 1 --2- 172.21.15.12:0/2628922674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed778cf40 0x555ed7787630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.257+0000 7fb3816d8640 1 --2- 172.21.15.12:0/2628922674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed7792aa0 0x555ed779e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.257+0000 7fb37bfff640 1 -- 172.21.15.12:0/2628922674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed7792aa0 msgr2=0x555ed779e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.257+0000 7fb37bfff640 1 --2- 172.21.15.12:0/2628922674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed7792aa0 0x555ed779e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.257+0000 7fb37bfff640 1 -- 172.21.15.12:0/2628922674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed778cf40 msgr2=0x555ed7787630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.257+0000 7fb37bfff640 1 --2- 172.21.15.12:0/2628922674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed778cf40 0x555ed7787630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.257+0000 7fb37bfff640 1 -- 172.21.15.12:0/2628922674 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ed7788150 con 0x555ed778d930 2024-05-09T09:27:19.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.257+0000 7fb37bfff640 1 --2- 172.21.15.12:0/2628922674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed7787b70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb3680ed180 tx=0x7fb368003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d523d8db74586d11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.257+0000 7fb3667fc640 1 -- 172.21.15.12:0/2628922674 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368049040 con 0x555ed778d930 2024-05-09T09:27:19.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.257+0000 7fb3667fc640 1 -- 172.21.15.12:0/2628922674 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb3680bc5e0 con 0x555ed778d930 2024-05-09T09:27:19.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.257+0000 7fb3667fc640 1 -- 172.21.15.12:0/2628922674 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3680b8550 con 0x555ed778d930 2024-05-09T09:27:19.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.258+0000 7fb385e46740 1 -- 172.21.15.12:0/2628922674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 msgr2=0x555ed7787b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.258+0000 7fb385e46740 1 --2- 172.21.15.12:0/2628922674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed7787b70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb3680ed180 tx=0x7fb368003a50 comp rx=0 tx=0).stop 2024-05-09T09:27:19.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.258+0000 7fb385e46740 1 -- 172.21.15.12:0/2628922674 shutdown_connections 2024-05-09T09:27:19.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.258+0000 7fb385e46740 1 --2- 172.21.15.12:0/2628922674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed7792aa0 0x555ed779e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.258+0000 7fb385e46740 1 --2- 172.21.15.12:0/2628922674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed7787b70 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.258+0000 7fb385e46740 1 --2- 172.21.15.12:0/2628922674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed778cf40 0x555ed7787630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.258+0000 7fb385e46740 1 -- 172.21.15.12:0/2628922674 >> 172.21.15.12:0/2628922674 conn(0x555ed77885b0 msgr2=0x555ed7786f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.258+0000 7fb385e46740 1 -- 172.21.15.12:0/2628922674 shutdown_connections 2024-05-09T09:27:19.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.258+0000 7fb385e46740 1 -- 172.21.15.12:0/2628922674 wait complete. 2024-05-09T09:27:19.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.258+0000 7fb385e46740 1 -- start start 2024-05-09T09:27:19.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.258+0000 7fb385e46740 1 Processor -- start 2024-05-09T09:27:19.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb385e46740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 0x555ed779c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb385e46740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed779c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb385e46740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed7792aa0 0x555ed779d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb385e46740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1abb60 con 0x555ed778d930 2024-05-09T09:27:19.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb385e46740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1ab490 con 0x555ed7792aa0 2024-05-09T09:27:19.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb385e46740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1a86d0 con 0x555ed778cf40 2024-05-09T09:27:19.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb385e46740 10 client.0 _ll_register_callbacks cb 0x7fb374009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:19.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb37bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed779c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb37bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed779c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38862/0 (socket says 172.21.15.12:38862) 2024-05-09T09:27:19.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb37bfff640 1 -- 172.21.15.12:0/3869636177 learned_addr learned my addr 172.21.15.12:0/3869636177 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb380ed7640 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 0x555ed779c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb3816d8640 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed7792aa0 0x555ed779d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb37bfff640 1 -- 172.21.15.12:0/3869636177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 msgr2=0x555ed779c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb37bfff640 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 0x555ed779c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb37bfff640 1 -- 172.21.15.12:0/3869636177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed7792aa0 msgr2=0x555ed779d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb37bfff640 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed7792aa0 0x555ed779d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb37bfff640 1 -- 172.21.15.12:0/3869636177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ed779dda0 con 0x555ed778d930 2024-05-09T09:27:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb3816d8640 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed7792aa0 0x555ed779d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:27:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb37bfff640 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed779c840 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb3680c4620 tx=0x7fb368005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82efb8ee84f2e62 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb35bfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:19.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb35bfff640 1 -- 172.21.15.12:0/3869636177 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368049070 con 0x555ed778d930 2024-05-09T09:27:19.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb35bfff640 1 -- 172.21.15.12:0/3869636177 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fb3680b8550 con 0x555ed778d930 2024-05-09T09:27:19.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.259+0000 7fb385e46740 1 -- 172.21.15.12:0/3869636177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3680bc5e0 con 0x555ed778d930 2024-05-09T09:27:19.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.260+0000 7fb35bfff640 1 -- 172.21.15.12:0/3869636177 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb3680c0290 con 0x555ed778d930 2024-05-09T09:27:19.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.260+0000 7fb385e46740 10 client.16338 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:19.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.260+0000 7fb385e46740 1 -- 172.21.15.12:0/3869636177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555ed7809d30 con 0x555ed778d930 2024-05-09T09:27:19.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.260+0000 7fb35bfff640 1 -- 172.21.15.12:0/3869636177 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb3680bc5e0 con 0x555ed778d930 2024-05-09T09:27:19.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.260+0000 7fb35bfff640 1 -- 172.21.15.12:0/3869636177 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fb3680c47c0 con 0x555ed778d930 2024-05-09T09:27:19.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.260+0000 7fb385e46740 10 client.16338 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:19.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.260+0000 7fb385e46740 10 client.16338 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:27:19.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.260+0000 7fb385e46740 20 client.16338 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:19.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.260+0000 7fb385e46740 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555ed780a680 0x555ed780caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.260+0000 7fb385e46740 4 client.16338 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:27:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.260+0000 7fb385e46740 1 -- 172.21.15.12:0/3869636177 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ed76f96c0 con 0x555ed780a680 2024-05-09T09:27:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.260+0000 7fb380ed7640 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555ed780a680 0x555ed780caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb380ed7640 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555ed780a680 0x555ed780caa0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cd56f22cd91cad1 server_cookie=88e0cb78191b90a2 in_seq=0 out_seq=0 2024-05-09T09:27:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb35bfff640 10 client.16338 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:27:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb35bfff640 1 -- 172.21.15.12:0/3869636177 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ed76f96c0 con 0x555ed780a680 2024-05-09T09:27:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb35bfff640 10 client.16338 handle_command_reply: tid=0 2024-05-09T09:27:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb385e46740 1 client.16338 shutdown 2024-05-09T09:27:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb385e46740 1 -- 172.21.15.12:0/3869636177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 msgr2=0x555ed779c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb385e46740 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed779c840 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb3680c4620 tx=0x7fb368005bc0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb385e46740 1 -- 172.21.15.12:0/3869636177 shutdown_connections 2024-05-09T09:27:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb385e46740 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555ed780a680 0x555ed780caa0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb385e46740 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ed7792aa0 0x555ed779d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb35bfff640 0 client.16338 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:27:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb385e46740 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ed778d930 0x555ed779c840 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb385e46740 1 --2- 172.21.15.12:0/3869636177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ed778cf40 0x555ed779c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb385e46740 1 -- 172.21.15.12:0/3869636177 >> 172.21.15.12:0/3869636177 conn(0x555ed77885b0 msgr2=0x555ed778bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb385e46740 1 -- 172.21.15.12:0/3869636177 shutdown_connections 2024-05-09T09:27:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb385e46740 1 -- 172.21.15.12:0/3869636177 wait complete. 2024-05-09T09:27:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.261+0000 7fb385e46740 20 client.16338 trim_cache size 0 max 16384 2024-05-09T09:27:19.289 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:27:19.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.262+0000 7fb38313b640 1 -- 172.21.15.12:0/2866521361 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb36c0a2c10 msgr2=0x7fb36c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.262+0000 7fb38313b640 1 --2- 172.21.15.12:0/2866521361 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb36c0a2c10 0x7fb36c0a50d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x555ed778e190 tx=0x7fb374004a40 comp rx=0 tx=0).stop 2024-05-09T09:27:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.262+0000 7fb38313b640 1 -- 172.21.15.12:0/2866521361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb37c104c80 msgr2=0x7fb37c198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.262+0000 7fb38313b640 1 --2- 172.21.15.12:0/2866521361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb37c104c80 0x7fb37c198120 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb368005ab0 tx=0x7fb3680042f0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.262+0000 7fb38313b640 1 -- 172.21.15.12:0/2866521361 shutdown_connections 2024-05-09T09:27:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.262+0000 7fb38313b640 1 --2- 172.21.15.12:0/2866521361 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb36c0a2c10 0x7fb36c0a50d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.262+0000 7fb38313b640 1 --2- 172.21.15.12:0/2866521361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb37c1075b0 0x7fb37c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.262+0000 7fb38313b640 1 --2- 172.21.15.12:0/2866521361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb37c104c80 0x7fb37c198120 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.262+0000 7fb38313b640 1 --2- 172.21.15.12:0/2866521361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37c102350 0x7fb37c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.262+0000 7fb38313b640 1 -- 172.21.15.12:0/2866521361 >> 172.21.15.12:0/2866521361 conn(0x7fb37c0fbce0 msgr2=0x7fb37c102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.262+0000 7fb38313b640 1 -- 172.21.15.12:0/2866521361 shutdown_connections 2024-05-09T09:27:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.262+0000 7fb38313b640 1 -- 172.21.15.12:0/2866521361 wait complete. 2024-05-09T09:27:19.292 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:27:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 -- 172.21.15.12:0/1824871468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7010103430 msgr2=0x7f7010109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 --2- 172.21.15.12:0/1824871468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7010103430 0x7f7010109cc0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7000009ca0 tx=0x7f700002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 -- 172.21.15.12:0/1824871468 shutdown_connections 2024-05-09T09:27:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 --2- 172.21.15.12:0/1824871468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7010103430 0x7f7010109cc0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 --2- 172.21.15.12:0/1824871468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7010102a70 0x7f7010102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 --2- 172.21.15.12:0/1824871468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010101880 0x7f7010101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 -- 172.21.15.12:0/1824871468 >> 172.21.15.12:0/1824871468 conn(0x7f70100fce10 msgr2=0x7f70100ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 -- 172.21.15.12:0/1824871468 shutdown_connections 2024-05-09T09:27:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 -- 172.21.15.12:0/1824871468 wait complete. 2024-05-09T09:27:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 Processor -- start 2024-05-09T09:27:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 -- start start 2024-05-09T09:27:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7010101880 0x7f7010197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010102a70 0x7f7010198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.400+0000 7f701821c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7010103430 0x7f701019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f701821c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f701010d4e0 con 0x7f7010103430 2024-05-09T09:27:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f701821c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f701010d360 con 0x7f7010101880 2024-05-09T09:27:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f701821c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f701010d660 con 0x7f7010102a70 2024-05-09T09:27:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f70167b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7010103430 0x7f701019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f70167b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7010103430 0x7f701019f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38876/0 (socket says 172.21.15.12:38876) 2024-05-09T09:27:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f70167b9640 1 -- 172.21.15.12:0/496078868 learned_addr learned my addr 172.21.15.12:0/496078868 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f70157b7640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010102a70 0x7f7010198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f7015fb8640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7010101880 0x7f7010197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f70167b9640 1 -- 172.21.15.12:0/496078868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010102a70 msgr2=0x7f7010198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f70167b9640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010102a70 0x7f7010198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f70167b9640 1 -- 172.21.15.12:0/496078868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7010101880 msgr2=0x7f7010197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f70167b9640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7010101880 0x7f7010197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f70167b9640 1 -- 172.21.15.12:0/496078868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701019f770 con 0x7f7010103430 2024-05-09T09:27:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f70157b7640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010102a70 0x7f7010198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f70167b9640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7010103430 0x7f701019f100 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7000004ad0 tx=0x7f700002f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f700effd640 1 -- 172.21.15.12:0/496078868 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000003cf0 con 0x7f7010103430 2024-05-09T09:27:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f700effd640 1 -- 172.21.15.12:0/496078868 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7000004510 con 0x7f7010103430 2024-05-09T09:27:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f701821c640 1 -- 172.21.15.12:0/496078868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701019fa00 con 0x7f7010103430 2024-05-09T09:27:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f700effd640 1 -- 172.21.15.12:0/496078868 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700002fd10 con 0x7f7010103430 2024-05-09T09:27:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.401+0000 7f701821c640 1 -- 172.21.15.12:0/496078868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701019ff50 con 0x7f7010103430 2024-05-09T09:27:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.404+0000 7f700effd640 1 -- 172.21.15.12:0/496078868 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7000004040 con 0x7f7010103430 2024-05-09T09:27:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.404+0000 7f701b00a740 1 Processor -- start 2024-05-09T09:27:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.404+0000 7f701b00a740 1 -- start start 2024-05-09T09:27:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.404+0000 7f701b00a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722a350 0x56434722a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.404+0000 7f701b00a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722ad40 0x56434722f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.404+0000 7f701b00a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 0x564347232340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.404+0000 7f701b00a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56434718ac40 con 0x56434722feb0 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.404+0000 7f701b00a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564347132580 con 0x56434722ad40 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.404+0000 7f701b00a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564347186c00 con 0x56434722a350 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.404+0000 7f70157b7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 0x564347232340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.404+0000 7f70157b7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 0x564347232340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38890/0 (socket says 172.21.15.12:38890) 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.404+0000 7f70157b7640 1 -- 172.21.15.12:0/3258595926 learned_addr learned my addr 172.21.15.12:0/3258595926 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.404+0000 7f7015fb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722ad40 0x56434722f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f700effd640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6ff80a2b90 0x7f6ff80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f70157b7640 1 --2- 172.21.15.12:0/3258595926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722a350 0x56434722a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f70157b7640 1 -- 172.21.15.12:0/3258595926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722a350 msgr2=0x56434722a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f70157b7640 1 --2- 172.21.15.12:0/3258595926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722a350 0x56434722a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f70157b7640 1 -- 172.21.15.12:0/3258595926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722ad40 msgr2=0x56434722f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f70157b7640 1 --2- 172.21.15.12:0/3258595926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722ad40 0x56434722f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f70157b7640 1 -- 172.21.15.12:0/3258595926 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564346e9e3f0 con 0x56434722feb0 2024-05-09T09:27:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f70167b9640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6ff80a2b90 0x7f6ff80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f700effd640 1 -- 172.21.15.12:0/496078868 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f70000f0c80 con 0x7f7010103430 2024-05-09T09:27:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f70157b7640 1 --2- 172.21.15.12:0/3258595926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 0x564347232340 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7004005bc0 tx=0x7f700400b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a30e14c9599479 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f6ff7fff640 1 -- 172.21.15.12:0/3258595926 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004019070 con 0x56434722feb0 2024-05-09T09:27:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f6ff7fff640 1 -- 172.21.15.12:0/3258595926 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f70040047a0 con 0x56434722feb0 2024-05-09T09:27:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f6ff7fff640 1 -- 172.21.15.12:0/3258595926 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004011850 con 0x56434722feb0 2024-05-09T09:27:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.405+0000 7f70167b9640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6ff80a2b90 0x7f6ff80a5050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f70000bce10 tx=0x7f70000b25f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.406+0000 7f701b00a740 1 -- 172.21.15.12:0/3258595926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 msgr2=0x564347232340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.406+0000 7f701b00a740 1 --2- 172.21.15.12:0/3258595926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 0x564347232340 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7004005bc0 tx=0x7f700400b720 comp rx=0 tx=0).stop 2024-05-09T09:27:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.406+0000 7f701b00a740 1 -- 172.21.15.12:0/3258595926 shutdown_connections 2024-05-09T09:27:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.406+0000 7f701b00a740 1 --2- 172.21.15.12:0/3258595926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 0x564347232340 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.406+0000 7f701b00a740 1 --2- 172.21.15.12:0/3258595926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722ad40 0x56434722f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.406+0000 7f701b00a740 1 --2- 172.21.15.12:0/3258595926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722a350 0x56434722a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.406+0000 7f701b00a740 1 -- 172.21.15.12:0/3258595926 >> 172.21.15.12:0/3258595926 conn(0x5643472259c0 msgr2=0x564347225dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.406+0000 7f701b00a740 1 -- 172.21.15.12:0/3258595926 shutdown_connections 2024-05-09T09:27:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.406+0000 7f701b00a740 1 -- 172.21.15.12:0/3258595926 wait complete. 2024-05-09T09:27:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.406+0000 7f701b00a740 1 -- start start 2024-05-09T09:27:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.406+0000 7f701b00a740 1 Processor -- start 2024-05-09T09:27:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f701b00a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722a350 0x5643472347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f701b00a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722ad40 0x564347234d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f701b00a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722feb0 0x56434723bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f701b00a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564347232e60 con 0x56434722ad40 2024-05-09T09:27:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f701b00a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564347145c40 con 0x56434722a350 2024-05-09T09:27:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f701b00a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564347232fe0 con 0x56434722feb0 2024-05-09T09:27:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f701b00a740 10 client.0 _ll_register_callbacks cb 0x564346e9dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f7015fb8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722ad40 0x564347234d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f7015fb8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722ad40 0x564347234d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38904/0 (socket says 172.21.15.12:38904) 2024-05-09T09:27:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f7015fb8640 1 -- 172.21.15.12:0/3320506581 learned_addr learned my addr 172.21.15.12:0/3320506581 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f70157b7640 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722a350 0x5643472347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f70157b7640 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722feb0 0x56434723bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f7015fb8640 1 -- 172.21.15.12:0/3320506581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722feb0 msgr2=0x56434723bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f7015fb8640 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722feb0 0x56434723bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f7015fb8640 1 -- 172.21.15.12:0/3320506581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722a350 msgr2=0x5643472347d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f7015fb8640 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722a350 0x5643472347d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f7015fb8640 1 -- 172.21.15.12:0/3320506581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564346f687d0 con 0x56434722ad40 2024-05-09T09:27:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f70157b7640 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722feb0 0x56434723bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:27:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f7015fb8640 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722ad40 0x564347234d10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f700800ccd0 tx=0x7f70080039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d4acb55bdbd571 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.407+0000 7f6ff57fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f6ff57fa640 1 -- 172.21.15.12:0/3320506581 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008010840 con 0x56434722ad40 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f6ff57fa640 1 -- 172.21.15.12:0/3320506581 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f70080109e0 con 0x56434722ad40 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f701b00a740 1 -- 172.21.15.12:0/3320506581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70040047a0 con 0x56434722ad40 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f6ff57fa640 1 -- 172.21.15.12:0/3320506581 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7008010be0 con 0x56434722ad40 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f701b00a740 10 client.16350 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f701b00a740 1 -- 172.21.15.12:0/3320506581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56434723f190 con 0x56434722ad40 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f6ff57fa640 1 -- 172.21.15.12:0/3320506581 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f70040047a0 con 0x56434722ad40 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f6ff57fa640 1 -- 172.21.15.12:0/3320506581 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f7008018020 con 0x56434722ad40 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f701b00a740 10 client.16350 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f701b00a740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f701b00a740 20 client.16350 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f701b00a740 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564347240860 0x564347242ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f701b00a740 4 client.16350 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.408+0000 7f701b00a740 1 -- 172.21.15.12:0/3320506581 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564347198370 con 0x564347240860 2024-05-09T09:27:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.409+0000 7f70157b7640 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564347240860 0x564347242ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.409+0000 7f6ff57fa640 10 client.16350 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.409+0000 7f70157b7640 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564347240860 0x564347242ce0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c750d77cfba69e12 server_cookie=341e4ca211f9c596 in_seq=0 out_seq=0 2024-05-09T09:27:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f6ff57fa640 1 -- 172.21.15.12:0/3320506581 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564347198370 con 0x564347240860 2024-05-09T09:27:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f6ff57fa640 10 client.16350 handle_command_reply: tid=0 2024-05-09T09:27:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f701b00a740 1 client.16350 shutdown 2024-05-09T09:27:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f701b00a740 1 -- 172.21.15.12:0/3320506581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722ad40 msgr2=0x564347234d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f701b00a740 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722ad40 0x564347234d10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f700800ccd0 tx=0x7f70080039c0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f701b00a740 1 -- 172.21.15.12:0/3320506581 shutdown_connections 2024-05-09T09:27:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f701b00a740 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564347240860 0x564347242ce0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f701b00a740 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722feb0 0x56434723bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f701b00a740 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722ad40 0x564347234d10 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f701b00a740 1 --2- 172.21.15.12:0/3320506581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722a350 0x5643472347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f6ff57fa640 0 client.16350 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:19.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f701b00a740 1 -- 172.21.15.12:0/3320506581 >> 172.21.15.12:0/3320506581 conn(0x5643472259c0 msgr2=0x5643472207b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f701b00a740 1 -- 172.21.15.12:0/3320506581 shutdown_connections 2024-05-09T09:27:19.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f701b00a740 1 -- 172.21.15.12:0/3320506581 wait complete. 2024-05-09T09:27:19.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.410+0000 7f701b00a740 20 client.16350 trim_cache size 0 max 16384 2024-05-09T09:27:19.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.423+0000 7f701b00a740 1 Processor -- start 2024-05-09T09:27:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.423+0000 7f701b00a740 1 -- start start 2024-05-09T09:27:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f701b00a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722a350 0x56434722c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f701b00a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722ad40 0x56434723bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f701b00a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 0x56434722c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f701b00a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f70101a8380 con 0x56434722feb0 2024-05-09T09:27:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f701b00a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f70101a9f50 con 0x56434722ad40 2024-05-09T09:27:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f701b00a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f70101a8a50 con 0x56434722a350 2024-05-09T09:27:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f70157b7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 0x56434722c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f70157b7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 0x56434722c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38914/0 (socket says 172.21.15.12:38914) 2024-05-09T09:27:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f70157b7640 1 -- 172.21.15.12:0/75486738 learned_addr learned my addr 172.21.15.12:0/75486738 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f70157b7640 1 --2- 172.21.15.12:0/75486738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722a350 0x56434722c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f70157b7640 1 -- 172.21.15.12:0/75486738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722a350 msgr2=0x56434722c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f70157b7640 1 --2- 172.21.15.12:0/75486738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722a350 0x56434722c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f70157b7640 1 -- 172.21.15.12:0/75486738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722ad40 msgr2=0x56434723bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f7015fb8640 1 --2- 172.21.15.12:0/75486738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722ad40 0x56434723bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f70157b7640 1 --2- 172.21.15.12:0/75486738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722ad40 0x56434723bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f70157b7640 1 -- 172.21.15.12:0/75486738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56434722b120 con 0x56434722feb0 2024-05-09T09:27:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f7015fb8640 1 --2- 172.21.15.12:0/75486738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722ad40 0x56434723bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:27:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f70157b7640 1 --2- 172.21.15.12:0/75486738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 0x56434722c6e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f70040135e0 tx=0x7f7004015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458442e14675c5f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f6ff77fe640 1 -- 172.21.15.12:0/75486738 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004030070 con 0x56434722feb0 2024-05-09T09:27:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f6ff77fe640 1 -- 172.21.15.12:0/75486738 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7004029790 con 0x56434722feb0 2024-05-09T09:27:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.424+0000 7f6ff77fe640 1 -- 172.21.15.12:0/75486738 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004019050 con 0x56434722feb0 2024-05-09T09:27:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.425+0000 7f701b00a740 1 -- 172.21.15.12:0/75486738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 msgr2=0x56434722c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.425+0000 7f701b00a740 1 --2- 172.21.15.12:0/75486738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 0x56434722c6e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f70040135e0 tx=0x7f7004015040 comp rx=0 tx=0).stop 2024-05-09T09:27:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.425+0000 7f701b00a740 1 -- 172.21.15.12:0/75486738 shutdown_connections 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.425+0000 7f701b00a740 1 --2- 172.21.15.12:0/75486738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722feb0 0x56434722c6e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.425+0000 7f701b00a740 1 --2- 172.21.15.12:0/75486738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722ad40 0x56434723bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.425+0000 7f701b00a740 1 --2- 172.21.15.12:0/75486738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722a350 0x56434722c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.425+0000 7f701b00a740 1 -- 172.21.15.12:0/75486738 >> 172.21.15.12:0/75486738 conn(0x5643472259c0 msgr2=0x5643472347d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.425+0000 7f701b00a740 1 -- 172.21.15.12:0/75486738 shutdown_connections 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.425+0000 7f701b00a740 1 -- 172.21.15.12:0/75486738 wait complete. 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.425+0000 7f701b00a740 1 -- start start 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.425+0000 7f701b00a740 1 Processor -- start 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f701b00a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722a350 0x564347239490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f701b00a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722ad40 0x5643472399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f701b00a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722feb0 0x56434723a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f701b00a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f70101a97f0 con 0x56434722a350 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f701b00a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f70101a9120 con 0x56434722feb0 2024-05-09T09:27:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f701b00a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f70101a6360 con 0x56434722ad40 2024-05-09T09:27:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f701b00a740 10 client.0 _ll_register_callbacks cb 0x7f7008004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f70157b7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722a350 0x564347239490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f70157b7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722a350 0x564347239490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38928/0 (socket says 172.21.15.12:38928) 2024-05-09T09:27:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f70157b7640 1 -- 172.21.15.12:0/354491294 learned_addr learned my addr 172.21.15.12:0/354491294 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f7015fb8640 1 --2- 172.21.15.12:0/354491294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722ad40 0x5643472399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f70157b7640 1 --2- 172.21.15.12:0/354491294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722feb0 0x56434723a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f70157b7640 1 -- 172.21.15.12:0/354491294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722ad40 msgr2=0x5643472399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f70157b7640 1 --2- 172.21.15.12:0/354491294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722ad40 0x5643472399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f70157b7640 1 -- 172.21.15.12:0/354491294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722feb0 msgr2=0x56434723a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f70157b7640 1 --2- 172.21.15.12:0/354491294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722feb0 0x56434723a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f70157b7640 1 -- 172.21.15.12:0/354491294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434723afd0 con 0x56434722a350 2024-05-09T09:27:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f70157b7640 1 --2- 172.21.15.12:0/354491294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722a350 0x564347239490 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f700400b500 tx=0x7f7004011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d436fbbd7c46de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.426+0000 7f6ff57fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f6ff57fa640 1 -- 172.21.15.12:0/354491294 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004019070 con 0x56434722a350 2024-05-09T09:27:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f6ff57fa640 1 -- 172.21.15.12:0/354491294 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f7004004f70 con 0x56434722a350 2024-05-09T09:27:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f701b00a740 1 -- 172.21.15.12:0/354491294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7004029790 con 0x56434722a350 2024-05-09T09:27:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f6ff57fa640 1 -- 172.21.15.12:0/354491294 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7004030040 con 0x56434722a350 2024-05-09T09:27:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f701b00a740 10 client.16356 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f701b00a740 1 -- 172.21.15.12:0/354491294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5643472a6e60 con 0x56434722a350 2024-05-09T09:27:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f6ff57fa640 1 -- 172.21.15.12:0/354491294 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7004029790 con 0x56434722a350 2024-05-09T09:27:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f6ff57fa640 1 -- 172.21.15.12:0/354491294 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f70040035f0 con 0x56434722a350 2024-05-09T09:27:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f701b00a740 10 client.16356 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f701b00a740 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f701b00a740 20 client.16356 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f701b00a740 1 --2- 172.21.15.12:0/354491294 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5643472a7630 0x5643472a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f701b00a740 4 client.16356 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:27:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.427+0000 7f701b00a740 1 -- 172.21.15.12:0/354491294 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643471926c0 con 0x5643472a7630 2024-05-09T09:27:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f7015fb8640 1 --2- 172.21.15.12:0/354491294 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5643472a7630 0x5643472a9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f7015fb8640 1 --2- 172.21.15.12:0/354491294 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5643472a7630 0x5643472a9a50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27eed13c401adb9a server_cookie=f322c6c713c9160e in_seq=0 out_seq=0 2024-05-09T09:27:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f6ff57fa640 10 client.16356 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f6ff57fa640 1 -- 172.21.15.12:0/354491294 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643471926c0 con 0x5643472a7630 2024-05-09T09:27:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f6ff57fa640 10 client.16356 handle_command_reply: tid=0 2024-05-09T09:27:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f701b00a740 1 client.16356 shutdown 2024-05-09T09:27:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f701b00a740 1 -- 172.21.15.12:0/354491294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722a350 msgr2=0x564347239490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f701b00a740 1 --2- 172.21.15.12:0/354491294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722a350 0x564347239490 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f700400b500 tx=0x7f7004011440 comp rx=0 tx=0).stop 2024-05-09T09:27:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f701b00a740 1 -- 172.21.15.12:0/354491294 shutdown_connections 2024-05-09T09:27:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f701b00a740 1 --2- 172.21.15.12:0/354491294 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5643472a7630 0x5643472a9a50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f701b00a740 1 --2- 172.21.15.12:0/354491294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434722feb0 0x56434723a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f701b00a740 1 --2- 172.21.15.12:0/354491294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56434722ad40 0x5643472399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f6ff57fa640 0 client.16356 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:19.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f701b00a740 1 --2- 172.21.15.12:0/354491294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56434722a350 0x564347239490 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.428+0000 7f701b00a740 1 -- 172.21.15.12:0/354491294 >> 172.21.15.12:0/354491294 conn(0x5643472259c0 msgr2=0x5643472347b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701b00a740 1 -- 172.21.15.12:0/354491294 shutdown_connections 2024-05-09T09:27:19.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701b00a740 1 -- 172.21.15.12:0/354491294 wait complete. 2024-05-09T09:27:19.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701b00a740 20 client.16356 trim_cache size 0 max 16384 2024-05-09T09:27:19.435 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:27:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701821c640 1 -- 172.21.15.12:0/496078868 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6ff80a2b90 msgr2=0x7f6ff80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701821c640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6ff80a2b90 0x7f6ff80a5050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f70000bce10 tx=0x7f70000b25f0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701821c640 1 -- 172.21.15.12:0/496078868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7010103430 msgr2=0x7f701019f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701821c640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7010103430 0x7f701019f100 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7000004ad0 tx=0x7f700002f450 comp rx=0 tx=0).stop 2024-05-09T09:27:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701821c640 1 -- 172.21.15.12:0/496078868 shutdown_connections 2024-05-09T09:27:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701821c640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6ff80a2b90 0x7f6ff80a5050 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701821c640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7010103430 0x7f701019f100 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701821c640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010102a70 0x7f7010198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701821c640 1 --2- 172.21.15.12:0/496078868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7010101880 0x7f7010197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:19.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701821c640 1 -- 172.21.15.12:0/496078868 >> 172.21.15.12:0/496078868 conn(0x7f70100fce10 msgr2=0x7f7010105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:19.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701821c640 1 -- 172.21.15.12:0/496078868 shutdown_connections 2024-05-09T09:27:19.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:19.429+0000 7f701821c640 1 -- 172.21.15.12:0/496078868 wait complete. 2024-05-09T09:27:19.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:19 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1880569615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:27:20.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:20 smithi016 ceph-mon[33335]: pgmap v420: 129 pgs: 129 active+clean; 104 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 2.1 MiB/s wr, 1.01k op/s 2024-05-09T09:27:20.440 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:27:20.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:27:20.440 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:27:20.465 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:20 smithi012 ceph-mon[30140]: pgmap v420: 129 pgs: 129 active+clean; 104 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 2.1 MiB/s wr, 1.01k op/s 2024-05-09T09:27:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:20 smithi170 ceph-mon[33632]: pgmap v420: 129 pgs: 129 active+clean; 104 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 2.1 MiB/s wr, 1.01k op/s 2024-05-09T09:27:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.570+0000 7f2adb1c4640 1 -- 172.21.15.12:0/3930555344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ad4102320 msgr2=0x7f2ad4108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.571+0000 7f2adb1c4640 1 --2- 172.21.15.12:0/3930555344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ad4102320 0x7f2ad4108bb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009ca0 tx=0x7f2acc02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.571+0000 7f2adb1c4640 1 -- 172.21.15.12:0/3930555344 shutdown_connections 2024-05-09T09:27:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.571+0000 7f2adb1c4640 1 --2- 172.21.15.12:0/3930555344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ad4102320 0x7f2ad4108bb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.571+0000 7f2adb1c4640 1 --2- 172.21.15.12:0/3930555344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad4101960 0x7f2ad4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.571+0000 7f2adb1c4640 1 --2- 172.21.15.12:0/3930555344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad4100770 0x7f2ad4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.571+0000 7f2adb1c4640 1 -- 172.21.15.12:0/3930555344 >> 172.21.15.12:0/3930555344 conn(0x7f2ad40fbce0 msgr2=0x7f2ad40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.571+0000 7f2adb1c4640 1 -- 172.21.15.12:0/3930555344 shutdown_connections 2024-05-09T09:27:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.571+0000 7f2adb1c4640 1 -- 172.21.15.12:0/3930555344 wait complete. 2024-05-09T09:27:20.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.571+0000 7f2adb1c4640 1 Processor -- start 2024-05-09T09:27:20.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.571+0000 7f2adb1c4640 1 -- start start 2024-05-09T09:27:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2adb1c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ad4100770 0x7f2ad410af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2adb1c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad4101960 0x7f2ad41095b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2adb1c4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad4102320 0x7f2ad4109af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2adb1c4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad410d4e0 con 0x7f2ad4100770 2024-05-09T09:27:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2adb1c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad410d360 con 0x7f2ad4101960 2024-05-09T09:27:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2adb1c4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad410d660 con 0x7f2ad4102320 2024-05-09T09:27:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2ad8f60640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ad4100770 0x7f2ad410af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2ad8f60640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ad4100770 0x7f2ad410af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38956/0 (socket says 172.21.15.12:38956) 2024-05-09T09:27:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2ad8f60640 1 -- 172.21.15.12:0/1759417046 learned_addr learned my addr 172.21.15.12:0/1759417046 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2ad9761640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad4102320 0x7f2ad4109af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2ad3fff640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad4101960 0x7f2ad41095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2ad8f60640 1 -- 172.21.15.12:0/1759417046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad4102320 msgr2=0x7f2ad4109af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2ad8f60640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad4102320 0x7f2ad4109af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2ad8f60640 1 -- 172.21.15.12:0/1759417046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad4101960 msgr2=0x7f2ad41095b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2ad8f60640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad4101960 0x7f2ad41095b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2ad8f60640 1 -- 172.21.15.12:0/1759417046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad410a380 con 0x7f2ad4100770 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2ad3fff640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad4101960 0x7f2ad41095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2ad9761640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad4102320 0x7f2ad4109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.572+0000 7f2ad8f60640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ad4100770 0x7f2ad410af70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800e9c0 tx=0x7f2ac800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.573+0000 7f2ad1ffb640 1 -- 172.21.15.12:0/1759417046 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac800cd80 con 0x7f2ad4100770 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.573+0000 7f2adb1c4640 1 -- 172.21.15.12:0/1759417046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad41a1cd0 con 0x7f2ad4100770 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.573+0000 7f2ad1ffb640 1 -- 172.21.15.12:0/1759417046 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2ac80035f0 con 0x7f2ad4100770 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.573+0000 7f2ad1ffb640 1 -- 172.21.15.12:0/1759417046 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8010690 con 0x7f2ad4100770 2024-05-09T09:27:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.573+0000 7f2adb1c4640 1 -- 172.21.15.12:0/1759417046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad41a21c0 con 0x7f2ad4100770 2024-05-09T09:27:20.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.575+0000 7f2addfa8740 1 Processor -- start 2024-05-09T09:27:20.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2addfa8740 1 -- start start 2024-05-09T09:27:20.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2ad1ffb640 1 -- 172.21.15.12:0/1759417046 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ac80037a0 con 0x7f2ad4100770 2024-05-09T09:27:20.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2addfa8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de54b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2addfa8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54beb0 0x55c6de5509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2addfa8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de5534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2addfa8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c6de4af990 con 0x55c6de551020 2024-05-09T09:27:20.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2addfa8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c6de4575b0 con 0x55c6de54beb0 2024-05-09T09:27:20.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2addfa8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c6de4ab950 con 0x55c6de54b4c0 2024-05-09T09:27:20.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2ad3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de5534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2ad9761640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de54b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2ad3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de5534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38958/0 (socket says 172.21.15.12:38958) 2024-05-09T09:27:20.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2ad3fff640 1 -- 172.21.15.12:0/724929855 learned_addr learned my addr 172.21.15.12:0/724929855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:20.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2ad8f60640 1 --2- 172.21.15.12:0/724929855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54beb0 0x55c6de5509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2ad9761640 1 -- 172.21.15.12:0/724929855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54beb0 msgr2=0x55c6de5509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2ad9761640 1 --2- 172.21.15.12:0/724929855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54beb0 0x55c6de5509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2ad9761640 1 -- 172.21.15.12:0/724929855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 msgr2=0x55c6de5534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2ad9761640 1 --2- 172.21.15.12:0/724929855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de5534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.576+0000 7f2ad9761640 1 -- 172.21.15.12:0/724929855 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6de28b7e0 con 0x55c6de54b4c0 2024-05-09T09:27:20.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.577+0000 7f2ad3fff640 1 --2- 172.21.15.12:0/724929855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de5534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:20.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.577+0000 7f2ad1ffb640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ac40a2b90 0x7f2ac40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.577+0000 7f2ad9761640 1 --2- 172.21.15.12:0/724929855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de54b8e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2acc02ade0 tx=0x7f2acc0093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e73f1954224cd9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.577+0000 7f2ad8f60640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ac40a2b90 0x7f2ac40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.577+0000 7f2abeffd640 1 -- 172.21.15.12:0/724929855 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc031450 con 0x55c6de54b4c0 2024-05-09T09:27:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.577+0000 7f2ad1ffb640 1 -- 172.21.15.12:0/1759417046 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f2ac80cf990 con 0x7f2ad4100770 2024-05-09T09:27:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.577+0000 7f2abeffd640 1 -- 172.21.15.12:0/724929855 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2acc036040 con 0x55c6de54b4c0 2024-05-09T09:27:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.577+0000 7f2abeffd640 1 -- 172.21.15.12:0/724929855 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc0045e0 con 0x55c6de54b4c0 2024-05-09T09:27:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.578+0000 7f2addfa8740 1 -- 172.21.15.12:0/724929855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 msgr2=0x55c6de54b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.578+0000 7f2ad8f60640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ac40a2b90 0x7f2ac40a5050 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80cd430 tx=0x7f2ac8003ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.578+0000 7f2addfa8740 1 --2- 172.21.15.12:0/724929855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de54b8e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2acc02ade0 tx=0x7f2acc0093e0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.578+0000 7f2addfa8740 1 -- 172.21.15.12:0/724929855 shutdown_connections 2024-05-09T09:27:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.578+0000 7f2addfa8740 1 --2- 172.21.15.12:0/724929855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de5534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.578+0000 7f2addfa8740 1 --2- 172.21.15.12:0/724929855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54beb0 0x55c6de5509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.578+0000 7f2addfa8740 1 --2- 172.21.15.12:0/724929855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de54b8e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.578+0000 7f2addfa8740 1 -- 172.21.15.12:0/724929855 >> 172.21.15.12:0/724929855 conn(0x55c6de5130a0 msgr2=0x55c6de546f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.578+0000 7f2addfa8740 1 -- 172.21.15.12:0/724929855 shutdown_connections 2024-05-09T09:27:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.578+0000 7f2addfa8740 1 -- 172.21.15.12:0/724929855 wait complete. 2024-05-09T09:27:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.578+0000 7f2addfa8740 1 -- start start 2024-05-09T09:27:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.578+0000 7f2addfa8740 1 Processor -- start 2024-05-09T09:27:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2addfa8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54b4c0 0x55c6de5558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2addfa8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54beb0 0x55c6de555e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2addfa8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de55ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2addfa8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c6de553f70 con 0x55c6de551020 2024-05-09T09:27:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2addfa8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c6de46ab40 con 0x55c6de54b4c0 2024-05-09T09:27:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2addfa8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c6de5540f0 con 0x55c6de54beb0 2024-05-09T09:27:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2addfa8740 10 client.0 _ll_register_callbacks cb 0x55c6de45f0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2ad9761640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de55ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2ad3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54beb0 0x55c6de555e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2ad9761640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de55ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38968/0 (socket says 172.21.15.12:38968) 2024-05-09T09:27:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2ad9761640 1 -- 172.21.15.12:0/3984926109 learned_addr learned my addr 172.21.15.12:0/3984926109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2ad9761640 1 --2- 172.21.15.12:0/3984926109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54b4c0 0x55c6de5558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2ad3fff640 1 -- 172.21.15.12:0/3984926109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54b4c0 msgr2=0x55c6de5558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2ad3fff640 1 --2- 172.21.15.12:0/3984926109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54b4c0 0x55c6de5558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2ad3fff640 1 -- 172.21.15.12:0/3984926109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 msgr2=0x55c6de55ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2ad3fff640 1 --2- 172.21.15.12:0/3984926109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de55ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2ad3fff640 1 -- 172.21.15.12:0/3984926109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6de2ebf70 con 0x55c6de54beb0 2024-05-09T09:27:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.579+0000 7f2ad9761640 1 --2- 172.21.15.12:0/3984926109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54b4c0 0x55c6de5558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.580+0000 7f2ad3fff640 1 --2- 172.21.15.12:0/3984926109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54beb0 0x55c6de555e20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00136e0 tx=0x7f2ac0013ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f32d855c966aadd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.580+0000 7f2ab3fff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:27:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.580+0000 7f2ab3fff640 1 -- 172.21.15.12:0/3984926109 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0011e30 con 0x55c6de54beb0 2024-05-09T09:27:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.580+0000 7f2ab3fff640 1 -- 172.21.15.12:0/3984926109 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f2ac0004020 con 0x55c6de54beb0 2024-05-09T09:27:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.580+0000 7f2addfa8740 1 -- 172.21.15.12:0/3984926109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2acc036040 con 0x55c6de54beb0 2024-05-09T09:27:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.580+0000 7f2ab3fff640 1 -- 172.21.15.12:0/3984926109 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2ac0004220 con 0x55c6de54beb0 2024-05-09T09:27:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.580+0000 7f2addfa8740 10 client.25927 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.580+0000 7f2addfa8740 1 -- 172.21.15.12:0/3984926109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c6de183170 con 0x55c6de54beb0 2024-05-09T09:27:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.580+0000 7f2ab3fff640 1 -- 172.21.15.12:0/3984926109 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2acc036040 con 0x55c6de54beb0 2024-05-09T09:27:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.581+0000 7f2ab3fff640 1 -- 172.21.15.12:0/3984926109 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f2ac001a070 con 0x55c6de54beb0 2024-05-09T09:27:20.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.581+0000 7f2addfa8740 10 client.25927 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:20.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.581+0000 7f2addfa8740 10 client.25927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:20.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.581+0000 7f2addfa8740 20 client.25927 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:20.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.581+0000 7f2addfa8740 1 --2- 172.21.15.12:0/3984926109 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c6de561590 0x55c6de5639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.581+0000 7f2addfa8740 4 client.25927 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:20.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.581+0000 7f2addfa8740 1 -- 172.21.15.12:0/3984926109 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6de4bd0c0 con 0x55c6de561590 2024-05-09T09:27:20.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.581+0000 7f2ad9761640 1 --2- 172.21.15.12:0/3984926109 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c6de561590 0x55c6de5639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.581+0000 7f2ad9761640 1 --2- 172.21.15.12:0/3984926109 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c6de561590 0x55c6de5639b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3709122f8cee4bfb server_cookie=b9f26278c8772de7 in_seq=0 out_seq=0 2024-05-09T09:27:20.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.581+0000 7f2ab3fff640 10 client.25927 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:20.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2ab3fff640 1 -- 172.21.15.12:0/3984926109 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6de4bd0c0 con 0x55c6de561590 2024-05-09T09:27:20.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2ab3fff640 10 client.25927 handle_command_reply: tid=0 2024-05-09T09:27:20.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2addfa8740 1 client.25927 shutdown 2024-05-09T09:27:20.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2addfa8740 1 -- 172.21.15.12:0/3984926109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54beb0 msgr2=0x55c6de555e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2addfa8740 1 --2- 172.21.15.12:0/3984926109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54beb0 0x55c6de555e20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00136e0 tx=0x7f2ac0013ba0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2addfa8740 1 -- 172.21.15.12:0/3984926109 shutdown_connections 2024-05-09T09:27:20.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2addfa8740 1 --2- 172.21.15.12:0/3984926109 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c6de561590 0x55c6de5639b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2addfa8740 1 --2- 172.21.15.12:0/3984926109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de55ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2addfa8740 1 --2- 172.21.15.12:0/3984926109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54beb0 0x55c6de555e20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2ab3fff640 0 client.25927 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:20.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2addfa8740 1 --2- 172.21.15.12:0/3984926109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54b4c0 0x55c6de5558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2addfa8740 1 -- 172.21.15.12:0/3984926109 >> 172.21.15.12:0/3984926109 conn(0x55c6de5130a0 msgr2=0x55c6de546080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2addfa8740 1 -- 172.21.15.12:0/3984926109 shutdown_connections 2024-05-09T09:27:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2addfa8740 1 -- 172.21.15.12:0/3984926109 wait complete. 2024-05-09T09:27:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.582+0000 7f2addfa8740 20 client.25927 trim_cache size 0 max 16384 2024-05-09T09:27:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.607+0000 7f2addfa8740 1 Processor -- start 2024-05-09T09:27:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.607+0000 7f2addfa8740 1 -- start start 2024-05-09T09:27:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.607+0000 7f2addfa8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de549e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.607+0000 7f2addfa8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54beb0 0x55c6de55ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.607+0000 7f2addfa8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de54a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.607+0000 7f2addfa8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41ac860 con 0x55c6de551020 2024-05-09T09:27:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.607+0000 7f2addfa8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41ae430 con 0x55c6de54beb0 2024-05-09T09:27:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.607+0000 7f2addfa8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41acf30 con 0x55c6de54b4c0 2024-05-09T09:27:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.607+0000 7f2ad9761640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de549e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.608+0000 7f2ad9761640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de549e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:53274/0 (socket says 172.21.15.12:53274) 2024-05-09T09:27:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.608+0000 7f2ad9761640 1 -- 172.21.15.12:0/2630899072 learned_addr learned my addr 172.21.15.12:0/2630899072 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.608+0000 7f2ad9761640 1 --2- 172.21.15.12:0/2630899072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de54a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.608+0000 7f2ad3fff640 1 --2- 172.21.15.12:0/2630899072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54beb0 0x55c6de55ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.608+0000 7f2ad9761640 1 -- 172.21.15.12:0/2630899072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54beb0 msgr2=0x55c6de55ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.608+0000 7f2ad9761640 1 --2- 172.21.15.12:0/2630899072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54beb0 0x55c6de55ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.608+0000 7f2ad9761640 1 -- 172.21.15.12:0/2630899072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 msgr2=0x55c6de54a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.608+0000 7f2ad9761640 1 --2- 172.21.15.12:0/2630899072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de54a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.608+0000 7f2ad9761640 1 -- 172.21.15.12:0/2630899072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6de547bf0 con 0x55c6de54b4c0 2024-05-09T09:27:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.608+0000 7f2ad9761640 1 --2- 172.21.15.12:0/2630899072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de549e00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009ca0 tx=0x7f2acc02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8622d1f097811627 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.608+0000 7f2abe7fc640 1 -- 172.21.15.12:0/2630899072 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc0045c0 con 0x55c6de54b4c0 2024-05-09T09:27:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.608+0000 7f2abe7fc640 1 -- 172.21.15.12:0/2630899072 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2acc036070 con 0x55c6de54b4c0 2024-05-09T09:27:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.608+0000 7f2abe7fc640 1 -- 172.21.15.12:0/2630899072 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc02f660 con 0x55c6de54b4c0 2024-05-09T09:27:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.609+0000 7f2addfa8740 1 -- 172.21.15.12:0/2630899072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 msgr2=0x55c6de549e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.609+0000 7f2addfa8740 1 --2- 172.21.15.12:0/2630899072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de549e00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009ca0 tx=0x7f2acc02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.609+0000 7f2addfa8740 1 -- 172.21.15.12:0/2630899072 shutdown_connections 2024-05-09T09:27:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.609+0000 7f2addfa8740 1 --2- 172.21.15.12:0/2630899072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de551020 0x55c6de54a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.609+0000 7f2addfa8740 1 --2- 172.21.15.12:0/2630899072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de54beb0 0x55c6de55ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.609+0000 7f2addfa8740 1 --2- 172.21.15.12:0/2630899072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de549e00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.609+0000 7f2addfa8740 1 -- 172.21.15.12:0/2630899072 >> 172.21.15.12:0/2630899072 conn(0x55c6de5130a0 msgr2=0x55c6de54d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.609+0000 7f2addfa8740 1 -- 172.21.15.12:0/2630899072 shutdown_connections 2024-05-09T09:27:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.609+0000 7f2addfa8740 1 -- 172.21.15.12:0/2630899072 wait complete. 2024-05-09T09:27:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.609+0000 7f2addfa8740 1 -- start start 2024-05-09T09:27:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.609+0000 7f2addfa8740 1 Processor -- start 2024-05-09T09:27:20.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2addfa8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de55a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2addfa8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de54beb0 0x55c6de55ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2addfa8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de551020 0x55c6de55bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2addfa8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41adcd0 con 0x55c6de54beb0 2024-05-09T09:27:20.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2addfa8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41ad600 con 0x55c6de551020 2024-05-09T09:27:20.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2addfa8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41aa840 con 0x55c6de54b4c0 2024-05-09T09:27:20.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2addfa8740 10 client.0 _ll_register_callbacks cb 0x7f2ac0009730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:20.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2ad9761640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de55a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2ad3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de54beb0 0x55c6de55ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2ad9761640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de55a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:53284/0 (socket says 172.21.15.12:53284) 2024-05-09T09:27:20.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2ad9761640 1 -- 172.21.15.12:0/1871507883 learned_addr learned my addr 172.21.15.12:0/1871507883 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:20.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2ad9761640 1 --2- 172.21.15.12:0/1871507883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de551020 0x55c6de55bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2ad9761640 1 -- 172.21.15.12:0/1871507883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de551020 msgr2=0x55c6de55bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2ad9761640 1 --2- 172.21.15.12:0/1871507883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de551020 0x55c6de55bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2ad9761640 1 -- 172.21.15.12:0/1871507883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de54beb0 msgr2=0x55c6de55ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2ad9761640 1 --2- 172.21.15.12:0/1871507883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de54beb0 0x55c6de55ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.610+0000 7f2ad9761640 1 -- 172.21.15.12:0/1871507883 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6de55c330 con 0x55c6de54b4c0 2024-05-09T09:27:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2ad9761640 1 --2- 172.21.15.12:0/1871507883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de55a840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2acc031ab0 tx=0x7f2acc05b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43be8b221647e9ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2ab3fff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:27:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2ab3fff640 1 -- 172.21.15.12:0/1871507883 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc0317c0 con 0x55c6de54b4c0 2024-05-09T09:27:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2addfa8740 1 -- 172.21.15.12:0/1871507883 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2acc036070 con 0x55c6de54b4c0 2024-05-09T09:27:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2ab3fff640 1 -- 172.21.15.12:0/1871507883 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f2acc003d10 con 0x55c6de54b4c0 2024-05-09T09:27:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2ab3fff640 1 -- 172.21.15.12:0/1871507883 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2acc003f10 con 0x55c6de54b4c0 2024-05-09T09:27:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2addfa8740 10 client.25933 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2addfa8740 1 -- 172.21.15.12:0/1871507883 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c6de5c7d90 con 0x55c6de54b4c0 2024-05-09T09:27:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2ab3fff640 1 -- 172.21.15.12:0/1871507883 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2acc036070 con 0x55c6de54b4c0 2024-05-09T09:27:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2ab3fff640 1 -- 172.21.15.12:0/1871507883 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f2acc050040 con 0x55c6de54b4c0 2024-05-09T09:27:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2addfa8740 10 client.25933 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2addfa8740 10 client.25933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2addfa8740 20 client.25933 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2addfa8740 1 --2- 172.21.15.12:0/1871507883 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c6de5c8360 0x55c6de5ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2addfa8740 4 client.25933 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:27:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.611+0000 7f2addfa8740 1 -- 172.21.15.12:0/1871507883 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c6de4b7410 con 0x55c6de5c8360 2024-05-09T09:27:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.612+0000 7f2ad3fff640 1 --2- 172.21.15.12:0/1871507883 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c6de5c8360 0x55c6de5ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.612+0000 7f2ab3fff640 10 client.25933 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:20.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.612+0000 7f2ad3fff640 1 --2- 172.21.15.12:0/1871507883 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c6de5c8360 0x55c6de5ca750 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f25a5dde4c3c33 server_cookie=e27b5c2517d34c25 in_seq=0 out_seq=0 2024-05-09T09:27:20.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2ab3fff640 1 -- 172.21.15.12:0/1871507883 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c6de4b7410 con 0x55c6de5c8360 2024-05-09T09:27:20.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2ab3fff640 10 client.25933 handle_command_reply: tid=0 2024-05-09T09:27:20.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2addfa8740 1 client.25933 shutdown 2024-05-09T09:27:20.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2addfa8740 1 -- 172.21.15.12:0/1871507883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 msgr2=0x55c6de55a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2addfa8740 1 --2- 172.21.15.12:0/1871507883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de55a840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2acc031ab0 tx=0x7f2acc05b040 comp rx=0 tx=0).stop 2024-05-09T09:27:20.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2addfa8740 1 -- 172.21.15.12:0/1871507883 shutdown_connections 2024-05-09T09:27:20.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2addfa8740 1 --2- 172.21.15.12:0/1871507883 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55c6de5c8360 0x55c6de5ca750 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2addfa8740 1 --2- 172.21.15.12:0/1871507883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6de551020 0x55c6de55bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2addfa8740 1 --2- 172.21.15.12:0/1871507883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6de54beb0 0x55c6de55ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2addfa8740 1 --2- 172.21.15.12:0/1871507883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6de54b4c0 0x55c6de55a840 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2addfa8740 1 -- 172.21.15.12:0/1871507883 >> 172.21.15.12:0/1871507883 conn(0x55c6de5130a0 msgr2=0x55c6de549e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:20.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2ab3fff640 0 client.25933 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:20.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2addfa8740 1 -- 172.21.15.12:0/1871507883 shutdown_connections 2024-05-09T09:27:20.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.613+0000 7f2addfa8740 1 -- 172.21.15.12:0/1871507883 wait complete. 2024-05-09T09:27:20.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.614+0000 7f2addfa8740 20 client.25933 trim_cache size 0 max 16384 2024-05-09T09:27:20.620 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:27:20.620 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T09:27:20.620 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "32af2082-fd82-4318-82e8-5fdc47b2b926", 2024-05-09T09:27:20.621 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T09:27:20.621 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:27:20.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.614+0000 7f2adb1c4640 1 -- 172.21.15.12:0/1759417046 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ac40a2b90 msgr2=0x7f2ac40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.614+0000 7f2adb1c4640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ac40a2b90 0x7f2ac40a5050 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80cd430 tx=0x7f2ac8003ea0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.614+0000 7f2adb1c4640 1 -- 172.21.15.12:0/1759417046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ad4100770 msgr2=0x7f2ad410af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.614+0000 7f2adb1c4640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ad4100770 0x7f2ad410af70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800e9c0 tx=0x7f2ac800ee80 comp rx=0 tx=0).stop 2024-05-09T09:27:20.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.614+0000 7f2adb1c4640 1 -- 172.21.15.12:0/1759417046 shutdown_connections 2024-05-09T09:27:20.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.614+0000 7f2adb1c4640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2ac40a2b90 0x7f2ac40a5050 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.614+0000 7f2adb1c4640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad4102320 0x7f2ad4109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.614+0000 7f2adb1c4640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad4101960 0x7f2ad41095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.614+0000 7f2adb1c4640 1 --2- 172.21.15.12:0/1759417046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ad4100770 0x7f2ad410af70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.614+0000 7f2adb1c4640 1 -- 172.21.15.12:0/1759417046 >> 172.21.15.12:0/1759417046 conn(0x7f2ad40fbce0 msgr2=0x7f2ad4104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:20.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.614+0000 7f2adb1c4640 1 -- 172.21.15.12:0/1759417046 shutdown_connections 2024-05-09T09:27:20.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.614+0000 7f2adb1c4640 1 -- 172.21.15.12:0/1759417046 wait complete. 2024-05-09T09:27:20.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:27:20.627 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:27:20.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.819+0000 7f07e6ed4640 1 -- 172.21.15.12:0/3152816714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07e0101960 msgr2=0x7f07e0101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.819+0000 7f07e6ed4640 1 --2- 172.21.15.12:0/3152816714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07e0101960 0x7f07e0101de0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009900 tx=0x7f07cc02a410 comp rx=0 tx=0).stop 2024-05-09T09:27:20.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.820+0000 7f07e6ed4640 1 -- 172.21.15.12:0/3152816714 shutdown_connections 2024-05-09T09:27:20.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.820+0000 7f07e6ed4640 1 --2- 172.21.15.12:0/3152816714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07e0102320 0x7f07e0108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.820+0000 7f07e6ed4640 1 --2- 172.21.15.12:0/3152816714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07e0101960 0x7f07e0101de0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.820+0000 7f07e6ed4640 1 --2- 172.21.15.12:0/3152816714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07e0100770 0x7f07e0100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.820+0000 7f07e6ed4640 1 -- 172.21.15.12:0/3152816714 >> 172.21.15.12:0/3152816714 conn(0x7f07e00fbce0 msgr2=0x7f07e00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.820+0000 7f07e6ed4640 1 -- 172.21.15.12:0/3152816714 shutdown_connections 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.820+0000 7f07e6ed4640 1 -- 172.21.15.12:0/3152816714 wait complete. 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.820+0000 7f07e6ed4640 1 Processor -- start 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.820+0000 7f07e6ed4640 1 -- start start 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e6ed4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07e0100770 0x7f07e0199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e6ed4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07e0101960 0x7f07e019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e6ed4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07e0102320 0x7f07e01a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e6ed4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f07e010c3d0 con 0x7f07e0102320 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e6ed4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07e010c250 con 0x7f07e0100770 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e6ed4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f07e010c550 con 0x7f07e0101960 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e5471640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07e0102320 0x7f07e01a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e4c70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07e0100770 0x7f07e0199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e5471640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07e0102320 0x7f07e01a1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39010/0 (socket says 172.21.15.12:39010) 2024-05-09T09:27:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e5471640 1 -- 172.21.15.12:0/2547756385 learned_addr learned my addr 172.21.15.12:0/2547756385 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07dffff640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07e0101960 0x7f07e019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e4c70640 1 -- 172.21.15.12:0/2547756385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07e0101960 msgr2=0x7f07e019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e4c70640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07e0101960 0x7f07e019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e4c70640 1 -- 172.21.15.12:0/2547756385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07e0102320 msgr2=0x7f07e01a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e4c70640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07e0102320 0x7f07e01a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e4c70640 1 -- 172.21.15.12:0/2547756385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e01a1a00 con 0x7f07e0100770 2024-05-09T09:27:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e5471640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07e0102320 0x7f07e01a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.821+0000 7f07e4c70640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07e0100770 0x7f07e0199de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07d400e9c0 tx=0x7f07d400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.822+0000 7f07ddffb640 1 -- 172.21.15.12:0/2547756385 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d400ccc0 con 0x7f07e0100770 2024-05-09T09:27:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.822+0000 7f07e6ed4640 1 -- 172.21.15.12:0/2547756385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e01a1cf0 con 0x7f07e0100770 2024-05-09T09:27:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.822+0000 7f07ddffb640 1 -- 172.21.15.12:0/2547756385 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f07d40035f0 con 0x7f07e0100770 2024-05-09T09:27:20.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.822+0000 7f07ddffb640 1 -- 172.21.15.12:0/2547756385 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d400ce60 con 0x7f07e0100770 2024-05-09T09:27:20.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.822+0000 7f07e6ed4640 1 -- 172.21.15.12:0/2547756385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e01a21b0 con 0x7f07e0100770 2024-05-09T09:27:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.826+0000 7f07e9be8740 1 Processor -- start 2024-05-09T09:27:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.826+0000 7f07ddffb640 1 -- 172.21.15.12:0/2547756385 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f07d40037a0 con 0x7f07e0100770 2024-05-09T09:27:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.826+0000 7f07e9be8740 1 -- start start 2024-05-09T09:27:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e9be8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025eab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e9be8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025eb130 0x55f8025efc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e9be8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025f02a0 0x55f8025f2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e9be8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f802546d00 con 0x55f8025f02a0 2024-05-09T09:27:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e9be8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f8024ee4c0 con 0x55f8025eb130 2024-05-09T09:27:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e9be8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f802542cc0 con 0x55f8025ea740 2024-05-09T09:27:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e5471640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e5471640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025eab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:53304/0 (socket says 172.21.15.12:53304) 2024-05-09T09:27:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e5471640 1 -- 172.21.15.12:0/3275226335 learned_addr learned my addr 172.21.15.12:0/3275226335 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07dffff640 1 --2- 172.21.15.12:0/3275226335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025f02a0 0x55f8025f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e4c70640 1 --2- 172.21.15.12:0/3275226335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025eb130 0x55f8025efc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e5471640 1 -- 172.21.15.12:0/3275226335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025eb130 msgr2=0x55f8025efc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e5471640 1 --2- 172.21.15.12:0/3275226335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025eb130 0x55f8025efc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e5471640 1 -- 172.21.15.12:0/3275226335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025f02a0 msgr2=0x55f8025f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.827+0000 7f07e5471640 1 --2- 172.21.15.12:0/3275226335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025f02a0 0x55f8025f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.828+0000 7f07e5471640 1 -- 172.21.15.12:0/3275226335 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8023247d0 con 0x55f8025ea740 2024-05-09T09:27:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.828+0000 7f07ddffb640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f07d00a2c90 0x7f07d00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.828+0000 7f07e5471640 1 --2- 172.21.15.12:0/3275226335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025eab60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f07d80118b0 tx=0x7f07d8011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e50320524c750abf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.828+0000 7f07e4c70640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f07d00a2c90 0x7f07d00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.828+0000 7f07caffd640 1 -- 172.21.15.12:0/3275226335 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d801e070 con 0x55f8025ea740 2024-05-09T09:27:20.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.829+0000 7f07ddffb640 1 -- 172.21.15.12:0/2547756385 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f07d40cf820 con 0x7f07e0100770 2024-05-09T09:27:20.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.829+0000 7f07caffd640 1 -- 172.21.15.12:0/3275226335 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f07d800fe90 con 0x55f8025ea740 2024-05-09T09:27:20.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.829+0000 7f07caffd640 1 -- 172.21.15.12:0/3275226335 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8010c00 con 0x55f8025ea740 2024-05-09T09:27:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.830+0000 7f07e4c70640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f07d00a2c90 0x7f07d00a5150 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f07d4098620 tx=0x7f07d4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.830+0000 7f07e9be8740 1 -- 172.21.15.12:0/3275226335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 msgr2=0x55f8025eab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.830+0000 7f07e9be8740 1 --2- 172.21.15.12:0/3275226335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025eab60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f07d80118b0 tx=0x7f07d8011d70 comp rx=0 tx=0).stop 2024-05-09T09:27:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.830+0000 7f07e9be8740 1 -- 172.21.15.12:0/3275226335 shutdown_connections 2024-05-09T09:27:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.830+0000 7f07e9be8740 1 --2- 172.21.15.12:0/3275226335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025f02a0 0x55f8025f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.830+0000 7f07e9be8740 1 --2- 172.21.15.12:0/3275226335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025eb130 0x55f8025efc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.830+0000 7f07e9be8740 1 --2- 172.21.15.12:0/3275226335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025eab60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.830+0000 7f07e9be8740 1 -- 172.21.15.12:0/3275226335 >> 172.21.15.12:0/3275226335 conn(0x55f8025e5db0 msgr2=0x55f8025e61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.830+0000 7f07e9be8740 1 -- 172.21.15.12:0/3275226335 shutdown_connections 2024-05-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.830+0000 7f07e9be8740 1 -- 172.21.15.12:0/3275226335 wait complete. 2024-05-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07e9be8740 1 -- start start 2024-05-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07e9be8740 1 Processor -- start 2024-05-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07e9be8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07e9be8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025eb130 0x55f8025f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07e9be8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025f02a0 0x55f8025fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07e9be8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f8025f3250 con 0x55f8025eb130 2024-05-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07e9be8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f802501b90 con 0x55f8025f02a0 2024-05-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07e9be8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f8025f33d0 con 0x55f8025ea740 2024-05-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07e9be8740 10 client.0 _ll_register_callbacks cb 0x55f80253dda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07e5471640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025f02a0 0x55f8025fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07e5471640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07e5471640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025f48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:53314/0 (socket says 172.21.15.12:53314) 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07dffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025eb130 0x55f8025f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.831+0000 7f07e5471640 1 -- 172.21.15.12:0/3308968054 learned_addr learned my addr 172.21.15.12:0/3308968054 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.832+0000 7f07e5471640 1 -- 172.21.15.12:0/3308968054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 msgr2=0x55f8025f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.832+0000 7f07e5471640 1 --2- 172.21.15.12:0/3308968054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.832+0000 7f07e5471640 1 -- 172.21.15.12:0/3308968054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025eb130 msgr2=0x55f8025f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.832+0000 7f07e5471640 1 --2- 172.21.15.12:0/3308968054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025eb130 0x55f8025f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.832+0000 7f07e5471640 1 -- 172.21.15.12:0/3308968054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f80225a3f0 con 0x55f8025f02a0 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.832+0000 7f07dffff640 1 --2- 172.21.15.12:0/3308968054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025eb130 0x55f8025f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.832+0000 7f07e5471640 1 --2- 172.21.15.12:0/3308968054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025f02a0 0x55f8025fbe50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f07d8011880 tx=0x7f07d8011d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3661399f789801af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.832+0000 7f07bffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.832+0000 7f07bffff640 1 -- 172.21.15.12:0/3308968054 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d801e070 con 0x55f8025f02a0 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.832+0000 7f07bffff640 1 -- 172.21.15.12:0/3308968054 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f07d8010de0 con 0x55f8025f02a0 2024-05-09T09:27:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.833+0000 7f07e9be8740 1 -- 172.21.15.12:0/3308968054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d800fe90 con 0x55f8025f02a0 2024-05-09T09:27:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.833+0000 7f07bffff640 1 -- 172.21.15.12:0/3308968054 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f07d8034050 con 0x55f8025f02a0 2024-05-09T09:27:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.833+0000 7f07e9be8740 10 client.25838 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.833+0000 7f07e9be8740 1 -- 172.21.15.12:0/3308968054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f80221a0f0 con 0x55f8025f02a0 2024-05-09T09:27:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.833+0000 7f07bffff640 1 -- 172.21.15.12:0/3308968054 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f07d800fe90 con 0x55f8025f02a0 2024-05-09T09:27:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.834+0000 7f07bffff640 1 -- 172.21.15.12:0/3308968054 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f07d8005670 con 0x55f8025f02a0 2024-05-09T09:27:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.834+0000 7f07e9be8740 10 client.25838 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.834+0000 7f07e9be8740 10 client.25838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.834+0000 7f07e9be8740 20 client.25838 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.834+0000 7f07e9be8740 1 --2- 172.21.15.12:0/3308968054 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f802600550 0x55f802602970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.834+0000 7f07e9be8740 4 client.25838 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.834+0000 7f07e9be8740 1 -- 172.21.15.12:0/3308968054 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f802554430 con 0x55f802600550 2024-05-09T09:27:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.834+0000 7f07dffff640 1 --2- 172.21.15.12:0/3308968054 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f802600550 0x55f802602970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.835+0000 7f07dffff640 1 --2- 172.21.15.12:0/3308968054 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f802600550 0x55f802602970 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a0b67cdfd6b19f9 server_cookie=98532ff39997dead in_seq=0 out_seq=0 2024-05-09T09:27:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.835+0000 7f07bffff640 10 client.25838 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.836+0000 7f07bffff640 1 -- 172.21.15.12:0/3308968054 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f802554430 con 0x55f802600550 2024-05-09T09:27:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.836+0000 7f07bffff640 10 client.25838 handle_command_reply: tid=0 2024-05-09T09:27:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.837+0000 7f07e9be8740 1 client.25838 shutdown 2024-05-09T09:27:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.837+0000 7f07e9be8740 1 -- 172.21.15.12:0/3308968054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025f02a0 msgr2=0x55f8025fbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.837+0000 7f07e9be8740 1 --2- 172.21.15.12:0/3308968054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025f02a0 0x55f8025fbe50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f07d8011880 tx=0x7f07d8011d70 comp rx=0 tx=0).stop 2024-05-09T09:27:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.837+0000 7f07e9be8740 1 -- 172.21.15.12:0/3308968054 shutdown_connections 2024-05-09T09:27:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.837+0000 7f07e9be8740 1 --2- 172.21.15.12:0/3308968054 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f802600550 0x55f802602970 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.837+0000 7f07e9be8740 1 --2- 172.21.15.12:0/3308968054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025f02a0 0x55f8025fbe50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.837+0000 7f07e9be8740 1 --2- 172.21.15.12:0/3308968054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025eb130 0x55f8025f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.837+0000 7f07e9be8740 1 --2- 172.21.15.12:0/3308968054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.837+0000 7f07bffff640 0 client.25838 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.837+0000 7f07e9be8740 1 -- 172.21.15.12:0/3308968054 >> 172.21.15.12:0/3308968054 conn(0x55f8025e5db0 msgr2=0x55f8025dc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.837+0000 7f07e9be8740 1 -- 172.21.15.12:0/3308968054 shutdown_connections 2024-05-09T09:27:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.837+0000 7f07e9be8740 1 -- 172.21.15.12:0/3308968054 wait complete. 2024-05-09T09:27:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.837+0000 7f07e9be8740 20 client.25838 trim_cache size 0 max 16384 2024-05-09T09:27:20.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.860+0000 7f07e9be8740 1 Processor -- start 2024-05-09T09:27:20.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.860+0000 7f07e9be8740 1 -- start start 2024-05-09T09:27:20.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.860+0000 7f07e9be8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025e5db0 0x55f8025ec2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.860+0000 7f07e9be8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025ea740 0x55f8025fbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.860+0000 7f07e9be8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025eb130 0x55f8025ec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.860+0000 7f07e9be8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01a83e0 con 0x55f8025e5db0 2024-05-09T09:27:20.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.860+0000 7f07e9be8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01a9fb0 con 0x55f8025ea740 2024-05-09T09:27:20.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.860+0000 7f07e9be8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01a8ab0 con 0x55f8025eb130 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.860+0000 7f07e5471640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025e5db0 0x55f8025ec2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.860+0000 7f07e5471640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025e5db0 0x55f8025ec2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39044/0 (socket says 172.21.15.12:39044) 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.860+0000 7f07e5471640 1 -- 172.21.15.12:0/2762748143 learned_addr learned my addr 172.21.15.12:0/2762748143 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.860+0000 7f07dffff640 1 --2- 172.21.15.12:0/2762748143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025ea740 0x55f8025fbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.861+0000 7f07e5471640 1 --2- 172.21.15.12:0/2762748143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025eb130 0x55f8025ec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.861+0000 7f07dffff640 1 -- 172.21.15.12:0/2762748143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025eb130 msgr2=0x55f8025ec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.861+0000 7f07dffff640 1 --2- 172.21.15.12:0/2762748143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025eb130 0x55f8025ec7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.861+0000 7f07dffff640 1 -- 172.21.15.12:0/2762748143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025e5db0 msgr2=0x55f8025ec2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.861+0000 7f07dffff640 1 --2- 172.21.15.12:0/2762748143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025e5db0 0x55f8025ec2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.861+0000 7f07dffff640 1 -- 172.21.15.12:0/2762748143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8025ea380 con 0x55f8025ea740 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.861+0000 7f07dffff640 1 --2- 172.21.15.12:0/2762748143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025ea740 0x55f8025fbe10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f07cc005ab0 tx=0x7f07cc031510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb5e1ecc32fb0e88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.861+0000 7f07ca7fc640 1 -- 172.21.15.12:0/2762748143 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc035040 con 0x55f8025ea740 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.861+0000 7f07ca7fc640 1 -- 172.21.15.12:0/2762748143 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f07cc046dc0 con 0x55f8025ea740 2024-05-09T09:27:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.861+0000 7f07ca7fc640 1 -- 172.21.15.12:0/2762748143 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0467d0 con 0x55f8025ea740 2024-05-09T09:27:20.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.862+0000 7f07e9be8740 1 -- 172.21.15.12:0/2762748143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025ea740 msgr2=0x55f8025fbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.862+0000 7f07e9be8740 1 --2- 172.21.15.12:0/2762748143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025ea740 0x55f8025fbe10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f07cc005ab0 tx=0x7f07cc031510 comp rx=0 tx=0).stop 2024-05-09T09:27:20.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.862+0000 7f07e9be8740 1 -- 172.21.15.12:0/2762748143 shutdown_connections 2024-05-09T09:27:20.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.862+0000 7f07e9be8740 1 --2- 172.21.15.12:0/2762748143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025eb130 0x55f8025ec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.862+0000 7f07e9be8740 1 --2- 172.21.15.12:0/2762748143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025ea740 0x55f8025fbe10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.862+0000 7f07e9be8740 1 --2- 172.21.15.12:0/2762748143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025e5db0 0x55f8025ec2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.862+0000 7f07e9be8740 1 -- 172.21.15.12:0/2762748143 >> 172.21.15.12:0/2762748143 conn(0x55f8025f3ae0 msgr2=0x55f8025f48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.862+0000 7f07e9be8740 1 -- 172.21.15.12:0/2762748143 shutdown_connections 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.862+0000 7f07e9be8740 1 -- 172.21.15.12:0/2762748143 wait complete. 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.863+0000 7f07e9be8740 1 -- start start 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.863+0000 7f07e9be8740 1 Processor -- start 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.863+0000 7f07e9be8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025e5db0 0x55f8025f9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.863+0000 7f07e9be8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025f9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.863+0000 7f07e9be8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025eb130 0x55f8025fa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.863+0000 7f07e9be8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01a9850 con 0x55f8025eb130 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.863+0000 7f07e9be8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01a9180 con 0x55f8025e5db0 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.863+0000 7f07e9be8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01a6400 con 0x55f8025ea740 2024-05-09T09:27:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.863+0000 7f07e9be8740 10 client.0 _ll_register_callbacks cb 0x7f07d800fc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.863+0000 7f07e5471640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025e5db0 0x55f8025f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.863+0000 7f07e5471640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025eb130 0x55f8025fa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.863+0000 7f07e5471640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025eb130 0x55f8025fa790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39054/0 (socket says 172.21.15.12:39054) 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.863+0000 7f07e5471640 1 -- 172.21.15.12:0/336661498 learned_addr learned my addr 172.21.15.12:0/336661498 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.864+0000 7f07dffff640 1 --2- 172.21.15.12:0/336661498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.864+0000 7f07e5471640 1 -- 172.21.15.12:0/336661498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 msgr2=0x55f8025f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.864+0000 7f07e5471640 1 --2- 172.21.15.12:0/336661498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.864+0000 7f07e5471640 1 -- 172.21.15.12:0/336661498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025e5db0 msgr2=0x55f8025f9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.864+0000 7f07e5471640 1 --2- 172.21.15.12:0/336661498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025e5db0 0x55f8025f9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.864+0000 7f07e5471640 1 -- 172.21.15.12:0/336661498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8025fad80 con 0x55f8025eb130 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.864+0000 7f07e5471640 1 --2- 172.21.15.12:0/336661498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025eb130 0x55f8025fa790 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f07d8029600 tx=0x7f07d8009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67fbf303fc62ff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.864+0000 7f07bffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.864+0000 7f07bffff640 1 -- 172.21.15.12:0/336661498 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d801e070 con 0x55f8025eb130 2024-05-09T09:27:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.864+0000 7f07bffff640 1 -- 172.21.15.12:0/336661498 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f07d80299a0 con 0x55f8025eb130 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.864+0000 7f07e9be8740 1 -- 172.21.15.12:0/336661498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07cc046dc0 con 0x55f8025eb130 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.865+0000 7f07bffff640 1 -- 172.21.15.12:0/336661498 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f07d8029ba0 con 0x55f8025eb130 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.865+0000 7f07e9be8740 10 client.16392 fetch_fsmap learned FSMap version 31 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.865+0000 7f07e9be8740 1 -- 172.21.15.12:0/336661498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f8025fc090 con 0x55f8025eb130 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.865+0000 7f07bffff640 1 -- 172.21.15.12:0/336661498 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f07cc046dc0 con 0x55f8025eb130 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.865+0000 7f07bffff640 1 -- 172.21.15.12:0/336661498 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f07d801aa70 con 0x55f8025eb130 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.866+0000 7f07e9be8740 10 client.16392 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.866+0000 7f07e9be8740 10 client.16392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.866+0000 7f07e9be8740 20 client.16392 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.866+0000 7f07e9be8740 1 --2- 172.21.15.12:0/336661498 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f8025f02a0 0x55f802669250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.866+0000 7f07e9be8740 4 client.16392 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.866+0000 7f07e9be8740 1 -- 172.21.15.12:0/336661498 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f80254e780 con 0x55f8025f02a0 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.866+0000 7f07dffff640 1 --2- 172.21.15.12:0/336661498 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f8025f02a0 0x55f802669250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.866+0000 7f07dffff640 1 --2- 172.21.15.12:0/336661498 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f8025f02a0 0x55f802669250 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e38ac1cbc75fea1 server_cookie=9f6794f4952cdce0 in_seq=0 out_seq=0 2024-05-09T09:27:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.866+0000 7f07bffff640 10 client.16392 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.867+0000 7f07bffff640 1 -- 172.21.15.12:0/336661498 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55f80254e780 con 0x55f8025f02a0 2024-05-09T09:27:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.867+0000 7f07bffff640 10 client.16392 handle_command_reply: tid=0 2024-05-09T09:27:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.867+0000 7f07e9be8740 1 client.16392 shutdown 2024-05-09T09:27:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.868+0000 7f07e9be8740 1 -- 172.21.15.12:0/336661498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025eb130 msgr2=0x55f8025fa790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.868+0000 7f07e9be8740 1 --2- 172.21.15.12:0/336661498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025eb130 0x55f8025fa790 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f07d8029600 tx=0x7f07d8009fa0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.868+0000 7f07e9be8740 1 -- 172.21.15.12:0/336661498 shutdown_connections 2024-05-09T09:27:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.868+0000 7f07e9be8740 1 --2- 172.21.15.12:0/336661498 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f8025f02a0 0x55f802669250 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.868+0000 7f07e9be8740 1 --2- 172.21.15.12:0/336661498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8025eb130 0x55f8025fa790 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.868+0000 7f07bffff640 0 client.16392 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.868+0000 7f07e9be8740 1 --2- 172.21.15.12:0/336661498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8025ea740 0x55f8025f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.868+0000 7f07e9be8740 1 --2- 172.21.15.12:0/336661498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8025e5db0 0x55f8025f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.868+0000 7f07e9be8740 1 -- 172.21.15.12:0/336661498 >> 172.21.15.12:0/336661498 conn(0x55f8025f3ae0 msgr2=0x55f8025f47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:20.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.868+0000 7f07e9be8740 1 -- 172.21.15.12:0/336661498 shutdown_connections 2024-05-09T09:27:20.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.868+0000 7f07e9be8740 1 -- 172.21.15.12:0/336661498 wait complete. 2024-05-09T09:27:20.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.868+0000 7f07e9be8740 20 client.16392 trim_cache size 0 max 16384 2024-05-09T09:27:20.872 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:27:20.872 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (674 inodes in the stack)", 2024-05-09T09:27:20.872 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T09:27:20.872 INFO:teuthology.orchestra.run.smithi012.stdout: "32af2082-fd82-4318-82e8-5fdc47b2b926": { 2024-05-09T09:27:20.872 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T09:27:20.872 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "32af2082-fd82-4318-82e8-5fdc47b2b926", 2024-05-09T09:27:20.872 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T09:27:20.873 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:27:20.873 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:27:20.873 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:27:20.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.869+0000 7f07e6ed4640 1 -- 172.21.15.12:0/2547756385 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f07d00a2c90 msgr2=0x7f07d00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.869+0000 7f07e6ed4640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f07d00a2c90 0x7f07d00a5150 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f07d4098620 tx=0x7f07d4000bc0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.869+0000 7f07e6ed4640 1 -- 172.21.15.12:0/2547756385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07e0100770 msgr2=0x7f07e0199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:20.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.869+0000 7f07e6ed4640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07e0100770 0x7f07e0199de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07d400e9c0 tx=0x7f07d400ee80 comp rx=0 tx=0).stop 2024-05-09T09:27:20.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.869+0000 7f07e6ed4640 1 -- 172.21.15.12:0/2547756385 shutdown_connections 2024-05-09T09:27:20.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.869+0000 7f07e6ed4640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f07d00a2c90 0x7f07d00a5150 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.869+0000 7f07e6ed4640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07e0102320 0x7f07e01a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.869+0000 7f07e6ed4640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07e0101960 0x7f07e019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.869+0000 7f07e6ed4640 1 --2- 172.21.15.12:0/2547756385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07e0100770 0x7f07e0199de0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:20.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.869+0000 7f07e6ed4640 1 -- 172.21.15.12:0/2547756385 >> 172.21.15.12:0/2547756385 conn(0x7f07e00fbce0 msgr2=0x7f07e00fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:20.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.870+0000 7f07e6ed4640 1 -- 172.21.15.12:0/2547756385 shutdown_connections 2024-05-09T09:27:20.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:20.870+0000 7f07e6ed4640 1 -- 172.21.15.12:0/2547756385 wait complete. 2024-05-09T09:27:20.882 INFO:tasks.cephfs.filesystem:scrub status for tag:32af2082-fd82-4318-82e8-5fdc47b2b926 - {'path': '/', 'tag': '32af2082-fd82-4318-82e8-5fdc47b2b926', 'options': 'recursive,force'} 2024-05-09T09:27:21.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:21 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:21.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:21 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:21.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:21 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:22 smithi012 ceph-mon[30140]: scrub queued for path: / 2024-05-09T09:27:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:22 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:22 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:22 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:22 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:22 smithi012 ceph-mon[30140]: pgmap v421: 129 pgs: 129 active+clean; 103 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 1.03k op/s 2024-05-09T09:27:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:22 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:22 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:22 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:22 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:22 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:27:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:22 smithi170 ceph-mon[33632]: scrub queued for path: / 2024-05-09T09:27:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:22 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:22 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:22 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:22 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:22 smithi170 ceph-mon[33632]: pgmap v421: 129 pgs: 129 active+clean; 103 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 1.03k op/s 2024-05-09T09:27:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:22 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:22 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:22 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:22 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:22 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:27:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:22 smithi016 ceph-mon[33335]: scrub queued for path: / 2024-05-09T09:27:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:22 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:22 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:22 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:22 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:22 smithi016 ceph-mon[33335]: pgmap v421: 129 pgs: 129 active+clean; 103 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 1.03k op/s 2024-05-09T09:27:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:22 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:22 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:22 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:22 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:22.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:22 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:27:24.454 INFO:tasks.workunit.client.0.smithi012.stderr:+ tar xf doc.tar 2024-05-09T09:27:24.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:24 smithi012 ceph-mon[30140]: pgmap v422: 129 pgs: 129 active+clean; 104 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.6 MiB/s wr, 1.04k op/s 2024-05-09T09:27:24.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:24 smithi170 ceph-mon[33632]: pgmap v422: 129 pgs: 129 active+clean; 104 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.6 MiB/s wr, 1.04k op/s 2024-05-09T09:27:24.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:24 smithi016 ceph-mon[33335]: pgmap v422: 129 pgs: 129 active+clean; 104 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.6 MiB/s wr, 1.04k op/s 2024-05-09T09:27:26.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:26 smithi012 ceph-mon[30140]: pgmap v423: 129 pgs: 129 active+clean; 113 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.9 MiB/s wr, 1.14k op/s 2024-05-09T09:27:26.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:26 smithi170 ceph-mon[33632]: pgmap v423: 129 pgs: 129 active+clean; 113 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.9 MiB/s wr, 1.14k op/s 2024-05-09T09:27:26.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:26 smithi016 ceph-mon[33335]: pgmap v423: 129 pgs: 129 active+clean; 113 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.9 MiB/s wr, 1.14k op/s 2024-05-09T09:27:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.406+0000 7fbc23a5d640 1 -- 172.21.15.12:0/3695978622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc1c1018c0 msgr2=0x7fbc1c101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.406+0000 7fbc23a5d640 1 --2- 172.21.15.12:0/3695978622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc1c1018c0 0x7fbc1c101ce0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009900 tx=0x7fbc1802a410 comp rx=0 tx=0).stop 2024-05-09T09:27:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.406+0000 7fbc23a5d640 1 -- 172.21.15.12:0/3695978622 shutdown_connections 2024-05-09T09:27:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.406+0000 7fbc23a5d640 1 --2- 172.21.15.12:0/3695978622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1c103470 0x7fbc1c109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.406+0000 7fbc23a5d640 1 --2- 172.21.15.12:0/3695978622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc1c102ab0 0x7fbc1c102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.406+0000 7fbc23a5d640 1 --2- 172.21.15.12:0/3695978622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc1c1018c0 0x7fbc1c101ce0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.406+0000 7fbc23a5d640 1 -- 172.21.15.12:0/3695978622 >> 172.21.15.12:0/3695978622 conn(0x7fbc1c0fce50 msgr2=0x7fbc1c0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.406+0000 7fbc23a5d640 1 -- 172.21.15.12:0/3695978622 shutdown_connections 2024-05-09T09:27:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.406+0000 7fbc23a5d640 1 -- 172.21.15.12:0/3695978622 wait complete. 2024-05-09T09:27:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.406+0000 7fbc23a5d640 1 Processor -- start 2024-05-09T09:27:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.406+0000 7fbc23a5d640 1 -- start start 2024-05-09T09:27:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc23a5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc1c1018c0 0x7fbc1c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc23a5d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1c102ab0 0x7fbc1c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc23a5d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc1c103470 0x7fbc1c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc23a5d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c10d520 con 0x7fbc1c103470 2024-05-09T09:27:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc23a5d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c10d3a0 con 0x7fbc1c1018c0 2024-05-09T09:27:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc23a5d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c10d6a0 con 0x7fbc1c102ab0 2024-05-09T09:27:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc217f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc1c1018c0 0x7fbc1c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc217f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc1c1018c0 0x7fbc1c197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:46892/0 (socket says 172.21.15.12:46892) 2024-05-09T09:27:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc21ffa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc1c103470 0x7fbc1c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc20ff8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1c102ab0 0x7fbc1c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc217f9640 1 -- 172.21.15.12:0/1535962229 learned_addr learned my addr 172.21.15.12:0/1535962229 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc21ffa640 1 -- 172.21.15.12:0/1535962229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1c102ab0 msgr2=0x7fbc1c198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc21ffa640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1c102ab0 0x7fbc1c198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc21ffa640 1 -- 172.21.15.12:0/1535962229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc1c1018c0 msgr2=0x7fbc1c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc21ffa640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc1c1018c0 0x7fbc1c197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc21ffa640 1 -- 172.21.15.12:0/1535962229 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c19f750 con 0x7fbc1c103470 2024-05-09T09:27:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc217f9640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc1c1018c0 0x7fbc1c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1c102ab0 0x7fbc1c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.407+0000 7fbc21ffa640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc1c103470 0x7fbc1c19f0e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009fd0 tx=0x7fbc1400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.408+0000 7fbc127fc640 1 -- 172.21.15.12:0/1535962229 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1400ce20 con 0x7fbc1c103470 2024-05-09T09:27:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.408+0000 7fbc127fc640 1 -- 172.21.15.12:0/1535962229 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc14005020 con 0x7fbc1c103470 2024-05-09T09:27:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.408+0000 7fbc23a5d640 1 -- 172.21.15.12:0/1535962229 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1c19fa40 con 0x7fbc1c103470 2024-05-09T09:27:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.408+0000 7fbc127fc640 1 -- 172.21.15.12:0/1535962229 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc140106b0 con 0x7fbc1c103470 2024-05-09T09:27:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.408+0000 7fbc23a5d640 1 -- 172.21.15.12:0/1535962229 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1c19ff90 con 0x7fbc1c103470 2024-05-09T09:27:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.410+0000 7fbc127fc640 1 -- 172.21.15.12:0/1535962229 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbc140051d0 con 0x7fbc1c103470 2024-05-09T09:27:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc2676d740 1 Processor -- start 2024-05-09T09:27:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc2676d740 1 -- start start 2024-05-09T09:27:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc2676d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dbb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc2676d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dbbc40 0x55f966dc0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc2676d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dc0db0 0x55f966dc3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc2676d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f966d1f340 con 0x55f966dbbc40 2024-05-09T09:27:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc2676d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f966cc7290 con 0x55f966dc0db0 2024-05-09T09:27:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc2676d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f966d1b300 con 0x55f966dbb250 2024-05-09T09:27:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc20ff8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dbb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc20ff8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dbb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:39504/0 (socket says 172.21.15.12:39504) 2024-05-09T09:27:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc20ff8640 1 -- 172.21.15.12:0/2732014084 learned_addr learned my addr 172.21.15.12:0/2732014084 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc127fc640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbc080a2c30 0x7fbc080a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc217f9640 1 --2- 172.21.15.12:0/2732014084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dbbc40 0x55f966dc0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/2732014084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dc0db0 0x55f966dc3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc20ff8640 1 -- 172.21.15.12:0/2732014084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dc0db0 msgr2=0x55f966dc3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.411+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/2732014084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dc0db0 0x55f966dc3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.412+0000 7fbc20ff8640 1 -- 172.21.15.12:0/2732014084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dbbc40 msgr2=0x55f966dc0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.412+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/2732014084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dbbc40 0x55f966dc0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.412+0000 7fbc21ffa640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbc080a2c30 0x7fbc080a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.412+0000 7fbc20ff8640 1 -- 172.21.15.12:0/2732014084 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f966ceb8e0 con 0x55f966dbb250 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.412+0000 7fbc217f9640 1 --2- 172.21.15.12:0/2732014084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dbbc40 0x55f966dc0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.412+0000 7fbc127fc640 1 -- 172.21.15.12:0/1535962229 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbc140cfad0 con 0x7fbc1c103470 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.412+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/2732014084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dbb670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c005ca0 tx=0x7fbc0c00bbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e61e8e2b2798d79c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.412+0000 7fbc21ffa640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbc080a2c30 0x7fbc080a50f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbc1409bef0 tx=0x7fbc140bc040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.412+0000 7fbc077fe640 1 -- 172.21.15.12:0/2732014084 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c012cd0 con 0x55f966dbb250 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.412+0000 7fbc077fe640 1 -- 172.21.15.12:0/2732014084 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc0c010c00 con 0x55f966dbb250 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.412+0000 7fbc077fe640 1 -- 172.21.15.12:0/2732014084 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c003b60 con 0x55f966dbb250 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.413+0000 7fbc2676d740 1 -- 172.21.15.12:0/2732014084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 msgr2=0x55f966dbb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.413+0000 7fbc2676d740 1 --2- 172.21.15.12:0/2732014084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dbb670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c005ca0 tx=0x7fbc0c00bbf0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.413+0000 7fbc2676d740 1 -- 172.21.15.12:0/2732014084 shutdown_connections 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.413+0000 7fbc2676d740 1 --2- 172.21.15.12:0/2732014084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dc0db0 0x55f966dc3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.413+0000 7fbc2676d740 1 --2- 172.21.15.12:0/2732014084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dbbc40 0x55f966dc0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.413+0000 7fbc2676d740 1 --2- 172.21.15.12:0/2732014084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dbb670 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.413+0000 7fbc2676d740 1 -- 172.21.15.12:0/2732014084 >> 172.21.15.12:0/2732014084 conn(0x55f966db68c0 msgr2=0x55f966db6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.413+0000 7fbc2676d740 1 -- 172.21.15.12:0/2732014084 shutdown_connections 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.413+0000 7fbc2676d740 1 -- 172.21.15.12:0/2732014084 wait complete. 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.413+0000 7fbc2676d740 1 -- start start 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.413+0000 7fbc2676d740 1 Processor -- start 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc2676d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dc5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc2676d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dbbc40 0x55f966dc5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc2676d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 0x55f966dccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc2676d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f966dc3d60 con 0x55f966dc0db0 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc2676d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f966cda790 con 0x55f966dbbc40 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc2676d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f966dc3ee0 con 0x55f966dbb250 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc2676d740 10 client.0 _ll_register_callbacks cb 0x55f966d15a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc20ff8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dc5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc20ff8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 0x55f966dccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc20ff8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 0x55f966dccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60900/0 (socket says 172.21.15.12:60900) 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc20ff8640 1 -- 172.21.15.12:0/1917598352 learned_addr learned my addr 172.21.15.12:0/1917598352 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc217f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dbbc40 0x55f966dc5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc20ff8640 1 -- 172.21.15.12:0/1917598352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 msgr2=0x55f966dc5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/1917598352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dc5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc20ff8640 1 -- 172.21.15.12:0/1917598352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dbbc40 msgr2=0x55f966dc5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/1917598352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dbbc40 0x55f966dc5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc20ff8640 1 -- 172.21.15.12:0/1917598352 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f966a33210 con 0x55f966dc0db0 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.414+0000 7fbc217f9640 1 --2- 172.21.15.12:0/1917598352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dbbc40 0x55f966dc5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.415+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/1917598352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 0x55f966dccaa0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00bfd0 tx=0x7fbc0c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7c8aca550933c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.415+0000 7fbc04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.415+0000 7fbc04ff9640 1 -- 172.21.15.12:0/1917598352 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c027040 con 0x55f966dc0db0 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.415+0000 7fbc2676d740 1 -- 172.21.15.12:0/1917598352 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc0c010c00 con 0x55f966dc0db0 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.415+0000 7fbc04ff9640 1 -- 172.21.15.12:0/1917598352 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbc0c005520 con 0x55f966dc0db0 2024-05-09T09:27:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.415+0000 7fbc04ff9640 1 -- 172.21.15.12:0/1917598352 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbc0c035050 con 0x55f966dc0db0 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.415+0000 7fbc2676d740 10 client.16404 fetch_fsmap learned FSMap version 32 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.415+0000 7fbc2676d740 1 -- 172.21.15.12:0/1917598352 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f966aff620 con 0x55f966dc0db0 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.415+0000 7fbc04ff9640 1 -- 172.21.15.12:0/1917598352 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc0c010c00 con 0x55f966dc0db0 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.415+0000 7fbc04ff9640 1 -- 172.21.15.12:0/1917598352 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fbc0c0207a0 con 0x55f966dc0db0 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.416+0000 7fbc2676d740 10 client.16404 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.416+0000 7fbc2676d740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.416+0000 7fbc2676d740 20 client.16404 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.416+0000 7fbc2676d740 1 --2- 172.21.15.12:0/1917598352 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f966dd0df0 0x55f966dd3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.416+0000 7fbc2676d740 4 client.16404 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.416+0000 7fbc2676d740 1 -- 172.21.15.12:0/1917598352 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f966d2ca70 con 0x55f966dd0df0 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.416+0000 7fbc217f9640 1 --2- 172.21.15.12:0/1917598352 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f966dd0df0 0x55f966dd3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.416+0000 7fbc04ff9640 10 client.16404 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.416+0000 7fbc217f9640 1 --2- 172.21.15.12:0/1917598352 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f966dd0df0 0x55f966dd3200 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de37c9cf4868f8e9 server_cookie=dcfa29b163a954f7 in_seq=0 out_seq=0 2024-05-09T09:27:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc04ff9640 1 -- 172.21.15.12:0/1917598352 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f966d2ca70 con 0x55f966dd0df0 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc04ff9640 10 client.16404 handle_command_reply: tid=0 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc2676d740 1 client.16404 shutdown 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc2676d740 1 -- 172.21.15.12:0/1917598352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 msgr2=0x55f966dccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc2676d740 1 --2- 172.21.15.12:0/1917598352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 0x55f966dccaa0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00bfd0 tx=0x7fbc0c0035f0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc2676d740 1 -- 172.21.15.12:0/1917598352 shutdown_connections 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc2676d740 1 --2- 172.21.15.12:0/1917598352 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f966dd0df0 0x55f966dd3200 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc2676d740 1 --2- 172.21.15.12:0/1917598352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 0x55f966dccaa0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc2676d740 1 --2- 172.21.15.12:0/1917598352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dbbc40 0x55f966dc5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc2676d740 1 --2- 172.21.15.12:0/1917598352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dc5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc2676d740 1 -- 172.21.15.12:0/1917598352 >> 172.21.15.12:0/1917598352 conn(0x55f966db68c0 msgr2=0x55f966db5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc04ff9640 0 client.16404 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc2676d740 1 -- 172.21.15.12:0/1917598352 shutdown_connections 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.417+0000 7fbc2676d740 1 -- 172.21.15.12:0/1917598352 wait complete. 2024-05-09T09:27:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.418+0000 7fbc2676d740 20 client.16404 trim_cache size 0 max 16384 2024-05-09T09:27:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc2676d740 1 Processor -- start 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc2676d740 1 -- start start 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc2676d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dbb250 0x55f966dd8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc2676d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbbc40 0x55f966dd9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc2676d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dc0db0 0x55f966dcca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc2676d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1aa5f0 con 0x55f966dbb250 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc2676d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1ac1f0 con 0x55f966dc0db0 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc2676d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1aacc0 con 0x55f966dbbc40 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc20ff8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dbb250 0x55f966dd8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc20ff8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dbb250 0x55f966dd8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60916/0 (socket says 172.21.15.12:60916) 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc20ff8640 1 -- 172.21.15.12:0/2870774989 learned_addr learned my addr 172.21.15.12:0/2870774989 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc217f9640 1 --2- 172.21.15.12:0/2870774989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbbc40 0x55f966dd9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc20ff8640 1 -- 172.21.15.12:0/2870774989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbbc40 msgr2=0x55f966dd9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/2870774989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbbc40 0x55f966dd9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc20ff8640 1 -- 172.21.15.12:0/2870774989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dc0db0 msgr2=0x55f966dcca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/2870774989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dc0db0 0x55f966dcca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc20ff8640 1 -- 172.21.15.12:0/2870774989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f966dc45e0 con 0x55f966dbb250 2024-05-09T09:27:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.438+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/2870774989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dbb250 0x55f966dd8b10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0124b0 tx=0x7fbc0c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c818e77a960270 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.439+0000 7fbc06ffd640 1 -- 172.21.15.12:0/2870774989 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c020d90 con 0x55f966dbb250 2024-05-09T09:27:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.439+0000 7fbc06ffd640 1 -- 172.21.15.12:0/2870774989 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc0c016070 con 0x55f966dbb250 2024-05-09T09:27:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.439+0000 7fbc06ffd640 1 -- 172.21.15.12:0/2870774989 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c004d90 con 0x55f966dbb250 2024-05-09T09:27:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.439+0000 7fbc2676d740 1 -- 172.21.15.12:0/2870774989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dbb250 msgr2=0x55f966dd8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.439+0000 7fbc2676d740 1 --2- 172.21.15.12:0/2870774989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dbb250 0x55f966dd8b10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0124b0 tx=0x7fbc0c00bf20 comp rx=0 tx=0).stop 2024-05-09T09:27:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.439+0000 7fbc2676d740 1 -- 172.21.15.12:0/2870774989 shutdown_connections 2024-05-09T09:27:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.439+0000 7fbc2676d740 1 --2- 172.21.15.12:0/2870774989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dc0db0 0x55f966dcca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.439+0000 7fbc2676d740 1 --2- 172.21.15.12:0/2870774989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbbc40 0x55f966dd9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.439+0000 7fbc2676d740 1 --2- 172.21.15.12:0/2870774989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dbb250 0x55f966dd8b10 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.439+0000 7fbc2676d740 1 -- 172.21.15.12:0/2870774989 >> 172.21.15.12:0/2870774989 conn(0x55f966db68c0 msgr2=0x55f966db6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.440+0000 7fbc2676d740 1 -- 172.21.15.12:0/2870774989 shutdown_connections 2024-05-09T09:27:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.440+0000 7fbc2676d740 1 -- 172.21.15.12:0/2870774989 wait complete. 2024-05-09T09:27:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.440+0000 7fbc2676d740 1 -- start start 2024-05-09T09:27:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.440+0000 7fbc2676d740 1 Processor -- start 2024-05-09T09:27:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.440+0000 7fbc2676d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dc99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.440+0000 7fbc2676d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dbbc40 0x55f966dc9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.440+0000 7fbc2676d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 0x55f966e35370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.440+0000 7fbc2676d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1aba60 con 0x55f966dc0db0 2024-05-09T09:27:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.440+0000 7fbc2676d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1ab390 con 0x55f966dbbc40 2024-05-09T09:27:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.440+0000 7fbc2676d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1a85d0 con 0x55f966dbb250 2024-05-09T09:27:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.440+0000 7fbc2676d740 10 client.0 _ll_register_callbacks cb 0x7fbc0c0126e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.440+0000 7fbc20ff8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 0x55f966e35370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc217f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dbbc40 0x55f966dc9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc20ff8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 0x55f966e35370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60920/0 (socket says 172.21.15.12:60920) 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc20ff8640 1 -- 172.21.15.12:0/431922164 learned_addr learned my addr 172.21.15.12:0/431922164 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc217f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dbbc40 0x55f966dc9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:46944/0 (socket says 172.21.15.12:46944) 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/431922164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dc99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc217f9640 1 -- 172.21.15.12:0/431922164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 msgr2=0x55f966dc99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc217f9640 1 --2- 172.21.15.12:0/431922164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dc99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc217f9640 1 -- 172.21.15.12:0/431922164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 msgr2=0x55f966e35370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc217f9640 1 --2- 172.21.15.12:0/431922164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 0x55f966e35370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc217f9640 1 -- 172.21.15.12:0/431922164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f966e358b0 con 0x55f966dbbc40 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/431922164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 0x55f966e35370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc217f9640 1 --2- 172.21.15.12:0/431922164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dbbc40 0x55f966dc9f20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55f966dbc6b0 tx=0x7fbc180042c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6af197ed8a962918 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc04ff9640 1 -- 172.21.15.12:0/431922164 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc18035040 con 0x55f966dbbc40 2024-05-09T09:27:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc04ff9640 1 -- 172.21.15.12:0/431922164 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbc180467b0 con 0x55f966dbbc40 2024-05-09T09:27:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.441+0000 7fbc2676d740 1 -- 172.21.15.12:0/431922164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc0c016070 con 0x55f966dbbc40 2024-05-09T09:27:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.442+0000 7fbc04ff9640 1 -- 172.21.15.12:0/431922164 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbc18054050 con 0x55f966dbbc40 2024-05-09T09:27:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.442+0000 7fbc2676d740 10 client.25859 fetch_fsmap learned FSMap version 32 2024-05-09T09:27:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.442+0000 7fbc2676d740 1 -- 172.21.15.12:0/431922164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f966e36c90 con 0x55f966dbbc40 2024-05-09T09:27:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.442+0000 7fbc04ff9640 1 -- 172.21.15.12:0/431922164 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc0c016070 con 0x55f966dbbc40 2024-05-09T09:27:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.442+0000 7fbc04ff9640 1 -- 172.21.15.12:0/431922164 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fbc18005330 con 0x55f966dbbc40 2024-05-09T09:27:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.442+0000 7fbc2676d740 10 client.25859 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T09:27:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.442+0000 7fbc2676d740 10 client.25859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.442+0000 7fbc2676d740 20 client.25859 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.442+0000 7fbc2676d740 1 --2- 172.21.15.12:0/431922164 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f966e376d0 0x55f966e39af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.442+0000 7fbc2676d740 4 client.25859 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:27:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.442+0000 7fbc2676d740 1 -- 172.21.15.12:0/431922164 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f966d26dc0 con 0x55f966e376d0 2024-05-09T09:27:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.442+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/431922164 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f966e376d0 0x55f966e39af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.443+0000 7fbc04ff9640 10 client.25859 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.443+0000 7fbc20ff8640 1 --2- 172.21.15.12:0/431922164 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f966e376d0 0x55f966e39af0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a2d5b5de715543 server_cookie=4c823108bcbadb08 in_seq=0 out_seq=0 2024-05-09T09:27:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.553+0000 7fbc04ff9640 1 -- 172.21.15.12:0/431922164 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f966d26dc0 con 0x55f966e376d0 2024-05-09T09:27:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.553+0000 7fbc04ff9640 10 client.25859 handle_command_reply: tid=0 2024-05-09T09:27:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.554+0000 7fbc2676d740 1 client.25859 shutdown 2024-05-09T09:27:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.554+0000 7fbc2676d740 1 -- 172.21.15.12:0/431922164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dbbc40 msgr2=0x55f966dc9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.554+0000 7fbc2676d740 1 --2- 172.21.15.12:0/431922164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dbbc40 0x55f966dc9f20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55f966dbc6b0 tx=0x7fbc180042c0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.554+0000 7fbc2676d740 1 -- 172.21.15.12:0/431922164 shutdown_connections 2024-05-09T09:27:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.554+0000 7fbc2676d740 1 --2- 172.21.15.12:0/431922164 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55f966e376d0 0x55f966e39af0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.554+0000 7fbc2676d740 1 --2- 172.21.15.12:0/431922164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f966dc0db0 0x55f966e35370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.554+0000 7fbc2676d740 1 --2- 172.21.15.12:0/431922164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f966dbbc40 0x55f966dc9f20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.554+0000 7fbc2676d740 1 --2- 172.21.15.12:0/431922164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f966dbb250 0x55f966dc99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.554+0000 7fbc2676d740 1 -- 172.21.15.12:0/431922164 >> 172.21.15.12:0/431922164 conn(0x55f966db68c0 msgr2=0x55f966dc5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.554+0000 7fbc04ff9640 0 client.25859 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.554+0000 7fbc2676d740 1 -- 172.21.15.12:0/431922164 shutdown_connections 2024-05-09T09:27:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.554+0000 7fbc2676d740 1 -- 172.21.15.12:0/431922164 wait complete. 2024-05-09T09:27:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.554+0000 7fbc2676d740 20 client.25859 trim_cache size 0 max 16384 2024-05-09T09:27:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:27:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:27:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:27:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:27:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.555+0000 7fbc23a5d640 1 -- 172.21.15.12:0/1535962229 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbc080a2c30 msgr2=0x7fbc080a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.555+0000 7fbc23a5d640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbc080a2c30 0x7fbc080a50f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbc1409bef0 tx=0x7fbc140bc040 comp rx=0 tx=0).stop 2024-05-09T09:27:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.555+0000 7fbc23a5d640 1 -- 172.21.15.12:0/1535962229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc1c103470 msgr2=0x7fbc1c19f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.555+0000 7fbc23a5d640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc1c103470 0x7fbc1c19f0e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009fd0 tx=0x7fbc1400ef10 comp rx=0 tx=0).stop 2024-05-09T09:27:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.555+0000 7fbc23a5d640 1 -- 172.21.15.12:0/1535962229 shutdown_connections 2024-05-09T09:27:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.555+0000 7fbc23a5d640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbc080a2c30 0x7fbc080a50f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.555+0000 7fbc23a5d640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc1c103470 0x7fbc1c19f0e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.555+0000 7fbc23a5d640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1c102ab0 0x7fbc1c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.555+0000 7fbc23a5d640 1 --2- 172.21.15.12:0/1535962229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc1c1018c0 0x7fbc1c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.555+0000 7fbc23a5d640 1 -- 172.21.15.12:0/1535962229 >> 172.21.15.12:0/1535962229 conn(0x7fbc1c0fce50 msgr2=0x7fbc1c105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.555+0000 7fbc23a5d640 1 -- 172.21.15.12:0/1535962229 shutdown_connections 2024-05-09T09:27:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:27.555+0000 7fbc23a5d640 1 -- 172.21.15.12:0/1535962229 wait complete. 2024-05-09T09:27:28.285 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:28 smithi012 ceph-mon[30140]: pgmap v424: 129 pgs: 129 active+clean; 124 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 2.3 MiB/s wr, 876 op/s 2024-05-09T09:27:28.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:28 smithi170 ceph-mon[33632]: pgmap v424: 129 pgs: 129 active+clean; 124 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 2.3 MiB/s wr, 876 op/s 2024-05-09T09:27:28.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:28 smithi016 ceph-mon[33335]: pgmap v424: 129 pgs: 129 active+clean; 124 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 2.3 MiB/s wr, 876 op/s 2024-05-09T09:27:30.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:30 smithi012 ceph-mon[30140]: pgmap v425: 129 pgs: 129 active+clean; 124 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 2.4 MiB/s wr, 499 op/s 2024-05-09T09:27:30.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:30 smithi170 ceph-mon[33632]: pgmap v425: 129 pgs: 129 active+clean; 124 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 2.4 MiB/s wr, 499 op/s 2024-05-09T09:27:30.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:30 smithi016 ceph-mon[33335]: pgmap v425: 129 pgs: 129 active+clean; 124 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 2.4 MiB/s wr, 499 op/s 2024-05-09T09:27:32.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:32 smithi012 ceph-mon[30140]: pgmap v426: 129 pgs: 129 active+clean; 126 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.6 MiB/s wr, 510 op/s 2024-05-09T09:27:32.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:32 smithi170 ceph-mon[33632]: pgmap v426: 129 pgs: 129 active+clean; 126 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.6 MiB/s wr, 510 op/s 2024-05-09T09:27:32.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:32 smithi016 ceph-mon[33335]: pgmap v426: 129 pgs: 129 active+clean; 126 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.6 MiB/s wr, 510 op/s 2024-05-09T09:27:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:34 smithi012 ceph-mon[30140]: pgmap v427: 129 pgs: 129 active+clean; 126 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 2.4 MiB/s wr, 528 op/s 2024-05-09T09:27:34.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:34 smithi170 ceph-mon[33632]: pgmap v427: 129 pgs: 129 active+clean; 126 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 2.4 MiB/s wr, 528 op/s 2024-05-09T09:27:34.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:34 smithi016 ceph-mon[33335]: pgmap v427: 129 pgs: 129 active+clean; 126 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 2.4 MiB/s wr, 528 op/s 2024-05-09T09:27:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:36 smithi012 ceph-mon[30140]: pgmap v428: 129 pgs: 129 active+clean; 130 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 2.7 MiB/s wr, 532 op/s 2024-05-09T09:27:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:36 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:27:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:36 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:27:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:36 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:27:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:36 smithi170 ceph-mon[33632]: pgmap v428: 129 pgs: 129 active+clean; 130 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 2.7 MiB/s wr, 532 op/s 2024-05-09T09:27:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:36 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:27:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:36 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:27:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:36 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:27:36.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:36 smithi016 ceph-mon[33335]: pgmap v428: 129 pgs: 129 active+clean; 130 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 2.7 MiB/s wr, 532 op/s 2024-05-09T09:27:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:36 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:27:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:36 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:27:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:36 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:27:37.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:37 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:27:37.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:37 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:27:37.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:37 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:27:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:38 smithi012 ceph-mon[30140]: pgmap v429: 129 pgs: 129 active+clean; 131 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.0 MiB/s wr, 462 op/s 2024-05-09T09:27:38.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:38 smithi170 ceph-mon[33632]: pgmap v429: 129 pgs: 129 active+clean; 131 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.0 MiB/s wr, 462 op/s 2024-05-09T09:27:38.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:38 smithi016 ceph-mon[33335]: pgmap v429: 129 pgs: 129 active+clean; 131 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.0 MiB/s wr, 462 op/s 2024-05-09T09:27:40.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:40 smithi012 ceph-mon[30140]: pgmap v430: 129 pgs: 129 active+clean; 131 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.1 MiB/s wr, 399 op/s 2024-05-09T09:27:40.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:40 smithi170 ceph-mon[33632]: pgmap v430: 129 pgs: 129 active+clean; 131 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.1 MiB/s wr, 399 op/s 2024-05-09T09:27:40.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:40 smithi016 ceph-mon[33335]: pgmap v430: 129 pgs: 129 active+clean; 131 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.1 MiB/s wr, 399 op/s 2024-05-09T09:27:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.657+0000 7eff933d5640 1 -- 172.21.15.12:0/1528683085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff8c107570 msgr2=0x7eff8c1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.657+0000 7eff933d5640 1 --2- 172.21.15.12:0/1528683085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff8c107570 0x7eff8c1099c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7eff84009ca0 tx=0x7eff8402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.657+0000 7eff933d5640 1 -- 172.21.15.12:0/1528683085 shutdown_connections 2024-05-09T09:27:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.657+0000 7eff933d5640 1 --2- 172.21.15.12:0/1528683085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff8c107570 0x7eff8c1099c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.657+0000 7eff933d5640 1 --2- 172.21.15.12:0/1528683085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff8c104c40 0x7eff8c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.657+0000 7eff933d5640 1 --2- 172.21.15.12:0/1528683085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff8c102310 0x7eff8c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.657+0000 7eff933d5640 1 -- 172.21.15.12:0/1528683085 >> 172.21.15.12:0/1528683085 conn(0x7eff8c0fbd00 msgr2=0x7eff8c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.657+0000 7eff933d5640 1 -- 172.21.15.12:0/1528683085 shutdown_connections 2024-05-09T09:27:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.657+0000 7eff933d5640 1 -- 172.21.15.12:0/1528683085 wait complete. 2024-05-09T09:27:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff933d5640 1 Processor -- start 2024-05-09T09:27:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff933d5640 1 -- start start 2024-05-09T09:27:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff933d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff8c102310 0x7eff8c101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff933d5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff8c104c40 0x7eff8c1002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff933d5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff8c107570 0x7eff8c100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff933d5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10d4f0 con 0x7eff8c107570 2024-05-09T09:27:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff933d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10d370 con 0x7eff8c102310 2024-05-09T09:27:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff933d5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10d670 con 0x7eff8c104c40 2024-05-09T09:27:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff91171640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff8c102310 0x7eff8c101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff91972640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff8c107570 0x7eff8c100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff91972640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff8c107570 0x7eff8c100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45160/0 (socket says 172.21.15.12:45160) 2024-05-09T09:27:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff90970640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff8c104c40 0x7eff8c1002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff91972640 1 -- 172.21.15.12:0/852653776 learned_addr learned my addr 172.21.15.12:0/852653776 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff91972640 1 -- 172.21.15.12:0/852653776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff8c104c40 msgr2=0x7eff8c1002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff91972640 1 --2- 172.21.15.12:0/852653776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff8c104c40 0x7eff8c1002c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff91972640 1 -- 172.21.15.12:0/852653776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff8c102310 msgr2=0x7eff8c101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff91972640 1 --2- 172.21.15.12:0/852653776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff8c102310 0x7eff8c101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff91972640 1 -- 172.21.15.12:0/852653776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c101090 con 0x7eff8c107570 2024-05-09T09:27:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.658+0000 7eff91972640 1 --2- 172.21.15.12:0/852653776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff8c107570 0x7eff8c100800 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7eff84009430 tx=0x7eff8402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.659+0000 7eff827fc640 1 -- 172.21.15.12:0/852653776 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84003bf0 con 0x7eff8c107570 2024-05-09T09:27:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.659+0000 7eff827fc640 1 -- 172.21.15.12:0/852653776 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7eff84003d90 con 0x7eff8c107570 2024-05-09T09:27:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.659+0000 7eff933d5640 1 -- 172.21.15.12:0/852653776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c1a1c10 con 0x7eff8c107570 2024-05-09T09:27:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.659+0000 7eff827fc640 1 -- 172.21.15.12:0/852653776 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8402fc90 con 0x7eff8c107570 2024-05-09T09:27:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.659+0000 7eff933d5640 1 -- 172.21.15.12:0/852653776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c1a21b0 con 0x7eff8c107570 2024-05-09T09:27:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.661+0000 7eff961c4740 1 Processor -- start 2024-05-09T09:27:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.662+0000 7eff827fc640 1 -- 172.21.15.12:0/852653776 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff84004040 con 0x7eff8c107570 2024-05-09T09:27:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.662+0000 7eff961c4740 1 -- start start 2024-05-09T09:27:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.662+0000 7eff961c4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95e250 0x556d8d95e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.662+0000 7eff961c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d95ec40 0x556d8d963740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.662+0000 7eff961c4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d963db0 0x556d8d966240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.662+0000 7eff961c4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556d8d8c2340 con 0x556d8d963db0 2024-05-09T09:27:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.662+0000 7eff961c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556d8d86a290 con 0x556d8d95ec40 2024-05-09T09:27:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.662+0000 7eff961c4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556d8d8be300 con 0x556d8d95e250 2024-05-09T09:27:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.662+0000 7eff90970640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d963db0 0x556d8d966240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.662+0000 7eff90970640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d963db0 0x556d8d966240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45162/0 (socket says 172.21.15.12:45162) 2024-05-09T09:27:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.662+0000 7eff90970640 1 -- 172.21.15.12:0/1664323276 learned_addr learned my addr 172.21.15.12:0/1664323276 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.662+0000 7eff90970640 1 --2- 172.21.15.12:0/1664323276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95e250 0x556d8d95e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.662+0000 7eff91171640 1 --2- 172.21.15.12:0/1664323276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d95ec40 0x556d8d963740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.663+0000 7eff90970640 1 -- 172.21.15.12:0/1664323276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95e250 msgr2=0x556d8d95e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.663+0000 7eff90970640 1 --2- 172.21.15.12:0/1664323276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95e250 0x556d8d95e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.663+0000 7eff90970640 1 -- 172.21.15.12:0/1664323276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d95ec40 msgr2=0x556d8d963740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.663+0000 7eff90970640 1 --2- 172.21.15.12:0/1664323276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d95ec40 0x556d8d963740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.663+0000 7eff90970640 1 -- 172.21.15.12:0/1664323276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d8d88e8e0 con 0x556d8d963db0 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.663+0000 7eff827fc640 1 --2- 172.21.15.12:0/852653776 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7eff780a2cb0 0x7eff780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.663+0000 7eff90970640 1 --2- 172.21.15.12:0/1664323276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d963db0 0x556d8d966240 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009fd0 tx=0x7eff7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b706ad6f52b0f9a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.663+0000 7eff91171640 1 --2- 172.21.15.12:0/852653776 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7eff780a2cb0 0x7eff780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.663+0000 7eff827fc640 1 -- 172.21.15.12:0/852653776 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7eff840f09f0 con 0x7eff8c107570 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.663+0000 7eff777fe640 1 -- 172.21.15.12:0/1664323276 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c004050 con 0x556d8d963db0 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.663+0000 7eff777fe640 1 -- 172.21.15.12:0/1664323276 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7eff7c0054f0 con 0x556d8d963db0 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.663+0000 7eff777fe640 1 -- 172.21.15.12:0/1664323276 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c004c60 con 0x556d8d963db0 2024-05-09T09:27:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.664+0000 7eff91171640 1 --2- 172.21.15.12:0/852653776 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7eff780a2cb0 0x7eff780a5170 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x556d8d95f040 tx=0x7eff8800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.664+0000 7eff961c4740 1 -- 172.21.15.12:0/1664323276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d963db0 msgr2=0x556d8d966240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.664+0000 7eff961c4740 1 --2- 172.21.15.12:0/1664323276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d963db0 0x556d8d966240 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009fd0 tx=0x7eff7c004730 comp rx=0 tx=0).stop 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.664+0000 7eff961c4740 1 -- 172.21.15.12:0/1664323276 shutdown_connections 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.664+0000 7eff961c4740 1 --2- 172.21.15.12:0/1664323276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d963db0 0x556d8d966240 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.664+0000 7eff961c4740 1 --2- 172.21.15.12:0/1664323276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d95ec40 0x556d8d963740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.664+0000 7eff961c4740 1 --2- 172.21.15.12:0/1664323276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95e250 0x556d8d95e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.664+0000 7eff961c4740 1 -- 172.21.15.12:0/1664323276 >> 172.21.15.12:0/1664323276 conn(0x556d8d9598c0 msgr2=0x556d8d959cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.664+0000 7eff961c4740 1 -- 172.21.15.12:0/1664323276 shutdown_connections 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.664+0000 7eff961c4740 1 -- 172.21.15.12:0/1664323276 wait complete. 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff961c4740 1 -- start start 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff961c4740 1 Processor -- start 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff961c4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95e250 0x556d8d9684f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff961c4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95ec40 0x556d8d968a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff961c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d963db0 0x556d8d96fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff961c4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556d8d966d60 con 0x556d8d95e250 2024-05-09T09:27:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff961c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556d8d87d790 con 0x556d8d963db0 2024-05-09T09:27:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff961c4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556d8d966ee0 con 0x556d8d95ec40 2024-05-09T09:27:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff961c4740 10 client.0 _ll_register_callbacks cb 0x556d8d8b8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff91171640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95e250 0x556d8d9684f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff90970640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95ec40 0x556d8d968a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff90970640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95ec40 0x556d8d968a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:41754/0 (socket says 172.21.15.12:41754) 2024-05-09T09:27:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff91171640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95e250 0x556d8d9684f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45166/0 (socket says 172.21.15.12:45166) 2024-05-09T09:27:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff90970640 1 -- 172.21.15.12:0/1331866583 learned_addr learned my addr 172.21.15.12:0/1331866583 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.665+0000 7eff91972640 1 --2- 172.21.15.12:0/1331866583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d963db0 0x556d8d96fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff90970640 1 -- 172.21.15.12:0/1331866583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d963db0 msgr2=0x556d8d96fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff90970640 1 --2- 172.21.15.12:0/1331866583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d963db0 0x556d8d96fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff90970640 1 -- 172.21.15.12:0/1331866583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95e250 msgr2=0x556d8d9684f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff90970640 1 --2- 172.21.15.12:0/1331866583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95e250 0x556d8d9684f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff90970640 1 -- 172.21.15.12:0/1331866583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d8d5d6210 con 0x556d8d95ec40 2024-05-09T09:27:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff90970640 1 --2- 172.21.15.12:0/1331866583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95ec40 0x556d8d968a30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00fbe0 tx=0x7eff7c00fc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64aa4df03f20d91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:27:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff74ff9640 1 -- 172.21.15.12:0/1331866583 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c0049e0 con 0x556d8d95ec40 2024-05-09T09:27:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff961c4740 1 -- 172.21.15.12:0/1331866583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff7c0054f0 con 0x556d8d95ec40 2024-05-09T09:27:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff74ff9640 1 -- 172.21.15.12:0/1331866583 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7eff7c004b80 con 0x556d8d95ec40 2024-05-09T09:27:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff74ff9640 1 -- 172.21.15.12:0/1331866583 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7eff7c033050 con 0x556d8d95ec40 2024-05-09T09:27:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff961c4740 10 client.25981 fetch_fsmap learned FSMap version 32 2024-05-09T09:27:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff961c4740 1 -- 172.21.15.12:0/1331866583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556d8d6a2620 con 0x556d8d95ec40 2024-05-09T09:27:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.666+0000 7eff74ff9640 1 -- 172.21.15.12:0/1331866583 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7eff7c0054f0 con 0x556d8d95ec40 2024-05-09T09:27:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.667+0000 7eff74ff9640 1 -- 172.21.15.12:0/1331866583 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7eff7c01d020 con 0x556d8d95ec40 2024-05-09T09:27:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.667+0000 7eff961c4740 10 client.25981 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T09:27:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.667+0000 7eff961c4740 10 client.25981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.667+0000 7eff961c4740 20 client.25981 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.667+0000 7eff961c4740 1 --2- 172.21.15.12:0/1331866583 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556d8d973d40 0x556d8d976180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.667+0000 7eff961c4740 4 client.25981 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.667+0000 7eff961c4740 1 -- 172.21.15.12:0/1331866583 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d8d8cfa70 con 0x556d8d973d40 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.667+0000 7eff91171640 1 --2- 172.21.15.12:0/1331866583 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556d8d973d40 0x556d8d976180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.667+0000 7eff74ff9640 10 client.25981 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.667+0000 7eff91171640 1 --2- 172.21.15.12:0/1331866583 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556d8d973d40 0x556d8d976180 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2bbddba6cc79e00 server_cookie=40f0ca66fd2e7ffc in_seq=0 out_seq=0 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.668+0000 7eff74ff9640 1 -- 172.21.15.12:0/1331866583 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d8d8cfa70 con 0x556d8d973d40 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.668+0000 7eff74ff9640 10 client.25981 handle_command_reply: tid=0 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.668+0000 7eff961c4740 1 client.25981 shutdown 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.668+0000 7eff961c4740 1 -- 172.21.15.12:0/1331866583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95ec40 msgr2=0x556d8d968a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.668+0000 7eff961c4740 1 --2- 172.21.15.12:0/1331866583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95ec40 0x556d8d968a30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00fbe0 tx=0x7eff7c00fc10 comp rx=0 tx=0).stop 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.668+0000 7eff961c4740 1 -- 172.21.15.12:0/1331866583 shutdown_connections 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.668+0000 7eff961c4740 1 --2- 172.21.15.12:0/1331866583 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556d8d973d40 0x556d8d976180 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.668+0000 7eff961c4740 1 --2- 172.21.15.12:0/1331866583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d963db0 0x556d8d96fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.668+0000 7eff74ff9640 0 client.25981 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.668+0000 7eff961c4740 1 --2- 172.21.15.12:0/1331866583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95ec40 0x556d8d968a30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.668+0000 7eff961c4740 1 --2- 172.21.15.12:0/1331866583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95e250 0x556d8d9684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.668+0000 7eff961c4740 1 -- 172.21.15.12:0/1331866583 >> 172.21.15.12:0/1331866583 conn(0x556d8d9598c0 msgr2=0x556d8d958a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.668+0000 7eff961c4740 1 -- 172.21.15.12:0/1331866583 shutdown_connections 2024-05-09T09:27:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.669+0000 7eff961c4740 1 -- 172.21.15.12:0/1331866583 wait complete. 2024-05-09T09:27:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.669+0000 7eff961c4740 20 client.25981 trim_cache size 0 max 16384 2024-05-09T09:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.687+0000 7eff961c4740 1 Processor -- start 2024-05-09T09:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff961c4740 1 -- start start 2024-05-09T09:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff961c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d95e250 0x556d8d97ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff961c4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95ec40 0x556d8d97bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff961c4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d963db0 0x556d8d96fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff961c4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1ac6f0 con 0x556d8d95ec40 2024-05-09T09:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff961c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1ae2f0 con 0x556d8d95e250 2024-05-09T09:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff961c4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1acdc0 con 0x556d8d963db0 2024-05-09T09:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff90970640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95ec40 0x556d8d97bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff91171640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d95e250 0x556d8d97ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff90970640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95ec40 0x556d8d97bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45178/0 (socket says 172.21.15.12:45178) 2024-05-09T09:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff90970640 1 -- 172.21.15.12:0/503441320 learned_addr learned my addr 172.21.15.12:0/503441320 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff91972640 1 --2- 172.21.15.12:0/503441320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d963db0 0x556d8d96fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff91171640 1 -- 172.21.15.12:0/503441320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d963db0 msgr2=0x556d8d96fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff91171640 1 --2- 172.21.15.12:0/503441320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d963db0 0x556d8d96fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff91171640 1 -- 172.21.15.12:0/503441320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95ec40 msgr2=0x556d8d97bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff91171640 1 --2- 172.21.15.12:0/503441320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95ec40 0x556d8d97bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff91171640 1 -- 172.21.15.12:0/503441320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d8d966c40 con 0x556d8d95e250 2024-05-09T09:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.688+0000 7eff91171640 1 --2- 172.21.15.12:0/503441320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d95e250 0x556d8d97ba50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff8800bc30 tx=0x7eff88019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eb7695fbbf0f6e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.689+0000 7eff76ffd640 1 -- 172.21.15.12:0/503441320 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8802a420 con 0x556d8d95e250 2024-05-09T09:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.689+0000 7eff76ffd640 1 -- 172.21.15.12:0/503441320 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7eff8802f070 con 0x556d8d95e250 2024-05-09T09:27:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.689+0000 7eff76ffd640 1 -- 172.21.15.12:0/503441320 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8801ddd0 con 0x556d8d95e250 2024-05-09T09:27:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.689+0000 7eff961c4740 1 -- 172.21.15.12:0/503441320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d95e250 msgr2=0x556d8d97ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.689+0000 7eff961c4740 1 --2- 172.21.15.12:0/503441320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d95e250 0x556d8d97ba50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff8800bc30 tx=0x7eff88019000 comp rx=0 tx=0).stop 2024-05-09T09:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.689+0000 7eff961c4740 1 -- 172.21.15.12:0/503441320 shutdown_connections 2024-05-09T09:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.689+0000 7eff961c4740 1 --2- 172.21.15.12:0/503441320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d963db0 0x556d8d96fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.689+0000 7eff961c4740 1 --2- 172.21.15.12:0/503441320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95ec40 0x556d8d97bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.689+0000 7eff961c4740 1 --2- 172.21.15.12:0/503441320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d95e250 0x556d8d97ba50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.689+0000 7eff961c4740 1 -- 172.21.15.12:0/503441320 >> 172.21.15.12:0/503441320 conn(0x556d8d9598c0 msgr2=0x556d8d9672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.690+0000 7eff961c4740 1 -- 172.21.15.12:0/503441320 shutdown_connections 2024-05-09T09:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.690+0000 7eff961c4740 1 -- 172.21.15.12:0/503441320 wait complete. 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.690+0000 7eff961c4740 1 -- start start 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.690+0000 7eff961c4740 1 Processor -- start 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.690+0000 7eff961c4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95e250 0x556d8d96cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.690+0000 7eff961c4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95ec40 0x556d8d96d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.690+0000 7eff961c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d963db0 0x556d8d96e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.690+0000 7eff961c4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1adb60 con 0x556d8d95e250 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.690+0000 7eff961c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1ad490 con 0x556d8d963db0 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.690+0000 7eff961c4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1aa6d0 con 0x556d8d95ec40 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff961c4740 10 client.0 _ll_register_callbacks cb 0x7eff8800b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff91171640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95e250 0x556d8d96cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff91171640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95e250 0x556d8d96cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45186/0 (socket says 172.21.15.12:45186) 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff91171640 1 -- 172.21.15.12:0/57694765 learned_addr learned my addr 172.21.15.12:0/57694765 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff90970640 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95ec40 0x556d8d96d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff91972640 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d963db0 0x556d8d96e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff90970640 1 -- 172.21.15.12:0/57694765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d963db0 msgr2=0x556d8d96e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff90970640 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d963db0 0x556d8d96e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff90970640 1 -- 172.21.15.12:0/57694765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95e250 msgr2=0x556d8d96cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff90970640 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95e250 0x556d8d96cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff90970640 1 -- 172.21.15.12:0/57694765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d8d96e650 con 0x556d8d95ec40 2024-05-09T09:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff91972640 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d963db0 0x556d8d96e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff90970640 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95ec40 0x556d8d96d050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7eff7c006010 tx=0x7eff7c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=435548239a92506d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff74ff9640 1 -- 172.21.15.12:0/57694765 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00fc80 con 0x556d8d95ec40 2024-05-09T09:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff74ff9640 1 -- 172.21.15.12:0/57694765 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7eff7c025030 con 0x556d8d95ec40 2024-05-09T09:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff961c4740 1 -- 172.21.15.12:0/57694765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff8802f070 con 0x556d8d95ec40 2024-05-09T09:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.691+0000 7eff74ff9640 1 -- 172.21.15.12:0/57694765 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7eff7c009e00 con 0x556d8d95ec40 2024-05-09T09:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.692+0000 7eff961c4740 10 client.25987 fetch_fsmap learned FSMap version 32 2024-05-09T09:27:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.692+0000 7eff961c4740 1 -- 172.21.15.12:0/57694765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556d8d96f960 con 0x556d8d95ec40 2024-05-09T09:27:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.692+0000 7eff74ff9640 1 -- 172.21.15.12:0/57694765 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7eff8802f070 con 0x556d8d95ec40 2024-05-09T09:27:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.692+0000 7eff74ff9640 1 -- 172.21.15.12:0/57694765 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7eff7c01d050 con 0x556d8d95ec40 2024-05-09T09:27:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.692+0000 7eff961c4740 10 client.25987 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T09:27:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.692+0000 7eff961c4740 10 client.25987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.692+0000 7eff961c4740 20 client.25987 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.692+0000 7eff961c4740 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556d8d973d40 0x556d8d9dc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.692+0000 7eff961c4740 4 client.25987 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:27:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.692+0000 7eff961c4740 1 -- 172.21.15.12:0/57694765 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d8d8c9dc0 con 0x556d8d973d40 2024-05-09T09:27:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.692+0000 7eff91171640 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556d8d973d40 0x556d8d9dc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.693+0000 7eff91171640 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556d8d973d40 0x556d8d9dc660 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c0344870504ab79 server_cookie=488c21f32de27a41 in_seq=0 out_seq=0 2024-05-09T09:27:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.693+0000 7eff74ff9640 10 client.25987 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:40.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.802+0000 7eff74ff9640 1 -- 172.21.15.12:0/57694765 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d8d8c9dc0 con 0x556d8d973d40 2024-05-09T09:27:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.802+0000 7eff74ff9640 10 client.25987 handle_command_reply: tid=0 2024-05-09T09:27:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.802+0000 7eff961c4740 1 client.25987 shutdown 2024-05-09T09:27:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.802+0000 7eff961c4740 1 -- 172.21.15.12:0/57694765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95ec40 msgr2=0x556d8d96d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.802+0000 7eff961c4740 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95ec40 0x556d8d96d050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7eff7c006010 tx=0x7eff7c005270 comp rx=0 tx=0).stop 2024-05-09T09:27:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.802+0000 7eff961c4740 1 -- 172.21.15.12:0/57694765 shutdown_connections 2024-05-09T09:27:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.802+0000 7eff961c4740 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556d8d973d40 0x556d8d9dc660 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.802+0000 7eff961c4740 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d8d963db0 0x556d8d96e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.802+0000 7eff74ff9640 0 client.25987 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.802+0000 7eff961c4740 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d8d95ec40 0x556d8d96d050 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.803+0000 7eff961c4740 1 --2- 172.21.15.12:0/57694765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d8d95e250 0x556d8d96cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.803+0000 7eff961c4740 1 -- 172.21.15.12:0/57694765 >> 172.21.15.12:0/57694765 conn(0x556d8d9598c0 msgr2=0x556d8d9684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.803+0000 7eff961c4740 1 -- 172.21.15.12:0/57694765 shutdown_connections 2024-05-09T09:27:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.803+0000 7eff961c4740 1 -- 172.21.15.12:0/57694765 wait complete. 2024-05-09T09:27:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.803+0000 7eff961c4740 20 client.25987 trim_cache size 0 max 16384 2024-05-09T09:27:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:27:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:27:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:27:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:27:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.804+0000 7eff933d5640 1 -- 172.21.15.12:0/852653776 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7eff780a2cb0 msgr2=0x7eff780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.804+0000 7eff933d5640 1 --2- 172.21.15.12:0/852653776 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7eff780a2cb0 0x7eff780a5170 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x556d8d95f040 tx=0x7eff8800b410 comp rx=0 tx=0).stop 2024-05-09T09:27:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.804+0000 7eff933d5640 1 -- 172.21.15.12:0/852653776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff8c107570 msgr2=0x7eff8c100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.804+0000 7eff933d5640 1 --2- 172.21.15.12:0/852653776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff8c107570 0x7eff8c100800 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7eff84009430 tx=0x7eff8402f380 comp rx=0 tx=0).stop 2024-05-09T09:27:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.805+0000 7eff933d5640 1 -- 172.21.15.12:0/852653776 shutdown_connections 2024-05-09T09:27:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.805+0000 7eff933d5640 1 --2- 172.21.15.12:0/852653776 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7eff780a2cb0 0x7eff780a5170 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.805+0000 7eff933d5640 1 --2- 172.21.15.12:0/852653776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff8c107570 0x7eff8c100800 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.805+0000 7eff933d5640 1 --2- 172.21.15.12:0/852653776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff8c104c40 0x7eff8c1002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.805+0000 7eff933d5640 1 --2- 172.21.15.12:0/852653776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff8c102310 0x7eff8c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.805+0000 7eff933d5640 1 -- 172.21.15.12:0/852653776 >> 172.21.15.12:0/852653776 conn(0x7eff8c0fbd00 msgr2=0x7eff8c102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.805+0000 7eff933d5640 1 -- 172.21.15.12:0/852653776 shutdown_connections 2024-05-09T09:27:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:40.805+0000 7eff933d5640 1 -- 172.21.15.12:0/852653776 wait complete. 2024-05-09T09:27:41.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:41 smithi016 ceph-mon[33335]: daemon mds.h restarted 2024-05-09T09:27:41.254 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:27:40 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h[63713]: ignoring --setuser ceph since I am not root 2024-05-09T09:27:41.254 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:27:40 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h[63713]: ignoring --setgroup ceph since I am not root 2024-05-09T09:27:41.254 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:27:40 smithi016 ceph-mds[63717]: -- 172.21.15.16:0/334658507 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55991db47a40 con 0x55991db1b000 2024-05-09T09:27:41.254 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:27:40 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h[63713]: starting mds.h at 2024-05-09T09:27:41.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:41 smithi012 ceph-mon[30140]: daemon mds.h restarted 2024-05-09T09:27:41.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:41 smithi170 ceph-mon[33632]: daemon mds.h restarted 2024-05-09T09:27:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:42 smithi012 ceph-mon[30140]: pgmap v431: 129 pgs: 129 active+clean; 134 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.1 MiB/s wr, 495 op/s 2024-05-09T09:27:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:42 smithi012 ceph-mon[30140]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T09:27:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:42 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.16:6836/3067829157,v1:172.21.15.16:6837/3067829157] up:boot 2024-05-09T09:27:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:42 smithi012 ceph-mon[30140]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T09:27:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:42 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:27:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:42 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:27:42.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:42 smithi170 ceph-mon[33632]: pgmap v431: 129 pgs: 129 active+clean; 134 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.1 MiB/s wr, 495 op/s 2024-05-09T09:27:42.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:42 smithi170 ceph-mon[33632]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T09:27:42.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:42 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.16:6836/3067829157,v1:172.21.15.16:6837/3067829157] up:boot 2024-05-09T09:27:42.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:42 smithi170 ceph-mon[33632]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T09:27:42.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:42 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:27:42.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:42 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:27:42.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:42 smithi016 ceph-mon[33335]: pgmap v431: 129 pgs: 129 active+clean; 134 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.1 MiB/s wr, 495 op/s 2024-05-09T09:27:42.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:42 smithi016 ceph-mon[33335]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T09:27:42.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:42 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.16:6836/3067829157,v1:172.21.15.16:6837/3067829157] up:boot 2024-05-09T09:27:42.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:42 smithi016 ceph-mon[33335]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T09:27:42.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:42 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:27:42.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:42 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:27:44.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:44 smithi012 ceph-mon[30140]: pgmap v433: 129 pgs: 129 active+clean; 136 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.1 MiB/s wr, 470 op/s 2024-05-09T09:27:44.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:44 smithi170 ceph-mon[33632]: pgmap v433: 129 pgs: 129 active+clean; 136 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.1 MiB/s wr, 470 op/s 2024-05-09T09:27:44.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:44 smithi016 ceph-mon[33335]: pgmap v433: 129 pgs: 129 active+clean; 136 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.1 MiB/s wr, 470 op/s 2024-05-09T09:27:46.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:46 smithi012 ceph-mon[30140]: pgmap v434: 129 pgs: 129 active+clean; 129 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.0 MiB/s wr, 482 op/s 2024-05-09T09:27:46.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:46 smithi170 ceph-mon[33632]: pgmap v434: 129 pgs: 129 active+clean; 129 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.0 MiB/s wr, 482 op/s 2024-05-09T09:27:46.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:46 smithi016 ceph-mon[33335]: pgmap v434: 129 pgs: 129 active+clean; 129 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.0 MiB/s wr, 482 op/s 2024-05-09T09:27:48.294 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:48 smithi170 ceph-mon[33632]: pgmap v435: 129 pgs: 129 active+clean; 131 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.0 MiB/s wr, 487 op/s 2024-05-09T09:27:48.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:48 smithi012 ceph-mon[30140]: pgmap v435: 129 pgs: 129 active+clean; 131 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.0 MiB/s wr, 487 op/s 2024-05-09T09:27:48.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:48 smithi016 ceph-mon[33335]: pgmap v435: 129 pgs: 129 active+clean; 131 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.0 MiB/s wr, 487 op/s 2024-05-09T09:27:49.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:49 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:27:49.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:49 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:27:49.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:49 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:27:50.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:50 smithi012 ceph-mon[30140]: pgmap v436: 129 pgs: 129 active+clean; 132 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.2 MiB/s wr, 516 op/s 2024-05-09T09:27:50.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:50 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:27:50.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:50 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:27:50.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:50 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:27:50.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:50 smithi170 ceph-mon[33632]: pgmap v436: 129 pgs: 129 active+clean; 132 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.2 MiB/s wr, 516 op/s 2024-05-09T09:27:50.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:50 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:27:50.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:50 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:27:50.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:50 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:27:50.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:50 smithi016 ceph-mon[33335]: pgmap v436: 129 pgs: 129 active+clean; 132 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.2 MiB/s wr, 516 op/s 2024-05-09T09:27:50.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:50 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:27:50.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:50 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:27:50.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:50 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:27:50.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:27:50.883 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:27:51.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.004+0000 7ff82f51d640 1 -- 172.21.15.12:0/1050310190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff828102a70 msgr2=0x7ff828102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.004+0000 7ff82f51d640 1 --2- 172.21.15.12:0/1050310190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff828102a70 0x7ff828102ef0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff818009900 tx=0x7ff81802a410 comp rx=0 tx=0).stop 2024-05-09T09:27:51.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.004+0000 7ff82f51d640 1 -- 172.21.15.12:0/1050310190 shutdown_connections 2024-05-09T09:27:51.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.004+0000 7ff82f51d640 1 --2- 172.21.15.12:0/1050310190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828103430 0x7ff828109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.004+0000 7ff82f51d640 1 --2- 172.21.15.12:0/1050310190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff828102a70 0x7ff828102ef0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.004+0000 7ff82f51d640 1 --2- 172.21.15.12:0/1050310190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff828101880 0x7ff828101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.004+0000 7ff82f51d640 1 -- 172.21.15.12:0/1050310190 >> 172.21.15.12:0/1050310190 conn(0x7ff8280fce10 msgr2=0x7ff8280ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.004+0000 7ff82f51d640 1 -- 172.21.15.12:0/1050310190 shutdown_connections 2024-05-09T09:27:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.004+0000 7ff82f51d640 1 -- 172.21.15.12:0/1050310190 wait complete. 2024-05-09T09:27:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.005+0000 7ff82f51d640 1 Processor -- start 2024-05-09T09:27:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.005+0000 7ff82f51d640 1 -- start start 2024-05-09T09:27:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.005+0000 7ff82f51d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828101880 0x7ff828199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.005+0000 7ff82f51d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff828102a70 0x7ff82819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.005+0000 7ff82f51d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff828103430 0x7ff8281a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.005+0000 7ff82f51d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff82810d4e0 con 0x7ff828103430 2024-05-09T09:27:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.005+0000 7ff82f51d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff82810d360 con 0x7ff828101880 2024-05-09T09:27:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.005+0000 7ff82f51d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff82810d660 con 0x7ff828102a70 2024-05-09T09:27:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.005+0000 7ff82daba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff828103430 0x7ff8281a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.005+0000 7ff82d2b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828101880 0x7ff828199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.005+0000 7ff82daba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff828103430 0x7ff8281a13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50500/0 (socket says 172.21.15.12:50500) 2024-05-09T09:27:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.005+0000 7ff82daba640 1 -- 172.21.15.12:0/1544531809 learned_addr learned my addr 172.21.15.12:0/1544531809 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.005+0000 7ff82cab8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff828102a70 0x7ff82819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.006+0000 7ff82daba640 1 -- 172.21.15.12:0/1544531809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff828102a70 msgr2=0x7ff82819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.006+0000 7ff82daba640 1 --2- 172.21.15.12:0/1544531809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff828102a70 0x7ff82819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.006+0000 7ff82daba640 1 -- 172.21.15.12:0/1544531809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828101880 msgr2=0x7ff828199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.006+0000 7ff82daba640 1 --2- 172.21.15.12:0/1544531809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828101880 0x7ff828199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.006+0000 7ff82daba640 1 -- 172.21.15.12:0/1544531809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8281a1a10 con 0x7ff828103430 2024-05-09T09:27:51.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.006+0000 7ff82d2b9640 1 --2- 172.21.15.12:0/1544531809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828101880 0x7ff828199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:51.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.006+0000 7ff82daba640 1 --2- 172.21.15.12:0/1544531809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff828103430 0x7ff8281a13a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff82000eb10 tx=0x7ff82000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.006+0000 7ff81e7fc640 1 -- 172.21.15.12:0/1544531809 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820004b20 con 0x7ff828103430 2024-05-09T09:27:51.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.006+0000 7ff82f51d640 1 -- 172.21.15.12:0/1544531809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8281a1d00 con 0x7ff828103430 2024-05-09T09:27:51.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.006+0000 7ff81e7fc640 1 -- 172.21.15.12:0/1544531809 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff820004cc0 con 0x7ff828103430 2024-05-09T09:27:51.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.006+0000 7ff81e7fc640 1 -- 172.21.15.12:0/1544531809 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820010700 con 0x7ff828103430 2024-05-09T09:27:51.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.006+0000 7ff82f51d640 1 -- 172.21.15.12:0/1544531809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8281a21d0 con 0x7ff828103430 2024-05-09T09:27:51.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.009+0000 7ff832220740 1 Processor -- start 2024-05-09T09:27:51.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.009+0000 7ff81e7fc640 1 -- 172.21.15.12:0/1544531809 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff820004e70 con 0x7ff828103430 2024-05-09T09:27:51.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.009+0000 7ff832220740 1 -- start start 2024-05-09T09:27:51.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff832220740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf4f30 0x56534fcf5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff832220740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 0x56534fcfa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff832220740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcfaa90 0x56534fcfcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff832220740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56534fc59d00 con 0x56534fcfaa90 2024-05-09T09:27:51.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff832220740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56534fc014c0 con 0x56534fcf5920 2024-05-09T09:27:51.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff832220740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56534fc55cc0 con 0x56534fcf4f30 2024-05-09T09:27:51.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff82d2b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 0x56534fcfa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff82cab8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf4f30 0x56534fcf5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff82d2b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 0x56534fcfa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:45722/0 (socket says 172.21.15.12:45722) 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff82d2b9640 1 -- 172.21.15.12:0/9003761 learned_addr learned my addr 172.21.15.12:0/9003761 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff82cab8640 1 --2- 172.21.15.12:0/9003761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcfaa90 0x56534fcfcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff82d2b9640 1 -- 172.21.15.12:0/9003761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf4f30 msgr2=0x56534fcf5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff82d2b9640 1 --2- 172.21.15.12:0/9003761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf4f30 0x56534fcf5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff82d2b9640 1 -- 172.21.15.12:0/9003761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcfaa90 msgr2=0x56534fcfcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff82d2b9640 1 --2- 172.21.15.12:0/9003761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcfaa90 0x56534fcfcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff82d2b9640 1 -- 172.21.15.12:0/9003761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534fa377d0 con 0x56534fcf5920 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff82cab8640 1 --2- 172.21.15.12:0/9003761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcfaa90 0x56534fcfcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff81e7fc640 1 --2- 172.21.15.12:0/1544531809 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff8140a2b90 0x7ff8140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff82cab8640 1 --2- 172.21.15.12:0/1544531809 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff8140a2b90 0x7ff8140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff82d2b9640 1 --2- 172.21.15.12:0/9003761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 0x56534fcfa420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff824010450 tx=0x7ff824010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb8b0f6b3f19f9ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.010+0000 7ff81e7fc640 1 -- 172.21.15.12:0/1544531809 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff8200cf9b0 con 0x7ff828103430 2024-05-09T09:27:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.011+0000 7ff8137fe640 1 -- 172.21.15.12:0/9003761 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824011760 con 0x56534fcf5920 2024-05-09T09:27:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.011+0000 7ff8137fe640 1 -- 172.21.15.12:0/9003761 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff824011dd0 con 0x56534fcf5920 2024-05-09T09:27:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.011+0000 7ff8137fe640 1 -- 172.21.15.12:0/9003761 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82401a380 con 0x56534fcf5920 2024-05-09T09:27:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.011+0000 7ff832220740 1 -- 172.21.15.12:0/9003761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 msgr2=0x56534fcfa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.011+0000 7ff82cab8640 1 --2- 172.21.15.12:0/1544531809 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff8140a2b90 0x7ff8140a5050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff818000c00 tx=0x7ff818031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.011+0000 7ff832220740 1 --2- 172.21.15.12:0/9003761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 0x56534fcfa420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff824010450 tx=0x7ff824010910 comp rx=0 tx=0).stop 2024-05-09T09:27:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.011+0000 7ff832220740 1 -- 172.21.15.12:0/9003761 shutdown_connections 2024-05-09T09:27:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.011+0000 7ff832220740 1 --2- 172.21.15.12:0/9003761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcfaa90 0x56534fcfcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.011+0000 7ff832220740 1 --2- 172.21.15.12:0/9003761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 0x56534fcfa420 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.011+0000 7ff832220740 1 --2- 172.21.15.12:0/9003761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf4f30 0x56534fcf5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.011+0000 7ff832220740 1 -- 172.21.15.12:0/9003761 >> 172.21.15.12:0/9003761 conn(0x56534fcf05a0 msgr2=0x56534fcf09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.011+0000 7ff832220740 1 -- 172.21.15.12:0/9003761 shutdown_connections 2024-05-09T09:27:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff832220740 1 -- 172.21.15.12:0/9003761 wait complete. 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff832220740 1 -- start start 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff832220740 1 Processor -- start 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff832220740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fcff300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff832220740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 0x56534fcff840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff832220740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcfaa90 0x56534fd06880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff832220740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56534fcfda40 con 0x56534fcf4f30 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff832220740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56534fc14b90 con 0x56534fcf5920 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff832220740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56534fcfdbc0 con 0x56534fcfaa90 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff832220740 10 client.0 _ll_register_callbacks cb 0x56534fc50da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff82d2b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fcff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff82cab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 0x56534fcff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff82d2b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fcff300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50510/0 (socket says 172.21.15.12:50510) 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff82cab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 0x56534fcff840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:45730/0 (socket says 172.21.15.12:45730) 2024-05-09T09:27:51.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.012+0000 7ff82d2b9640 1 -- 172.21.15.12:0/4242926798 learned_addr learned my addr 172.21.15.12:0/4242926798 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff82daba640 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcfaa90 0x56534fd06880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff82cab8640 1 -- 172.21.15.12:0/4242926798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcfaa90 msgr2=0x56534fd06880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff82cab8640 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcfaa90 0x56534fd06880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff82cab8640 1 -- 172.21.15.12:0/4242926798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 msgr2=0x56534fcff300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff82cab8640 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fcff300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff82cab8640 1 -- 172.21.15.12:0/4242926798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534f96d3f0 con 0x56534fcf5920 2024-05-09T09:27:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff82d2b9640 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fcff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff82daba640 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcfaa90 0x56534fd06880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff82cab8640 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 0x56534fcff840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff818005ab0 tx=0x7ff81802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a160393d631d755a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff810ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:27:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff810ff9640 1 -- 172.21.15.12:0/4242926798 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81804e030 con 0x56534fcf5920 2024-05-09T09:27:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff810ff9640 1 -- 172.21.15.12:0/4242926798 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff818005100 con 0x56534fcf5920 2024-05-09T09:27:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff832220740 1 -- 172.21.15.12:0/4242926798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff824011dd0 con 0x56534fcf5920 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff810ff9640 1 -- 172.21.15.12:0/4242926798 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff81805b050 con 0x56534fcf5920 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff832220740 10 client.25895 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.013+0000 7ff832220740 1 -- 172.21.15.12:0/4242926798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56534f92d0f0 con 0x56534fcf5920 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.014+0000 7ff810ff9640 1 -- 172.21.15.12:0/4242926798 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff824011dd0 con 0x56534fcf5920 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.014+0000 7ff810ff9640 1 -- 172.21.15.12:0/4242926798 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7ff818047050 con 0x56534fcf5920 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.014+0000 7ff832220740 10 client.25895 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.014+0000 7ff832220740 10 client.25895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.014+0000 7ff832220740 20 client.25895 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.014+0000 7ff832220740 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56534fd0b240 0x56534fd0d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.014+0000 7ff832220740 4 client.25895 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.014+0000 7ff832220740 1 -- 172.21.15.12:0/4242926798 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56534fc67430 con 0x56534fd0b240 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.014+0000 7ff82d2b9640 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56534fd0b240 0x56534fd0d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.014+0000 7ff82d2b9640 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56534fd0b240 0x56534fd0d6a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dffe55ffe59c45b server_cookie=28d9874b1f1f372 in_seq=0 out_seq=0 2024-05-09T09:27:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.014+0000 7ff810ff9640 10 client.25895 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.015+0000 7ff810ff9640 1 -- 172.21.15.12:0/4242926798 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56534fc67430 con 0x56534fd0b240 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.015+0000 7ff810ff9640 10 client.25895 handle_command_reply: tid=0 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.015+0000 7ff832220740 1 client.25895 shutdown 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.015+0000 7ff832220740 1 -- 172.21.15.12:0/4242926798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 msgr2=0x56534fcff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.015+0000 7ff832220740 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 0x56534fcff840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff818005ab0 tx=0x7ff81802a410 comp rx=0 tx=0).stop 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.015+0000 7ff832220740 1 -- 172.21.15.12:0/4242926798 shutdown_connections 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.015+0000 7ff832220740 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56534fd0b240 0x56534fd0d6a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.015+0000 7ff832220740 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcfaa90 0x56534fd06880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.015+0000 7ff832220740 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcf5920 0x56534fcff840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.015+0000 7ff832220740 1 --2- 172.21.15.12:0/4242926798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fcff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.015+0000 7ff810ff9640 0 client.25895 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.015+0000 7ff832220740 1 -- 172.21.15.12:0/4242926798 >> 172.21.15.12:0/4242926798 conn(0x56534fcf05a0 msgr2=0x56534fcef7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.016+0000 7ff832220740 1 -- 172.21.15.12:0/4242926798 shutdown_connections 2024-05-09T09:27:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.016+0000 7ff832220740 1 -- 172.21.15.12:0/4242926798 wait complete. 2024-05-09T09:27:51.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.016+0000 7ff832220740 20 client.25895 trim_cache size 0 max 16384 2024-05-09T09:27:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff832220740 1 Processor -- start 2024-05-09T09:27:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff832220740 1 -- start start 2024-05-09T09:27:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff832220740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fcf6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff832220740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf5920 0x56534fcf7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff832220740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcfaa90 0x56534fcfe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff832220740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff8281ac7f0 con 0x56534fcf4f30 2024-05-09T09:27:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff832220740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8281ae3f0 con 0x56534fcfaa90 2024-05-09T09:27:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff832220740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8281acec0 con 0x56534fcf5920 2024-05-09T09:27:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff82d2b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fcf6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff82d2b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fcf6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50516/0 (socket says 172.21.15.12:50516) 2024-05-09T09:27:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff82d2b9640 1 -- 172.21.15.12:0/2069086337 learned_addr learned my addr 172.21.15.12:0/2069086337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff82daba640 1 --2- 172.21.15.12:0/2069086337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcfaa90 0x56534fcfe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff82cab8640 1 --2- 172.21.15.12:0/2069086337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf5920 0x56534fcf7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff82daba640 1 -- 172.21.15.12:0/2069086337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf5920 msgr2=0x56534fcf7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff82daba640 1 --2- 172.21.15.12:0/2069086337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf5920 0x56534fcf7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff82daba640 1 -- 172.21.15.12:0/2069086337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 msgr2=0x56534fcf6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff82daba640 1 --2- 172.21.15.12:0/2069086337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fcf6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.035+0000 7ff82daba640 1 -- 172.21.15.12:0/2069086337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534fd0dd20 con 0x56534fcfaa90 2024-05-09T09:27:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.036+0000 7ff82daba640 1 --2- 172.21.15.12:0/2069086337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcfaa90 0x56534fcfe5f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff8200cd130 tx=0x7ff82009ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ba267ea1849f8f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.036+0000 7ff812ffd640 1 -- 172.21.15.12:0/2069086337 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82009e040 con 0x56534fcfaa90 2024-05-09T09:27:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.036+0000 7ff812ffd640 1 -- 172.21.15.12:0/2069086337 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff82009c540 con 0x56534fcfaa90 2024-05-09T09:27:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.036+0000 7ff812ffd640 1 -- 172.21.15.12:0/2069086337 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82009c300 con 0x56534fcfaa90 2024-05-09T09:27:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.037+0000 7ff832220740 1 -- 172.21.15.12:0/2069086337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcfaa90 msgr2=0x56534fcfe5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.037+0000 7ff832220740 1 --2- 172.21.15.12:0/2069086337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcfaa90 0x56534fcfe5f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff8200cd130 tx=0x7ff82009ba40 comp rx=0 tx=0).stop 2024-05-09T09:27:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.037+0000 7ff832220740 1 -- 172.21.15.12:0/2069086337 shutdown_connections 2024-05-09T09:27:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.037+0000 7ff832220740 1 --2- 172.21.15.12:0/2069086337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcfaa90 0x56534fcfe5f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.037+0000 7ff832220740 1 --2- 172.21.15.12:0/2069086337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf5920 0x56534fcf7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.037+0000 7ff832220740 1 --2- 172.21.15.12:0/2069086337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fcf6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.037+0000 7ff832220740 1 -- 172.21.15.12:0/2069086337 >> 172.21.15.12:0/2069086337 conn(0x56534fcf05a0 msgr2=0x56534fcf0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.037+0000 7ff832220740 1 -- 172.21.15.12:0/2069086337 shutdown_connections 2024-05-09T09:27:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.037+0000 7ff832220740 1 -- 172.21.15.12:0/2069086337 wait complete. 2024-05-09T09:27:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.037+0000 7ff832220740 1 -- start start 2024-05-09T09:27:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.037+0000 7ff832220740 1 Processor -- start 2024-05-09T09:27:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff832220740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fd02f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff832220740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf5920 0x56534fd03450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff832220740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcfaa90 0x56534fd04460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff832220740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff8281add20 con 0x56534fcf4f30 2024-05-09T09:27:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff832220740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8281ad590 con 0x56534fcfaa90 2024-05-09T09:27:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff832220740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8281aa7d0 con 0x56534fcf5920 2024-05-09T09:27:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff832220740 10 client.0 _ll_register_callbacks cb 0x7ff818031750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82d2b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fd02f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82cab8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf5920 0x56534fd03450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82daba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcfaa90 0x56534fd04460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82cab8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf5920 0x56534fd03450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:57862/0 (socket says 172.21.15.12:57862) 2024-05-09T09:27:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82cab8640 1 -- 172.21.15.12:0/1428549282 learned_addr learned my addr 172.21.15.12:0/1428549282 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82d2b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fd02f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50532/0 (socket says 172.21.15.12:50532) 2024-05-09T09:27:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82daba640 1 -- 172.21.15.12:0/1428549282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf5920 msgr2=0x56534fd03450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82daba640 1 --2- 172.21.15.12:0/1428549282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf5920 0x56534fd03450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82daba640 1 -- 172.21.15.12:0/1428549282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 msgr2=0x56534fd02f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82daba640 1 --2- 172.21.15.12:0/1428549282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fd02f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82daba640 1 -- 172.21.15.12:0/1428549282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534fd04a50 con 0x56534fcfaa90 2024-05-09T09:27:51.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82cab8640 1 --2- 172.21.15.12:0/1428549282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf5920 0x56534fd03450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:51.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82d2b9640 1 --2- 172.21.15.12:0/1428549282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fd02f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:51.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff82daba640 1 --2- 172.21.15.12:0/1428549282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcfaa90 0x56534fd04460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff82009bdd0 tx=0x7ff8200bc050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=647062c10b06fe0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff810ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:27:51.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff810ff9640 1 -- 172.21.15.12:0/1428549282 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82009e070 con 0x56534fcfaa90 2024-05-09T09:27:51.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff832220740 1 -- 172.21.15.12:0/1428549282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff82009c540 con 0x56534fcfaa90 2024-05-09T09:27:51.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.038+0000 7ff810ff9640 1 -- 172.21.15.12:0/1428549282 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff8200a4820 con 0x56534fcfaa90 2024-05-09T09:27:51.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.039+0000 7ff810ff9640 1 -- 172.21.15.12:0/1428549282 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff8200a4ab0 con 0x56534fcfaa90 2024-05-09T09:27:51.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.039+0000 7ff832220740 10 client.25901 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:51.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.039+0000 7ff832220740 1 -- 172.21.15.12:0/1428549282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56534fd05d60 con 0x56534fcfaa90 2024-05-09T09:27:51.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.039+0000 7ff810ff9640 1 -- 172.21.15.12:0/1428549282 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff82009c540 con 0x56534fcfaa90 2024-05-09T09:27:51.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.039+0000 7ff810ff9640 1 -- 172.21.15.12:0/1428549282 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7ff82009ad00 con 0x56534fcfaa90 2024-05-09T09:27:51.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.039+0000 7ff832220740 10 client.25901 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.039+0000 7ff832220740 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.039+0000 7ff832220740 20 client.25901 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.039+0000 7ff832220740 1 --2- 172.21.15.12:0/1428549282 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56534fd71c30 0x56534fd74050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.039+0000 7ff832220740 4 client.25901 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.039+0000 7ff832220740 1 -- 172.21.15.12:0/1428549282 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56534fc61780 con 0x56534fd71c30 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.039+0000 7ff82d2b9640 1 --2- 172.21.15.12:0/1428549282 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56534fd71c30 0x56534fd74050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff82d2b9640 1 --2- 172.21.15.12:0/1428549282 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56534fd71c30 0x56534fd74050 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7cf6c0ec677c1c server_cookie=87a4454144ee9e77 in_seq=0 out_seq=0 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff810ff9640 10 client.25901 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff810ff9640 1 -- 172.21.15.12:0/1428549282 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56534fc61780 con 0x56534fd71c30 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff810ff9640 10 client.25901 handle_command_reply: tid=0 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff832220740 1 client.25901 shutdown 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff832220740 1 -- 172.21.15.12:0/1428549282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcfaa90 msgr2=0x56534fd04460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff832220740 1 --2- 172.21.15.12:0/1428549282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcfaa90 0x56534fd04460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff82009bdd0 tx=0x7ff8200bc050 comp rx=0 tx=0).stop 2024-05-09T09:27:51.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff832220740 1 -- 172.21.15.12:0/1428549282 shutdown_connections 2024-05-09T09:27:51.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff832220740 1 --2- 172.21.15.12:0/1428549282 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x56534fd71c30 0x56534fd74050 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff832220740 1 --2- 172.21.15.12:0/1428549282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56534fcfaa90 0x56534fd04460 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff832220740 1 --2- 172.21.15.12:0/1428549282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56534fcf5920 0x56534fd03450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff832220740 1 --2- 172.21.15.12:0/1428549282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56534fcf4f30 0x56534fd02f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff810ff9640 0 client.25901 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:51.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff832220740 1 -- 172.21.15.12:0/1428549282 >> 172.21.15.12:0/1428549282 conn(0x56534fcf05a0 msgr2=0x56534fceef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff832220740 1 -- 172.21.15.12:0/1428549282 shutdown_connections 2024-05-09T09:27:51.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff832220740 1 -- 172.21.15.12:0/1428549282 wait complete. 2024-05-09T09:27:51.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.040+0000 7ff832220740 20 client.25901 trim_cache size 0 max 16384 2024-05-09T09:27:51.047 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:27:51.048 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T09:27:51.048 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T09:27:51.048 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:27:51.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.041+0000 7ff82f51d640 1 -- 172.21.15.12:0/1544531809 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff8140a2b90 msgr2=0x7ff8140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.041+0000 7ff82f51d640 1 --2- 172.21.15.12:0/1544531809 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff8140a2b90 0x7ff8140a5050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff818000c00 tx=0x7ff818031510 comp rx=0 tx=0).stop 2024-05-09T09:27:51.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.041+0000 7ff82f51d640 1 -- 172.21.15.12:0/1544531809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff828103430 msgr2=0x7ff8281a13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.041+0000 7ff82f51d640 1 --2- 172.21.15.12:0/1544531809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff828103430 0x7ff8281a13a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff82000eb10 tx=0x7ff82000c2d0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.041+0000 7ff82f51d640 1 -- 172.21.15.12:0/1544531809 shutdown_connections 2024-05-09T09:27:51.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.041+0000 7ff82f51d640 1 --2- 172.21.15.12:0/1544531809 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff8140a2b90 0x7ff8140a5050 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.041+0000 7ff82f51d640 1 --2- 172.21.15.12:0/1544531809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff828103430 0x7ff8281a13a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.041+0000 7ff82f51d640 1 --2- 172.21.15.12:0/1544531809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff828102a70 0x7ff82819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.041+0000 7ff82f51d640 1 --2- 172.21.15.12:0/1544531809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828101880 0x7ff828199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.041+0000 7ff82f51d640 1 -- 172.21.15.12:0/1544531809 >> 172.21.15.12:0/1544531809 conn(0x7ff8280fce10 msgr2=0x7ff828073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.041+0000 7ff82f51d640 1 -- 172.21.15.12:0/1544531809 shutdown_connections 2024-05-09T09:27:51.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.041+0000 7ff82f51d640 1 -- 172.21.15.12:0/1544531809 wait complete. 2024-05-09T09:27:51.054 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:27:51.054 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:27:51.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.170+0000 7fd510af5640 1 -- 172.21.15.12:0/2985154376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c102320 msgr2=0x7fd50c108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.170+0000 7fd510af5640 1 --2- 172.21.15.12:0/2985154376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c102320 0x7fd50c108bb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd504009ca0 tx=0x7fd50402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.170+0000 7fd510af5640 1 -- 172.21.15.12:0/2985154376 shutdown_connections 2024-05-09T09:27:51.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.170+0000 7fd510af5640 1 --2- 172.21.15.12:0/2985154376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c102320 0x7fd50c108bb0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.170+0000 7fd510af5640 1 --2- 172.21.15.12:0/2985154376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd50c101960 0x7fd50c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.170+0000 7fd510af5640 1 --2- 172.21.15.12:0/2985154376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd50c100770 0x7fd50c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.170+0000 7fd510af5640 1 -- 172.21.15.12:0/2985154376 >> 172.21.15.12:0/2985154376 conn(0x7fd50c0fbce0 msgr2=0x7fd50c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.170+0000 7fd510af5640 1 -- 172.21.15.12:0/2985154376 shutdown_connections 2024-05-09T09:27:51.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.170+0000 7fd510af5640 1 -- 172.21.15.12:0/2985154376 wait complete. 2024-05-09T09:27:51.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd510af5640 1 Processor -- start 2024-05-09T09:27:51.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd510af5640 1 -- start start 2024-05-09T09:27:51.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd510af5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd50c100770 0x7fd50c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd510af5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd50c101960 0x7fd50c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd510af5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c102320 0x7fd50c1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd510af5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c10c3c0 con 0x7fd50c102320 2024-05-09T09:27:51.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd510af5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c10c240 con 0x7fd50c100770 2024-05-09T09:27:51.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd510af5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c10c540 con 0x7fd50c101960 2024-05-09T09:27:51.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd50ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c102320 0x7fd50c1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd503fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd50c101960 0x7fd50c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd50ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c102320 0x7fd50c1a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50550/0 (socket says 172.21.15.12:50550) 2024-05-09T09:27:51.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd50ad9d640 1 -- 172.21.15.12:0/3796181623 learned_addr learned my addr 172.21.15.12:0/3796181623 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd50a59c640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd50c100770 0x7fd50c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd503fff640 1 -- 172.21.15.12:0/3796181623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd50c100770 msgr2=0x7fd50c069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd503fff640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd50c100770 0x7fd50c069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd503fff640 1 -- 172.21.15.12:0/3796181623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c102320 msgr2=0x7fd50c1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd503fff640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c102320 0x7fd50c1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd503fff640 1 -- 172.21.15.12:0/3796181623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50c1a1990 con 0x7fd50c101960 2024-05-09T09:27:51.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.171+0000 7fd50ad9d640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c102320 0x7fd50c1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:51.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.172+0000 7fd503fff640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd50c101960 0x7fd50c19ec50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4005fd0 tx=0x7fd4f400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.172+0000 7fd5037fe640 1 -- 172.21.15.12:0/3796181623 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4009c30 con 0x7fd50c101960 2024-05-09T09:27:51.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.172+0000 7fd510af5640 1 -- 172.21.15.12:0/3796181623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50c1a1c20 con 0x7fd50c101960 2024-05-09T09:27:51.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.172+0000 7fd5037fe640 1 -- 172.21.15.12:0/3796181623 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd4f4009dd0 con 0x7fd50c101960 2024-05-09T09:27:51.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.172+0000 7fd510af5640 1 -- 172.21.15.12:0/3796181623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd50c1a21c0 con 0x7fd50c101960 2024-05-09T09:27:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.172+0000 7fd5037fe640 1 -- 172.21.15.12:0/3796181623 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4020060 con 0x7fd50c101960 2024-05-09T09:27:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.173+0000 7fd510af5640 1 -- 172.21.15.12:0/3796181623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd50c107a90 con 0x7fd50c101960 2024-05-09T09:27:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.174+0000 7fd5037fe640 1 -- 172.21.15.12:0/3796181623 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd4f4003e70 con 0x7fd50c101960 2024-05-09T09:27:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.175+0000 7fd5037fe640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4f80a2bc0 0x7fd4f80a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.175+0000 7fd50a59c640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4f80a2bc0 0x7fd4f80a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.175+0000 7fd5037fe640 1 -- 172.21.15.12:0/3796181623 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd4f40cf540 con 0x7fd50c101960 2024-05-09T09:27:51.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.175+0000 7fd50a59c640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4f80a2bc0 0x7fd4f80a5080 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc0095f0 tx=0x7fd4fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.179+0000 7fd5037fe640 1 -- 172.21.15.12:0/3796181623 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd4f4017020 con 0x7fd50c101960 2024-05-09T09:27:51.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.445+0000 7fd510af5640 1 -- 172.21.15.12:0/3796181623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd50c02d260 con 0x7fd50c101960 2024-05-09T09:27:51.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.445+0000 7fd5037fe640 1 -- 172.21.15.12:0/3796181623 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11938 (secure 0 0 0) 0x7fd4f409ba20 con 0x7fd50c101960 2024-05-09T09:27:51.445 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:27:51.445 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:27:41.047243+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25883":{"gid":25883,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3067829157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3067829157},{"type":"v1","addr":"172.21.15.16:6837","nonce":3067829157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[24502,24523,24515,14628,24484]},"id":1}]} 2024-05-09T09:27:51.446 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2024-05-09T09:27:51.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.448+0000 7fd510af5640 1 -- 172.21.15.12:0/3796181623 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4f80a2bc0 msgr2=0x7fd4f80a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.448+0000 7fd510af5640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4f80a2bc0 0x7fd4f80a5080 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc0095f0 tx=0x7fd4fc005f90 comp rx=0 tx=0).stop 2024-05-09T09:27:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.448+0000 7fd510af5640 1 -- 172.21.15.12:0/3796181623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd50c101960 msgr2=0x7fd50c19ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.448+0000 7fd510af5640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd50c101960 0x7fd50c19ec50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4005fd0 tx=0x7fd4f400dde0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.448+0000 7fd510af5640 1 -- 172.21.15.12:0/3796181623 shutdown_connections 2024-05-09T09:27:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.448+0000 7fd510af5640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4f80a2bc0 0x7fd4f80a5080 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.448+0000 7fd510af5640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c102320 0x7fd50c1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.448+0000 7fd510af5640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd50c101960 0x7fd50c19ec50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.448+0000 7fd510af5640 1 --2- 172.21.15.12:0/3796181623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd50c100770 0x7fd50c069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.448+0000 7fd510af5640 1 -- 172.21.15.12:0/3796181623 >> 172.21.15.12:0/3796181623 conn(0x7fd50c0fbce0 msgr2=0x7fd50c104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.448+0000 7fd510af5640 1 -- 172.21.15.12:0/3796181623 shutdown_connections 2024-05-09T09:27:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.448+0000 7fd510af5640 1 -- 172.21.15.12:0/3796181623 wait complete. 2024-05-09T09:27:51.459 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-09T09:27:51.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.579+0000 7ff9bd5c8640 1 -- 172.21.15.12:0/714749979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9b8101980 msgr2=0x7ff9b8101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.579+0000 7ff9bd5c8640 1 --2- 172.21.15.12:0/714749979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9b8101980 0x7ff9b8101e00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009900 tx=0x7ff9a002a410 comp rx=0 tx=0).stop 2024-05-09T09:27:51.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.579+0000 7ff9bd5c8640 1 -- 172.21.15.12:0/714749979 shutdown_connections 2024-05-09T09:27:51.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.579+0000 7ff9bd5c8640 1 --2- 172.21.15.12:0/714749979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b8102340 0x7ff9b8108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.579+0000 7ff9bd5c8640 1 --2- 172.21.15.12:0/714749979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9b8101980 0x7ff9b8101e00 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.579+0000 7ff9bd5c8640 1 --2- 172.21.15.12:0/714749979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8100790 0x7ff9b8100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.579+0000 7ff9bd5c8640 1 -- 172.21.15.12:0/714749979 >> 172.21.15.12:0/714749979 conn(0x7ff9b80fbd20 msgr2=0x7ff9b80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.580+0000 7ff9bd5c8640 1 -- 172.21.15.12:0/714749979 shutdown_connections 2024-05-09T09:27:51.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.580+0000 7ff9bd5c8640 1 -- 172.21.15.12:0/714749979 wait complete. 2024-05-09T09:27:51.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.580+0000 7ff9bd5c8640 1 Processor -- start 2024-05-09T09:27:51.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.580+0000 7ff9bd5c8640 1 -- start start 2024-05-09T09:27:51.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.580+0000 7ff9bd5c8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b8100790 0x7ff9b8197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.580+0000 7ff9bd5c8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9b8101980 0x7ff9b81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.580+0000 7ff9bd5c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8102340 0x7ff9b819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.580+0000 7ff9bd5c8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b810d500 con 0x7ff9b8101980 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.580+0000 7ff9bd5c8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b810d380 con 0x7ff9b8102340 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.580+0000 7ff9bd5c8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b810d680 con 0x7ff9b8100790 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.580+0000 7ff9affff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9b8101980 0x7ff9b81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.580+0000 7ff9affff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9b8101980 0x7ff9b81980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50570/0 (socket says 172.21.15.12:50570) 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.581+0000 7ff9affff640 1 -- 172.21.15.12:0/2055852300 learned_addr learned my addr 172.21.15.12:0/2055852300 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.581+0000 7ff9b6ffd640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b8100790 0x7ff9b8197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.581+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8102340 0x7ff9b819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.581+0000 7ff9affff640 1 -- 172.21.15.12:0/2055852300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b8100790 msgr2=0x7ff9b8197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.581+0000 7ff9affff640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b8100790 0x7ff9b8197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.581+0000 7ff9affff640 1 -- 172.21.15.12:0/2055852300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8102340 msgr2=0x7ff9b819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.581+0000 7ff9affff640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8102340 0x7ff9b819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.581+0000 7ff9affff640 1 -- 172.21.15.12:0/2055852300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b819f7d0 con 0x7ff9b8101980 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.581+0000 7ff9affff640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9b8101980 0x7ff9b81980f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009230 tx=0x7ff9a0031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.581+0000 7ff9b4ff9640 1 -- 172.21.15.12:0/2055852300 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a003f030 con 0x7ff9b8101980 2024-05-09T09:27:51.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.581+0000 7ff9bd5c8640 1 -- 172.21.15.12:0/2055852300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b819fa60 con 0x7ff9b8101980 2024-05-09T09:27:51.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.581+0000 7ff9b4ff9640 1 -- 172.21.15.12:0/2055852300 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff9a0003ac0 con 0x7ff9b8101980 2024-05-09T09:27:51.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.581+0000 7ff9bd5c8640 1 -- 172.21.15.12:0/2055852300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b819ffa0 con 0x7ff9b8101980 2024-05-09T09:27:51.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.582+0000 7ff9b4ff9640 1 -- 172.21.15.12:0/2055852300 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0049060 con 0x7ff9b8101980 2024-05-09T09:27:51.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.585+0000 7ff9c03b3740 1 Processor -- start 2024-05-09T09:27:51.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.585+0000 7ff9b4ff9640 1 -- 172.21.15.12:0/2055852300 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff9a0042030 con 0x7ff9b8101980 2024-05-09T09:27:51.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.585+0000 7ff9c03b3740 1 -- start start 2024-05-09T09:27:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.585+0000 7ff9c03b3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e568350 0x55868e568770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9c03b3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568d40 0x55868e56d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9c03b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e56deb0 0x55868e570340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9c03b3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55868e4c8c40 con 0x55868e568d40 2024-05-09T09:27:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9c03b3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55868e470580 con 0x55868e568350 2024-05-09T09:27:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9c03b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55868e4c4c00 con 0x55868e56deb0 2024-05-09T09:27:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e568350 0x55868e568770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9b6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568d40 0x55868e56d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e568350 0x55868e568770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:45778/0 (socket says 172.21.15.12:45778) 2024-05-09T09:27:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9b77fe640 1 -- 172.21.15.12:0/4193672522 learned_addr learned my addr 172.21.15.12:0/4193672522 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9affff640 1 --2- 172.21.15.12:0/4193672522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e56deb0 0x55868e570340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9b6ffd640 1 -- 172.21.15.12:0/4193672522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e56deb0 msgr2=0x55868e570340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9b6ffd640 1 --2- 172.21.15.12:0/4193672522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e56deb0 0x55868e570340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9b6ffd640 1 -- 172.21.15.12:0/4193672522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e568350 msgr2=0x55868e568770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9b6ffd640 1 --2- 172.21.15.12:0/4193672522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e568350 0x55868e568770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9b4ff9640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff9a40a2b90 0x7ff9a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9b6ffd640 1 -- 172.21.15.12:0/4193672522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868e1dc3f0 con 0x55868e568d40 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/4193672522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e568350 0x55868e568770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9affff640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff9a40a2b90 0x7ff9a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9b6ffd640 1 --2- 172.21.15.12:0/4193672522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568d40 0x55868e56d840 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff9b81017e0 tx=0x7ff9a800bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8eb1fdac54817c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9b4ff9640 1 -- 172.21.15.12:0/2055852300 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff9a00efa30 con 0x7ff9b8101980 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.586+0000 7ff9ad7fa640 1 -- 172.21.15.12:0/4193672522 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8003dd0 con 0x55868e568d40 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.587+0000 7ff9ad7fa640 1 -- 172.21.15.12:0/4193672522 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff9a8015070 con 0x55868e568d40 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.587+0000 7ff9ad7fa640 1 -- 172.21.15.12:0/4193672522 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80116a0 con 0x55868e568d40 2024-05-09T09:27:51.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.587+0000 7ff9c03b3740 1 -- 172.21.15.12:0/4193672522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568d40 msgr2=0x55868e56d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.587+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/4193672522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568d40 0x55868e56d840 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff9b81017e0 tx=0x7ff9a800bbe0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.587+0000 7ff9affff640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff9a40a2b90 0x7ff9a40a5050 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00c4600 tx=0x7ff9a00f3510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.587+0000 7ff9c03b3740 1 -- 172.21.15.12:0/4193672522 shutdown_connections 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.587+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/4193672522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e56deb0 0x55868e570340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.587+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/4193672522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568d40 0x55868e56d840 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.587+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/4193672522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e568350 0x55868e568770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.587+0000 7ff9c03b3740 1 -- 172.21.15.12:0/4193672522 >> 172.21.15.12:0/4193672522 conn(0x55868e55f5b0 msgr2=0x55868e55f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.587+0000 7ff9c03b3740 1 -- 172.21.15.12:0/4193672522 shutdown_connections 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9c03b3740 1 -- 172.21.15.12:0/4193672522 wait complete. 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9c03b3740 1 -- start start 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9c03b3740 1 Processor -- start 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9c03b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568350 0x55868e572770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9c03b3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568d40 0x55868e572cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9c03b3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 0x55868e579cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9c03b3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55868e570e60 con 0x55868e568d40 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9c03b3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55868e483c40 con 0x55868e56deb0 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9c03b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55868e570fe0 con 0x55868e568350 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9c03b3740 10 client.0 _ll_register_callbacks cb 0x55868e1dbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9b6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568d40 0x55868e572cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9b77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568350 0x55868e572770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9b6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568d40 0x55868e572cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50596/0 (socket says 172.21.15.12:50596) 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9b77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568350 0x55868e572770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:57900/0 (socket says 172.21.15.12:57900) 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9b6ffd640 1 -- 172.21.15.12:0/3934593207 learned_addr learned my addr 172.21.15.12:0/3934593207 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.588+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/3934593207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 0x55868e579cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff9b77fe640 1 -- 172.21.15.12:0/3934593207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 msgr2=0x55868e579cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/3934593207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 0x55868e579cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff9b77fe640 1 -- 172.21.15.12:0/3934593207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568d40 msgr2=0x55868e572cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/3934593207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568d40 0x55868e572cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff9b77fe640 1 -- 172.21.15.12:0/3934593207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55868e2a67d0 con 0x55868e568350 2024-05-09T09:27:51.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff9b6ffd640 1 --2- 172.21.15.12:0/3934593207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568d40 0x55868e572cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/3934593207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568350 0x55868e572770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55868e5695a0 tx=0x7ff9b0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a7144261a9fd098 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff99e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff99e7fc640 1 -- 172.21.15.12:0/3934593207 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0013070 con 0x55868e568350 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff99e7fc640 1 -- 172.21.15.12:0/3934593207 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff9b0013210 con 0x55868e568350 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3934593207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a8015070 con 0x55868e568350 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff99e7fc640 1 -- 172.21.15.12:0/3934593207 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff9b0028050 con 0x55868e568350 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff9c03b3740 10 client.26029 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.589+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3934593207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55868e57d0d0 con 0x55868e568350 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.590+0000 7ff99e7fc640 1 -- 172.21.15.12:0/3934593207 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff9a8015070 con 0x55868e568350 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.590+0000 7ff99e7fc640 1 -- 172.21.15.12:0/3934593207 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7ff9b0004660 con 0x55868e568350 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.590+0000 7ff9c03b3740 10 client.26029 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.590+0000 7ff9c03b3740 10 client.26029 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.590+0000 7ff9c03b3740 20 client.26029 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.590+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3934593207 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55868e57e6f0 0x55868e580bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.590+0000 7ff9c03b3740 4 client.26029 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.590+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3934593207 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55868e4d6370 con 0x55868e57e6f0 2024-05-09T09:27:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.590+0000 7ff9b6ffd640 1 --2- 172.21.15.12:0/3934593207 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55868e57e6f0 0x55868e580bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.590+0000 7ff9b6ffd640 1 --2- 172.21.15.12:0/3934593207 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55868e57e6f0 0x55868e580bb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=953666bbf84e4af1 server_cookie=22aca23e5ced46c9 in_seq=0 out_seq=0 2024-05-09T09:27:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.590+0000 7ff99e7fc640 10 client.26029 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:27:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff99e7fc640 1 -- 172.21.15.12:0/3934593207 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55868e4d6370 con 0x55868e57e6f0 2024-05-09T09:27:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff99e7fc640 10 client.26029 handle_command_reply: tid=0 2024-05-09T09:27:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff9c03b3740 1 client.26029 shutdown 2024-05-09T09:27:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3934593207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568350 msgr2=0x55868e572770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3934593207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568350 0x55868e572770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55868e5695a0 tx=0x7ff9b0004a40 comp rx=0 tx=0).stop 2024-05-09T09:27:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3934593207 shutdown_connections 2024-05-09T09:27:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3934593207 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55868e57e6f0 0x55868e580bb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3934593207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 0x55868e579cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff99e7fc640 0 client.26029 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:27:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3934593207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568d40 0x55868e572cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3934593207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568350 0x55868e572770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3934593207 >> 172.21.15.12:0/3934593207 conn(0x55868e55f5b0 msgr2=0x55868e55e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3934593207 shutdown_connections 2024-05-09T09:27:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3934593207 wait complete. 2024-05-09T09:27:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.591+0000 7ff9c03b3740 20 client.26029 trim_cache size 0 max 16384 2024-05-09T09:27:51.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.607+0000 7ff9c03b3740 1 Processor -- start 2024-05-09T09:27:51.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.608+0000 7ff9c03b3740 1 -- start start 2024-05-09T09:27:51.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.608+0000 7ff9c03b3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 0x55868e572a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.608+0000 7ff9c03b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568d40 0x55868e579cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.608+0000 7ff9c03b3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 0x55868e562330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.608+0000 7ff9b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 0x55868e572a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.608+0000 7ff9c03b3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81a8440 con 0x55868e568350 2024-05-09T09:27:51.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.608+0000 7ff9c03b3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81aa040 con 0x55868e56deb0 2024-05-09T09:27:51.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.608+0000 7ff9c03b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81a8b10 con 0x55868e568d40 2024-05-09T09:27:51.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.608+0000 7ff9b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 0x55868e572a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50602/0 (socket says 172.21.15.12:50602) 2024-05-09T09:27:51.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.608+0000 7ff9b77fe640 1 -- 172.21.15.12:0/3992607573 learned_addr learned my addr 172.21.15.12:0/3992607573 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.608+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/3992607573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 0x55868e562330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.608+0000 7ff9b6ffd640 1 --2- 172.21.15.12:0/3992607573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568d40 0x55868e579cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.609+0000 7ff9b77fe640 1 -- 172.21.15.12:0/3992607573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568d40 msgr2=0x55868e579cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.609+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/3992607573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568d40 0x55868e579cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.609+0000 7ff9b77fe640 1 -- 172.21.15.12:0/3992607573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 msgr2=0x55868e562330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.609+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/3992607573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 0x55868e562330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.609+0000 7ff9b77fe640 1 -- 172.21.15.12:0/3992607573 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868e569120 con 0x55868e568350 2024-05-09T09:27:51.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.609+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/3992607573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 0x55868e572a90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005d70 tx=0x7ff9b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2d2b6d6c4af91a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.609+0000 7ff9acff9640 1 -- 172.21.15.12:0/3992607573 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0013070 con 0x55868e568350 2024-05-09T09:27:51.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.609+0000 7ff9acff9640 1 -- 172.21.15.12:0/3992607573 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff9b0013210 con 0x55868e568350 2024-05-09T09:27:51.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.609+0000 7ff9acff9640 1 -- 172.21.15.12:0/3992607573 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0005040 con 0x55868e568350 2024-05-09T09:27:51.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.610+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3992607573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 msgr2=0x55868e572a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.610+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3992607573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 0x55868e572a90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005d70 tx=0x7ff9b0017000 comp rx=0 tx=0).stop 2024-05-09T09:27:51.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.610+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3992607573 shutdown_connections 2024-05-09T09:27:51.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.610+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3992607573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 0x55868e562330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.610+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3992607573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568d40 0x55868e579cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.610+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3992607573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 0x55868e572a90 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.610+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3992607573 >> 172.21.15.12:0/3992607573 conn(0x55868e55f5b0 msgr2=0x55868e572380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.610+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3992607573 shutdown_connections 2024-05-09T09:27:51.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.610+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3992607573 wait complete. 2024-05-09T09:27:51.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9c03b3740 1 -- start start 2024-05-09T09:27:51.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9c03b3740 1 Processor -- start 2024-05-09T09:27:51.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9c03b3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 0x55868e576c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9c03b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568d40 0x55868e577160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9c03b3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 0x55868e5e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9c03b3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81a9970 con 0x55868e568350 2024-05-09T09:27:51.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9c03b3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81a91e0 con 0x55868e56deb0 2024-05-09T09:27:51.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9c03b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81a6420 con 0x55868e568d40 2024-05-09T09:27:51.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9c03b3740 10 client.0 _ll_register_callbacks cb 0x7ff9b0009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:51.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 0x55868e576c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 0x55868e576c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50614/0 (socket says 172.21.15.12:50614) 2024-05-09T09:27:51.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9b77fe640 1 -- 172.21.15.12:0/3750810391 learned_addr learned my addr 172.21.15.12:0/3750810391 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9b6ffd640 1 --2- 172.21.15.12:0/3750810391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568d40 0x55868e577160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/3750810391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 0x55868e5e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9b77fe640 1 -- 172.21.15.12:0/3750810391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568d40 msgr2=0x55868e577160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/3750810391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568d40 0x55868e577160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9b77fe640 1 -- 172.21.15.12:0/3750810391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 msgr2=0x55868e5e2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/3750810391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 0x55868e5e2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9b77fe640 1 -- 172.21.15.12:0/3750810391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55868e5e32d0 con 0x55868e568350 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff9b77fe640 1 --2- 172.21.15.12:0/3750810391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 0x55868e576c20 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00114c0 tx=0x7ff9b0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b136b1deb6d708c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.611+0000 7ff99e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff99e7fc640 1 -- 172.21.15.12:0/3750810391 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0021430 con 0x55868e568350 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff99e7fc640 1 -- 172.21.15.12:0/3750810391 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff9b001c070 con 0x55868e568350 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3750810391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9b0013210 con 0x55868e568350 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff99e7fc640 1 -- 172.21.15.12:0/3750810391 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff9b0042050 con 0x55868e568350 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff9c03b3740 10 client.16476 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3750810391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55868e5e4570 con 0x55868e568350 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff99e7fc640 1 -- 172.21.15.12:0/3750810391 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff9b0013210 con 0x55868e568350 2024-05-09T09:27:51.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff99e7fc640 1 -- 172.21.15.12:0/3750810391 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7ff9b0039410 con 0x55868e568350 2024-05-09T09:27:51.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff9c03b3740 10 client.16476 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:51.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff9c03b3740 10 client.16476 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:27:51.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff9c03b3740 20 client.16476 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:51.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3750810391 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55868e5e4ec0 0x55868e5e72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff9c03b3740 4 client.16476 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:27:51.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.612+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3750810391 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55868e4d06c0 con 0x55868e5e4ec0 2024-05-09T09:27:51.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.613+0000 7ff9b6ffd640 1 --2- 172.21.15.12:0/3750810391 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55868e5e4ec0 0x55868e5e72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.613+0000 7ff9b6ffd640 1 --2- 172.21.15.12:0/3750810391 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55868e5e4ec0 0x55868e5e72e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=592fb98133ba4604 server_cookie=569416b64e6f53a9 in_seq=0 out_seq=0 2024-05-09T09:27:51.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.613+0000 7ff99e7fc640 10 client.16476 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:27:51.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff99e7fc640 1 -- 172.21.15.12:0/3750810391 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55868e4d06c0 con 0x55868e5e4ec0 2024-05-09T09:27:51.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff99e7fc640 10 client.16476 handle_command_reply: tid=0 2024-05-09T09:27:51.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff9c03b3740 1 client.16476 shutdown 2024-05-09T09:27:51.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3750810391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 msgr2=0x55868e576c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3750810391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 0x55868e576c20 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00114c0 tx=0x7ff9b0000f40 comp rx=0 tx=0).stop 2024-05-09T09:27:51.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3750810391 shutdown_connections 2024-05-09T09:27:51.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3750810391 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x55868e5e4ec0 0x55868e5e72e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3750810391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868e56deb0 0x55868e5e2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3750810391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55868e568d40 0x55868e577160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff9c03b3740 1 --2- 172.21.15.12:0/3750810391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55868e568350 0x55868e576c20 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3750810391 >> 172.21.15.12:0/3750810391 conn(0x55868e55f5b0 msgr2=0x55868e55df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff99e7fc640 0 client.16476 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:27:51.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3750810391 shutdown_connections 2024-05-09T09:27:51.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff9c03b3740 1 -- 172.21.15.12:0/3750810391 wait complete. 2024-05-09T09:27:51.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.614+0000 7ff9c03b3740 20 client.16476 trim_cache size 0 max 16384 2024-05-09T09:27:51.622 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:27:51.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.615+0000 7ff9bd5c8640 1 -- 172.21.15.12:0/2055852300 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff9a40a2b90 msgr2=0x7ff9a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.615+0000 7ff9bd5c8640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff9a40a2b90 0x7ff9a40a5050 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00c4600 tx=0x7ff9a00f3510 comp rx=0 tx=0).stop 2024-05-09T09:27:51.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.615+0000 7ff9bd5c8640 1 -- 172.21.15.12:0/2055852300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9b8101980 msgr2=0x7ff9b81980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.615+0000 7ff9bd5c8640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9b8101980 0x7ff9b81980f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009230 tx=0x7ff9a0031ac0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.615+0000 7ff9bd5c8640 1 -- 172.21.15.12:0/2055852300 shutdown_connections 2024-05-09T09:27:51.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.615+0000 7ff9bd5c8640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff9a40a2b90 0x7ff9a40a5050 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.615+0000 7ff9bd5c8640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8102340 0x7ff9b819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.615+0000 7ff9bd5c8640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9b8101980 0x7ff9b81980f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.615+0000 7ff9bd5c8640 1 --2- 172.21.15.12:0/2055852300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b8100790 0x7ff9b8197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.615+0000 7ff9bd5c8640 1 -- 172.21.15.12:0/2055852300 >> 172.21.15.12:0/2055852300 conn(0x7ff9b80fbd20 msgr2=0x7ff9b8104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.615+0000 7ff9bd5c8640 1 -- 172.21.15.12:0/2055852300 shutdown_connections 2024-05-09T09:27:51.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.615+0000 7ff9bd5c8640 1 -- 172.21.15.12:0/2055852300 wait complete. 2024-05-09T09:27:51.627 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-09T09:27:51.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.753+0000 7f219861b640 1 -- 172.21.15.12:0/1262953779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21901023b0 msgr2=0x7f21901047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.753+0000 7f219861b640 1 --2- 172.21.15.12:0/1262953779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21901023b0 0x7f21901047a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f218c009900 tx=0x7f218c02a410 comp rx=0 tx=0).stop 2024-05-09T09:27:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.753+0000 7f219861b640 1 -- 172.21.15.12:0/1262953779 shutdown_connections 2024-05-09T09:27:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.753+0000 7f219861b640 1 --2- 172.21.15.12:0/1262953779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2190107610 0x7f2190109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.753+0000 7f219861b640 1 --2- 172.21.15.12:0/1262953779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2190104ce0 0x7f21901070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.753+0000 7f219861b640 1 --2- 172.21.15.12:0/1262953779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21901023b0 0x7f21901047a0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.753+0000 7f219861b640 1 -- 172.21.15.12:0/1262953779 >> 172.21.15.12:0/1262953779 conn(0x7f21900fbce0 msgr2=0x7f21900fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.753+0000 7f219861b640 1 -- 172.21.15.12:0/1262953779 shutdown_connections 2024-05-09T09:27:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.753+0000 7f219861b640 1 -- 172.21.15.12:0/1262953779 wait complete. 2024-05-09T09:27:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.754+0000 7f219861b640 1 Processor -- start 2024-05-09T09:27:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.754+0000 7f219861b640 1 -- start start 2024-05-09T09:27:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f219861b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21901023b0 0x7f2190197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f219861b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2190104ce0 0x7f2190198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f219861b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2190107610 0x7f219019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f219861b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f219010d500 con 0x7f21901023b0 2024-05-09T09:27:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f219861b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f219010d380 con 0x7f2190104ce0 2024-05-09T09:27:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f219861b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f219010d680 con 0x7f2190107610 2024-05-09T09:27:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f21963b7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21901023b0 0x7f2190197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f21963b7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21901023b0 0x7f2190197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46096/0 (socket says 172.21.15.12:46096) 2024-05-09T09:27:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f2196bb8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2190107610 0x7f219019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f21963b7640 1 -- 172.21.15.12:0/1165389069 learned_addr learned my addr 172.21.15.12:0/1165389069 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f2195bb6640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2190104ce0 0x7f2190198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f21963b7640 1 -- 172.21.15.12:0/1165389069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2190107610 msgr2=0x7f219019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f21963b7640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2190107610 0x7f219019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f21963b7640 1 -- 172.21.15.12:0/1165389069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2190104ce0 msgr2=0x7f2190198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f21963b7640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2190104ce0 0x7f2190198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f21963b7640 1 -- 172.21.15.12:0/1165389069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219019f830 con 0x7f21901023b0 2024-05-09T09:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f2195bb6640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2190104ce0 0x7f2190198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.755+0000 7f2196bb8640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2190107610 0x7f219019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.756+0000 7f21963b7640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21901023b0 0x7f2190197c20 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f218c009230 tx=0x7f218c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.756+0000 7f21877fe640 1 -- 172.21.15.12:0/1165389069 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c03f030 con 0x7f21901023b0 2024-05-09T09:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.756+0000 7f21877fe640 1 -- 172.21.15.12:0/1165389069 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f218c031980 con 0x7f21901023b0 2024-05-09T09:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.756+0000 7f219861b640 1 -- 172.21.15.12:0/1165389069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219019fac0 con 0x7f21901023b0 2024-05-09T09:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.756+0000 7f21877fe640 1 -- 172.21.15.12:0/1165389069 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c035070 con 0x7f21901023b0 2024-05-09T09:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.756+0000 7f219861b640 1 -- 172.21.15.12:0/1165389069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219019ffb0 con 0x7f21901023b0 2024-05-09T09:27:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.758+0000 7f21877fe640 1 -- 172.21.15.12:0/1165389069 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f218c042030 con 0x7f21901023b0 2024-05-09T09:27:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.759+0000 7f219b3fd740 1 Processor -- start 2024-05-09T09:27:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.759+0000 7f219b3fd740 1 -- start start 2024-05-09T09:27:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.759+0000 7f219b3fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f2d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.759+0000 7f219b3fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 0x55ac02f32430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.759+0000 7f21877fe640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f217c0a2b90 0x7f217c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.759+0000 7f219b3fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f32aa0 0x55ac02f34f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.759+0000 7f219b3fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ac02e91c40 con 0x55ac02f32aa0 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.759+0000 7f219b3fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac02e39580 con 0x55ac02f2cf40 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.759+0000 7f219b3fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ac02e8dc00 con 0x55ac02f2d930 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f2196bb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f2d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f2196bb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f2d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:40994/0 (socket says 172.21.15.12:40994) 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f2196bb8640 1 -- 172.21.15.12:0/1350243530 learned_addr learned my addr 172.21.15.12:0/1350243530 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f2196bb8640 1 --2- 172.21.15.12:0/1350243530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f32aa0 0x55ac02f34f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f21877fe640 1 -- 172.21.15.12:0/1165389069 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f218c0efa20 con 0x7f21901023b0 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f2195bb6640 1 --2- 172.21.15.12:0/1350243530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 0x55ac02f32430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f21963b7640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f217c0a2b90 0x7f217c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f2196bb8640 1 -- 172.21.15.12:0/1350243530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 msgr2=0x55ac02f32430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f2196bb8640 1 --2- 172.21.15.12:0/1350243530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 0x55ac02f32430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f2196bb8640 1 -- 172.21.15.12:0/1350243530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f32aa0 msgr2=0x55ac02f34f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f2196bb8640 1 --2- 172.21.15.12:0/1350243530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f32aa0 0x55ac02f34f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f2196bb8640 1 -- 172.21.15.12:0/1350243530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac02ba53f0 con 0x55ac02f2cf40 2024-05-09T09:27:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f2195bb6640 1 --2- 172.21.15.12:0/1350243530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 0x55ac02f32430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f2196bb8640 1 --2- 172.21.15.12:0/1350243530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f2d360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2188011f00 tx=0x7f2188009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16aae18f7ed1fae2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f2184bfe640 1 -- 172.21.15.12:0/1350243530 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218800fd10 con 0x55ac02f2cf40 2024-05-09T09:27:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.760+0000 7f21963b7640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f217c0a2b90 0x7f217c0a5050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f218c032b60 tx=0x7f218c03b460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.761+0000 7f2184bfe640 1 -- 172.21.15.12:0/1350243530 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2188003e40 con 0x55ac02f2cf40 2024-05-09T09:27:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.761+0000 7f2184bfe640 1 -- 172.21.15.12:0/1350243530 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188004820 con 0x55ac02f2cf40 2024-05-09T09:27:51.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.761+0000 7f219b3fd740 1 -- 172.21.15.12:0/1350243530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 msgr2=0x55ac02f2d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.761+0000 7f219b3fd740 1 --2- 172.21.15.12:0/1350243530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f2d360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2188011f00 tx=0x7f2188009f30 comp rx=0 tx=0).stop 2024-05-09T09:27:51.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.761+0000 7f219b3fd740 1 -- 172.21.15.12:0/1350243530 shutdown_connections 2024-05-09T09:27:51.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.761+0000 7f219b3fd740 1 --2- 172.21.15.12:0/1350243530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f32aa0 0x55ac02f34f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.761+0000 7f219b3fd740 1 --2- 172.21.15.12:0/1350243530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 0x55ac02f32430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.761+0000 7f219b3fd740 1 --2- 172.21.15.12:0/1350243530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f2d360 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.761+0000 7f219b3fd740 1 -- 172.21.15.12:0/1350243530 >> 172.21.15.12:0/1350243530 conn(0x55ac02f285b0 msgr2=0x55ac02f289b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f219b3fd740 1 -- 172.21.15.12:0/1350243530 shutdown_connections 2024-05-09T09:27:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f219b3fd740 1 -- 172.21.15.12:0/1350243530 wait complete. 2024-05-09T09:27:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f219b3fd740 1 -- start start 2024-05-09T09:27:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f219b3fd740 1 Processor -- start 2024-05-09T09:27:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f219b3fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f2cf40 0x55ac02f373d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f219b3fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 0x55ac02f37910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f219b3fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f32aa0 0x55ac02f3e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f219b3fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ac02f35a50 con 0x55ac02f2cf40 2024-05-09T09:27:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f219b3fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac02e4cc40 con 0x55ac02f32aa0 2024-05-09T09:27:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f219b3fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ac02f35bd0 con 0x55ac02f2d930 2024-05-09T09:27:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f219b3fd740 10 client.0 _ll_register_callbacks cb 0x55ac02ba4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f2196bb8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f2cf40 0x55ac02f373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f2196bb8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f2cf40 0x55ac02f373d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46114/0 (socket says 172.21.15.12:46114) 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f2196bb8640 1 -- 172.21.15.12:0/3116677009 learned_addr learned my addr 172.21.15.12:0/3116677009 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f2195bb6640 1 --2- 172.21.15.12:0/3116677009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 0x55ac02f37910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.762+0000 7f2196bb8640 1 --2- 172.21.15.12:0/3116677009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f32aa0 0x55ac02f3e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.763+0000 7f2196bb8640 1 -- 172.21.15.12:0/3116677009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 msgr2=0x55ac02f37910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.763+0000 7f2196bb8640 1 --2- 172.21.15.12:0/3116677009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 0x55ac02f37910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.763+0000 7f2196bb8640 1 -- 172.21.15.12:0/3116677009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f32aa0 msgr2=0x55ac02f3e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.763+0000 7f2196bb8640 1 --2- 172.21.15.12:0/3116677009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f32aa0 0x55ac02f3e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.763+0000 7f2196bb8640 1 -- 172.21.15.12:0/3116677009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac02c6f7d0 con 0x55ac02f2cf40 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.763+0000 7f2196bb8640 1 --2- 172.21.15.12:0/3116677009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f2cf40 0x55ac02f373d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2188011820 tx=0x7f2188005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2674d83c686e21bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.763+0000 7f2179ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.763+0000 7f2179ffb640 1 -- 172.21.15.12:0/3116677009 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188026030 con 0x55ac02f2cf40 2024-05-09T09:27:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.763+0000 7f2179ffb640 1 -- 172.21.15.12:0/3116677009 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f218801f440 con 0x55ac02f2cf40 2024-05-09T09:27:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.763+0000 7f219b3fd740 1 -- 172.21.15.12:0/3116677009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ac02e9f370 con 0x55ac02f2cf40 2024-05-09T09:27:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.763+0000 7f2179ffb640 1 -- 172.21.15.12:0/3116677009 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2188035050 con 0x55ac02f2cf40 2024-05-09T09:27:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.763+0000 7f219b3fd740 10 client.16488 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.763+0000 7f219b3fd740 1 -- 172.21.15.12:0/3116677009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ac02f41cc0 con 0x55ac02f2cf40 2024-05-09T09:27:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.764+0000 7f2179ffb640 1 -- 172.21.15.12:0/3116677009 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55ac02e9f370 con 0x55ac02f2cf40 2024-05-09T09:27:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.764+0000 7f2179ffb640 1 -- 172.21.15.12:0/3116677009 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f218801d020 con 0x55ac02f2cf40 2024-05-09T09:27:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.764+0000 7f219b3fd740 10 client.16488 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.764+0000 7f219b3fd740 10 client.16488 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:27:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.764+0000 7f219b3fd740 20 client.16488 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.764+0000 7f219b3fd740 1 --2- 172.21.15.12:0/3116677009 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ac02f43080 0x55ac02f454c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.764+0000 7f219b3fd740 4 client.16488 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.764+0000 7f219b3fd740 1 -- 172.21.15.12:0/3116677009 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac02e996c0 con 0x55ac02f43080 2024-05-09T09:27:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.764+0000 7f2195bb6640 1 --2- 172.21.15.12:0/3116677009 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ac02f43080 0x55ac02f454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.765+0000 7f2195bb6640 1 --2- 172.21.15.12:0/3116677009 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ac02f43080 0x55ac02f454c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2b80799f516034f server_cookie=2617ef04423acd87 in_seq=0 out_seq=0 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.765+0000 7f2179ffb640 10 client.16488 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.765+0000 7f2179ffb640 1 -- 172.21.15.12:0/3116677009 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac02e996c0 con 0x55ac02f43080 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.765+0000 7f2179ffb640 10 client.16488 handle_command_reply: tid=0 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.765+0000 7f219b3fd740 1 client.16488 shutdown 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.765+0000 7f219b3fd740 1 -- 172.21.15.12:0/3116677009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f2cf40 msgr2=0x55ac02f373d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.765+0000 7f219b3fd740 1 --2- 172.21.15.12:0/3116677009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f2cf40 0x55ac02f373d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2188011820 tx=0x7f2188005bd0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.766+0000 7f219b3fd740 1 -- 172.21.15.12:0/3116677009 shutdown_connections 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.766+0000 7f219b3fd740 1 --2- 172.21.15.12:0/3116677009 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ac02f43080 0x55ac02f454c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.766+0000 7f2179ffb640 0 client.16488 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.766+0000 7f219b3fd740 1 --2- 172.21.15.12:0/3116677009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f32aa0 0x55ac02f3e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.766+0000 7f219b3fd740 1 --2- 172.21.15.12:0/3116677009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 0x55ac02f37910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.766+0000 7f219b3fd740 1 --2- 172.21.15.12:0/3116677009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f2cf40 0x55ac02f373d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.766+0000 7f219b3fd740 1 -- 172.21.15.12:0/3116677009 >> 172.21.15.12:0/3116677009 conn(0x55ac02f285b0 msgr2=0x55ac02f277b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.766+0000 7f219b3fd740 1 -- 172.21.15.12:0/3116677009 shutdown_connections 2024-05-09T09:27:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.766+0000 7f219b3fd740 1 -- 172.21.15.12:0/3116677009 wait complete. 2024-05-09T09:27:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.766+0000 7f219b3fd740 20 client.16488 trim_cache size 0 max 16384 2024-05-09T09:27:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.782+0000 7f219b3fd740 1 Processor -- start 2024-05-09T09:27:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.782+0000 7f219b3fd740 1 -- start start 2024-05-09T09:27:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.782+0000 7f219b3fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f376f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.782+0000 7f219b3fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f2d930 0x55ac02f3e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.782+0000 7f219b3fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f32aa0 0x55ac02f26f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.782+0000 7f219b3fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f21901aa660 con 0x55ac02f2d930 2024-05-09T09:27:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.782+0000 7f219b3fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21901ac260 con 0x55ac02f2cf40 2024-05-09T09:27:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.782+0000 7f219b3fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f21901aad30 con 0x55ac02f32aa0 2024-05-09T09:27:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.782+0000 7f2196bb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.782+0000 7f2196bb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f376f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41012/0 (socket says 172.21.15.12:41012) 2024-05-09T09:27:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.782+0000 7f2196bb8640 1 -- 172.21.15.12:0/2639724789 learned_addr learned my addr 172.21.15.12:0/2639724789 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.783+0000 7f2195bb6640 1 --2- 172.21.15.12:0/2639724789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f2d930 0x55ac02f3e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.783+0000 7f2196bb8640 1 --2- 172.21.15.12:0/2639724789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f32aa0 0x55ac02f26f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.783+0000 7f2196bb8640 1 -- 172.21.15.12:0/2639724789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f32aa0 msgr2=0x55ac02f26f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.783+0000 7f2196bb8640 1 --2- 172.21.15.12:0/2639724789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f32aa0 0x55ac02f26f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.783+0000 7f2196bb8640 1 -- 172.21.15.12:0/2639724789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f2d930 msgr2=0x55ac02f3e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.783+0000 7f2196bb8640 1 --2- 172.21.15.12:0/2639724789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f2d930 0x55ac02f3e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.783+0000 7f2196bb8640 1 -- 172.21.15.12:0/2639724789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac02f19d20 con 0x55ac02f2cf40 2024-05-09T09:27:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.783+0000 7f2196bb8640 1 --2- 172.21.15.12:0/2639724789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f376f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2188006010 tx=0x7f218800e930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f49ed89dfeeb6d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.783+0000 7f217bfff640 1 -- 172.21.15.12:0/2639724789 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188026040 con 0x55ac02f2cf40 2024-05-09T09:27:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.783+0000 7f217bfff640 1 -- 172.21.15.12:0/2639724789 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2188003b80 con 0x55ac02f2cf40 2024-05-09T09:27:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.783+0000 7f217bfff640 1 -- 172.21.15.12:0/2639724789 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188011680 con 0x55ac02f2cf40 2024-05-09T09:27:51.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.784+0000 7f219b3fd740 1 -- 172.21.15.12:0/2639724789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 msgr2=0x55ac02f376f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.784+0000 7f219b3fd740 1 --2- 172.21.15.12:0/2639724789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f376f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2188006010 tx=0x7f218800e930 comp rx=0 tx=0).stop 2024-05-09T09:27:51.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 -- 172.21.15.12:0/2639724789 shutdown_connections 2024-05-09T09:27:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 --2- 172.21.15.12:0/2639724789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f32aa0 0x55ac02f26f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 --2- 172.21.15.12:0/2639724789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f2d930 0x55ac02f3e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 --2- 172.21.15.12:0/2639724789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f376f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 -- 172.21.15.12:0/2639724789 >> 172.21.15.12:0/2639724789 conn(0x55ac02f285b0 msgr2=0x55ac02f36fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 -- 172.21.15.12:0/2639724789 shutdown_connections 2024-05-09T09:27:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 -- 172.21.15.12:0/2639724789 wait complete. 2024-05-09T09:27:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 -- start start 2024-05-09T09:27:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 Processor -- start 2024-05-09T09:27:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f39570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 0x55ac02f39ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f32aa0 0x55ac02faba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f21901abb90 con 0x55ac02f32aa0 2024-05-09T09:27:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21901ab400 con 0x55ac02f2cf40 2024-05-09T09:27:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.785+0000 7f219b3fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f21901a8640 con 0x55ac02f2d930 2024-05-09T09:27:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.786+0000 7f219b3fd740 10 client.0 _ll_register_callbacks cb 0x7f21880119f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.786+0000 7f2196bb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f39570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.786+0000 7f2196bb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f39570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41016/0 (socket says 172.21.15.12:41016) 2024-05-09T09:27:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.786+0000 7f2196bb8640 1 -- 172.21.15.12:0/1403849851 learned_addr learned my addr 172.21.15.12:0/1403849851 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.786+0000 7f2196bb8640 1 --2- 172.21.15.12:0/1403849851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f32aa0 0x55ac02faba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.786+0000 7f2195bb6640 1 --2- 172.21.15.12:0/1403849851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 0x55ac02f39ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.786+0000 7f2196bb8640 1 -- 172.21.15.12:0/1403849851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 msgr2=0x55ac02f39ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.786+0000 7f2196bb8640 1 --2- 172.21.15.12:0/1403849851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 0x55ac02f39ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.786+0000 7f2196bb8640 1 -- 172.21.15.12:0/1403849851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f32aa0 msgr2=0x55ac02faba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.786+0000 7f2196bb8640 1 --2- 172.21.15.12:0/1403849851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f32aa0 0x55ac02faba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.786+0000 7f2196bb8640 1 -- 172.21.15.12:0/1403849851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac02f3a0a0 con 0x55ac02f2cf40 2024-05-09T09:27:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.786+0000 7f2196bb8640 1 --2- 172.21.15.12:0/1403849851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f39570 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f218800e8b0 tx=0x7f218800e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5b07b88d43d9829 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.786+0000 7f2179ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:27:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.787+0000 7f2179ffb640 1 -- 172.21.15.12:0/1403849851 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218800e5d0 con 0x55ac02f2cf40 2024-05-09T09:27:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.787+0000 7f2179ffb640 1 -- 172.21.15.12:0/1403849851 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2188004310 con 0x55ac02f2cf40 2024-05-09T09:27:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.787+0000 7f219b3fd740 1 -- 172.21.15.12:0/1403849851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2188003b80 con 0x55ac02f2cf40 2024-05-09T09:27:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.787+0000 7f2179ffb640 1 -- 172.21.15.12:0/1403849851 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f218800eaf0 con 0x55ac02f2cf40 2024-05-09T09:27:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.787+0000 7f219b3fd740 10 client.25943 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.787+0000 7f219b3fd740 1 -- 172.21.15.12:0/1403849851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ac02facfb0 con 0x55ac02f2cf40 2024-05-09T09:27:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.787+0000 7f2179ffb640 1 -- 172.21.15.12:0/1403849851 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2188003b80 con 0x55ac02f2cf40 2024-05-09T09:27:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.787+0000 7f2179ffb640 1 -- 172.21.15.12:0/1403849851 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f218801d070 con 0x55ac02f2cf40 2024-05-09T09:27:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.787+0000 7f219b3fd740 10 client.25943 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.787+0000 7f219b3fd740 10 client.25943 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:27:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.788+0000 7f219b3fd740 20 client.25943 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.788+0000 7f219b3fd740 1 --2- 172.21.15.12:0/1403849851 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ac02f43080 0x55ac02faf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.788+0000 7f219b3fd740 4 client.25943 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:27:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.788+0000 7f219b3fd740 1 -- 172.21.15.12:0/1403849851 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac02e16820 con 0x55ac02f43080 2024-05-09T09:27:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.788+0000 7f2195bb6640 1 --2- 172.21.15.12:0/1403849851 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ac02f43080 0x55ac02faf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.789+0000 7f2195bb6640 1 --2- 172.21.15.12:0/1403849851 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ac02f43080 0x55ac02faf8e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c40ee131c98cc667 server_cookie=f83c9421c1540898 in_seq=0 out_seq=0 2024-05-09T09:27:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.789+0000 7f2179ffb640 10 client.25943 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:27:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f2179ffb640 1 -- 172.21.15.12:0/1403849851 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac02e16820 con 0x55ac02f43080 2024-05-09T09:27:51.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f2179ffb640 10 client.25943 handle_command_reply: tid=0 2024-05-09T09:27:51.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f219b3fd740 1 client.25943 shutdown 2024-05-09T09:27:51.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f219b3fd740 1 -- 172.21.15.12:0/1403849851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 msgr2=0x55ac02f39570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f219b3fd740 1 --2- 172.21.15.12:0/1403849851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f39570 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f218800e8b0 tx=0x7f218800e3c0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f219b3fd740 1 -- 172.21.15.12:0/1403849851 shutdown_connections 2024-05-09T09:27:51.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f219b3fd740 1 --2- 172.21.15.12:0/1403849851 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55ac02f43080 0x55ac02faf8e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f219b3fd740 1 --2- 172.21.15.12:0/1403849851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac02f32aa0 0x55ac02faba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f219b3fd740 1 --2- 172.21.15.12:0/1403849851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac02f2d930 0x55ac02f39ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f2179ffb640 0 client.25943 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:27:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f219b3fd740 1 --2- 172.21.15.12:0/1403849851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac02f2cf40 0x55ac02f39570 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f219b3fd740 1 -- 172.21.15.12:0/1403849851 >> 172.21.15.12:0/1403849851 conn(0x55ac02f285b0 msgr2=0x55ac02f2b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f219b3fd740 1 -- 172.21.15.12:0/1403849851 shutdown_connections 2024-05-09T09:27:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f219b3fd740 1 -- 172.21.15.12:0/1403849851 wait complete. 2024-05-09T09:27:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.790+0000 7f219b3fd740 20 client.25943 trim_cache size 0 max 16384 2024-05-09T09:27:51.797 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:27:51.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.791+0000 7f219861b640 1 -- 172.21.15.12:0/1165389069 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f217c0a2b90 msgr2=0x7f217c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.791+0000 7f219861b640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f217c0a2b90 0x7f217c0a5050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f218c032b60 tx=0x7f218c03b460 comp rx=0 tx=0).stop 2024-05-09T09:27:51.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.791+0000 7f219861b640 1 -- 172.21.15.12:0/1165389069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21901023b0 msgr2=0x7f2190197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.791+0000 7f219861b640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21901023b0 0x7f2190197c20 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f218c009230 tx=0x7f218c0042f0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.791+0000 7f219861b640 1 -- 172.21.15.12:0/1165389069 shutdown_connections 2024-05-09T09:27:51.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.791+0000 7f219861b640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f217c0a2b90 0x7f217c0a5050 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.791+0000 7f219861b640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2190107610 0x7f219019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.791+0000 7f219861b640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2190104ce0 0x7f2190198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.791+0000 7f219861b640 1 --2- 172.21.15.12:0/1165389069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21901023b0 0x7f2190197c20 secure :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f218c009230 tx=0x7f218c0042f0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.791+0000 7f219861b640 1 -- 172.21.15.12:0/1165389069 >> 172.21.15.12:0/1165389069 conn(0x7f21900fbce0 msgr2=0x7f2190102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.791+0000 7f219861b640 1 -- 172.21.15.12:0/1165389069 shutdown_connections 2024-05-09T09:27:51.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.791+0000 7f219861b640 1 -- 172.21.15.12:0/1165389069 wait complete. 2024-05-09T09:27:51.809 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:27:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.939+0000 7f56d2177640 1 -- 172.21.15.12:0/1982062428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56cc1075f0 msgr2=0x7f56cc1099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.939+0000 7f56d2177640 1 --2- 172.21.15.12:0/1982062428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56cc1075f0 0x7f56cc1099e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009ca0 tx=0x7f56c402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.939+0000 7f56d2177640 1 -- 172.21.15.12:0/1982062428 shutdown_connections 2024-05-09T09:27:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.939+0000 7f56d2177640 1 --2- 172.21.15.12:0/1982062428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56cc1075f0 0x7f56cc1099e0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.939+0000 7f56d2177640 1 --2- 172.21.15.12:0/1982062428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56cc104cc0 0x7f56cc1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.939+0000 7f56d2177640 1 --2- 172.21.15.12:0/1982062428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc102390 0x7f56cc104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.939+0000 7f56d2177640 1 -- 172.21.15.12:0/1982062428 >> 172.21.15.12:0/1982062428 conn(0x7f56cc0fbce0 msgr2=0x7f56cc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.939+0000 7f56d2177640 1 -- 172.21.15.12:0/1982062428 shutdown_connections 2024-05-09T09:27:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.939+0000 7f56d2177640 1 -- 172.21.15.12:0/1982062428 wait complete. 2024-05-09T09:27:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.939+0000 7f56d2177640 1 Processor -- start 2024-05-09T09:27:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56d2177640 1 -- start start 2024-05-09T09:27:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56d2177640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc102390 0x7f56cc197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56d2177640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56cc104cc0 0x7f56cc1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56d2177640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56cc1075f0 0x7f56cc19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56d2177640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc10d4f0 con 0x7f56cc104cc0 2024-05-09T09:27:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56d2177640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc10d370 con 0x7f56cc102390 2024-05-09T09:27:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56d2177640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc10d670 con 0x7f56cc1075f0 2024-05-09T09:27:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56caffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56cc104cc0 0x7f56cc1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc102390 0x7f56cc197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56caffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56cc104cc0 0x7f56cc1980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46158/0 (socket says 172.21.15.12:46158) 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc102390 0x7f56cc197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41034/0 (socket says 172.21.15.12:41034) 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56caffd640 1 -- 172.21.15.12:0/542870735 learned_addr learned my addr 172.21.15.12:0/542870735 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56cbfff640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56cc1075f0 0x7f56cc19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56cb7fe640 1 -- 172.21.15.12:0/542870735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56cc1075f0 msgr2=0x7f56cc19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56cc1075f0 0x7f56cc19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56cb7fe640 1 -- 172.21.15.12:0/542870735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56cc104cc0 msgr2=0x7f56cc1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56cc104cc0 0x7f56cc1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56cb7fe640 1 -- 172.21.15.12:0/542870735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc19f7c0 con 0x7f56cc102390 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56cbfff640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56cc1075f0 0x7f56cc19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.940+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc102390 0x7f56cc197ba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00d6d0 tx=0x7f56bc00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.941+0000 7f56c8ff9640 1 -- 172.21.15.12:0/542870735 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc017070 con 0x7f56cc102390 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.941+0000 7f56d2177640 1 -- 172.21.15.12:0/542870735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56cc19fab0 con 0x7f56cc102390 2024-05-09T09:27:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.941+0000 7f56c8ff9640 1 -- 172.21.15.12:0/542870735 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f56bc009820 con 0x7f56cc102390 2024-05-09T09:27:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.941+0000 7f56c8ff9640 1 -- 172.21.15.12:0/542870735 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc004cd0 con 0x7f56cc102390 2024-05-09T09:27:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.941+0000 7f56d2177640 1 -- 172.21.15.12:0/542870735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56cc1a0000 con 0x7f56cc102390 2024-05-09T09:27:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.944+0000 7f56c8ff9640 1 -- 172.21.15.12:0/542870735 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f56bc0099d0 con 0x7f56cc102390 2024-05-09T09:27:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.944+0000 7f56d4f67740 1 Processor -- start 2024-05-09T09:27:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.944+0000 7f56d4f67740 1 -- start start 2024-05-09T09:27:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.944+0000 7f56d4f67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 0x558db124c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.944+0000 7f56d4f67740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124c930 0x558db1251430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.944+0000 7f56d4f67740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db1251aa0 0x558db1253f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.944+0000 7f56d4f67740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558db11b0c40 con 0x558db124c930 2024-05-09T09:27:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.944+0000 7f56d4f67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558db1158580 con 0x558db124bf40 2024-05-09T09:27:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.944+0000 7f56d4f67740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558db11acc00 con 0x558db1251aa0 2024-05-09T09:27:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124c930 0x558db1251430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 0x558db124c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56caffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db1251aa0 0x558db1253f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 0x558db124c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41042/0 (socket says 172.21.15.12:41042) 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124c930 0x558db1251430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46160/0 (socket says 172.21.15.12:46160) 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56cbfff640 1 -- 172.21.15.12:0/3555042073 learned_addr learned my addr 172.21.15.12:0/3555042073 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56c8ff9640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f56b40a2b90 0x7f56b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56cbfff640 1 -- 172.21.15.12:0/3555042073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db1251aa0 msgr2=0x558db1253f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56cbfff640 1 --2- 172.21.15.12:0/3555042073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db1251aa0 0x558db1253f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56cbfff640 1 -- 172.21.15.12:0/3555042073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124c930 msgr2=0x558db1251430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56cbfff640 1 --2- 172.21.15.12:0/3555042073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124c930 0x558db1251430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56cbfff640 1 -- 172.21.15.12:0/3555042073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558db0ec43f0 con 0x558db124bf40 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/3555042073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124c930 0x558db1251430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56cbfff640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f56b40a2b90 0x7f56b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56c8ff9640 1 -- 172.21.15.12:0/542870735 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f56bc0cee90 con 0x7f56cc102390 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.945+0000 7f56cbfff640 1 --2- 172.21.15.12:0/3555042073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 0x558db124c360 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f56c4035000 tx=0x7f56c4005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c82666c081b192f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.946+0000 7f56b1ffb640 1 -- 172.21.15.12:0/3555042073 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4043040 con 0x558db124bf40 2024-05-09T09:27:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.946+0000 7f56b1ffb640 1 -- 172.21.15.12:0/3555042073 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f56c4036040 con 0x558db124bf40 2024-05-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.946+0000 7f56b1ffb640 1 -- 172.21.15.12:0/3555042073 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c402fa20 con 0x558db124bf40 2024-05-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.946+0000 7f56cbfff640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f56b40a2b90 0x7f56b40a5050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f56c404e000 tx=0x7f56c404f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.946+0000 7f56d4f67740 1 -- 172.21.15.12:0/3555042073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 msgr2=0x558db124c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.946+0000 7f56d4f67740 1 --2- 172.21.15.12:0/3555042073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 0x558db124c360 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f56c4035000 tx=0x7f56c4005fa0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.946+0000 7f56d4f67740 1 -- 172.21.15.12:0/3555042073 shutdown_connections 2024-05-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.947+0000 7f56d4f67740 1 --2- 172.21.15.12:0/3555042073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db1251aa0 0x558db1253f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.947+0000 7f56d4f67740 1 --2- 172.21.15.12:0/3555042073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124c930 0x558db1251430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.947+0000 7f56d4f67740 1 --2- 172.21.15.12:0/3555042073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 0x558db124c360 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.947+0000 7f56d4f67740 1 -- 172.21.15.12:0/3555042073 >> 172.21.15.12:0/3555042073 conn(0x558db12475b0 msgr2=0x558db12479b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.947+0000 7f56d4f67740 1 -- 172.21.15.12:0/3555042073 shutdown_connections 2024-05-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.947+0000 7f56d4f67740 1 -- 172.21.15.12:0/3555042073 wait complete. 2024-05-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.947+0000 7f56d4f67740 1 -- start start 2024-05-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.947+0000 7f56d4f67740 1 Processor -- start 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56d4f67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 0x558db1256340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56d4f67740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db124c930 0x558db1256880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56d4f67740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db1251aa0 0x558db125d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56d4f67740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558db1254a50 con 0x558db1251aa0 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56d4f67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558db116bc40 con 0x558db124bf40 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56d4f67740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558db1254bd0 con 0x558db124c930 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56d4f67740 10 client.0 _ll_register_callbacks cb 0x558db0ec3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db1251aa0 0x558db125d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 0x558db1256340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56caffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db124c930 0x558db1256880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db1251aa0 0x558db125d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46168/0 (socket says 172.21.15.12:46168) 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56cbfff640 1 -- 172.21.15.12:0/2069406615 learned_addr learned my addr 172.21.15.12:0/2069406615 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56caffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db124c930 0x558db1256880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:33402/0 (socket says 172.21.15.12:33402) 2024-05-09T09:27:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 0x558db1256340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41056/0 (socket says 172.21.15.12:41056) 2024-05-09T09:27:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56cbfff640 1 -- 172.21.15.12:0/2069406615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db124c930 msgr2=0x558db1256880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56cbfff640 1 --2- 172.21.15.12:0/2069406615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db124c930 0x558db1256880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56cbfff640 1 -- 172.21.15.12:0/2069406615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 msgr2=0x558db1256340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56cbfff640 1 --2- 172.21.15.12:0/2069406615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 0x558db1256340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56cbfff640 1 -- 172.21.15.12:0/2069406615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558db0f8e7d0 con 0x558db1251aa0 2024-05-09T09:27:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56caffd640 1 --2- 172.21.15.12:0/2069406615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db124c930 0x558db1256880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/2069406615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 0x558db1256340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56cbfff640 1 --2- 172.21.15.12:0/2069406615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db1251aa0 0x558db125d8c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f56c4006010 tx=0x7f56c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf425ad0bdaa04ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.948+0000 7f56aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56aaffd640 1 -- 172.21.15.12:0/2069406615 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c405a020 con 0x558db1251aa0 2024-05-09T09:27:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56aaffd640 1 -- 172.21.15.12:0/2069406615 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f56c4004040 con 0x558db1251aa0 2024-05-09T09:27:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56d4f67740 1 -- 172.21.15.12:0/2069406615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56c4036040 con 0x558db1251aa0 2024-05-09T09:27:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56aaffd640 1 -- 172.21.15.12:0/2069406615 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f56c4065050 con 0x558db1251aa0 2024-05-09T09:27:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56d4f67740 10 client.16506 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56d4f67740 1 -- 172.21.15.12:0/2069406615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558db1254a50 con 0x558db1251aa0 2024-05-09T09:27:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56aaffd640 1 -- 172.21.15.12:0/2069406615 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f56c4036040 con 0x558db1251aa0 2024-05-09T09:27:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56aaffd640 1 -- 172.21.15.12:0/2069406615 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f56c4030a40 con 0x558db1251aa0 2024-05-09T09:27:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56d4f67740 10 client.16506 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56d4f67740 10 client.16506 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:27:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56d4f67740 20 client.16506 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56d4f67740 1 --2- 172.21.15.12:0/2069406615 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x558db1261f30 0x558db1264340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56d4f67740 4 client.16506 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.949+0000 7f56d4f67740 1 -- 172.21.15.12:0/2069406615 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558db11be370 con 0x558db1261f30 2024-05-09T09:27:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.950+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/2069406615 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x558db1261f30 0x558db1264340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.950+0000 7f56aaffd640 10 client.16506 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:27:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.950+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/2069406615 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x558db1261f30 0x558db1264340 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d72327a12d8bdc6 server_cookie=5c6cf7bc7a467764 in_seq=0 out_seq=0 2024-05-09T09:27:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56aaffd640 1 -- 172.21.15.12:0/2069406615 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558db11be370 con 0x558db1261f30 2024-05-09T09:27:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56aaffd640 10 client.16506 handle_command_reply: tid=0 2024-05-09T09:27:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56d4f67740 1 client.16506 shutdown 2024-05-09T09:27:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56d4f67740 1 -- 172.21.15.12:0/2069406615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db1251aa0 msgr2=0x558db125d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56d4f67740 1 --2- 172.21.15.12:0/2069406615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db1251aa0 0x558db125d8c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f56c4006010 tx=0x7f56c4004a40 comp rx=0 tx=0).stop 2024-05-09T09:27:51.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56d4f67740 1 -- 172.21.15.12:0/2069406615 shutdown_connections 2024-05-09T09:27:51.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56d4f67740 1 --2- 172.21.15.12:0/2069406615 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x558db1261f30 0x558db1264340 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56d4f67740 1 --2- 172.21.15.12:0/2069406615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db1251aa0 0x558db125d8c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56d4f67740 1 --2- 172.21.15.12:0/2069406615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db124c930 0x558db1256880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56d4f67740 1 --2- 172.21.15.12:0/2069406615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124bf40 0x558db1256340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56d4f67740 1 -- 172.21.15.12:0/2069406615 >> 172.21.15.12:0/2069406615 conn(0x558db12475b0 msgr2=0x558db12467b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56aaffd640 0 client.16506 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:27:51.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56d4f67740 1 -- 172.21.15.12:0/2069406615 shutdown_connections 2024-05-09T09:27:51.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56d4f67740 1 -- 172.21.15.12:0/2069406615 wait complete. 2024-05-09T09:27:51.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.951+0000 7f56d4f67740 20 client.16506 trim_cache size 0 max 16384 2024-05-09T09:27:51.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.966+0000 7f56d4f67740 1 Processor -- start 2024-05-09T09:27:51.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.966+0000 7f56d4f67740 1 -- start start 2024-05-09T09:27:51.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.966+0000 7f56d4f67740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 0x558db1255960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.966+0000 7f56d4f67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124c930 0x558db125d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.966+0000 7f56d4f67740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db1251aa0 0x558db124dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.966+0000 7f56d4f67740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc1ac7b0 con 0x558db124bf40 2024-05-09T09:27:51.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.966+0000 7f56d4f67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc1ae3b0 con 0x558db124c930 2024-05-09T09:27:51.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.966+0000 7f56d4f67740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc1ace80 con 0x558db1251aa0 2024-05-09T09:27:51.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.966+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 0x558db1255960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.966+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 0x558db1255960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46176/0 (socket says 172.21.15.12:46176) 2024-05-09T09:27:51.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.966+0000 7f56cb7fe640 1 -- 172.21.15.12:0/1345715916 learned_addr learned my addr 172.21.15.12:0/1345715916 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.967+0000 7f56caffd640 1 --2- 172.21.15.12:0/1345715916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124c930 0x558db125d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.967+0000 7f56cbfff640 1 --2- 172.21.15.12:0/1345715916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db1251aa0 0x558db124dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.967+0000 7f56cb7fe640 1 -- 172.21.15.12:0/1345715916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db1251aa0 msgr2=0x558db124dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.967+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/1345715916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db1251aa0 0x558db124dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.967+0000 7f56cb7fe640 1 -- 172.21.15.12:0/1345715916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124c930 msgr2=0x558db125d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.967+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/1345715916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124c930 0x558db125d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.967+0000 7f56cb7fe640 1 -- 172.21.15.12:0/1345715916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558db1238d20 con 0x558db124bf40 2024-05-09T09:27:51.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.967+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/1345715916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 0x558db1255960 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0cc930 tx=0x7f56bc0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5174db425834f28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.967+0000 7f56b17fa640 1 -- 172.21.15.12:0/1345715916 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc020040 con 0x558db124bf40 2024-05-09T09:27:51.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.967+0000 7f56b17fa640 1 -- 172.21.15.12:0/1345715916 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f56bc0b42e0 con 0x558db124bf40 2024-05-09T09:27:51.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.967+0000 7f56b17fa640 1 -- 172.21.15.12:0/1345715916 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc097a40 con 0x558db124bf40 2024-05-09T09:27:51.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.968+0000 7f56d4f67740 1 -- 172.21.15.12:0/1345715916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 msgr2=0x558db1255960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.968+0000 7f56d4f67740 1 --2- 172.21.15.12:0/1345715916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 0x558db1255960 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0cc930 tx=0x7f56bc0bb040 comp rx=0 tx=0).stop 2024-05-09T09:27:51.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.968+0000 7f56d4f67740 1 -- 172.21.15.12:0/1345715916 shutdown_connections 2024-05-09T09:27:51.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.968+0000 7f56d4f67740 1 --2- 172.21.15.12:0/1345715916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db1251aa0 0x558db124dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.968+0000 7f56d4f67740 1 --2- 172.21.15.12:0/1345715916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db124c930 0x558db125d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.968+0000 7f56d4f67740 1 --2- 172.21.15.12:0/1345715916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 0x558db1255960 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.968+0000 7f56d4f67740 1 -- 172.21.15.12:0/1345715916 >> 172.21.15.12:0/1345715916 conn(0x558db12475b0 msgr2=0x558db1256340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.969+0000 7f56d4f67740 1 -- 172.21.15.12:0/1345715916 shutdown_connections 2024-05-09T09:27:51.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.969+0000 7f56d4f67740 1 -- 172.21.15.12:0/1345715916 wait complete. 2024-05-09T09:27:51.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.969+0000 7f56d4f67740 1 -- start start 2024-05-09T09:27:51.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.969+0000 7f56d4f67740 1 Processor -- start 2024-05-09T09:27:51.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.969+0000 7f56d4f67740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 0x558db125a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.969+0000 7f56d4f67740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db124c930 0x558db125abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.969+0000 7f56d4f67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db1251aa0 0x558db12c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.969+0000 7f56d4f67740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc1adce0 con 0x558db124bf40 2024-05-09T09:27:51.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.969+0000 7f56d4f67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc1ad550 con 0x558db1251aa0 2024-05-09T09:27:51.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.969+0000 7f56d4f67740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc1aa790 con 0x558db124c930 2024-05-09T09:27:51.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.969+0000 7f56d4f67740 10 client.0 _ll_register_callbacks cb 0x7f56bc0976f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:51.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 0x558db125a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 0x558db125a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46182/0 (socket says 172.21.15.12:46182) 2024-05-09T09:27:51.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56cb7fe640 1 -- 172.21.15.12:0/54435077 learned_addr learned my addr 172.21.15.12:0/54435077 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:51.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db1251aa0 0x558db12c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56caffd640 1 --2- 172.21.15.12:0/54435077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db124c930 0x558db125abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56cb7fe640 1 -- 172.21.15.12:0/54435077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db124c930 msgr2=0x558db125abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/54435077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db124c930 0x558db125abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56cb7fe640 1 -- 172.21.15.12:0/54435077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db1251aa0 msgr2=0x558db12c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:51.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/54435077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db1251aa0 0x558db12c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56cb7fe640 1 -- 172.21.15.12:0/54435077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558db125b270 con 0x558db124bf40 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56cb7fe640 1 --2- 172.21.15.12:0/54435077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 0x558db125a690 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0cc610 tx=0x7f56bc009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e70d913ebdd126 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56aaffd640 1 -- 172.21.15.12:0/54435077 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc020050 con 0x558db124bf40 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56aaffd640 1 -- 172.21.15.12:0/54435077 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f56bc09aa20 con 0x558db124bf40 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56d4f67740 1 -- 172.21.15.12:0/54435077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56bc0b42e0 con 0x558db124bf40 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56aaffd640 1 -- 172.21.15.12:0/54435077 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f56bc09f6d0 con 0x558db124bf40 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56d4f67740 10 client.16512 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.970+0000 7f56d4f67740 1 -- 172.21.15.12:0/54435077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558db12c7ae0 con 0x558db124bf40 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.971+0000 7f56aaffd640 1 -- 172.21.15.12:0/54435077 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f56bc0b42e0 con 0x558db124bf40 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.971+0000 7f56aaffd640 1 -- 172.21.15.12:0/54435077 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f56bc09f220 con 0x558db124bf40 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.971+0000 7f56d4f67740 10 client.16512 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.971+0000 7f56d4f67740 10 client.16512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.971+0000 7f56d4f67740 20 client.16512 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:51.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.971+0000 7f56d4f67740 1 --2- 172.21.15.12:0/54435077 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x558db12c8330 0x558db12ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.971+0000 7f56d4f67740 4 client.16512 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.971+0000 7f56d4f67740 1 -- 172.21.15.12:0/54435077 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558db11b86c0 con 0x558db12c8330 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.971+0000 7f56caffd640 1 --2- 172.21.15.12:0/54435077 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x558db12c8330 0x558db12ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.971+0000 7f56aaffd640 10 client.16512 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.971+0000 7f56caffd640 1 --2- 172.21.15.12:0/54435077 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x558db12c8330 0x558db12ca750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f772455f008d460 server_cookie=1dd5d8e20dccf93b in_seq=0 out_seq=0 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56aaffd640 1 -- 172.21.15.12:0/54435077 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558db11b86c0 con 0x558db12c8330 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56aaffd640 10 client.16512 handle_command_reply: tid=0 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56d4f67740 1 client.16512 shutdown 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56d4f67740 1 -- 172.21.15.12:0/54435077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 msgr2=0x558db125a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56d4f67740 1 --2- 172.21.15.12:0/54435077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 0x558db125a690 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0cc610 tx=0x7f56bc009e70 comp rx=0 tx=0).stop 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56d4f67740 1 -- 172.21.15.12:0/54435077 shutdown_connections 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56d4f67740 1 --2- 172.21.15.12:0/54435077 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x558db12c8330 0x558db12ca750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56d4f67740 1 --2- 172.21.15.12:0/54435077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558db1251aa0 0x558db12c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56d4f67740 1 --2- 172.21.15.12:0/54435077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558db124c930 0x558db125abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56d4f67740 1 --2- 172.21.15.12:0/54435077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558db124bf40 0x558db125a690 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56d4f67740 1 -- 172.21.15.12:0/54435077 >> 172.21.15.12:0/54435077 conn(0x558db12475b0 msgr2=0x558db1256340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56aaffd640 0 client.16512 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:27:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56d4f67740 1 -- 172.21.15.12:0/54435077 shutdown_connections 2024-05-09T09:27:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56d4f67740 1 -- 172.21.15.12:0/54435077 wait complete. 2024-05-09T09:27:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.972+0000 7f56d4f67740 20 client.16512 trim_cache size 0 max 16384 2024-05-09T09:27:51.978 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:27:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.973+0000 7f56d2177640 1 -- 172.21.15.12:0/542870735 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f56b40a2b90 msgr2=0x7f56b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.973+0000 7f56d2177640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f56b40a2b90 0x7f56b40a5050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f56c404e000 tx=0x7f56c404f000 comp rx=0 tx=0).stop 2024-05-09T09:27:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.973+0000 7f56d2177640 1 -- 172.21.15.12:0/542870735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc102390 msgr2=0x7f56cc197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.973+0000 7f56d2177640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc102390 0x7f56cc197ba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00d6d0 tx=0x7f56bc00db90 comp rx=0 tx=0).stop 2024-05-09T09:27:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.973+0000 7f56d2177640 1 -- 172.21.15.12:0/542870735 shutdown_connections 2024-05-09T09:27:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.973+0000 7f56d2177640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f56b40a2b90 0x7f56b40a5050 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.973+0000 7f56d2177640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56cc1075f0 0x7f56cc19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.973+0000 7f56d2177640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56cc104cc0 0x7f56cc1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.973+0000 7f56d2177640 1 --2- 172.21.15.12:0/542870735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc102390 0x7f56cc197ba0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:51.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.973+0000 7f56d2177640 1 -- 172.21.15.12:0/542870735 >> 172.21.15.12:0/542870735 conn(0x7f56cc0fbce0 msgr2=0x7f56cc102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:51.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.973+0000 7f56d2177640 1 -- 172.21.15.12:0/542870735 shutdown_connections 2024-05-09T09:27:51.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:51.973+0000 7f56d2177640 1 -- 172.21.15.12:0/542870735 wait complete. 2024-05-09T09:27:51.984 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:27:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.103+0000 7f1fa1b4f640 1 -- 172.21.15.12:0/1050974778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f9c104c80 msgr2=0x7f1f9c107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.103+0000 7f1fa1b4f640 1 --2- 172.21.15.12:0/1050974778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f9c104c80 0x7f1f9c107070 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009900 tx=0x7f1f8402a410 comp rx=0 tx=0).stop 2024-05-09T09:27:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.104+0000 7f1fa1b4f640 1 -- 172.21.15.12:0/1050974778 shutdown_connections 2024-05-09T09:27:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.104+0000 7f1fa1b4f640 1 --2- 172.21.15.12:0/1050974778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f9c1075b0 0x7f1f9c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.104+0000 7f1fa1b4f640 1 --2- 172.21.15.12:0/1050974778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f9c104c80 0x7f1f9c107070 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.104+0000 7f1fa1b4f640 1 --2- 172.21.15.12:0/1050974778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c102350 0x7f1f9c104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.104+0000 7f1fa1b4f640 1 -- 172.21.15.12:0/1050974778 >> 172.21.15.12:0/1050974778 conn(0x7f1f9c0fbce0 msgr2=0x7f1f9c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.104+0000 7f1fa1b4f640 1 -- 172.21.15.12:0/1050974778 shutdown_connections 2024-05-09T09:27:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.104+0000 7f1fa1b4f640 1 -- 172.21.15.12:0/1050974778 wait complete. 2024-05-09T09:27:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.104+0000 7f1fa1b4f640 1 Processor -- start 2024-05-09T09:27:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.104+0000 7f1fa1b4f640 1 -- start start 2024-05-09T09:27:52.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1fa1b4f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f9c102350 0x7f1f9c199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1fa1b4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c104c80 0x7f1f9c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1fa1b4f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f9c1075b0 0x7f1f9c1a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1fa1b4f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10d4d0 con 0x7f1f9c1075b0 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1fa1b4f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10d350 con 0x7f1f9c104c80 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1fa1b4f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10d650 con 0x7f1f9c102350 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f9c1075b0 0x7f1f9c1a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1f9b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f9c102350 0x7f1f9c199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1f9b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f9c102350 0x7f1f9c199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:33436/0 (socket says 172.21.15.12:33436) 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1f9b7fe640 1 -- 172.21.15.12:0/3087834472 learned_addr learned my addr 172.21.15.12:0/3087834472 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1f92ffd640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c104c80 0x7f1f9c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1f9b7fe640 1 -- 172.21.15.12:0/3087834472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c104c80 msgr2=0x7f1f9c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1f9b7fe640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c104c80 0x7f1f9c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1f9b7fe640 1 -- 172.21.15.12:0/3087834472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f9c1075b0 msgr2=0x7f1f9c1a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1f9b7fe640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f9c1075b0 0x7f1f9c1a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1f9b7fe640 1 -- 172.21.15.12:0/3087834472 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c1a1a10 con 0x7f1f9c102350 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f9c1075b0 0x7f1f9c1a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1f9b7fe640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f9c102350 0x7f1f9c199df0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00ea60 tx=0x7f1f8c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:52.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.105+0000 7f1f997fa640 1 -- 172.21.15.12:0/3087834472 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c00cde0 con 0x7f1f9c102350 2024-05-09T09:27:52.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.106+0000 7f1fa1b4f640 1 -- 172.21.15.12:0/3087834472 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c1a1d00 con 0x7f1f9c102350 2024-05-09T09:27:52.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.106+0000 7f1f997fa640 1 -- 172.21.15.12:0/3087834472 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1f8c0035f0 con 0x7f1f9c102350 2024-05-09T09:27:52.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.106+0000 7f1f997fa640 1 -- 172.21.15.12:0/3087834472 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c010690 con 0x7f1f9c102350 2024-05-09T09:27:52.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.106+0000 7f1fa1b4f640 1 -- 172.21.15.12:0/3087834472 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c1a21c0 con 0x7f1f9c102350 2024-05-09T09:27:52.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.108+0000 7f1f997fa640 1 -- 172.21.15.12:0/3087834472 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1f8c0037a0 con 0x7f1f9c102350 2024-05-09T09:27:52.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.108+0000 7f1fa485a740 1 Processor -- start 2024-05-09T09:27:52.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.108+0000 7f1fa485a740 1 -- start start 2024-05-09T09:27:52.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1fa485a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 0x55c15ead8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1fa485a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead8930 0x55c15eadd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1fa485a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 0x55c15eadff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1fa485a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c15ea3cc40 con 0x55c15ead7f40 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1fa485a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c15e9e4580 con 0x55c15ead8930 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1fa485a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c15ea38c00 con 0x55c15eaddaa0 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 0x55c15ead8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1f9b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead8930 0x55c15eadd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 0x55c15ead8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46216/0 (socket says 172.21.15.12:46216) 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1f9bfff640 1 -- 172.21.15.12:0/2470078031 learned_addr learned my addr 172.21.15.12:0/2470078031 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1f92ffd640 1 --2- 172.21.15.12:0/2470078031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 0x55c15eadff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1f9bfff640 1 -- 172.21.15.12:0/2470078031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 msgr2=0x55c15eadff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/2470078031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 0x55c15eadff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1f9bfff640 1 -- 172.21.15.12:0/2470078031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead8930 msgr2=0x55c15eadd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/2470078031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead8930 0x55c15eadd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1f9bfff640 1 -- 172.21.15.12:0/2470078031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c15e7503f0 con 0x55c15ead7f40 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1f997fa640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1f880a2b90 0x7f1f880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/2470078031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 0x55c15ead8360 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1f940117e0 tx=0x7f1f94011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea57b0b89d02a57 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.109+0000 7f1f9b7fe640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1f880a2b90 0x7f1f880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.110+0000 7f1f997fa640 1 -- 172.21.15.12:0/3087834472 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f1f8c0cf670 con 0x7f1f9c102350 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.110+0000 7f1f91ffb640 1 -- 172.21.15.12:0/2470078031 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94011310 con 0x55c15ead7f40 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.110+0000 7f1f91ffb640 1 -- 172.21.15.12:0/2470078031 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1f94003e40 con 0x55c15ead7f40 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.110+0000 7f1f91ffb640 1 -- 172.21.15.12:0/2470078031 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94004580 con 0x55c15ead7f40 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.110+0000 7f1fa485a740 1 -- 172.21.15.12:0/2470078031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 msgr2=0x55c15ead8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.110+0000 7f1fa485a740 1 --2- 172.21.15.12:0/2470078031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 0x55c15ead8360 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1f940117e0 tx=0x7f1f94011ca0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.110+0000 7f1f9b7fe640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1f880a2b90 0x7f1f880a5050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0cd110 tx=0x7f1f8c0d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.110+0000 7f1fa485a740 1 -- 172.21.15.12:0/2470078031 shutdown_connections 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.110+0000 7f1fa485a740 1 --2- 172.21.15.12:0/2470078031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 0x55c15eadff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.110+0000 7f1fa485a740 1 --2- 172.21.15.12:0/2470078031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead8930 0x55c15eadd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.110+0000 7f1fa485a740 1 --2- 172.21.15.12:0/2470078031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 0x55c15ead8360 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.110+0000 7f1fa485a740 1 -- 172.21.15.12:0/2470078031 >> 172.21.15.12:0/2470078031 conn(0x55c15ead35b0 msgr2=0x55c15ead39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.111+0000 7f1fa485a740 1 -- 172.21.15.12:0/2470078031 shutdown_connections 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.111+0000 7f1fa485a740 1 -- 172.21.15.12:0/2470078031 wait complete. 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.111+0000 7f1fa485a740 1 -- start start 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.111+0000 7f1fa485a740 1 Processor -- start 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.111+0000 7f1fa485a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead7f40 0x55c15eae2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.111+0000 7f1fa485a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15ead8930 0x55c15eae28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.111+0000 7f1fa485a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15eaddaa0 0x55c15eae98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.111+0000 7f1fa485a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c15eae0a50 con 0x55c15eaddaa0 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.111+0000 7f1fa485a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c15e9f7c40 con 0x55c15ead7f40 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.111+0000 7f1fa485a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c15eae0bd0 con 0x55c15ead8930 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.111+0000 7f1fa485a740 10 client.0 _ll_register_callbacks cb 0x55c15e74fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15eaddaa0 0x55c15eae98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f92ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15ead8930 0x55c15eae28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15eaddaa0 0x55c15eae98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46222/0 (socket says 172.21.15.12:46222) 2024-05-09T09:27:52.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f92ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15ead8930 0x55c15eae28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:33446/0 (socket says 172.21.15.12:33446) 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f9bfff640 1 -- 172.21.15.12:0/4138193128 learned_addr learned my addr 172.21.15.12:0/4138193128 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/4138193128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead7f40 0x55c15eae2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f9bfff640 1 -- 172.21.15.12:0/4138193128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15ead8930 msgr2=0x55c15eae28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/4138193128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15ead8930 0x55c15eae28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f9bfff640 1 -- 172.21.15.12:0/4138193128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead7f40 msgr2=0x55c15eae2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/4138193128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead7f40 0x55c15eae2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f9bfff640 1 -- 172.21.15.12:0/4138193128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c15e81a7d0 con 0x55c15eaddaa0 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/4138193128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15eaddaa0 0x55c15eae98e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1f94003d90 tx=0x7f1f94005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8548837c2240ca1a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f7affd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f7affd640 1 -- 172.21.15.12:0/4138193128 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94004c50 con 0x55c15eaddaa0 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f7affd640 1 -- 172.21.15.12:0/4138193128 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f1f94004df0 con 0x55c15eaddaa0 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1fa485a740 1 -- 172.21.15.12:0/4138193128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c15ea4a370 con 0x55c15eaddaa0 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1f7affd640 1 -- 172.21.15.12:0/4138193128 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1f94032050 con 0x55c15eaddaa0 2024-05-09T09:27:52.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1fa485a740 10 client.16524 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.112+0000 7f1fa485a740 1 -- 172.21.15.12:0/4138193128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c15eaecc50 con 0x55c15eaddaa0 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.113+0000 7f1f7affd640 1 -- 172.21.15.12:0/4138193128 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55c15ea4a370 con 0x55c15eaddaa0 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.113+0000 7f1f7affd640 1 -- 172.21.15.12:0/4138193128 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f1f94028040 con 0x55c15eaddaa0 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.113+0000 7f1fa485a740 10 client.16524 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.113+0000 7f1fa485a740 10 client.16524 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.113+0000 7f1fa485a740 20 client.16524 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.113+0000 7f1fa485a740 1 --2- 172.21.15.12:0/4138193128 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55c15eaee0e0 0x55c15eaf04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.113+0000 7f1fa485a740 4 client.16524 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.113+0000 7f1fa485a740 1 -- 172.21.15.12:0/4138193128 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c15ea446c0 con 0x55c15eaee0e0 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.113+0000 7f1f92ffd640 1 --2- 172.21.15.12:0/4138193128 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55c15eaee0e0 0x55c15eaf04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.113+0000 7f1f92ffd640 1 --2- 172.21.15.12:0/4138193128 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55c15eaee0e0 0x55c15eaf04d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a2df8fbd7b4b26a server_cookie=4c0801fa9355167f in_seq=0 out_seq=0 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.113+0000 7f1f7affd640 10 client.16524 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1f7affd640 1 -- 172.21.15.12:0/4138193128 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c15ea446c0 con 0x55c15eaee0e0 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1f7affd640 10 client.16524 handle_command_reply: tid=0 2024-05-09T09:27:52.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1fa485a740 1 client.16524 shutdown 2024-05-09T09:27:52.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1fa485a740 1 -- 172.21.15.12:0/4138193128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15eaddaa0 msgr2=0x55c15eae98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1fa485a740 1 --2- 172.21.15.12:0/4138193128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15eaddaa0 0x55c15eae98e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1f94003d90 tx=0x7f1f94005bd0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1fa485a740 1 -- 172.21.15.12:0/4138193128 shutdown_connections 2024-05-09T09:27:52.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1fa485a740 1 --2- 172.21.15.12:0/4138193128 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55c15eaee0e0 0x55c15eaf04d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1fa485a740 1 --2- 172.21.15.12:0/4138193128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15eaddaa0 0x55c15eae98e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1fa485a740 1 --2- 172.21.15.12:0/4138193128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15ead8930 0x55c15eae28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1fa485a740 1 --2- 172.21.15.12:0/4138193128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead7f40 0x55c15eae2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1f7affd640 0 client.16524 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:27:52.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1fa485a740 1 -- 172.21.15.12:0/4138193128 >> 172.21.15.12:0/4138193128 conn(0x55c15ead35b0 msgr2=0x55c15ead27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:52.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1fa485a740 1 -- 172.21.15.12:0/4138193128 shutdown_connections 2024-05-09T09:27:52.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1fa485a740 1 -- 172.21.15.12:0/4138193128 wait complete. 2024-05-09T09:27:52.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.114+0000 7f1fa485a740 20 client.16524 trim_cache size 0 max 16384 2024-05-09T09:27:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.129+0000 7f1fa485a740 1 Processor -- start 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.129+0000 7f1fa485a740 1 -- start start 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.129+0000 7f1fa485a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 0x55c15ead2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.129+0000 7f1fa485a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead8930 0x55c15eae98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.129+0000 7f1fa485a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 0x55c15ead2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.129+0000 7f1fa485a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1ac7e0 con 0x55c15ead7f40 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.129+0000 7f1fa485a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1ae3e0 con 0x55c15ead8930 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.129+0000 7f1fa485a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1aceb0 con 0x55c15eaddaa0 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.129+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 0x55c15ead2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.129+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 0x55c15ead2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46236/0 (socket says 172.21.15.12:46236) 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.129+0000 7f1f9bfff640 1 -- 172.21.15.12:0/180993058 learned_addr learned my addr 172.21.15.12:0/180993058 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.129+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/180993058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 0x55c15ead2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.130+0000 7f1f92ffd640 1 --2- 172.21.15.12:0/180993058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead8930 0x55c15eae98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.130+0000 7f1f9bfff640 1 -- 172.21.15.12:0/180993058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 msgr2=0x55c15ead2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.130+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/180993058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 0x55c15ead2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.130+0000 7f1f9bfff640 1 -- 172.21.15.12:0/180993058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead8930 msgr2=0x55c15eae98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.130+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/180993058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead8930 0x55c15eae98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.130+0000 7f1f9bfff640 1 -- 172.21.15.12:0/180993058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c15ead7d30 con 0x55c15ead7f40 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.130+0000 7f1f92ffd640 1 --2- 172.21.15.12:0/180993058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead8930 0x55c15eae98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.130+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/180993058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 0x55c15ead2630 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1f940055c0 tx=0x7f1f94009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cd4e93728f7e4f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.130+0000 7f1f917fa640 1 -- 172.21.15.12:0/180993058 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94003a60 con 0x55c15ead7f40 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.130+0000 7f1f917fa640 1 -- 172.21.15.12:0/180993058 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1f9400fc60 con 0x55c15ead7f40 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.130+0000 7f1f917fa640 1 -- 172.21.15.12:0/180993058 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f940056e0 con 0x55c15ead7f40 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 -- 172.21.15.12:0/180993058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 msgr2=0x55c15ead2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 --2- 172.21.15.12:0/180993058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 0x55c15ead2630 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1f940055c0 tx=0x7f1f94009520 comp rx=0 tx=0).stop 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 -- 172.21.15.12:0/180993058 shutdown_connections 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 --2- 172.21.15.12:0/180993058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 0x55c15ead2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 --2- 172.21.15.12:0/180993058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead8930 0x55c15eae98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 --2- 172.21.15.12:0/180993058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead7f40 0x55c15ead2630 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 -- 172.21.15.12:0/180993058 >> 172.21.15.12:0/180993058 conn(0x55c15ead35b0 msgr2=0x55c15ead1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:52.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 -- 172.21.15.12:0/180993058 shutdown_connections 2024-05-09T09:27:52.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 -- 172.21.15.12:0/180993058 wait complete. 2024-05-09T09:27:52.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 -- start start 2024-05-09T09:27:52.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 Processor -- start 2024-05-09T09:27:52.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead7f40 0x55c15eae6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead8930 0x55c15eae7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 0x55c15eae8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1add10 con 0x55c15ead8930 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1ad580 con 0x55c15ead7f40 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.131+0000 7f1fa485a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1aa7c0 con 0x55c15eaddaa0 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1fa485a740 10 client.0 _ll_register_callbacks cb 0x7f1f8c0d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead7f40 0x55c15eae6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead7f40 0x55c15eae6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41110/0 (socket says 172.21.15.12:41110) 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f9bfff640 1 -- 172.21.15.12:0/321154076 learned_addr learned my addr 172.21.15.12:0/321154076 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/321154076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 0x55c15eae8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f92ffd640 1 --2- 172.21.15.12:0/321154076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead8930 0x55c15eae7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f9bfff640 1 -- 172.21.15.12:0/321154076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 msgr2=0x55c15eae8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/321154076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 0x55c15eae8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f9bfff640 1 -- 172.21.15.12:0/321154076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead8930 msgr2=0x55c15eae7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/321154076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead8930 0x55c15eae7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f9bfff640 1 -- 172.21.15.12:0/321154076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c15eae8970 con 0x55c15ead7f40 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f9bfff640 1 --2- 172.21.15.12:0/321154076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead7f40 0x55c15eae6e30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1f94003e10 tx=0x7f1f94009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2ce2b90afe75476 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f7affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f7affd640 1 -- 172.21.15.12:0/321154076 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94011490 con 0x55c15ead7f40 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1fa485a740 1 -- 172.21.15.12:0/321154076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f9400fc60 con 0x55c15ead7f40 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f7affd640 1 -- 172.21.15.12:0/321154076 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f1f94004d30 con 0x55c15ead7f40 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1f7affd640 1 -- 172.21.15.12:0/321154076 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1f94004f30 con 0x55c15ead7f40 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1fa485a740 10 client.25976 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.132+0000 7f1fa485a740 1 -- 172.21.15.12:0/321154076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c15eae9be0 con 0x55c15ead7f40 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.133+0000 7f1f7affd640 1 -- 172.21.15.12:0/321154076 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1f9400fc60 con 0x55c15ead7f40 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.133+0000 7f1f7affd640 1 -- 172.21.15.12:0/321154076 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f1f94018020 con 0x55c15ead7f40 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.133+0000 7f1fa485a740 10 client.25976 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.133+0000 7f1fa485a740 10 client.25976 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:27:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.133+0000 7f1fa485a740 20 client.25976 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.133+0000 7f1fa485a740 1 --2- 172.21.15.12:0/321154076 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55c15eb54c60 0x55c15eb57080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.133+0000 7f1fa485a740 4 client.25976 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.133+0000 7f1fa485a740 1 -- 172.21.15.12:0/321154076 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c15e9c1820 con 0x55c15eb54c60 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.133+0000 7f1f92ffd640 1 --2- 172.21.15.12:0/321154076 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55c15eb54c60 0x55c15eb57080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1f92ffd640 1 --2- 172.21.15.12:0/321154076 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55c15eb54c60 0x55c15eb57080 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0aad941de930a05 server_cookie=a911f09f8d53e738 in_seq=0 out_seq=0 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1f7affd640 10 client.25976 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1f7affd640 1 -- 172.21.15.12:0/321154076 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c15e9c1820 con 0x55c15eb54c60 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1f7affd640 10 client.25976 handle_command_reply: tid=0 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1fa485a740 1 client.25976 shutdown 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1fa485a740 1 -- 172.21.15.12:0/321154076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead7f40 msgr2=0x55c15eae6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1fa485a740 1 --2- 172.21.15.12:0/321154076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead7f40 0x55c15eae6e30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1f94003e10 tx=0x7f1f94009610 comp rx=0 tx=0).stop 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1fa485a740 1 -- 172.21.15.12:0/321154076 shutdown_connections 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1fa485a740 1 --2- 172.21.15.12:0/321154076 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55c15eb54c60 0x55c15eb57080 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1fa485a740 1 --2- 172.21.15.12:0/321154076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c15eaddaa0 0x55c15eae8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1f7affd640 0 client.25976 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:27:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1fa485a740 1 --2- 172.21.15.12:0/321154076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c15ead8930 0x55c15eae7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1fa485a740 1 --2- 172.21.15.12:0/321154076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15ead7f40 0x55c15eae6e30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1fa485a740 1 -- 172.21.15.12:0/321154076 >> 172.21.15.12:0/321154076 conn(0x55c15ead35b0 msgr2=0x55c15ead6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1fa485a740 1 -- 172.21.15.12:0/321154076 shutdown_connections 2024-05-09T09:27:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1fa485a740 1 -- 172.21.15.12:0/321154076 wait complete. 2024-05-09T09:27:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.134+0000 7f1fa485a740 20 client.25976 trim_cache size 0 max 16384 2024-05-09T09:27:52.140 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:27:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.135+0000 7f1fa1b4f640 1 -- 172.21.15.12:0/3087834472 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1f880a2b90 msgr2=0x7f1f880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.135+0000 7f1fa1b4f640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1f880a2b90 0x7f1f880a5050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0cd110 tx=0x7f1f8c0d5550 comp rx=0 tx=0).stop 2024-05-09T09:27:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.135+0000 7f1fa1b4f640 1 -- 172.21.15.12:0/3087834472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f9c102350 msgr2=0x7f1f9c199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.135+0000 7f1fa1b4f640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f9c102350 0x7f1f9c199df0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00ea60 tx=0x7f1f8c00ef20 comp rx=0 tx=0).stop 2024-05-09T09:27:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.135+0000 7f1fa1b4f640 1 -- 172.21.15.12:0/3087834472 shutdown_connections 2024-05-09T09:27:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.135+0000 7f1fa1b4f640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1f880a2b90 0x7f1f880a5050 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.135+0000 7f1fa1b4f640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f9c1075b0 0x7f1f9c1a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.135+0000 7f1fa1b4f640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c104c80 0x7f1f9c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.135+0000 7f1fa1b4f640 1 --2- 172.21.15.12:0/3087834472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f9c102350 0x7f1f9c199df0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.135+0000 7f1fa1b4f640 1 -- 172.21.15.12:0/3087834472 >> 172.21.15.12:0/3087834472 conn(0x7f1f9c0fbce0 msgr2=0x7f1f9c101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.135+0000 7f1fa1b4f640 1 -- 172.21.15.12:0/3087834472 shutdown_connections 2024-05-09T09:27:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.135+0000 7f1fa1b4f640 1 -- 172.21.15.12:0/3087834472 wait complete. 2024-05-09T09:27:52.145 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:52 smithi012 ceph-mon[30140]: pgmap v437: 129 pgs: 129 active+clean; 136 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.4 MiB/s wr, 487 op/s 2024-05-09T09:27:52.145 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:52 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3796181623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:27:52.147 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:27:52.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.272+0000 7f7686314640 1 -- 172.21.15.12:0/2849253080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7680100770 msgr2=0x7f7680100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.272+0000 7f7686314640 1 --2- 172.21.15.12:0/2849253080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7680100770 0x7f7680100b90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7670009900 tx=0x7f767002a410 comp rx=0 tx=0).stop 2024-05-09T09:27:52.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.272+0000 7f7686314640 1 -- 172.21.15.12:0/2849253080 shutdown_connections 2024-05-09T09:27:52.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.272+0000 7f7686314640 1 --2- 172.21.15.12:0/2849253080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7680102320 0x7f7680108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.272+0000 7f7686314640 1 --2- 172.21.15.12:0/2849253080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7680101960 0x7f7680101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.272+0000 7f7686314640 1 --2- 172.21.15.12:0/2849253080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7680100770 0x7f7680100b90 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.272+0000 7f7686314640 1 -- 172.21.15.12:0/2849253080 >> 172.21.15.12:0/2849253080 conn(0x7f76800fbce0 msgr2=0x7f76800fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:52.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.272+0000 7f7686314640 1 -- 172.21.15.12:0/2849253080 shutdown_connections 2024-05-09T09:27:52.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.272+0000 7f7686314640 1 -- 172.21.15.12:0/2849253080 wait complete. 2024-05-09T09:27:52.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.273+0000 7f7686314640 1 Processor -- start 2024-05-09T09:27:52.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.273+0000 7f7686314640 1 -- start start 2024-05-09T09:27:52.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.273+0000 7f7686314640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7680100770 0x7f768010b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.273+0000 7f7686314640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7680101960 0x7f7680109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f7686314640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7680102320 0x7f7680109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f767ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7680100770 0x7f768010b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f767ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7680100770 0x7f768010b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41138/0 (socket says 172.21.15.12:41138) 2024-05-09T09:27:52.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f7677fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7680101960 0x7f7680109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f767ffff640 1 -- 172.21.15.12:0/2709640799 learned_addr learned my addr 172.21.15.12:0/2709640799 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:52.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f7686314640 1 -- 172.21.15.12:0/2709640799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f768010d4e0 con 0x7f7680102320 2024-05-09T09:27:52.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f7686314640 1 -- 172.21.15.12:0/2709640799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f768010d360 con 0x7f7680100770 2024-05-09T09:27:52.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f76848b1640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7680102320 0x7f7680109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f7686314640 1 -- 172.21.15.12:0/2709640799 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f768010d660 con 0x7f7680101960 2024-05-09T09:27:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f76848b1640 1 -- 172.21.15.12:0/2709640799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7680101960 msgr2=0x7f7680109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f76848b1640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7680101960 0x7f7680109730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f76848b1640 1 -- 172.21.15.12:0/2709640799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7680100770 msgr2=0x7f768010b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f76848b1640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7680100770 0x7f768010b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f76848b1640 1 -- 172.21.15.12:0/2709640799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768010a1b0 con 0x7f7680102320 2024-05-09T09:27:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f7677fff640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7680101960 0x7f7680109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.274+0000 7f76848b1640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7680102320 0x7f7680109c70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7678009fd0 tx=0x7f767800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.275+0000 7f767dffb640 1 -- 172.21.15.12:0/2709640799 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767800ce20 con 0x7f7680102320 2024-05-09T09:27:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.275+0000 7f767dffb640 1 -- 172.21.15.12:0/2709640799 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7678005020 con 0x7f7680102320 2024-05-09T09:27:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.275+0000 7f7686314640 1 -- 172.21.15.12:0/2709640799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768010a4a0 con 0x7f7680102320 2024-05-09T09:27:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.275+0000 7f767dffb640 1 -- 172.21.15.12:0/2709640799 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76780106b0 con 0x7f7680102320 2024-05-09T09:27:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.275+0000 7f7686314640 1 -- 172.21.15.12:0/2709640799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76801a6670 con 0x7f7680102320 2024-05-09T09:27:52.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.278+0000 7f7689025740 1 Processor -- start 2024-05-09T09:27:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.278+0000 7f767dffb640 1 -- 172.21.15.12:0/2709640799 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f76780051d0 con 0x7f7680102320 2024-05-09T09:27:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.278+0000 7f7689025740 1 -- start start 2024-05-09T09:27:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.278+0000 7f7689025740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8a5f40 0x55d44e8a6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.278+0000 7f7689025740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8ab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.278+0000 7f7689025740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8abaa0 0x55d44e8adf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.278+0000 7f7689025740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d44e80ac40 con 0x55d44e8abaa0 2024-05-09T09:27:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.278+0000 7f7689025740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d44e7b2580 con 0x55d44e8a6930 2024-05-09T09:27:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.278+0000 7f7689025740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d44e806c00 con 0x55d44e8a5f40 2024-05-09T09:27:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f7677fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8a5f40 0x55d44e8a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f767ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f7677fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8a5f40 0x55d44e8a6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:33500/0 (socket says 172.21.15.12:33500) 2024-05-09T09:27:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f7677fff640 1 -- 172.21.15.12:0/2487075136 learned_addr learned my addr 172.21.15.12:0/2487075136 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f7677fff640 1 --2- 172.21.15.12:0/2487075136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8abaa0 0x55d44e8adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f767ffff640 1 -- 172.21.15.12:0/2487075136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8a5f40 msgr2=0x55d44e8a6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f767dffb640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f766c0a2b90 0x7f766c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f767ffff640 1 --2- 172.21.15.12:0/2487075136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8a5f40 0x55d44e8a6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f767ffff640 1 -- 172.21.15.12:0/2487075136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8abaa0 msgr2=0x55d44e8adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f767ffff640 1 --2- 172.21.15.12:0/2487075136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8abaa0 0x55d44e8adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f767ffff640 1 -- 172.21.15.12:0/2487075136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d44e51e3f0 con 0x55d44e8a6930 2024-05-09T09:27:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f76848b1640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f766c0a2b90 0x7f766c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f767dffb640 1 -- 172.21.15.12:0/2709640799 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f76780cfbc0 con 0x7f7680102320 2024-05-09T09:27:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.279+0000 7f767ffff640 1 --2- 172.21.15.12:0/2487075136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8ab430 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7670005ab0 tx=0x7f7670005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a430d5fc7603df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.280+0000 7f76767fc640 1 -- 172.21.15.12:0/2487075136 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670040070 con 0x55d44e8a6930 2024-05-09T09:27:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.280+0000 7f76848b1640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f766c0a2b90 0x7f766c0a5050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f767800c760 tx=0x7f7678091530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.280+0000 7f76767fc640 1 -- 172.21.15.12:0/2487075136 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7670004030 con 0x55d44e8a6930 2024-05-09T09:27:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.280+0000 7f76767fc640 1 -- 172.21.15.12:0/2487075136 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670035040 con 0x55d44e8a6930 2024-05-09T09:27:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.280+0000 7f7689025740 1 -- 172.21.15.12:0/2487075136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 msgr2=0x55d44e8ab430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.280+0000 7f7689025740 1 --2- 172.21.15.12:0/2487075136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8ab430 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7670005ab0 tx=0x7f7670005df0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 -- 172.21.15.12:0/2487075136 shutdown_connections 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 --2- 172.21.15.12:0/2487075136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8abaa0 0x55d44e8adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 --2- 172.21.15.12:0/2487075136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8ab430 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 --2- 172.21.15.12:0/2487075136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8a5f40 0x55d44e8a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 -- 172.21.15.12:0/2487075136 >> 172.21.15.12:0/2487075136 conn(0x55d44e8a15b0 msgr2=0x55d44e8a19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 -- 172.21.15.12:0/2487075136 shutdown_connections 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 -- 172.21.15.12:0/2487075136 wait complete. 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 -- start start 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 Processor -- start 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8b0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 0x55d44e8b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d44e8ae9f0 con 0x55d44e8a5f40 2024-05-09T09:27:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d44e7c5c40 con 0x55d44e8a6930 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.281+0000 7f7689025740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d44e8aeb70 con 0x55d44e8abaa0 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f7689025740 10 client.0 _ll_register_callbacks cb 0x55d44e51dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f7677fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f7677fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46260/0 (socket says 172.21.15.12:46260) 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f7677fff640 1 -- 172.21.15.12:0/1990511505 learned_addr learned my addr 172.21.15.12:0/1990511505 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f767ffff640 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f7677fff640 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 0x55d44e8b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f767ffff640 1 -- 172.21.15.12:0/1990511505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 msgr2=0x55d44e8b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f767ffff640 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 0x55d44e8b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f767ffff640 1 -- 172.21.15.12:0/1990511505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 msgr2=0x55d44e8b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f767ffff640 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f767ffff640 1 -- 172.21.15.12:0/1990511505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d44e5e87d0 con 0x55d44e8a6930 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f7677fff640 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:52.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f767ffff640 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b0870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55d44e8a2520 tx=0x7f76700056a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184415d9dae00bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f765f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f765f7fe640 1 -- 172.21.15.12:0/1990511505 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670040070 con 0x55d44e8a6930 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f765f7fe640 1 -- 172.21.15.12:0/1990511505 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f7670031610 con 0x55d44e8a6930 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.282+0000 7f7689025740 1 -- 172.21.15.12:0/1990511505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7670004030 con 0x55d44e8a6930 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.283+0000 7f765f7fe640 1 -- 172.21.15.12:0/1990511505 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7670056050 con 0x55d44e8a6930 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.283+0000 7f7689025740 10 client.25988 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.283+0000 7f7689025740 1 -- 172.21.15.12:0/1990511505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d44e8bac20 con 0x55d44e8a6930 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.283+0000 7f765f7fe640 1 -- 172.21.15.12:0/1990511505 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7670004030 con 0x55d44e8a6930 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.283+0000 7f765f7fe640 1 -- 172.21.15.12:0/1990511505 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f76700035f0 con 0x55d44e8a6930 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.283+0000 7f7689025740 10 client.25988 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.283+0000 7f7689025740 10 client.25988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.283+0000 7f7689025740 20 client.25988 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.283+0000 7f7689025740 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d44e8bbf70 0x55d44e8be380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.283+0000 7f7689025740 4 client.25988 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.283+0000 7f7689025740 1 -- 172.21.15.12:0/1990511505 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d44e818370 con 0x55d44e8bbf70 2024-05-09T09:27:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.284+0000 7f7677fff640 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d44e8bbf70 0x55d44e8be380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.284+0000 7f7677fff640 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d44e8bbf70 0x55d44e8be380 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc04ca2cecdad9c5 server_cookie=f33e150314b66782 in_seq=0 out_seq=0 2024-05-09T09:27:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.284+0000 7f765f7fe640 10 client.25988 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f765f7fe640 1 -- 172.21.15.12:0/1990511505 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d44e818370 con 0x55d44e8bbf70 2024-05-09T09:27:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f765f7fe640 10 client.25988 handle_command_reply: tid=0 2024-05-09T09:27:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f7689025740 1 client.25988 shutdown 2024-05-09T09:27:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f7689025740 1 -- 172.21.15.12:0/1990511505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 msgr2=0x55d44e8b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f7689025740 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b0870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55d44e8a2520 tx=0x7f76700056a0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f7689025740 1 -- 172.21.15.12:0/1990511505 shutdown_connections 2024-05-09T09:27:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f7689025740 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d44e8bbf70 0x55d44e8be380 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f7689025740 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 0x55d44e8b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f765f7fe640 0 client.25988 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:52.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f7689025740 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b0870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f7689025740 1 --2- 172.21.15.12:0/1990511505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f7689025740 1 -- 172.21.15.12:0/1990511505 >> 172.21.15.12:0/1990511505 conn(0x55d44e8a15b0 msgr2=0x55d44e8a07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:52.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f7689025740 1 -- 172.21.15.12:0/1990511505 shutdown_connections 2024-05-09T09:27:52.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f7689025740 1 -- 172.21.15.12:0/1990511505 wait complete. 2024-05-09T09:27:52.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.285+0000 7f7689025740 20 client.25988 trim_cache size 0 max 16384 2024-05-09T09:27:52.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.299+0000 7f7689025740 1 Processor -- start 2024-05-09T09:27:52.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.299+0000 7f7689025740 1 -- start start 2024-05-09T09:27:52.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.299+0000 7f7689025740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8ade90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.299+0000 7f7689025740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.299+0000 7f7689025740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 0x55d44e8ab360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.299+0000 7f7689025740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f76801b0b80 con 0x55d44e8a5f40 2024-05-09T09:27:52.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.299+0000 7f7689025740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76801b2780 con 0x55d44e8a6930 2024-05-09T09:27:52.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.299+0000 7f7689025740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f76801b1250 con 0x55d44e8abaa0 2024-05-09T09:27:52.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.299+0000 7f7677fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.299+0000 7f7677fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46276/0 (socket says 172.21.15.12:46276) 2024-05-09T09:27:52.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.299+0000 7f7677fff640 1 -- 172.21.15.12:0/1721486313 learned_addr learned my addr 172.21.15.12:0/1721486313 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:52.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.299+0000 7f7677fff640 1 --2- 172.21.15.12:0/1721486313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 0x55d44e8ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.300+0000 7f767ffff640 1 --2- 172.21.15.12:0/1721486313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.300+0000 7f7677fff640 1 -- 172.21.15.12:0/1721486313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 msgr2=0x55d44e8ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.300+0000 7f7677fff640 1 --2- 172.21.15.12:0/1721486313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 0x55d44e8ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.300+0000 7f7677fff640 1 -- 172.21.15.12:0/1721486313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 msgr2=0x55d44e8b7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.300+0000 7f7677fff640 1 --2- 172.21.15.12:0/1721486313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.300+0000 7f7677fff640 1 -- 172.21.15.12:0/1721486313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d44e8bea00 con 0x55d44e8a5f40 2024-05-09T09:27:52.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.300+0000 7f7677fff640 1 --2- 172.21.15.12:0/1721486313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8ade90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55d44e8bec30 tx=0x7f7668005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c484350278c62f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:52.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.300+0000 7f7675ffb640 1 -- 172.21.15.12:0/1721486313 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76680054d0 con 0x55d44e8a5f40 2024-05-09T09:27:52.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.300+0000 7f7675ffb640 1 -- 172.21.15.12:0/1721486313 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7668004030 con 0x55d44e8a5f40 2024-05-09T09:27:52.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.300+0000 7f7675ffb640 1 -- 172.21.15.12:0/1721486313 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668003a80 con 0x55d44e8a5f40 2024-05-09T09:27:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.301+0000 7f7689025740 1 -- 172.21.15.12:0/1721486313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 msgr2=0x55d44e8ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.301+0000 7f7689025740 1 --2- 172.21.15.12:0/1721486313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8ade90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55d44e8bec30 tx=0x7f7668005f90 comp rx=0 tx=0).stop 2024-05-09T09:27:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.301+0000 7f7689025740 1 -- 172.21.15.12:0/1721486313 shutdown_connections 2024-05-09T09:27:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.301+0000 7f7689025740 1 --2- 172.21.15.12:0/1721486313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 0x55d44e8ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.301+0000 7f7689025740 1 --2- 172.21.15.12:0/1721486313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.301+0000 7f7689025740 1 --2- 172.21.15.12:0/1721486313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8ade90 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.301+0000 7f7689025740 1 -- 172.21.15.12:0/1721486313 >> 172.21.15.12:0/1721486313 conn(0x55d44e8a15b0 msgr2=0x55d44e8a7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.301+0000 7f7689025740 1 -- 172.21.15.12:0/1721486313 shutdown_connections 2024-05-09T09:27:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.301+0000 7f7689025740 1 -- 172.21.15.12:0/1721486313 wait complete. 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.301+0000 7f7689025740 1 -- start start 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.301+0000 7f7689025740 1 Processor -- start 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.301+0000 7f7689025740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7689025740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7689025740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 0x55d44e9204f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7689025740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f76801b20b0 con 0x55d44e8a5f40 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7689025740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76801b1920 con 0x55d44e8a6930 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7689025740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f76801aeb60 con 0x55d44e8abaa0 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7689025740 10 client.0 _ll_register_callbacks cb 0x7f7668011690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7677fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7677fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46278/0 (socket says 172.21.15.12:46278) 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7677fff640 1 -- 172.21.15.12:0/1835293180 learned_addr learned my addr 172.21.15.12:0/1835293180 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7677fff640 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 0x55d44e9204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f767ffff640 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7677fff640 1 -- 172.21.15.12:0/1835293180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 msgr2=0x55d44e9204f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7677fff640 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 0x55d44e9204f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7677fff640 1 -- 172.21.15.12:0/1835293180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 msgr2=0x55d44e8b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7677fff640 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7677fff640 1 -- 172.21.15.12:0/1835293180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d44e920a30 con 0x55d44e8a5f40 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f767ffff640 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7677fff640 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8b46a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7668025040 tx=0x7f7668004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acb798bfe057c42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f765f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f765f7fe640 1 -- 172.21.15.12:0/1835293180 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668003800 con 0x55d44e8a5f40 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f765f7fe640 1 -- 172.21.15.12:0/1835293180 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f76680039a0 con 0x55d44e8a5f40 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.302+0000 7f7689025740 1 -- 172.21.15.12:0/1835293180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7668004030 con 0x55d44e8a5f40 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.303+0000 7f765f7fe640 1 -- 172.21.15.12:0/1835293180 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7668003ba0 con 0x55d44e8a5f40 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.303+0000 7f7689025740 10 client.16548 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.303+0000 7f7689025740 1 -- 172.21.15.12:0/1835293180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d44e921bd0 con 0x55d44e8a5f40 2024-05-09T09:27:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.303+0000 7f765f7fe640 1 -- 172.21.15.12:0/1835293180 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7668004030 con 0x55d44e8a5f40 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.303+0000 7f765f7fe640 1 -- 172.21.15.12:0/1835293180 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f7668022470 con 0x55d44e8a5f40 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.303+0000 7f7689025740 10 client.16548 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.303+0000 7f7689025740 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.303+0000 7f7689025740 20 client.16548 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.303+0000 7f7689025740 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d44e9225c0 0x55d44e9249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.303+0000 7f7689025740 4 client.16548 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.303+0000 7f7689025740 1 -- 172.21.15.12:0/1835293180 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d44e8126c0 con 0x55d44e9225c0 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.303+0000 7f767ffff640 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d44e9225c0 0x55d44e9249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f767ffff640 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d44e9225c0 0x55d44e9249e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e05deecf6c5ead1d server_cookie=16a0a5353efa7ad4 in_seq=0 out_seq=0 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f765f7fe640 10 client.16548 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f765f7fe640 1 -- 172.21.15.12:0/1835293180 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d44e8126c0 con 0x55d44e9225c0 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f765f7fe640 10 client.16548 handle_command_reply: tid=0 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f7689025740 1 client.16548 shutdown 2024-05-09T09:27:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f7689025740 1 -- 172.21.15.12:0/1835293180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 msgr2=0x55d44e8b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f7689025740 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8b46a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7668025040 tx=0x7f7668004c40 comp rx=0 tx=0).stop 2024-05-09T09:27:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f7689025740 1 -- 172.21.15.12:0/1835293180 shutdown_connections 2024-05-09T09:27:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f7689025740 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d44e9225c0 0x55d44e9249e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f7689025740 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d44e8abaa0 0x55d44e9204f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f7689025740 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d44e8a6930 0x55d44e8b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f765f7fe640 0 client.16548 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f7689025740 1 --2- 172.21.15.12:0/1835293180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d44e8a5f40 0x55d44e8b46a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f7689025740 1 -- 172.21.15.12:0/1835293180 >> 172.21.15.12:0/1835293180 conn(0x55d44e8a15b0 msgr2=0x55d44e8b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f7689025740 1 -- 172.21.15.12:0/1835293180 shutdown_connections 2024-05-09T09:27:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f7689025740 1 -- 172.21.15.12:0/1835293180 wait complete. 2024-05-09T09:27:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.304+0000 7f7689025740 20 client.16548 trim_cache size 0 max 16384 2024-05-09T09:27:52.311 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:27:52.312 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:52 smithi016 ceph-mon[33335]: pgmap v437: 129 pgs: 129 active+clean; 136 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.4 MiB/s wr, 487 op/s 2024-05-09T09:27:52.312 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:52 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3796181623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:27:52.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.305+0000 7f7686314640 1 -- 172.21.15.12:0/2709640799 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f766c0a2b90 msgr2=0x7f766c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.305+0000 7f7686314640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f766c0a2b90 0x7f766c0a5050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f767800c760 tx=0x7f7678091530 comp rx=0 tx=0).stop 2024-05-09T09:27:52.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.305+0000 7f7686314640 1 -- 172.21.15.12:0/2709640799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7680102320 msgr2=0x7f7680109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:52.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.305+0000 7f7686314640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7680102320 0x7f7680109c70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7678009fd0 tx=0x7f767800ef10 comp rx=0 tx=0).stop 2024-05-09T09:27:52.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.305+0000 7f7686314640 1 -- 172.21.15.12:0/2709640799 shutdown_connections 2024-05-09T09:27:52.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.305+0000 7f7686314640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f766c0a2b90 0x7f766c0a5050 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.305+0000 7f7686314640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7680102320 0x7f7680109c70 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.305+0000 7f7686314640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7680101960 0x7f7680109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.305+0000 7f7686314640 1 --2- 172.21.15.12:0/2709640799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7680100770 0x7f768010b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:52.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.305+0000 7f7686314640 1 -- 172.21.15.12:0/2709640799 >> 172.21.15.12:0/2709640799 conn(0x7f76800fbce0 msgr2=0x7f7680104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:52.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.305+0000 7f7686314640 1 -- 172.21.15.12:0/2709640799 shutdown_connections 2024-05-09T09:27:52.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:52.305+0000 7f7686314640 1 -- 172.21.15.12:0/2709640799 wait complete. 2024-05-09T09:27:52.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:52 smithi170 ceph-mon[33632]: pgmap v437: 129 pgs: 129 active+clean; 136 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.4 MiB/s wr, 487 op/s 2024-05-09T09:27:52.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:52 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3796181623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:27:53.317 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:27:53.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:27:53.317 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:27:53.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.442+0000 7f0c6e5ee640 1 -- 172.21.15.12:0/2769764614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c68100770 msgr2=0x7f0c68100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.442+0000 7f0c6e5ee640 1 --2- 172.21.15.12:0/2769764614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c68100770 0x7f0c68100b90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009900 tx=0x7f0c5802a410 comp rx=0 tx=0).stop 2024-05-09T09:27:53.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.442+0000 7f0c6e5ee640 1 -- 172.21.15.12:0/2769764614 shutdown_connections 2024-05-09T09:27:53.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.442+0000 7f0c6e5ee640 1 --2- 172.21.15.12:0/2769764614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68102320 0x7f0c68108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.442+0000 7f0c6e5ee640 1 --2- 172.21.15.12:0/2769764614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c68101960 0x7f0c68101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.442+0000 7f0c6e5ee640 1 --2- 172.21.15.12:0/2769764614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c68100770 0x7f0c68100b90 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.442+0000 7f0c6e5ee640 1 -- 172.21.15.12:0/2769764614 >> 172.21.15.12:0/2769764614 conn(0x7f0c680fbce0 msgr2=0x7f0c680fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.442+0000 7f0c6e5ee640 1 -- 172.21.15.12:0/2769764614 shutdown_connections 2024-05-09T09:27:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.442+0000 7f0c6e5ee640 1 -- 172.21.15.12:0/2769764614 wait complete. 2024-05-09T09:27:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c6e5ee640 1 Processor -- start 2024-05-09T09:27:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c6e5ee640 1 -- start start 2024-05-09T09:27:53.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c6e5ee640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c68100770 0x7f0c6810b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c6e5ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68101960 0x7f0c68109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c6e5ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c68102320 0x7f0c68109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c6e5ee640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6810d4e0 con 0x7f0c68100770 2024-05-09T09:27:53.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c6e5ee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6810d360 con 0x7f0c68102320 2024-05-09T09:27:53.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c6e5ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6810d660 con 0x7f0c68101960 2024-05-09T09:27:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c68100770 0x7f0c6810b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c677fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68101960 0x7f0c68109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c68100770 0x7f0c6810b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46308/0 (socket says 172.21.15.12:46308) 2024-05-09T09:27:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c67fff640 1 -- 172.21.15.12:0/2741551902 learned_addr learned my addr 172.21.15.12:0/2741551902 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c6cb8b640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c68102320 0x7f0c68109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c677fe640 1 -- 172.21.15.12:0/2741551902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c68102320 msgr2=0x7f0c68109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c677fe640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c68102320 0x7f0c68109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.443+0000 7f0c677fe640 1 -- 172.21.15.12:0/2741551902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c68100770 msgr2=0x7f0c6810b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.444+0000 7f0c677fe640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c68100770 0x7f0c6810b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.444+0000 7f0c677fe640 1 -- 172.21.15.12:0/2741551902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6810a500 con 0x7f0c68101960 2024-05-09T09:27:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.444+0000 7f0c67fff640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c68100770 0x7f0c6810b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:53.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.444+0000 7f0c677fe640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68101960 0x7f0c68109730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c005fd0 tx=0x7f0c5c00dd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.444+0000 7f0c657fa640 1 -- 172.21.15.12:0/2741551902 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c009c30 con 0x7f0c68101960 2024-05-09T09:27:53.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.444+0000 7f0c6e5ee640 1 -- 172.21.15.12:0/2741551902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c681a1d20 con 0x7f0c68101960 2024-05-09T09:27:53.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.444+0000 7f0c657fa640 1 -- 172.21.15.12:0/2741551902 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0c5c00be30 con 0x7f0c68101960 2024-05-09T09:27:53.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.444+0000 7f0c657fa640 1 -- 172.21.15.12:0/2741551902 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c003de0 con 0x7f0c68101960 2024-05-09T09:27:53.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.444+0000 7f0c6e5ee640 1 -- 172.21.15.12:0/2741551902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c681a2200 con 0x7f0c68101960 2024-05-09T09:27:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c713db740 1 Processor -- start 2024-05-09T09:27:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c713db740 1 -- start start 2024-05-09T09:27:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c657fa640 1 -- 172.21.15.12:0/2741551902 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0c5c003fa0 con 0x7f0c68101960 2024-05-09T09:27:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c713db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f1f4c0 0x563489f1f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c713db740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1feb0 0x563489f249b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c713db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f25020 0x563489f274b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c713db740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563489e83990 con 0x563489f1feb0 2024-05-09T09:27:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c713db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563489e2b5b0 con 0x563489f1f4c0 2024-05-09T09:27:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c713db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563489e7f950 con 0x563489f25020 2024-05-09T09:27:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1feb0 0x563489f249b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c6cb8b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f1f4c0 0x563489f1f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c6cb8b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f1f4c0 0x563489f1f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41194/0 (socket says 172.21.15.12:41194) 2024-05-09T09:27:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c677fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f25020 0x563489f274b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c6cb8b640 1 -- 172.21.15.12:0/1436466760 learned_addr learned my addr 172.21.15.12:0/1436466760 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1feb0 0x563489f249b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46320/0 (socket says 172.21.15.12:46320) 2024-05-09T09:27:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c6cb8b640 1 -- 172.21.15.12:0/1436466760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f25020 msgr2=0x563489f274b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c6cb8b640 1 --2- 172.21.15.12:0/1436466760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f25020 0x563489f274b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c6cb8b640 1 -- 172.21.15.12:0/1436466760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1feb0 msgr2=0x563489f249b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c6cb8b640 1 --2- 172.21.15.12:0/1436466760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1feb0 0x563489f249b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c6cb8b640 1 -- 172.21.15.12:0/1436466760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563489c5f7e0 con 0x563489f1f4c0 2024-05-09T09:27:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.447+0000 7f0c67fff640 1 --2- 172.21.15.12:0/1436466760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1feb0 0x563489f249b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.448+0000 7f0c657fa640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0c500a2b90 0x7f0c500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.448+0000 7f0c6cb8b640 1 --2- 172.21.15.12:0/1436466760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f1f4c0 0x563489f1f8e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009f60 tx=0x7f0c60009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78bd7e7bde81c7f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.448+0000 7f0c67fff640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0c500a2b90 0x7f0c500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.448+0000 7f0c657fa640 1 -- 172.21.15.12:0/2741551902 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f0c5c0ceea0 con 0x7f0c68101960 2024-05-09T09:27:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.448+0000 7f0c4e7fc640 1 -- 172.21.15.12:0/1436466760 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6001e070 con 0x563489f1f4c0 2024-05-09T09:27:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.448+0000 7f0c4e7fc640 1 -- 172.21.15.12:0/1436466760 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0c60003c30 con 0x563489f1f4c0 2024-05-09T09:27:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.448+0000 7f0c4e7fc640 1 -- 172.21.15.12:0/1436466760 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6000f980 con 0x563489f1f4c0 2024-05-09T09:27:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.449+0000 7f0c67fff640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0c500a2b90 0x7f0c500a5050 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x563489f202b0 tx=0x7f0c58031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.449+0000 7f0c713db740 1 -- 172.21.15.12:0/1436466760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f1f4c0 msgr2=0x563489f1f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.449+0000 7f0c713db740 1 --2- 172.21.15.12:0/1436466760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f1f4c0 0x563489f1f8e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009f60 tx=0x7f0c60009f90 comp rx=0 tx=0).stop 2024-05-09T09:27:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.449+0000 7f0c713db740 1 -- 172.21.15.12:0/1436466760 shutdown_connections 2024-05-09T09:27:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.449+0000 7f0c713db740 1 --2- 172.21.15.12:0/1436466760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f25020 0x563489f274b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.449+0000 7f0c713db740 1 --2- 172.21.15.12:0/1436466760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1feb0 0x563489f249b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.449+0000 7f0c713db740 1 --2- 172.21.15.12:0/1436466760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f1f4c0 0x563489f1f8e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.449+0000 7f0c713db740 1 -- 172.21.15.12:0/1436466760 >> 172.21.15.12:0/1436466760 conn(0x563489ee70a0 msgr2=0x563489f1af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.449+0000 7f0c713db740 1 -- 172.21.15.12:0/1436466760 shutdown_connections 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.449+0000 7f0c713db740 1 -- 172.21.15.12:0/1436466760 wait complete. 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.449+0000 7f0c713db740 1 -- start start 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.449+0000 7f0c713db740 1 Processor -- start 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.450+0000 7f0c713db740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1f4c0 0x563489f29980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.450+0000 7f0c713db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f29ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.450+0000 7f0c713db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f25020 0x563489f30f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.450+0000 7f0c713db740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563489f27fd0 con 0x563489f1f4c0 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.450+0000 7f0c713db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563489e3eb40 con 0x563489f25020 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.450+0000 7f0c713db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563489f28150 con 0x563489f1feb0 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.450+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1f4c0 0x563489f29980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.450+0000 7f0c713db740 10 client.0 _ll_register_callbacks cb 0x563489e330f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.450+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1f4c0 0x563489f29980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46326/0 (socket says 172.21.15.12:46326) 2024-05-09T09:27:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.450+0000 7f0c677fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f29ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.450+0000 7f0c67fff640 1 -- 172.21.15.12:0/1552669793 learned_addr learned my addr 172.21.15.12:0/1552669793 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.450+0000 7f0c6cb8b640 1 --2- 172.21.15.12:0/1552669793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f25020 0x563489f30f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.451+0000 7f0c677fe640 1 -- 172.21.15.12:0/1552669793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f25020 msgr2=0x563489f30f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.451+0000 7f0c677fe640 1 --2- 172.21.15.12:0/1552669793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f25020 0x563489f30f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.451+0000 7f0c677fe640 1 -- 172.21.15.12:0/1552669793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1f4c0 msgr2=0x563489f29980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.451+0000 7f0c677fe640 1 --2- 172.21.15.12:0/1552669793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1f4c0 0x563489f29980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.451+0000 7f0c677fe640 1 -- 172.21.15.12:0/1552669793 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563489cbff70 con 0x563489f1feb0 2024-05-09T09:27:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.451+0000 7f0c677fe640 1 --2- 172.21.15.12:0/1552669793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f29ec0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0cc620 tx=0x7f0c5c0d2570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c49139a52287b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.451+0000 7f0c477fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:27:53.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.451+0000 7f0c477fe640 1 -- 172.21.15.12:0/1552669793 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c020050 con 0x563489f1feb0 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.451+0000 7f0c477fe640 1 -- 172.21.15.12:0/1552669793 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f0c5c0979a0 con 0x563489f1feb0 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.451+0000 7f0c713db740 1 -- 172.21.15.12:0/1552669793 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c60003c30 con 0x563489f1feb0 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.452+0000 7f0c477fe640 1 -- 172.21.15.12:0/1552669793 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0c5c09ac90 con 0x563489f1feb0 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.452+0000 7f0c713db740 10 client.26119 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.452+0000 7f0c713db740 1 -- 172.21.15.12:0/1552669793 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563489b57170 con 0x563489f1feb0 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.452+0000 7f0c477fe640 1 -- 172.21.15.12:0/1552669793 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0c60003c30 con 0x563489f1feb0 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.452+0000 7f0c477fe640 1 -- 172.21.15.12:0/1552669793 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f0c5c004290 con 0x563489f1feb0 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.452+0000 7f0c713db740 10 client.26119 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.452+0000 7f0c713db740 10 client.26119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.452+0000 7f0c713db740 20 client.26119 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.452+0000 7f0c713db740 1 --2- 172.21.15.12:0/1552669793 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x563489f33cb0 0x563489f36110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.452+0000 7f0c713db740 4 client.26119 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.452+0000 7f0c713db740 1 -- 172.21.15.12:0/1552669793 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563489e910c0 con 0x563489f33cb0 2024-05-09T09:27:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.452+0000 7f0c67fff640 1 --2- 172.21.15.12:0/1552669793 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x563489f33cb0 0x563489f36110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.453+0000 7f0c67fff640 1 --2- 172.21.15.12:0/1552669793 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x563489f33cb0 0x563489f36110 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbadc5aaec651641 server_cookie=c7df46e287d3c27b in_seq=0 out_seq=0 2024-05-09T09:27:53.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.453+0000 7f0c477fe640 10 client.26119 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:53.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.453+0000 7f0c477fe640 1 -- 172.21.15.12:0/1552669793 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563489e910c0 con 0x563489f33cb0 2024-05-09T09:27:53.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.453+0000 7f0c477fe640 10 client.26119 handle_command_reply: tid=0 2024-05-09T09:27:53.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.453+0000 7f0c713db740 1 client.26119 shutdown 2024-05-09T09:27:53.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.454+0000 7f0c713db740 1 -- 172.21.15.12:0/1552669793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 msgr2=0x563489f29ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.454+0000 7f0c713db740 1 --2- 172.21.15.12:0/1552669793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f29ec0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0cc620 tx=0x7f0c5c0d2570 comp rx=0 tx=0).stop 2024-05-09T09:27:53.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.454+0000 7f0c713db740 1 -- 172.21.15.12:0/1552669793 shutdown_connections 2024-05-09T09:27:53.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.454+0000 7f0c713db740 1 --2- 172.21.15.12:0/1552669793 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x563489f33cb0 0x563489f36110 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.454+0000 7f0c713db740 1 --2- 172.21.15.12:0/1552669793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f25020 0x563489f30f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.454+0000 7f0c713db740 1 --2- 172.21.15.12:0/1552669793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f29ec0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.454+0000 7f0c477fe640 0 client.26119 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:53.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.454+0000 7f0c713db740 1 --2- 172.21.15.12:0/1552669793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1f4c0 0x563489f29980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.454+0000 7f0c713db740 1 -- 172.21.15.12:0/1552669793 >> 172.21.15.12:0/1552669793 conn(0x563489ee70a0 msgr2=0x563489f1a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.454+0000 7f0c713db740 1 -- 172.21.15.12:0/1552669793 shutdown_connections 2024-05-09T09:27:53.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.454+0000 7f0c713db740 1 -- 172.21.15.12:0/1552669793 wait complete. 2024-05-09T09:27:53.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.454+0000 7f0c713db740 20 client.26119 trim_cache size 0 max 16384 2024-05-09T09:27:53.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c713db740 1 Processor -- start 2024-05-09T09:27:53.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c713db740 1 -- start start 2024-05-09T09:27:53.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c713db740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1f4c0 0x563489f1df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c713db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f1e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c713db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f25020 0x563489f405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c713db740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681a8470 con 0x563489f1f4c0 2024-05-09T09:27:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c713db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681aa070 con 0x563489f25020 2024-05-09T09:27:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c713db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681a8b40 con 0x563489f1feb0 2024-05-09T09:27:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1f4c0 0x563489f1df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c6cb8b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f25020 0x563489f405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c677fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f1e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1f4c0 0x563489f1df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46328/0 (socket says 172.21.15.12:46328) 2024-05-09T09:27:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c67fff640 1 -- 172.21.15.12:0/3058519102 learned_addr learned my addr 172.21.15.12:0/3058519102 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c6cb8b640 1 --2- 172.21.15.12:0/3058519102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f25020 0x563489f405d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41208/0 (socket says 172.21.15.12:41208) 2024-05-09T09:27:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c67fff640 1 -- 172.21.15.12:0/3058519102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 msgr2=0x563489f1e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c67fff640 1 --2- 172.21.15.12:0/3058519102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f1e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c67fff640 1 -- 172.21.15.12:0/3058519102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f25020 msgr2=0x563489f405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.476+0000 7f0c67fff640 1 --2- 172.21.15.12:0/3058519102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f25020 0x563489f405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.477+0000 7f0c67fff640 1 -- 172.21.15.12:0/3058519102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563489f36790 con 0x563489f1f4c0 2024-05-09T09:27:53.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.477+0000 7f0c677fe640 1 --2- 172.21.15.12:0/3058519102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f1e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:53.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.477+0000 7f0c6cb8b640 1 --2- 172.21.15.12:0/3058519102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f25020 0x563489f405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:53.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.477+0000 7f0c67fff640 1 --2- 172.21.15.12:0/3058519102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1f4c0 0x563489f1df10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0c58000c00 tx=0x7f0c5803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c54185bd8c7d5f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.477+0000 7f0c4dffb640 1 -- 172.21.15.12:0/3058519102 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58058070 con 0x563489f1f4c0 2024-05-09T09:27:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.477+0000 7f0c4dffb640 1 -- 172.21.15.12:0/3058519102 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0c580046c0 con 0x563489f1f4c0 2024-05-09T09:27:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.477+0000 7f0c4dffb640 1 -- 172.21.15.12:0/3058519102 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58003850 con 0x563489f1f4c0 2024-05-09T09:27:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.478+0000 7f0c713db740 1 -- 172.21.15.12:0/3058519102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1f4c0 msgr2=0x563489f1df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.478+0000 7f0c713db740 1 --2- 172.21.15.12:0/3058519102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1f4c0 0x563489f1df10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0c58000c00 tx=0x7f0c5803f000 comp rx=0 tx=0).stop 2024-05-09T09:27:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.478+0000 7f0c713db740 1 -- 172.21.15.12:0/3058519102 shutdown_connections 2024-05-09T09:27:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.478+0000 7f0c713db740 1 --2- 172.21.15.12:0/3058519102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f25020 0x563489f405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.478+0000 7f0c713db740 1 --2- 172.21.15.12:0/3058519102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f1e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.478+0000 7f0c713db740 1 --2- 172.21.15.12:0/3058519102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f1f4c0 0x563489f1df10 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.478+0000 7f0c713db740 1 -- 172.21.15.12:0/3058519102 >> 172.21.15.12:0/3058519102 conn(0x563489ee70a0 msgr2=0x563489f21350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.478+0000 7f0c713db740 1 -- 172.21.15.12:0/3058519102 shutdown_connections 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.478+0000 7f0c713db740 1 -- 172.21.15.12:0/3058519102 wait complete. 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.478+0000 7f0c713db740 1 -- start start 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c713db740 1 Processor -- start 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c713db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f1f4c0 0x563489f94180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c713db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f946c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c713db740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f25020 0x563489f9b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c713db740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681a99a0 con 0x563489f25020 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c713db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681a9210 con 0x563489f1f4c0 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c713db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681a6450 con 0x563489f1feb0 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c713db740 10 client.0 _ll_register_callbacks cb 0x7f0c5c005d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c6cb8b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f25020 0x563489f9b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f1f4c0 0x563489f94180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c6cb8b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f25020 0x563489f9b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46338/0 (socket says 172.21.15.12:46338) 2024-05-09T09:27:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c6cb8b640 1 -- 172.21.15.12:0/3042315488 learned_addr learned my addr 172.21.15.12:0/3042315488 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c677fe640 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.479+0000 7f0c67fff640 1 -- 172.21.15.12:0/3042315488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 msgr2=0x563489f946c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c67fff640 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f946c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c67fff640 1 -- 172.21.15.12:0/3042315488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f25020 msgr2=0x563489f9b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c67fff640 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f25020 0x563489f9b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c67fff640 1 -- 172.21.15.12:0/3042315488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563489f9b900 con 0x563489f1f4c0 2024-05-09T09:27:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c6cb8b640 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f25020 0x563489f9b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c677fe640 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f946c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c67fff640 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f1f4c0 0x563489f94180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0c580057a0 tx=0x7f0c58004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be62b53c8b496b1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c477fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:27:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c477fe640 1 -- 172.21.15.12:0/3042315488 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58058050 con 0x563489f1f4c0 2024-05-09T09:27:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c713db740 1 -- 172.21.15.12:0/3042315488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c580046c0 con 0x563489f1f4c0 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c477fe640 1 -- 172.21.15.12:0/3042315488 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f0c58004fa0 con 0x563489f1f4c0 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c477fe640 1 -- 172.21.15.12:0/3042315488 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0c580051e0 con 0x563489f1f4c0 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c713db740 10 client.26009 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c713db740 1 -- 172.21.15.12:0/3042315488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563489f9cc90 con 0x563489f1f4c0 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.480+0000 7f0c477fe640 1 -- 172.21.15.12:0/3042315488 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0c580046c0 con 0x563489f1f4c0 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.481+0000 7f0c477fe640 1 -- 172.21.15.12:0/3042315488 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f0c58043040 con 0x563489f1f4c0 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.481+0000 7f0c713db740 10 client.26009 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.481+0000 7f0c713db740 10 client.26009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.481+0000 7f0c713db740 20 client.26009 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.481+0000 7f0c713db740 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x563489f9d6c0 0x563489f9fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.481+0000 7f0c713db740 4 client.26009 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.481+0000 7f0c713db740 1 -- 172.21.15.12:0/3042315488 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563489e8b410 con 0x563489f9d6c0 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.481+0000 7f0c677fe640 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x563489f9d6c0 0x563489f9fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.481+0000 7f0c677fe640 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x563489f9d6c0 0x563489f9fae0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7447d0595c61c5e4 server_cookie=466968864af6178e in_seq=0 out_seq=0 2024-05-09T09:27:53.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.481+0000 7f0c477fe640 10 client.26009 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:53.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.482+0000 7f0c477fe640 1 -- 172.21.15.12:0/3042315488 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563489e8b410 con 0x563489f9d6c0 2024-05-09T09:27:53.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.482+0000 7f0c477fe640 10 client.26009 handle_command_reply: tid=0 2024-05-09T09:27:53.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.482+0000 7f0c713db740 1 client.26009 shutdown 2024-05-09T09:27:53.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.482+0000 7f0c713db740 1 -- 172.21.15.12:0/3042315488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f1f4c0 msgr2=0x563489f94180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.483+0000 7f0c713db740 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f1f4c0 0x563489f94180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0c580057a0 tx=0x7f0c58004870 comp rx=0 tx=0).stop 2024-05-09T09:27:53.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.483+0000 7f0c713db740 1 -- 172.21.15.12:0/3042315488 shutdown_connections 2024-05-09T09:27:53.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.483+0000 7f0c713db740 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x563489f9d6c0 0x563489f9fae0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.483+0000 7f0c713db740 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563489f25020 0x563489f9b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.483+0000 7f0c477fe640 0 client.26009 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:53.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.483+0000 7f0c713db740 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563489f1feb0 0x563489f946c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.483+0000 7f0c713db740 1 --2- 172.21.15.12:0/3042315488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563489f1f4c0 0x563489f94180 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.483+0000 7f0c713db740 1 -- 172.21.15.12:0/3042315488 >> 172.21.15.12:0/3042315488 conn(0x563489ee70a0 msgr2=0x563489f1df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.483+0000 7f0c713db740 1 -- 172.21.15.12:0/3042315488 shutdown_connections 2024-05-09T09:27:53.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.483+0000 7f0c713db740 1 -- 172.21.15.12:0/3042315488 wait complete. 2024-05-09T09:27:53.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.483+0000 7f0c713db740 20 client.26009 trim_cache size 0 max 16384 2024-05-09T09:27:53.489 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:27:53.489 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T09:27:53.489 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "4dc2d38a-1d3a-464a-8d5b-5f06a8f4958d", 2024-05-09T09:27:53.489 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T09:27:53.489 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:27:53.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.484+0000 7f0c6e5ee640 1 -- 172.21.15.12:0/2741551902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0c500a2b90 msgr2=0x7f0c500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.484+0000 7f0c6e5ee640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0c500a2b90 0x7f0c500a5050 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x563489f202b0 tx=0x7f0c58031510 comp rx=0 tx=0).stop 2024-05-09T09:27:53.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.484+0000 7f0c6e5ee640 1 -- 172.21.15.12:0/2741551902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68101960 msgr2=0x7f0c68109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.484+0000 7f0c6e5ee640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68101960 0x7f0c68109730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c005fd0 tx=0x7f0c5c00dd20 comp rx=0 tx=0).stop 2024-05-09T09:27:53.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.484+0000 7f0c6e5ee640 1 -- 172.21.15.12:0/2741551902 shutdown_connections 2024-05-09T09:27:53.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.484+0000 7f0c6e5ee640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f0c500a2b90 0x7f0c500a5050 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.484+0000 7f0c6e5ee640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c68102320 0x7f0c68109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.484+0000 7f0c6e5ee640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68101960 0x7f0c68109730 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.484+0000 7f0c6e5ee640 1 --2- 172.21.15.12:0/2741551902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c68100770 0x7f0c6810b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.484+0000 7f0c6e5ee640 1 -- 172.21.15.12:0/2741551902 >> 172.21.15.12:0/2741551902 conn(0x7f0c680fbce0 msgr2=0x7f0c68104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.484+0000 7f0c6e5ee640 1 -- 172.21.15.12:0/2741551902 shutdown_connections 2024-05-09T09:27:53.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.484+0000 7f0c6e5ee640 1 -- 172.21.15.12:0/2741551902 wait complete. 2024-05-09T09:27:53.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:27:53.496 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:27:53.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.628+0000 7f521a716640 1 -- 172.21.15.12:0/4251983790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5214100770 msgr2=0x7f5214100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.628+0000 7f521a716640 1 --2- 172.21.15.12:0/4251983790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5214100770 0x7f5214100b90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5204009900 tx=0x7f520402a410 comp rx=0 tx=0).stop 2024-05-09T09:27:53.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.628+0000 7f521a716640 1 -- 172.21.15.12:0/4251983790 shutdown_connections 2024-05-09T09:27:53.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.628+0000 7f521a716640 1 --2- 172.21.15.12:0/4251983790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5214102320 0x7f5214108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.628+0000 7f521a716640 1 --2- 172.21.15.12:0/4251983790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214101960 0x7f5214101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.628+0000 7f521a716640 1 --2- 172.21.15.12:0/4251983790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5214100770 0x7f5214100b90 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.628+0000 7f521a716640 1 -- 172.21.15.12:0/4251983790 >> 172.21.15.12:0/4251983790 conn(0x7f52140fbce0 msgr2=0x7f52140fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.628+0000 7f521a716640 1 -- 172.21.15.12:0/4251983790 shutdown_connections 2024-05-09T09:27:53.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.628+0000 7f521a716640 1 -- 172.21.15.12:0/4251983790 wait complete. 2024-05-09T09:27:53.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.629+0000 7f521a716640 1 Processor -- start 2024-05-09T09:27:53.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.629+0000 7f521a716640 1 -- start start 2024-05-09T09:27:53.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.629+0000 7f521a716640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5214100770 0x7f521419e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.629+0000 7f521a716640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214101960 0x7f521419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.629+0000 7f521a716640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5214102320 0x7f52141a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.629+0000 7f521a716640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f521410d4d0 con 0x7f5214102320 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.629+0000 7f521a716640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f521410d350 con 0x7f5214101960 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.629+0000 7f521a716640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f521410d650 con 0x7f5214100770 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.630+0000 7f5218cb3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5214102320 0x7f52141a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.630+0000 7f5218cb3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5214102320 0x7f52141a57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46354/0 (socket says 172.21.15.12:46354) 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.630+0000 7f5213fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5214100770 0x7f521419e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.630+0000 7f5218cb3640 1 -- 172.21.15.12:0/1119533370 learned_addr learned my addr 172.21.15.12:0/1119533370 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.630+0000 7f52137fe640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214101960 0x7f521419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.630+0000 7f5213fff640 1 -- 172.21.15.12:0/1119533370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214101960 msgr2=0x7f521419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.630+0000 7f5213fff640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214101960 0x7f521419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.630+0000 7f5213fff640 1 -- 172.21.15.12:0/1119533370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5214102320 msgr2=0x7f52141a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.630+0000 7f5213fff640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5214102320 0x7f52141a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.630+0000 7f5213fff640 1 -- 172.21.15.12:0/1119533370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52141a5e40 con 0x7f5214100770 2024-05-09T09:27:53.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.630+0000 7f52137fe640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214101960 0x7f521419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:53.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.630+0000 7f5218cb3640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5214102320 0x7f52141a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:53.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.631+0000 7f5213fff640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5214100770 0x7f521419e220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5204009230 tx=0x7f52040042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.631+0000 7f52117fa640 1 -- 172.21.15.12:0/1119533370 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520403f030 con 0x7f5214100770 2024-05-09T09:27:53.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.631+0000 7f52117fa640 1 -- 172.21.15.12:0/1119533370 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5204031980 con 0x7f5214100770 2024-05-09T09:27:53.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.631+0000 7f521a716640 1 -- 172.21.15.12:0/1119533370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52141a60d0 con 0x7f5214100770 2024-05-09T09:27:53.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.631+0000 7f52117fa640 1 -- 172.21.15.12:0/1119533370 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204035070 con 0x7f5214100770 2024-05-09T09:27:53.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.631+0000 7f521a716640 1 -- 172.21.15.12:0/1119533370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52141a65e0 con 0x7f5214100770 2024-05-09T09:27:53.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.634+0000 7f521d419740 1 Processor -- start 2024-05-09T09:27:53.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.634+0000 7f521d419740 1 -- start start 2024-05-09T09:27:53.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.634+0000 7f52117fa640 1 -- 172.21.15.12:0/1119533370 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5204042030 con 0x7f5214100770 2024-05-09T09:27:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.634+0000 7f521d419740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3d4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.634+0000 7f521d419740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d4920 0x559f7f3d9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.634+0000 7f521d419740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d9a90 0x559f7f3dbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.634+0000 7f521d419740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559f7f338d00 con 0x559f7f3d4920 2024-05-09T09:27:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.634+0000 7f521d419740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f7f2e04c0 con 0x559f7f3d9a90 2024-05-09T09:27:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.634+0000 7f521d419740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559f7f334cc0 con 0x559f7f3d3f30 2024-05-09T09:27:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f5213fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d4920 0x559f7f3d9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f52137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d9a90 0x559f7f3dbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f5213fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d4920 0x559f7f3d9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46370/0 (socket says 172.21.15.12:46370) 2024-05-09T09:27:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f52137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d9a90 0x559f7f3dbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41256/0 (socket says 172.21.15.12:41256) 2024-05-09T09:27:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f5213fff640 1 -- 172.21.15.12:0/2597656253 learned_addr learned my addr 172.21.15.12:0/2597656253 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f5218cb3640 1 --2- 172.21.15.12:0/2597656253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f52137fe640 1 -- 172.21.15.12:0/2597656253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 msgr2=0x559f7f3d4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f52137fe640 1 --2- 172.21.15.12:0/2597656253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3d4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f52137fe640 1 -- 172.21.15.12:0/2597656253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d4920 msgr2=0x559f7f3d9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f52137fe640 1 --2- 172.21.15.12:0/2597656253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d4920 0x559f7f3d9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f52137fe640 1 -- 172.21.15.12:0/2597656253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7f1167d0 con 0x559f7f3d9a90 2024-05-09T09:27:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f5218cb3640 1 --2- 172.21.15.12:0/2597656253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3d4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f52137fe640 1 --2- 172.21.15.12:0/2597656253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d9a90 0x559f7f3dbf20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5208009fa0 tx=0x7f52080118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d939aab64d0a2b90 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f52117fa640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f51fc0a2c10 0x7f51fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f5218cb3640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f51fc0a2c10 0x7f51fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.635+0000 7f51fa7fc640 1 -- 172.21.15.12:0/2597656253 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208004030 con 0x559f7f3d9a90 2024-05-09T09:27:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.636+0000 7f52117fa640 1 -- 172.21.15.12:0/1119533370 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f52040ef7f0 con 0x7f5214100770 2024-05-09T09:27:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.636+0000 7f51fa7fc640 1 -- 172.21.15.12:0/2597656253 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f52080054f0 con 0x559f7f3d9a90 2024-05-09T09:27:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.636+0000 7f51fa7fc640 1 -- 172.21.15.12:0/2597656253 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208004d70 con 0x559f7f3d9a90 2024-05-09T09:27:53.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.636+0000 7f521d419740 1 -- 172.21.15.12:0/2597656253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d9a90 msgr2=0x559f7f3dbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.636+0000 7f521d419740 1 --2- 172.21.15.12:0/2597656253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d9a90 0x559f7f3dbf20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5208009fa0 tx=0x7f52080118d0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.636+0000 7f5218cb3640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f51fc0a2c10 0x7f51fc0a50d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x559f7f3d5180 tx=0x7f520c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.636+0000 7f521d419740 1 -- 172.21.15.12:0/2597656253 shutdown_connections 2024-05-09T09:27:53.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.636+0000 7f521d419740 1 --2- 172.21.15.12:0/2597656253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d9a90 0x559f7f3dbf20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.636+0000 7f521d419740 1 --2- 172.21.15.12:0/2597656253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d4920 0x559f7f3d9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.636+0000 7f521d419740 1 --2- 172.21.15.12:0/2597656253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3d4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.636+0000 7f521d419740 1 -- 172.21.15.12:0/2597656253 >> 172.21.15.12:0/2597656253 conn(0x559f7f3cf5a0 msgr2=0x559f7f3cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.637+0000 7f521d419740 1 -- 172.21.15.12:0/2597656253 shutdown_connections 2024-05-09T09:27:53.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.637+0000 7f521d419740 1 -- 172.21.15.12:0/2597656253 wait complete. 2024-05-09T09:27:53.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.637+0000 7f521d419740 1 -- start start 2024-05-09T09:27:53.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.637+0000 7f521d419740 1 Processor -- start 2024-05-09T09:27:53.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f521d419740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d3f30 0x559f7f3de380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f521d419740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d4920 0x559f7f3de8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f521d419740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d9a90 0x559f7f3e5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f521d419740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559f7f3dca40 con 0x559f7f3d9a90 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f521d419740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f7f2f3b90 con 0x559f7f3d3f30 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f521d419740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559f7f3dcbc0 con 0x559f7f3d4920 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f521d419740 10 client.0 _ll_register_callbacks cb 0x559f7f32fda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f5213fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d3f30 0x559f7f3de380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f5218cb3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d9a90 0x559f7f3e5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f5213fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d3f30 0x559f7f3de380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41266/0 (socket says 172.21.15.12:41266) 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f5213fff640 1 -- 172.21.15.12:0/1049683789 learned_addr learned my addr 172.21.15.12:0/1049683789 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f52137fe640 1 --2- 172.21.15.12:0/1049683789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d4920 0x559f7f3de8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f5213fff640 1 -- 172.21.15.12:0/1049683789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d4920 msgr2=0x559f7f3de8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.638+0000 7f5213fff640 1 --2- 172.21.15.12:0/1049683789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d4920 0x559f7f3de8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.639+0000 7f5213fff640 1 -- 172.21.15.12:0/1049683789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d9a90 msgr2=0x559f7f3e5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.639+0000 7f5213fff640 1 --2- 172.21.15.12:0/1049683789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d9a90 0x559f7f3e5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.639+0000 7f5213fff640 1 -- 172.21.15.12:0/1049683789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7f04c3f0 con 0x559f7f3d3f30 2024-05-09T09:27:53.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.639+0000 7f5218cb3640 1 --2- 172.21.15.12:0/1049683789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d9a90 0x559f7f3e5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:53.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.640+0000 7f5213fff640 1 --2- 172.21.15.12:0/1049683789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d3f30 0x559f7f3de380 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f52040bb9d0 tx=0x7f5204009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7246c2483de0ffec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.640+0000 7f51f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:27:53.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.640+0000 7f51f37fe640 1 -- 172.21.15.12:0/1049683789 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204049070 con 0x559f7f3d3f30 2024-05-09T09:27:53.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.640+0000 7f521d419740 1 -- 172.21.15.12:0/1049683789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52080054f0 con 0x559f7f3d3f30 2024-05-09T09:27:53.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.640+0000 7f51f37fe640 1 -- 172.21.15.12:0/1049683789 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f52040b8340 con 0x559f7f3d3f30 2024-05-09T09:27:53.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.641+0000 7f51f37fe640 1 -- 172.21.15.12:0/1049683789 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f52040044e0 con 0x559f7f3d3f30 2024-05-09T09:27:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.641+0000 7f521d419740 10 client.26021 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.641+0000 7f521d419740 1 -- 172.21.15.12:0/1049683789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559f7f00c0f0 con 0x559f7f3d3f30 2024-05-09T09:27:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.641+0000 7f51f37fe640 1 -- 172.21.15.12:0/1049683789 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5204004780 con 0x559f7f3d3f30 2024-05-09T09:27:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.641+0000 7f51f37fe640 1 -- 172.21.15.12:0/1049683789 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f52040bbf30 con 0x559f7f3d3f30 2024-05-09T09:27:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.641+0000 7f521d419740 10 client.26021 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.641+0000 7f521d419740 10 client.26021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.641+0000 7f521d419740 20 client.26021 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.641+0000 7f521d419740 1 --2- 172.21.15.12:0/1049683789 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559f7f3ea2b0 0x559f7f3ec770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.642+0000 7f521d419740 4 client.26021 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.642+0000 7f521d419740 1 -- 172.21.15.12:0/1049683789 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f52080054f0 con 0x559f7f3ea2b0 2024-05-09T09:27:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.642+0000 7f52137fe640 1 --2- 172.21.15.12:0/1049683789 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559f7f3ea2b0 0x559f7f3ec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.643+0000 7f52137fe640 1 --2- 172.21.15.12:0/1049683789 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559f7f3ea2b0 0x559f7f3ec770 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c328c05646b997 server_cookie=700113736d457505 in_seq=0 out_seq=0 2024-05-09T09:27:53.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.643+0000 7f51f37fe640 10 client.26021 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:53.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.644+0000 7f51f37fe640 1 -- 172.21.15.12:0/1049683789 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f52080054f0 con 0x559f7f3ea2b0 2024-05-09T09:27:53.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.644+0000 7f51f37fe640 10 client.26021 handle_command_reply: tid=0 2024-05-09T09:27:53.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.644+0000 7f521d419740 1 client.26021 shutdown 2024-05-09T09:27:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.645+0000 7f521d419740 1 -- 172.21.15.12:0/1049683789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d3f30 msgr2=0x559f7f3de380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.645+0000 7f521d419740 1 --2- 172.21.15.12:0/1049683789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d3f30 0x559f7f3de380 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f52040bb9d0 tx=0x7f5204009680 comp rx=0 tx=0).stop 2024-05-09T09:27:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.645+0000 7f521d419740 1 -- 172.21.15.12:0/1049683789 shutdown_connections 2024-05-09T09:27:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.645+0000 7f521d419740 1 --2- 172.21.15.12:0/1049683789 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559f7f3ea2b0 0x559f7f3ec770 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.645+0000 7f521d419740 1 --2- 172.21.15.12:0/1049683789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d9a90 0x559f7f3e5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.645+0000 7f521d419740 1 --2- 172.21.15.12:0/1049683789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d4920 0x559f7f3de8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.645+0000 7f521d419740 1 --2- 172.21.15.12:0/1049683789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d3f30 0x559f7f3de380 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.645+0000 7f51f37fe640 0 client.26021 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.645+0000 7f521d419740 1 -- 172.21.15.12:0/1049683789 >> 172.21.15.12:0/1049683789 conn(0x559f7f3cf5a0 msgr2=0x559f7f3ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.645+0000 7f521d419740 1 -- 172.21.15.12:0/1049683789 shutdown_connections 2024-05-09T09:27:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.645+0000 7f521d419740 1 -- 172.21.15.12:0/1049683789 wait complete. 2024-05-09T09:27:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.645+0000 7f521d419740 20 client.26021 trim_cache size 0 max 16384 2024-05-09T09:27:53.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.676+0000 7f521d419740 1 Processor -- start 2024-05-09T09:27:53.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.676+0000 7f521d419740 1 -- start start 2024-05-09T09:27:53.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.676+0000 7f521d419740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3d5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.676+0000 7f521d419740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d4920 0x559f7f3e58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.676+0000 7f521d419740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d9a90 0x559f7f3d6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f521d419740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f52141b0cf0 con 0x559f7f3d4920 2024-05-09T09:27:53.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f521d419740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52141b28f0 con 0x559f7f3d9a90 2024-05-09T09:27:53.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f521d419740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f52141b13c0 con 0x559f7f3d3f30 2024-05-09T09:27:53.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f52137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d4920 0x559f7f3e58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f52137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d4920 0x559f7f3e58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46388/0 (socket says 172.21.15.12:46388) 2024-05-09T09:27:53.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f52137fe640 1 -- 172.21.15.12:0/2317215680 learned_addr learned my addr 172.21.15.12:0/2317215680 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f5218cb3640 1 --2- 172.21.15.12:0/2317215680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d9a90 0x559f7f3d6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f5213fff640 1 --2- 172.21.15.12:0/2317215680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3d5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f52137fe640 1 -- 172.21.15.12:0/2317215680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 msgr2=0x559f7f3d5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f52137fe640 1 --2- 172.21.15.12:0/2317215680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3d5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f52137fe640 1 -- 172.21.15.12:0/2317215680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d9a90 msgr2=0x559f7f3d6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f52137fe640 1 --2- 172.21.15.12:0/2317215680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d9a90 0x559f7f3d6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f52137fe640 1 -- 172.21.15.12:0/2317215680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7f3cf3d0 con 0x559f7f3d4920 2024-05-09T09:27:53.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.677+0000 7f52137fe640 1 --2- 172.21.15.12:0/2317215680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d4920 0x559f7f3e58c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5208009fd0 tx=0x7f52080096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a881f8d537feadd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.678+0000 7f51f9ffb640 1 -- 172.21.15.12:0/2317215680 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520802c030 con 0x559f7f3d4920 2024-05-09T09:27:53.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.678+0000 7f51f9ffb640 1 -- 172.21.15.12:0/2317215680 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f520801e040 con 0x559f7f3d4920 2024-05-09T09:27:53.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.678+0000 7f51f9ffb640 1 -- 172.21.15.12:0/2317215680 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208028e60 con 0x559f7f3d4920 2024-05-09T09:27:53.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.679+0000 7f521d419740 1 -- 172.21.15.12:0/2317215680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d4920 msgr2=0x559f7f3e58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.679+0000 7f521d419740 1 --2- 172.21.15.12:0/2317215680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d4920 0x559f7f3e58c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5208009fd0 tx=0x7f52080096c0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.679+0000 7f521d419740 1 -- 172.21.15.12:0/2317215680 shutdown_connections 2024-05-09T09:27:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.679+0000 7f521d419740 1 --2- 172.21.15.12:0/2317215680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3d9a90 0x559f7f3d6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.679+0000 7f521d419740 1 --2- 172.21.15.12:0/2317215680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d4920 0x559f7f3e58c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.679+0000 7f521d419740 1 --2- 172.21.15.12:0/2317215680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3d5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.679+0000 7f521d419740 1 -- 172.21.15.12:0/2317215680 >> 172.21.15.12:0/2317215680 conn(0x559f7f3cf5a0 msgr2=0x559f7f3ec2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.679+0000 7f521d419740 1 -- 172.21.15.12:0/2317215680 shutdown_connections 2024-05-09T09:27:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.679+0000 7f521d419740 1 -- 172.21.15.12:0/2317215680 wait complete. 2024-05-09T09:27:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.679+0000 7f521d419740 1 -- start start 2024-05-09T09:27:53.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.680+0000 7f521d419740 1 Processor -- start 2024-05-09T09:27:53.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.680+0000 7f521d419740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3e23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.680+0000 7f521d419740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d9a90 0x559f7f3e2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.680+0000 7f521d419740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3e58f0 0x559f7f44e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.680+0000 7f52137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d9a90 0x559f7f3e2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.680+0000 7f5213fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3e23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.680+0000 7f521d419740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f52141b2220 con 0x559f7f3d9a90 2024-05-09T09:27:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.680+0000 7f521d419740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52141b1a90 con 0x559f7f3e58f0 2024-05-09T09:27:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.680+0000 7f52137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d9a90 0x559f7f3e2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46392/0 (socket says 172.21.15.12:46392) 2024-05-09T09:27:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.680+0000 7f521d419740 1 -- 172.21.15.12:0/1853320955 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f52141aecd0 con 0x559f7f3d3f30 2024-05-09T09:27:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.680+0000 7f52137fe640 1 -- 172.21.15.12:0/1853320955 learned_addr learned my addr 172.21.15.12:0/1853320955 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.680+0000 7f5213fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3e23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:33648/0 (socket says 172.21.15.12:33648) 2024-05-09T09:27:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.680+0000 7f521d419740 10 client.0 _ll_register_callbacks cb 0x7f5204009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.681+0000 7f5218cb3640 1 --2- 172.21.15.12:0/1853320955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3e58f0 0x559f7f44e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.681+0000 7f52137fe640 1 -- 172.21.15.12:0/1853320955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 msgr2=0x559f7f3e23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.681+0000 7f52137fe640 1 --2- 172.21.15.12:0/1853320955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3e23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.681+0000 7f52137fe640 1 -- 172.21.15.12:0/1853320955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3e58f0 msgr2=0x559f7f44e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.681+0000 7f52137fe640 1 --2- 172.21.15.12:0/1853320955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3e58f0 0x559f7f44e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.681+0000 7f52137fe640 1 -- 172.21.15.12:0/1853320955 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7f44ede0 con 0x559f7f3d9a90 2024-05-09T09:27:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.681+0000 7f52137fe640 1 --2- 172.21.15.12:0/1853320955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d9a90 0x559f7f3e2920 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5208009f70 tx=0x7f5208011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7c4af1d648e56b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.681+0000 7f51f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.681+0000 7f51f37fe640 1 -- 172.21.15.12:0/1853320955 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520802c040 con 0x559f7f3d9a90 2024-05-09T09:27:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.681+0000 7f51f37fe640 1 -- 172.21.15.12:0/1853320955 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f52080280b0 con 0x559f7f3d9a90 2024-05-09T09:27:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.681+0000 7f521d419740 1 -- 172.21.15.12:0/1853320955 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f520801e040 con 0x559f7f3d9a90 2024-05-09T09:27:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.682+0000 7f51f37fe640 1 -- 172.21.15.12:0/1853320955 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5208040050 con 0x559f7f3d9a90 2024-05-09T09:27:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.682+0000 7f521d419740 10 client.16584 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.682+0000 7f521d419740 1 -- 172.21.15.12:0/1853320955 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559f7f450060 con 0x559f7f3d9a90 2024-05-09T09:27:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.682+0000 7f51f37fe640 1 -- 172.21.15.12:0/1853320955 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f520801e040 con 0x559f7f3d9a90 2024-05-09T09:27:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.682+0000 7f51f37fe640 1 -- 172.21.15.12:0/1853320955 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f520800f810 con 0x559f7f3d9a90 2024-05-09T09:27:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.682+0000 7f521d419740 10 client.16584 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.682+0000 7f521d419740 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.682+0000 7f521d419740 20 client.16584 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.682+0000 7f521d419740 1 --2- 172.21.15.12:0/1853320955 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559f7f4507c0 0x559f7f452be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.682+0000 7f521d419740 4 client.16584 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:27:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.682+0000 7f521d419740 1 -- 172.21.15.12:0/1853320955 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f7f346430 con 0x559f7f4507c0 2024-05-09T09:27:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.683+0000 7f5213fff640 1 --2- 172.21.15.12:0/1853320955 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559f7f4507c0 0x559f7f452be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.683+0000 7f5213fff640 1 --2- 172.21.15.12:0/1853320955 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559f7f4507c0 0x559f7f452be0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7578f8485c069f33 server_cookie=a7f84e631de60b1a in_seq=0 out_seq=0 2024-05-09T09:27:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.683+0000 7f51f37fe640 10 client.16584 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.684+0000 7f51f37fe640 1 -- 172.21.15.12:0/1853320955 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x559f7f346430 con 0x559f7f4507c0 2024-05-09T09:27:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.684+0000 7f51f37fe640 10 client.16584 handle_command_reply: tid=0 2024-05-09T09:27:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.684+0000 7f521d419740 1 client.16584 shutdown 2024-05-09T09:27:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.684+0000 7f521d419740 1 -- 172.21.15.12:0/1853320955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d9a90 msgr2=0x559f7f3e2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.684+0000 7f521d419740 1 --2- 172.21.15.12:0/1853320955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d9a90 0x559f7f3e2920 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5208009f70 tx=0x7f5208011e50 comp rx=0 tx=0).stop 2024-05-09T09:27:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f521d419740 1 -- 172.21.15.12:0/1853320955 shutdown_connections 2024-05-09T09:27:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f521d419740 1 --2- 172.21.15.12:0/1853320955 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559f7f4507c0 0x559f7f452be0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f521d419740 1 --2- 172.21.15.12:0/1853320955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f7f3e58f0 0x559f7f44e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f521d419740 1 --2- 172.21.15.12:0/1853320955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f7f3d9a90 0x559f7f3e2920 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f521d419740 1 --2- 172.21.15.12:0/1853320955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f3d3f30 0x559f7f3e23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f51f37fe640 0 client.16584 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f521d419740 1 -- 172.21.15.12:0/1853320955 >> 172.21.15.12:0/1853320955 conn(0x559f7f3cf5a0 msgr2=0x559f7f3d5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f521d419740 1 -- 172.21.15.12:0/1853320955 shutdown_connections 2024-05-09T09:27:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f521d419740 1 -- 172.21.15.12:0/1853320955 wait complete. 2024-05-09T09:27:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f521d419740 20 client.16584 trim_cache size 0 max 16384 2024-05-09T09:27:53.689 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:27:53.689 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (202 inodes in the stack)", 2024-05-09T09:27:53.689 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T09:27:53.689 INFO:teuthology.orchestra.run.smithi012.stdout: "4dc2d38a-1d3a-464a-8d5b-5f06a8f4958d": { 2024-05-09T09:27:53.689 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T09:27:53.689 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "4dc2d38a-1d3a-464a-8d5b-5f06a8f4958d", 2024-05-09T09:27:53.689 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T09:27:53.689 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:27:53.689 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:27:53.689 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:27:53.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f521a716640 1 -- 172.21.15.12:0/1119533370 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f51fc0a2c10 msgr2=0x7f51fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f521a716640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f51fc0a2c10 0x7f51fc0a50d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x559f7f3d5180 tx=0x7f520c009500 comp rx=0 tx=0).stop 2024-05-09T09:27:53.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f521a716640 1 -- 172.21.15.12:0/1119533370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5214100770 msgr2=0x7f521419e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.685+0000 7f521a716640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5214100770 0x7f521419e220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5204009230 tx=0x7f52040042f0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.686+0000 7f521a716640 1 -- 172.21.15.12:0/1119533370 shutdown_connections 2024-05-09T09:27:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.686+0000 7f521a716640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f51fc0a2c10 0x7f51fc0a50d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.686+0000 7f521a716640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5214102320 0x7f52141a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.686+0000 7f521a716640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214101960 0x7f521419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.686+0000 7f521a716640 1 --2- 172.21.15.12:0/1119533370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5214100770 0x7f521419e220 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.686+0000 7f521a716640 1 -- 172.21.15.12:0/1119533370 >> 172.21.15.12:0/1119533370 conn(0x7f52140fbce0 msgr2=0x7f52141099f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.686+0000 7f521a716640 1 -- 172.21.15.12:0/1119533370 shutdown_connections 2024-05-09T09:27:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:27:53.686+0000 7f521a716640 1 -- 172.21.15.12:0/1119533370 wait complete. 2024-05-09T09:27:53.700 INFO:tasks.cephfs.filesystem:scrub status for tag:4dc2d38a-1d3a-464a-8d5b-5f06a8f4958d - {'path': '/', 'tag': '4dc2d38a-1d3a-464a-8d5b-5f06a8f4958d', 'options': 'recursive,force'} 2024-05-09T09:27:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.948+0000 7f8522ffb640 1 -- 172.21.15.12:0/2256603951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c102340 msgr2=0x7f851c108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.948+0000 7f8522ffb640 1 --2- 172.21.15.12:0/2256603951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c102340 0x7f851c108bd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8514009be0 tx=0x7f851402c650 comp rx=0 tx=0).stop 2024-05-09T09:27:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.948+0000 7f8522ffb640 1 -- 172.21.15.12:0/2256603951 shutdown_connections 2024-05-09T09:27:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.948+0000 7f8522ffb640 1 --2- 172.21.15.12:0/2256603951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c102340 0x7f851c108bd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.948+0000 7f8522ffb640 1 --2- 172.21.15.12:0/2256603951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f851c101980 0x7f851c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.948+0000 7f8522ffb640 1 --2- 172.21.15.12:0/2256603951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f851c100790 0x7f851c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.948+0000 7f8522ffb640 1 -- 172.21.15.12:0/2256603951 >> 172.21.15.12:0/2256603951 conn(0x7f851c0fbd00 msgr2=0x7f851c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.948+0000 7f8522ffb640 1 -- 172.21.15.12:0/2256603951 shutdown_connections 2024-05-09T09:27:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.948+0000 7f8522ffb640 1 -- 172.21.15.12:0/2256603951 wait complete. 2024-05-09T09:27:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.949+0000 7f8522ffb640 1 Processor -- start 2024-05-09T09:27:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.949+0000 7f8522ffb640 1 -- start start 2024-05-09T09:27:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.949+0000 7f8522ffb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f851c100790 0x7f851c199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.949+0000 7f8522ffb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f851c101980 0x7f851c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.950+0000 7f8522ffb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c102340 0x7f851c1a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.950+0000 7f8522ffb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f851c10f6f0 con 0x7f851c100790 2024-05-09T09:27:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.950+0000 7f8522ffb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f851c10f570 con 0x7f851c101980 2024-05-09T09:27:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.950+0000 7f8522ffb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f851c10f870 con 0x7f851c102340 2024-05-09T09:27:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.950+0000 7f8513fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f851c101980 0x7f851c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.950+0000 7f8521598640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c102340 0x7f851c1a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.950+0000 7f8513fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f851c101980 0x7f851c19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41320/0 (socket says 172.21.15.12:41320) 2024-05-09T09:27:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.950+0000 7f8521598640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c102340 0x7f851c1a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:33660/0 (socket says 172.21.15.12:33660) 2024-05-09T09:27:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.950+0000 7f8513fff640 1 -- 172.21.15.12:0/1897227636 learned_addr learned my addr 172.21.15.12:0/1897227636 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.950+0000 7f8520d97640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f851c100790 0x7f851c199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.951+0000 7f8513fff640 1 -- 172.21.15.12:0/1897227636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c102340 msgr2=0x7f851c1a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.951+0000 7f8513fff640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c102340 0x7f851c1a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.951+0000 7f8513fff640 1 -- 172.21.15.12:0/1897227636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f851c100790 msgr2=0x7f851c199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.951+0000 7f8513fff640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f851c100790 0x7f851c199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.951+0000 7f8513fff640 1 -- 172.21.15.12:0/1897227636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c06b380 con 0x7f851c101980 2024-05-09T09:27:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.951+0000 7f8520d97640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f851c100790 0x7f851c199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.951+0000 7f8513fff640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f851c101980 0x7f851c19a240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e9c0 tx=0x7f850c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.951+0000 7f8511ffb640 1 -- 172.21.15.12:0/1897227636 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c00ccc0 con 0x7f851c101980 2024-05-09T09:27:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.951+0000 7f8522ffb640 1 -- 172.21.15.12:0/1897227636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851c06b670 con 0x7f851c101980 2024-05-09T09:27:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.951+0000 7f8511ffb640 1 -- 172.21.15.12:0/1897227636 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f850c0054f0 con 0x7f851c101980 2024-05-09T09:27:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.951+0000 7f8511ffb640 1 -- 172.21.15.12:0/1897227636 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c010430 con 0x7f851c101980 2024-05-09T09:27:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.952+0000 7f8522ffb640 1 -- 172.21.15.12:0/1897227636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851c06bc10 con 0x7f851c101980 2024-05-09T09:27:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.954+0000 7f8511ffb640 1 -- 172.21.15.12:0/1897227636 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f850c004f90 con 0x7f851c101980 2024-05-09T09:27:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.955+0000 7f8525ddf740 1 Processor -- start 2024-05-09T09:27:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.955+0000 7f8525ddf740 1 -- start start 2024-05-09T09:27:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.955+0000 7f8525ddf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228b93670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.955+0000 7f8525ddf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b93c40 0x55d228b98740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.955+0000 7f8525ddf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b98db0 0x55d228b9b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.955+0000 7f8525ddf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d228af7340 con 0x55d228b93250 2024-05-09T09:27:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.955+0000 7f8525ddf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d228a9f290 con 0x55d228b98db0 2024-05-09T09:27:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.955+0000 7f8525ddf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d228af3300 con 0x55d228b93c40 2024-05-09T09:27:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.955+0000 7f8521598640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228b93670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.955+0000 7f8521598640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228b93670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46428/0 (socket says 172.21.15.12:46428) 2024-05-09T09:27:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.955+0000 7f8513fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b98db0 0x55d228b9b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.955+0000 7f8521598640 1 -- 172.21.15.12:0/4238960761 learned_addr learned my addr 172.21.15.12:0/4238960761 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.955+0000 7f8520d97640 1 --2- 172.21.15.12:0/4238960761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b93c40 0x55d228b98740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.956+0000 7f8521598640 1 -- 172.21.15.12:0/4238960761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b93c40 msgr2=0x55d228b98740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.956+0000 7f8521598640 1 --2- 172.21.15.12:0/4238960761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b93c40 0x55d228b98740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.956+0000 7f8521598640 1 -- 172.21.15.12:0/4238960761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b98db0 msgr2=0x55d228b9b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.956+0000 7f8521598640 1 --2- 172.21.15.12:0/4238960761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b98db0 0x55d228b9b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.956+0000 7f8511ffb640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85040a2d00 0x7f85040a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.956+0000 7f8521598640 1 -- 172.21.15.12:0/4238960761 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d228ac38e0 con 0x55d228b93250 2024-05-09T09:27:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.956+0000 7f8520d97640 1 --2- 172.21.15.12:0/4238960761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b93c40 0x55d228b98740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.956+0000 7f8513fff640 1 --2- 172.21.15.12:0/4238960761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b98db0 0x55d228b9b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.956+0000 7f8520d97640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85040a2d00 0x7f85040a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.956+0000 7f8511ffb640 1 -- 172.21.15.12:0/1897227636 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f850c0cf800 con 0x7f851c101980 2024-05-09T09:27:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.956+0000 7f8521598640 1 --2- 172.21.15.12:0/4238960761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228b93670 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8514009be0 tx=0x7f851400d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46678134d10a857d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.957+0000 7f8502ffd640 1 -- 172.21.15.12:0/4238960761 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514042020 con 0x55d228b93250 2024-05-09T09:27:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.957+0000 7f8502ffd640 1 -- 172.21.15.12:0/4238960761 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8514039040 con 0x55d228b93250 2024-05-09T09:27:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.957+0000 7f8502ffd640 1 -- 172.21.15.12:0/4238960761 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514038030 con 0x55d228b93250 2024-05-09T09:27:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.958+0000 7f8525ddf740 1 -- 172.21.15.12:0/4238960761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 msgr2=0x55d228b93670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.958+0000 7f8525ddf740 1 --2- 172.21.15.12:0/4238960761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228b93670 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8514009be0 tx=0x7f851400d350 comp rx=0 tx=0).stop 2024-05-09T09:27:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.958+0000 7f8520d97640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85040a2d00 0x7f85040a51c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8518009fd0 tx=0x7f8518005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.958+0000 7f8525ddf740 1 -- 172.21.15.12:0/4238960761 shutdown_connections 2024-05-09T09:27:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.958+0000 7f8525ddf740 1 --2- 172.21.15.12:0/4238960761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b98db0 0x55d228b9b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.958+0000 7f8525ddf740 1 --2- 172.21.15.12:0/4238960761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b93c40 0x55d228b98740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.958+0000 7f8525ddf740 1 --2- 172.21.15.12:0/4238960761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228b93670 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.958+0000 7f8525ddf740 1 -- 172.21.15.12:0/4238960761 >> 172.21.15.12:0/4238960761 conn(0x55d228b8e8c0 msgr2=0x55d228b8ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.958+0000 7f8525ddf740 1 -- 172.21.15.12:0/4238960761 shutdown_connections 2024-05-09T09:27:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.958+0000 7f8525ddf740 1 -- 172.21.15.12:0/4238960761 wait complete. 2024-05-09T09:27:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.958+0000 7f8525ddf740 1 -- start start 2024-05-09T09:27:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.958+0000 7f8525ddf740 1 Processor -- start 2024-05-09T09:27:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8525ddf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228b9d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8525ddf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b93c40 0x55d228b9d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8525ddf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b98db0 0x55d228ba4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8525ddf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d228b9bd00 con 0x55d228b93250 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8525ddf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d228ab2790 con 0x55d228b98db0 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8525ddf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d228b9be80 con 0x55d228b93c40 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8525ddf740 10 client.0 _ll_register_callbacks cb 0x55d228aeda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8520d97640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228b9d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8520d97640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228b9d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46440/0 (socket says 172.21.15.12:46440) 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8520d97640 1 -- 172.21.15.12:0/592382710 learned_addr learned my addr 172.21.15.12:0/592382710 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8513fff640 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b93c40 0x55d228b9d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8521598640 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b98db0 0x55d228ba4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8520d97640 1 -- 172.21.15.12:0/592382710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b93c40 msgr2=0x55d228b9d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8520d97640 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b93c40 0x55d228b9d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8520d97640 1 -- 172.21.15.12:0/592382710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b98db0 msgr2=0x55d228ba4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8520d97640 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b98db0 0x55d228ba4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8520d97640 1 -- 172.21.15.12:0/592382710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d22880b210 con 0x55d228b93250 2024-05-09T09:27:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.959+0000 7f8521598640 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b98db0 0x55d228ba4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.960+0000 7f8520d97640 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228b9d4a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8518012d70 tx=0x7f8518004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c99b773cd50770e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.960+0000 7f84fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.960+0000 7f84fbfff640 1 -- 172.21.15.12:0/592382710 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518011940 con 0x55d228b93250 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.960+0000 7f8525ddf740 1 -- 172.21.15.12:0/592382710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8514039040 con 0x55d228b93250 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.960+0000 7f84fbfff640 1 -- 172.21.15.12:0/592382710 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8518011ae0 con 0x55d228b93250 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.960+0000 7f84fbfff640 1 -- 172.21.15.12:0/592382710 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8518011ce0 con 0x55d228b93250 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.960+0000 7f8525ddf740 10 client.16596 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.960+0000 7f8525ddf740 1 -- 172.21.15.12:0/592382710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d2288d7620 con 0x55d228b93250 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.961+0000 7f84fbfff640 1 -- 172.21.15.12:0/592382710 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8514039040 con 0x55d228b93250 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.961+0000 7f84fbfff640 1 -- 172.21.15.12:0/592382710 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f8518027070 con 0x55d228b93250 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.961+0000 7f8525ddf740 10 client.16596 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.961+0000 7f8525ddf740 10 client.16596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.961+0000 7f8525ddf740 20 client.16596 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.961+0000 7f8525ddf740 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d228ba9510 0x55d228bab990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.961+0000 7f8525ddf740 4 client.16596 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:27:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.961+0000 7f8525ddf740 1 -- 172.21.15.12:0/592382710 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d228b04a70 con 0x55d228ba9510 2024-05-09T09:27:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.962+0000 7f8513fff640 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d228ba9510 0x55d228bab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.962+0000 7f8513fff640 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d228ba9510 0x55d228bab990 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8744847d299c996d server_cookie=e0b3ebe885bde355 in_seq=0 out_seq=0 2024-05-09T09:27:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.962+0000 7f84fbfff640 10 client.16596 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f84fbfff640 1 -- 172.21.15.12:0/592382710 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d228b04a70 con 0x55d228ba9510 2024-05-09T09:27:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f84fbfff640 10 client.16596 handle_command_reply: tid=0 2024-05-09T09:27:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f8525ddf740 1 client.16596 shutdown 2024-05-09T09:27:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f8525ddf740 1 -- 172.21.15.12:0/592382710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 msgr2=0x55d228b9d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f8525ddf740 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228b9d4a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8518012d70 tx=0x7f8518004730 comp rx=0 tx=0).stop 2024-05-09T09:27:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f8525ddf740 1 -- 172.21.15.12:0/592382710 shutdown_connections 2024-05-09T09:27:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f8525ddf740 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d228ba9510 0x55d228bab990 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f8525ddf740 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b98db0 0x55d228ba4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f84fbfff640 0 client.16596 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f8525ddf740 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b93c40 0x55d228b9d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f8525ddf740 1 --2- 172.21.15.12:0/592382710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228b9d4a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f8525ddf740 1 -- 172.21.15.12:0/592382710 >> 172.21.15.12:0/592382710 conn(0x55d228b8e8c0 msgr2=0x55d228b8da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f8525ddf740 1 -- 172.21.15.12:0/592382710 shutdown_connections 2024-05-09T09:27:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.963+0000 7f8525ddf740 1 -- 172.21.15.12:0/592382710 wait complete. 2024-05-09T09:27:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.964+0000 7f8525ddf740 20 client.16596 trim_cache size 0 max 16384 2024-05-09T09:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.983+0000 7f8525ddf740 1 Processor -- start 2024-05-09T09:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.983+0000 7f8525ddf740 1 -- start start 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.983+0000 7f8525ddf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b93250 0x55d228b9d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.983+0000 7f8525ddf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93c40 0x55d228b9d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.983+0000 7f8525ddf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b98db0 0x55d228b8d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.983+0000 7f8525ddf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f851c1b5020 con 0x55d228b93c40 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.983+0000 7f8525ddf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f851c1b6c20 con 0x55d228b93250 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.983+0000 7f8525ddf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f851c1b56f0 con 0x55d228b98db0 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.984+0000 7f8520d97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b93250 0x55d228b9d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.983+0000 7f8513fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93c40 0x55d228b9d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.984+0000 7f8520d97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b93250 0x55d228b9d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:41334/0 (socket says 172.21.15.12:41334) 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.984+0000 7f8520d97640 1 -- 172.21.15.12:0/783116327 learned_addr learned my addr 172.21.15.12:0/783116327 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.984+0000 7f8521598640 1 --2- 172.21.15.12:0/783116327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b98db0 0x55d228b8d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.984+0000 7f8520d97640 1 -- 172.21.15.12:0/783116327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b98db0 msgr2=0x55d228b8d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.984+0000 7f8520d97640 1 --2- 172.21.15.12:0/783116327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b98db0 0x55d228b8d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.984+0000 7f8520d97640 1 -- 172.21.15.12:0/783116327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93c40 msgr2=0x55d228b9d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.984+0000 7f8520d97640 1 --2- 172.21.15.12:0/783116327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93c40 0x55d228b9d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.984+0000 7f8520d97640 1 -- 172.21.15.12:0/783116327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d228b8cf00 con 0x55d228b93250 2024-05-09T09:27:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.984+0000 7f8513fff640 1 --2- 172.21.15.12:0/783116327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93c40 0x55d228b9d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:27:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.984+0000 7f8520d97640 1 --2- 172.21.15.12:0/783116327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b93250 0x55d228b9d0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8518005880 tx=0x7f85180283f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35d81d0ea7578c38 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.985+0000 7f85027fc640 1 -- 172.21.15.12:0/783116327 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85180289b0 con 0x55d228b93250 2024-05-09T09:27:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.985+0000 7f85027fc640 1 -- 172.21.15.12:0/783116327 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f851801f420 con 0x55d228b93250 2024-05-09T09:27:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.985+0000 7f85027fc640 1 -- 172.21.15.12:0/783116327 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851801d650 con 0x55d228b93250 2024-05-09T09:27:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.985+0000 7f8525ddf740 1 -- 172.21.15.12:0/783116327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b93250 msgr2=0x55d228b9d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.985+0000 7f8525ddf740 1 --2- 172.21.15.12:0/783116327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b93250 0x55d228b9d0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8518005880 tx=0x7f85180283f0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.985+0000 7f8525ddf740 1 -- 172.21.15.12:0/783116327 shutdown_connections 2024-05-09T09:27:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.985+0000 7f8525ddf740 1 --2- 172.21.15.12:0/783116327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b98db0 0x55d228b8d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.985+0000 7f8525ddf740 1 --2- 172.21.15.12:0/783116327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93c40 0x55d228b9d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.985+0000 7f8525ddf740 1 --2- 172.21.15.12:0/783116327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b93250 0x55d228b9d0b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.985+0000 7f8525ddf740 1 -- 172.21.15.12:0/783116327 >> 172.21.15.12:0/783116327 conn(0x55d228b8e8c0 msgr2=0x55d228b8eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.986+0000 7f8525ddf740 1 -- 172.21.15.12:0/783116327 shutdown_connections 2024-05-09T09:27:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.986+0000 7f8525ddf740 1 -- 172.21.15.12:0/783116327 wait complete. 2024-05-09T09:27:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.986+0000 7f8525ddf740 1 -- start start 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.986+0000 7f8525ddf740 1 Processor -- start 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.986+0000 7f8525ddf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228bb0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.986+0000 7f8525ddf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b93c40 0x55d228bb0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.986+0000 7f8525ddf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b98db0 0x55d228c0db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8525ddf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f851c1b6550 con 0x55d228b93250 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8525ddf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f851c1b5dc0 con 0x55d228b93c40 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8525ddf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f851c1b3100 con 0x55d228b98db0 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8525ddf740 10 client.0 _ll_register_callbacks cb 0x7f8518009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8520d97640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228bb0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8520d97640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228bb0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46454/0 (socket says 172.21.15.12:46454) 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8520d97640 1 -- 172.21.15.12:0/2196817319 learned_addr learned my addr 172.21.15.12:0/2196817319 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8521598640 1 --2- 172.21.15.12:0/2196817319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b98db0 0x55d228c0db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8513fff640 1 --2- 172.21.15.12:0/2196817319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b93c40 0x55d228bb0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8520d97640 1 -- 172.21.15.12:0/2196817319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b98db0 msgr2=0x55d228c0db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8520d97640 1 --2- 172.21.15.12:0/2196817319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b98db0 0x55d228c0db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8520d97640 1 -- 172.21.15.12:0/2196817319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b93c40 msgr2=0x55d228bb0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8520d97640 1 --2- 172.21.15.12:0/2196817319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b93c40 0x55d228bb0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8520d97640 1 -- 172.21.15.12:0/2196817319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d228c0e100 con 0x55d228b93250 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f8520d97640 1 --2- 172.21.15.12:0/2196817319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228bb0880 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f851800f800 tx=0x7f8518011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bd0f9ce7a5ed76 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.987+0000 7f84fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.988+0000 7f84fbfff640 1 -- 172.21.15.12:0/2196817319 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518043060 con 0x55d228b93250 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.988+0000 7f8525ddf740 1 -- 172.21.15.12:0/2196817319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f851801f420 con 0x55d228b93250 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.988+0000 7f84fbfff640 1 -- 172.21.15.12:0/2196817319 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8518005450 con 0x55d228b93250 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.988+0000 7f84fbfff640 1 -- 172.21.15.12:0/2196817319 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f851804a050 con 0x55d228b93250 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.988+0000 7f8525ddf740 10 client.16602 fetch_fsmap learned FSMap version 34 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.988+0000 7f8525ddf740 1 -- 172.21.15.12:0/2196817319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d228c0f410 con 0x55d228b93250 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.988+0000 7f84fbfff640 1 -- 172.21.15.12:0/2196817319 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f851801f420 con 0x55d228b93250 2024-05-09T09:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.989+0000 7f84fbfff640 1 -- 172.21.15.12:0/2196817319 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f8518027030 con 0x55d228b93250 2024-05-09T09:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.989+0000 7f8525ddf740 10 client.16602 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.989+0000 7f8525ddf740 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.989+0000 7f8525ddf740 20 client.16602 populate_metadata read hostname 'smithi012' 2024-05-09T09:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.989+0000 7f8525ddf740 1 --2- 172.21.15.12:0/2196817319 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d228ba9510 0x55d228c11e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.989+0000 7f8525ddf740 4 client.16602 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.989+0000 7f8525ddf740 1 -- 172.21.15.12:0/2196817319 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d228afedc0 con 0x55d228ba9510 2024-05-09T09:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.989+0000 7f8513fff640 1 --2- 172.21.15.12:0/2196817319 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d228ba9510 0x55d228c11e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.989+0000 7f8513fff640 1 --2- 172.21.15.12:0/2196817319 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d228ba9510 0x55d228c11e20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c6a9028de9357b6 server_cookie=2bf2465aae095609 in_seq=0 out_seq=0 2024-05-09T09:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:53.990+0000 7f84fbfff640 10 client.16602 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:53.993 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:27:53 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i[64878]: 2024-05-09T09:27:53.655+0000 7f86071bb700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000553 [...2,head] /volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3/client.0/tmp/untar_tar/linux-4.0.5/Documentation/devicetree/bindings/arm/ auth{1=1,2=1,3=1,4=1} v9268 ap=2 f(v150 m2024-05-09T09:27:27.123182+0000 94=75+19) n(v2 rc2024-05-09T09:27:27.131422+0000 b207851 156=136+20) (inest sync->mix g=1,2,3,4) caps={15519=pAsLsXs/-@403} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x556d6ddc2b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000553:[<0x1000000054e/arm v4758>,<0x1000000054c/bindings v375>,<0x10000000203/devicetree v2637>,<0x10000000202/Documentation v700>,<0x10000000201/linux-4.0.5 v46>,<0x10000000006/untar_tar v63>,<0x10000000005/tmp v62>,<0x10000000003/client.0 v47>,<0x10000000002/addd6f8f-9193-4ea5-a44f-055563fddbc3 v59>,<0x10000000001/sv_0 v77>,<0x10000000000/qa v63>,<0x1/volumes v59>]//[]","memoryvalue":"(2)0x10000000553:[<0x1000000054e/arm v9268>,<0x1000000054c/bindings v2947>,<0x10000000203/devicetree v3640>,<0x10000000202/Documentation v1874>,<0x10000000201/linux-4.0.5 v497>,<0x10000000006/untar_tar v155>,<0x10000000005/tmp v144>,<0x10000000003/client.0 v125>,<0x10000000002/addd6f8f-9193-4ea5-a44f-055563fddbc3 v137>,<0x10000000001/sv_0 v156>,<0x10000000000/qa v138>,<0x1/volumes v128>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-09T09:27:27.115017+0000 93=74+19)","ondisk_value.rstat":"n(v0 rc2024-05-09T09:27:27.126048+0000 b207851 156=136+20)","memory_value.dirstat":"f(v150 m2024-05-09T09:27:27.123182+0000 94=75+19)","memory_value.rstat":"n(v2 rc2024-05-09T09:27:27.131422+0000 b207851 156=136+20)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-09T09:27:53.993 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:27:53 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i[64878]: 2024-05-09T09:27:53.659+0000 7f86071bb700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000b7b [...2,head] /volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3/client.0/tmp/untar_tar/linux-4.0.5/Documentation/devicetree/bindings/usb/ auth{1=1,2=1,3=1,4=1} v9701 ap=2 f(v88 m2024-05-09T09:27:28.294679+0000 44=44+0) n(v1 rc2024-05-09T09:27:28.301932+0000 b67190 45=44+1) (inest sync->mix g=1,2,3,4) caps={15519=pAsLsXs/-@231} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x556d6e59b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000b7b:[<0x1000000054e/usb v5752>,<0x1000000054c/bindings v375>,<0x10000000203/devicetree v2637>,<0x10000000202/Documentation v700>,<0x10000000201/linux-4.0.5 v46>,<0x10000000006/untar_tar v63>,<0x10000000005/tmp v62>,<0x10000000003/client.0 v47>,<0x10000000002/addd6f8f-9193-4ea5-a44f-055563fddbc3 v59>,<0x10000000001/sv_0 v77>,<0x10000000000/qa v63>,<0x1/volumes v59>]//[]","memoryvalue":"(2)0x10000000b7b:[<0x1000000054e/usb v9701>,<0x1000000054c/bindings v2947>,<0x10000000203/devicetree v3640>,<0x10000000202/Documentation v1874>,<0x10000000201/linux-4.0.5 v497>,<0x10000000006/untar_tar v155>,<0x10000000005/tmp v144>,<0x10000000003/client.0 v125>,<0x10000000002/addd6f8f-9193-4ea5-a44f-055563fddbc3 v137>,<0x10000000001/sv_0 v156>,<0x10000000000/qa v138>,<0x1/volumes v128>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-09T09:27:28.287044+0000 43=43+0)","ondisk_value.rstat":"n(v0 rc2024-05-09T09:27:28.296960+0000 b67190 45=44+1)","memory_value.dirstat":"f(v88 m2024-05-09T09:27:28.294679+0000 44=44+0)","memory_value.rstat":"n(v1 rc2024-05-09T09:27:28.301932+0000 b67190 45=44+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-09T09:27:53.993 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:27:53 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i[64878]: 2024-05-09T09:27:53.669+0000 7f86071bb700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008b2 [...2,head] /volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3/client.0/tmp/untar_tar/linux-4.0.5/Documentation/devicetree/bindings/net/ auth{1=1,2=1,3=1,4=1} v9959 ap=2 f(v122 m2024-05-09T09:27:29.635386+0000 66=61+5) n(v2 rc2024-05-09T09:27:29.643351+0000 b130960 88=82+6) (inest sync->mix g=1,2,3,4) caps={15519=pAsLsXs/-@320} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x556d6d514c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000008b2:[<0x1000000054e/net v4772>,<0x1000000054c/bindings v375>,<0x10000000203/devicetree v2637>,<0x10000000202/Documentation v700>,<0x10000000201/linux-4.0.5 v46>,<0x10000000006/untar_tar v63>,<0x10000000005/tmp v62>,<0x10000000003/client.0 v47>,<0x10000000002/addd6f8f-9193-4ea5-a44f-055563fddbc3 v59>,<0x10000000001/sv_0 v77>,<0x10000000000/qa v63>,<0x1/volumes v59>]//[]","memoryvalue":"(2)0x100000008b2:[<0x1000000054e/net v9959>,<0x1000000054c/bindings v2947>,<0x10000000203/devicetree v3640>,<0x10000000202/Documentation v1874>,<0x10000000201/linux-4.0.5 v497>,<0x10000000006/untar_tar v155>,<0x10000000005/tmp v144>,<0x10000000003/client.0 v125>,<0x10000000002/addd6f8f-9193-4ea5-a44f-055563fddbc3 v137>,<0x10000000001/sv_0 v156>,<0x10000000000/qa v138>,<0x1/volumes v128>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-09T09:27:29.620476+0000 65=60+5)","ondisk_value.rstat":"n(v0 rc2024-05-09T09:27:29.637710+0000 b130960 88=82+6)","memory_value.dirstat":"f(v122 m2024-05-09T09:27:29.635386+0000 66=61+5)","memory_value.rstat":"n(v2 rc2024-05-09T09:27:29.643351+0000 b130960 88=82+6)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-09T09:27:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.042+0000 7f84fbfff640 1 -- 172.21.15.12:0/2196817319 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d228afedc0 con 0x55d228ba9510 2024-05-09T09:27:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.042+0000 7f84fbfff640 10 client.16602 handle_command_reply: tid=0 2024-05-09T09:27:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.042+0000 7f8525ddf740 1 client.16602 shutdown 2024-05-09T09:27:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.042+0000 7f8525ddf740 1 -- 172.21.15.12:0/2196817319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 msgr2=0x55d228bb0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.042+0000 7f8525ddf740 1 --2- 172.21.15.12:0/2196817319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228bb0880 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f851800f800 tx=0x7f8518011a40 comp rx=0 tx=0).stop 2024-05-09T09:27:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.042+0000 7f8525ddf740 1 -- 172.21.15.12:0/2196817319 shutdown_connections 2024-05-09T09:27:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.042+0000 7f8525ddf740 1 --2- 172.21.15.12:0/2196817319 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55d228ba9510 0x55d228c11e20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.042+0000 7f84fbfff640 0 client.16602 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:27:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.042+0000 7f8525ddf740 1 --2- 172.21.15.12:0/2196817319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d228b98db0 0x55d228c0db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.042+0000 7f8525ddf740 1 --2- 172.21.15.12:0/2196817319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d228b93c40 0x55d228bb0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.042+0000 7f8525ddf740 1 --2- 172.21.15.12:0/2196817319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d228b93250 0x55d228bb0880 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.042+0000 7f8525ddf740 1 -- 172.21.15.12:0/2196817319 >> 172.21.15.12:0/2196817319 conn(0x55d228b8e8c0 msgr2=0x55d228b9d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.042+0000 7f8525ddf740 1 -- 172.21.15.12:0/2196817319 shutdown_connections 2024-05-09T09:27:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8525ddf740 1 -- 172.21.15.12:0/2196817319 wait complete. 2024-05-09T09:27:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8525ddf740 20 client.16602 trim_cache size 0 max 16384 2024-05-09T09:27:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:27:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:27:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:27:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:27:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8522ffb640 1 -- 172.21.15.12:0/1897227636 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85040a2d00 msgr2=0x7f85040a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8522ffb640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85040a2d00 0x7f85040a51c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8518009fd0 tx=0x7f8518005f90 comp rx=0 tx=0).stop 2024-05-09T09:27:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8522ffb640 1 -- 172.21.15.12:0/1897227636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f851c101980 msgr2=0x7f851c19a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:27:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8522ffb640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f851c101980 0x7f851c19a240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e9c0 tx=0x7f850c00ee80 comp rx=0 tx=0).stop 2024-05-09T09:27:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8522ffb640 1 -- 172.21.15.12:0/1897227636 shutdown_connections 2024-05-09T09:27:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8522ffb640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f85040a2d00 0x7f85040a51c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8522ffb640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c102340 0x7f851c1a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8522ffb640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f851c101980 0x7f851c19a240 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8522ffb640 1 --2- 172.21.15.12:0/1897227636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f851c100790 0x7f851c199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:27:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8522ffb640 1 -- 172.21.15.12:0/1897227636 >> 172.21.15.12:0/1897227636 conn(0x7f851c0fbd00 msgr2=0x7f851c104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:27:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8522ffb640 1 -- 172.21.15.12:0/1897227636 shutdown_connections 2024-05-09T09:27:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:27:54.043+0000 7f8522ffb640 1 -- 172.21.15.12:0/1897227636 wait complete. 2024-05-09T09:27:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:54 smithi012 ceph-mon[30140]: pgmap v438: 129 pgs: 129 active+clean; 137 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.4 MiB/s wr, 458 op/s 2024-05-09T09:27:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:54 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:54 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:54.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:54 smithi170 ceph-mon[33632]: pgmap v438: 129 pgs: 129 active+clean; 137 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.4 MiB/s wr, 458 op/s 2024-05-09T09:27:54.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:54 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:54.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:54 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:54.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:54 smithi016 ceph-mon[33335]: pgmap v438: 129 pgs: 129 active+clean; 137 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.4 MiB/s wr, 458 op/s 2024-05-09T09:27:54.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:54 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:54.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:54 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:55.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub queued for path: / 2024-05-09T09:27:55.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: Scrub error on inode 0x10000000553 (/volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3/client.0/tmp/untar_tar/linux-4.0.5/Documentation/devicetree/bindings/arm) see mds.i log and `damage ls` output for details 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: Scrub error on inode 0x10000000b7b (/volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3/client.0/tmp/untar_tar/linux-4.0.5/Documentation/devicetree/bindings/usb) see mds.i log and `damage ls` output for details 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: Scrub error on inode 0x100000008b2 (/volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3/client.0/tmp/untar_tar/linux-4.0.5/Documentation/devicetree/bindings/net) see mds.i log and `damage ls` output for details 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:55 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub queued for path: / 2024-05-09T09:27:55.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:55.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: Scrub error on inode 0x10000000553 (/volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3/client.0/tmp/untar_tar/linux-4.0.5/Documentation/devicetree/bindings/arm) see mds.i log and `damage ls` output for details 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: Scrub error on inode 0x10000000b7b (/volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3/client.0/tmp/untar_tar/linux-4.0.5/Documentation/devicetree/bindings/usb) see mds.i log and `damage ls` output for details 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: Scrub error on inode 0x100000008b2 (/volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3/client.0/tmp/untar_tar/linux-4.0.5/Documentation/devicetree/bindings/net) see mds.i log and `damage ls` output for details 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:55.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:55.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:55.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:55.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:55.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:27:55.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:55 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub queued for path: / 2024-05-09T09:27:55.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:55.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:55.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:55.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: Scrub error on inode 0x10000000553 (/volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3/client.0/tmp/untar_tar/linux-4.0.5/Documentation/devicetree/bindings/arm) see mds.i log and `damage ls` output for details 2024-05-09T09:27:55.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: Scrub error on inode 0x10000000b7b (/volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3/client.0/tmp/untar_tar/linux-4.0.5/Documentation/devicetree/bindings/usb) see mds.i log and `damage ls` output for details 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: Scrub error on inode 0x100000008b2 (/volumes/qa/sv_0/addd6f8f-9193-4ea5-a44f-055563fddbc3/client.0/tmp/untar_tar/linux-4.0.5/Documentation/devicetree/bindings/net) see mds.i log and `damage ls` output for details 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.505 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:55.505 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.505 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:55.505 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.505 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:27:55.505 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.505 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:55.505 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:55 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:27:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:56 smithi012 ceph-mon[30140]: pgmap v439: 129 pgs: 129 active+clean; 141 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.7 MiB/s wr, 490 op/s 2024-05-09T09:27:56.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:56 smithi170 ceph-mon[33632]: pgmap v439: 129 pgs: 129 active+clean; 141 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.7 MiB/s wr, 490 op/s 2024-05-09T09:27:56.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:56 smithi016 ceph-mon[33335]: pgmap v439: 129 pgs: 129 active+clean; 141 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.7 MiB/s wr, 490 op/s 2024-05-09T09:27:58.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:27:58 smithi012 ceph-mon[30140]: pgmap v440: 129 pgs: 129 active+clean; 142 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.8 MiB/s wr, 524 op/s 2024-05-09T09:27:58.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:27:58 smithi170 ceph-mon[33632]: pgmap v440: 129 pgs: 129 active+clean; 142 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.8 MiB/s wr, 524 op/s 2024-05-09T09:27:58.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:27:58 smithi016 ceph-mon[33335]: pgmap v440: 129 pgs: 129 active+clean; 142 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.8 MiB/s wr, 524 op/s 2024-05-09T09:28:00.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:00 smithi012 ceph-mon[30140]: pgmap v441: 129 pgs: 129 active+clean; 142 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.8 MiB/s wr, 533 op/s 2024-05-09T09:28:00.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:00 smithi170 ceph-mon[33632]: pgmap v441: 129 pgs: 129 active+clean; 142 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.8 MiB/s wr, 533 op/s 2024-05-09T09:28:00.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:00 smithi016 ceph-mon[33335]: pgmap v441: 129 pgs: 129 active+clean; 142 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.8 MiB/s wr, 533 op/s 2024-05-09T09:28:01.757 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2024-05-09T09:28:02.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:02 smithi012 ceph-mon[30140]: pgmap v442: 129 pgs: 129 active+clean; 145 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 564 op/s 2024-05-09T09:28:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:02 smithi170 ceph-mon[33632]: pgmap v442: 129 pgs: 129 active+clean; 145 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 564 op/s 2024-05-09T09:28:02.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:02 smithi016 ceph-mon[33335]: pgmap v442: 129 pgs: 129 active+clean; 145 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 564 op/s 2024-05-09T09:28:03.947 INFO:tasks.workunit.client.0.smithi012.stderr:+ tar c linux-4.0.5 2024-05-09T09:28:04.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:04 smithi012 ceph-mon[30140]: pgmap v443: 129 pgs: 129 active+clean; 146 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 521 op/s 2024-05-09T09:28:04.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:04 smithi170 ceph-mon[33632]: pgmap v443: 129 pgs: 129 active+clean; 146 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 521 op/s 2024-05-09T09:28:04.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:04 smithi016 ceph-mon[33335]: pgmap v443: 129 pgs: 129 active+clean; 146 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 521 op/s 2024-05-09T09:28:06.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:06 smithi012 ceph-mon[30140]: pgmap v444: 129 pgs: 129 active+clean; 149 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 520 op/s 2024-05-09T09:28:06.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:06 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:28:06.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:06 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:28:06.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:06 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:28:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:06 smithi170 ceph-mon[33632]: pgmap v444: 129 pgs: 129 active+clean; 149 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 520 op/s 2024-05-09T09:28:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:06 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:28:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:06 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:28:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:06 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:28:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:06 smithi016 ceph-mon[33335]: pgmap v444: 129 pgs: 129 active+clean; 149 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 520 op/s 2024-05-09T09:28:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:06 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:28:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:06 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:28:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:06 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:28:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.148+0000 7fe801514640 1 -- 172.21.15.12:0/1782076168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7fc102ab0 msgr2=0x7fe7fc102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.148+0000 7fe801514640 1 --2- 172.21.15.12:0/1782076168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7fc102ab0 0x7fe7fc102f30 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009900 tx=0x7fe7f002a410 comp rx=0 tx=0).stop 2024-05-09T09:28:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.148+0000 7fe801514640 1 -- 172.21.15.12:0/1782076168 shutdown_connections 2024-05-09T09:28:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.148+0000 7fe801514640 1 --2- 172.21.15.12:0/1782076168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7fc103470 0x7fe7fc109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.148+0000 7fe801514640 1 --2- 172.21.15.12:0/1782076168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7fc102ab0 0x7fe7fc102f30 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.148+0000 7fe801514640 1 --2- 172.21.15.12:0/1782076168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc1018c0 0x7fe7fc101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.148+0000 7fe801514640 1 -- 172.21.15.12:0/1782076168 >> 172.21.15.12:0/1782076168 conn(0x7fe7fc0fce50 msgr2=0x7fe7fc0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.148+0000 7fe801514640 1 -- 172.21.15.12:0/1782076168 shutdown_connections 2024-05-09T09:28:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.148+0000 7fe801514640 1 -- 172.21.15.12:0/1782076168 wait complete. 2024-05-09T09:28:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.149+0000 7fe801514640 1 Processor -- start 2024-05-09T09:28:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.149+0000 7fe801514640 1 -- start start 2024-05-09T09:28:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.149+0000 7fe801514640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc1018c0 0x7fe7fc199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.149+0000 7fe801514640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7fc102ab0 0x7fe7fc19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.149+0000 7fe801514640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7fc103470 0x7fe7fc1a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.149+0000 7fe801514640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc10d520 con 0x7fe7fc102ab0 2024-05-09T09:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.149+0000 7fe801514640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc10d3a0 con 0x7fe7fc1018c0 2024-05-09T09:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.149+0000 7fe801514640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc10d6a0 con 0x7fe7fc103470 2024-05-09T09:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7fa7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7fc102ab0 0x7fe7fc19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7fa7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7fc102ab0 0x7fe7fc19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47338/0 (socket says 172.21.15.12:47338) 2024-05-09T09:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7fa7fc640 1 -- 172.21.15.12:0/3931858309 learned_addr learned my addr 172.21.15.12:0/3931858309 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7fb7fe640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7fc103470 0x7fe7fc1a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7faffd640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc1018c0 0x7fe7fc199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7fa7fc640 1 -- 172.21.15.12:0/3931858309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7fc103470 msgr2=0x7fe7fc1a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7fc103470 0x7fe7fc1a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7fa7fc640 1 -- 172.21.15.12:0/3931858309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc1018c0 msgr2=0x7fe7fc199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc1018c0 0x7fe7fc199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7fa7fc640 1 -- 172.21.15.12:0/3931858309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7fc1a1970 con 0x7fe7fc102ab0 2024-05-09T09:28:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7faffd640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc1018c0 0x7fe7fc199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:28:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7fb7fe640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7fc103470 0x7fe7fc1a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7fc102ab0 0x7fe7fc19a290 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009230 tx=0x7fe7f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7e3fff640 1 -- 172.21.15.12:0/3931858309 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f003f030 con 0x7fe7fc102ab0 2024-05-09T09:28:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe801514640 1 -- 172.21.15.12:0/3931858309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7fc1a1c00 con 0x7fe7fc102ab0 2024-05-09T09:28:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7e3fff640 1 -- 172.21.15.12:0/3931858309 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe7f0031980 con 0x7fe7fc102ab0 2024-05-09T09:28:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe7e3fff640 1 -- 172.21.15.12:0/3931858309 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0035070 con 0x7fe7fc102ab0 2024-05-09T09:28:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.150+0000 7fe801514640 1 -- 172.21.15.12:0/3931858309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7fc1a2140 con 0x7fe7fc102ab0 2024-05-09T09:28:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.153+0000 7fe7e3fff640 1 -- 172.21.15.12:0/3931858309 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe7f0042030 con 0x7fe7fc102ab0 2024-05-09T09:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.153+0000 7fe804223740 1 Processor -- start 2024-05-09T09:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.153+0000 7fe804223740 1 -- start start 2024-05-09T09:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.153+0000 7fe804223740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd643670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.153+0000 7fe804223740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643c40 0x5605cd648740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.153+0000 7fe804223740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd648db0 0x5605cd64b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe804223740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5605cd5a7340 con 0x5605cd643250 2024-05-09T09:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe804223740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5605cd54f290 con 0x5605cd648db0 2024-05-09T09:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe804223740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5605cd5a3300 con 0x5605cd643c40 2024-05-09T09:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7fb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd643670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7fb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd643670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47342/0 (socket says 172.21.15.12:47342) 2024-05-09T09:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7faffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643c40 0x5605cd648740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7fb7fe640 1 -- 172.21.15.12:0/3818071138 learned_addr learned my addr 172.21.15.12:0/3818071138 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/3818071138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd648db0 0x5605cd64b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7fb7fe640 1 -- 172.21.15.12:0/3818071138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643c40 msgr2=0x5605cd648740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7fb7fe640 1 --2- 172.21.15.12:0/3818071138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643c40 0x5605cd648740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7fb7fe640 1 -- 172.21.15.12:0/3818071138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd648db0 msgr2=0x5605cd64b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7fb7fe640 1 --2- 172.21.15.12:0/3818071138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd648db0 0x5605cd64b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7fb7fe640 1 -- 172.21.15.12:0/3818071138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605cd5738e0 con 0x5605cd643250 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7e3fff640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe7e40a2c30 0x7fe7e40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7faffd640 1 --2- 172.21.15.12:0/3818071138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643c40 0x5605cd648740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7faffd640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe7e40a2c30 0x7fe7e40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7fb7fe640 1 --2- 172.21.15.12:0/3818071138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd643670 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4011790 tx=0x7fe7f4011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a68fa169a52cedd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7e3fff640 1 -- 172.21.15.12:0/3931858309 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe7f00ef990 con 0x7fe7fc102ab0 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.154+0000 7fe7e17fa640 1 -- 172.21.15.12:0/3818071138 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4003c30 con 0x5605cd643250 2024-05-09T09:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.155+0000 7fe7e17fa640 1 -- 172.21.15.12:0/3818071138 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe7f400fe70 con 0x5605cd643250 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.155+0000 7fe7e17fa640 1 -- 172.21.15.12:0/3818071138 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40193e0 con 0x5605cd643250 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.155+0000 7fe804223740 1 -- 172.21.15.12:0/3818071138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 msgr2=0x5605cd643670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.155+0000 7fe7faffd640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe7e40a2c30 0x7fe7e40a50f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5605cd644040 tx=0x7fe7ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.155+0000 7fe804223740 1 --2- 172.21.15.12:0/3818071138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd643670 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4011790 tx=0x7fe7f4011c50 comp rx=0 tx=0).stop 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.155+0000 7fe804223740 1 -- 172.21.15.12:0/3818071138 shutdown_connections 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.155+0000 7fe804223740 1 --2- 172.21.15.12:0/3818071138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd648db0 0x5605cd64b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.155+0000 7fe804223740 1 --2- 172.21.15.12:0/3818071138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643c40 0x5605cd648740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.155+0000 7fe804223740 1 --2- 172.21.15.12:0/3818071138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd643670 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.155+0000 7fe804223740 1 -- 172.21.15.12:0/3818071138 >> 172.21.15.12:0/3818071138 conn(0x5605cd63e8c0 msgr2=0x5605cd63ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.155+0000 7fe804223740 1 -- 172.21.15.12:0/3818071138 shutdown_connections 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.155+0000 7fe804223740 1 -- 172.21.15.12:0/3818071138 wait complete. 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe804223740 1 -- start start 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe804223740 1 Processor -- start 2024-05-09T09:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe804223740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd64d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe804223740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd643c40 0x5605cd64d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe804223740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd648db0 0x5605cd654a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe804223740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5605cd64bd60 con 0x5605cd643250 2024-05-09T09:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe804223740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5605cd562790 con 0x5605cd643c40 2024-05-09T09:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe804223740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5605cd64bee0 con 0x5605cd648db0 2024-05-09T09:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe804223740 10 client.0 _ll_register_callbacks cb 0x5605cd59da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe7faffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd64d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe7faffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd64d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47354/0 (socket says 172.21.15.12:47354) 2024-05-09T09:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe7faffd640 1 -- 172.21.15.12:0/2320116925 learned_addr learned my addr 172.21.15.12:0/2320116925 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd643c40 0x5605cd64d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.156+0000 7fe7fb7fe640 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd648db0 0x5605cd654a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe7fa7fc640 1 -- 172.21.15.12:0/2320116925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd648db0 msgr2=0x5605cd654a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd648db0 0x5605cd654a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe7fa7fc640 1 -- 172.21.15.12:0/2320116925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 msgr2=0x5605cd64d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd64d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe7fa7fc640 1 -- 172.21.15.12:0/2320116925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605cd2bb210 con 0x5605cd643c40 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe7faffd640 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd64d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd643c40 0x5605cd64d9d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00ed110 tx=0x7fe7f0005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7321a1d4fd8a84e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe7deff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe7deff5640 1 -- 172.21.15.12:0/2320116925 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0049050 con 0x5605cd643c40 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe7deff5640 1 -- 172.21.15.12:0/2320116925 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe7f0003cc0 con 0x5605cd643c40 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe804223740 1 -- 172.21.15.12:0/2320116925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f400fe70 con 0x5605cd643c40 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe7deff5640 1 -- 172.21.15.12:0/2320116925 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe7f00bb960 con 0x5605cd643c40 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe804223740 10 client.26057 fetch_fsmap learned FSMap version 34 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.157+0000 7fe804223740 1 -- 172.21.15.12:0/2320116925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5605cd387620 con 0x5605cd643c40 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.158+0000 7fe7deff5640 1 -- 172.21.15.12:0/2320116925 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe7f400fe70 con 0x5605cd643c40 2024-05-09T09:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.158+0000 7fe7deff5640 1 -- 172.21.15.12:0/2320116925 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fe7f00bc080 con 0x5605cd643c40 2024-05-09T09:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.158+0000 7fe804223740 10 client.26057 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.158+0000 7fe804223740 10 client.26057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.158+0000 7fe804223740 20 client.26057 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.158+0000 7fe804223740 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5605cd658f00 0x5605cd65b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.158+0000 7fe804223740 4 client.26057 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.158+0000 7fe804223740 1 -- 172.21.15.12:0/2320116925 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605cd5b4a70 con 0x5605cd658f00 2024-05-09T09:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.158+0000 7fe7faffd640 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5605cd658f00 0x5605cd65b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe7deff5640 10 client.26057 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe7faffd640 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5605cd658f00 0x5605cd65b380 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b46a129a6e313669 server_cookie=4440620aacc90324 in_seq=0 out_seq=0 2024-05-09T09:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe7deff5640 1 -- 172.21.15.12:0/2320116925 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605cd5b4a70 con 0x5605cd658f00 2024-05-09T09:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe7deff5640 10 client.26057 handle_command_reply: tid=0 2024-05-09T09:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe804223740 1 client.26057 shutdown 2024-05-09T09:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe804223740 1 -- 172.21.15.12:0/2320116925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd643c40 msgr2=0x5605cd64d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe804223740 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd643c40 0x5605cd64d9d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00ed110 tx=0x7fe7f0005b70 comp rx=0 tx=0).stop 2024-05-09T09:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe804223740 1 -- 172.21.15.12:0/2320116925 shutdown_connections 2024-05-09T09:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe804223740 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5605cd658f00 0x5605cd65b380 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe804223740 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd648db0 0x5605cd654a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe804223740 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd643c40 0x5605cd64d9d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe804223740 1 --2- 172.21.15.12:0/2320116925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd64d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe7deff5640 0 client.26057 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.159+0000 7fe804223740 1 -- 172.21.15.12:0/2320116925 >> 172.21.15.12:0/2320116925 conn(0x5605cd63e8c0 msgr2=0x5605cd63da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.160+0000 7fe804223740 1 -- 172.21.15.12:0/2320116925 shutdown_connections 2024-05-09T09:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.160+0000 7fe804223740 1 -- 172.21.15.12:0/2320116925 wait complete. 2024-05-09T09:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.160+0000 7fe804223740 20 client.26057 trim_cache size 0 max 16384 2024-05-09T09:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe804223740 1 Processor -- start 2024-05-09T09:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe804223740 1 -- start start 2024-05-09T09:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe804223740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd63e8c0 0x5605cd6444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe804223740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd644e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe804223740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643c40 0x5605cd6453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe804223740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc1ac700 con 0x5605cd643250 2024-05-09T09:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe804223740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc1ae300 con 0x5605cd63e8c0 2024-05-09T09:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe804223740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc1acdd0 con 0x5605cd643c40 2024-05-09T09:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe7fa7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd644e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe7fa7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd644e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47366/0 (socket says 172.21.15.12:47366) 2024-05-09T09:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe7fa7fc640 1 -- 172.21.15.12:0/2814695893 learned_addr learned my addr 172.21.15.12:0/2814695893 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe7fb7fe640 1 --2- 172.21.15.12:0/2814695893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643c40 0x5605cd6453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe7faffd640 1 --2- 172.21.15.12:0/2814695893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd63e8c0 0x5605cd6444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe7fa7fc640 1 -- 172.21.15.12:0/2814695893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643c40 msgr2=0x5605cd6453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/2814695893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643c40 0x5605cd6453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe7fa7fc640 1 -- 172.21.15.12:0/2814695893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd63e8c0 msgr2=0x5605cd6444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/2814695893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd63e8c0 0x5605cd6444d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe7fa7fc640 1 -- 172.21.15.12:0/2814695893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605cd65ba00 con 0x5605cd643250 2024-05-09T09:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.178+0000 7fe7faffd640 1 --2- 172.21.15.12:0/2814695893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd63e8c0 0x5605cd6444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.179+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/2814695893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd644e60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00b8280 tx=0x7fe7f00ba330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fd4335b69a7527 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.179+0000 7fe7e0ff9640 1 -- 172.21.15.12:0/2814695893 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0049070 con 0x5605cd643250 2024-05-09T09:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.179+0000 7fe7e0ff9640 1 -- 172.21.15.12:0/2814695893 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe7f00bc520 con 0x5605cd643250 2024-05-09T09:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.179+0000 7fe7e0ff9640 1 -- 172.21.15.12:0/2814695893 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00bb780 con 0x5605cd643250 2024-05-09T09:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.179+0000 7fe804223740 1 -- 172.21.15.12:0/2814695893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 msgr2=0x5605cd644e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.179+0000 7fe804223740 1 --2- 172.21.15.12:0/2814695893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd644e60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00b8280 tx=0x7fe7f00ba330 comp rx=0 tx=0).stop 2024-05-09T09:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 -- 172.21.15.12:0/2814695893 shutdown_connections 2024-05-09T09:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 --2- 172.21.15.12:0/2814695893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643c40 0x5605cd6453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 --2- 172.21.15.12:0/2814695893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643250 0x5605cd644e60 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 --2- 172.21.15.12:0/2814695893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd63e8c0 0x5605cd6444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 -- 172.21.15.12:0/2814695893 >> 172.21.15.12:0/2814695893 conn(0x5605cd6399b0 msgr2=0x5605cd648490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 -- 172.21.15.12:0/2814695893 shutdown_connections 2024-05-09T09:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 -- 172.21.15.12:0/2814695893 wait complete. 2024-05-09T09:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 -- start start 2024-05-09T09:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 Processor -- start 2024-05-09T09:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd63e8c0 0x5605cd6512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643250 0x5605cd651800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643c40 0x5605cd652810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc1adc30 con 0x5605cd643c40 2024-05-09T09:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc1ad4a0 con 0x5605cd63e8c0 2024-05-09T09:28:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc1aa6e0 con 0x5605cd643250 2024-05-09T09:28:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe804223740 10 client.0 _ll_register_callbacks cb 0x7fe7f00045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe7fb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643c40 0x5605cd652810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe7faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd63e8c0 0x5605cd6512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe7fb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643c40 0x5605cd652810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47382/0 (socket says 172.21.15.12:47382) 2024-05-09T09:28:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe7fb7fe640 1 -- 172.21.15.12:0/3864704401 learned_addr learned my addr 172.21.15.12:0/3864704401 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.180+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/3864704401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643250 0x5605cd651800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.181+0000 7fe7faffd640 1 -- 172.21.15.12:0/3864704401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643250 msgr2=0x5605cd651800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.181+0000 7fe7faffd640 1 --2- 172.21.15.12:0/3864704401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643250 0x5605cd651800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.181+0000 7fe7faffd640 1 -- 172.21.15.12:0/3864704401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643c40 msgr2=0x5605cd652810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.181+0000 7fe7faffd640 1 --2- 172.21.15.12:0/3864704401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643c40 0x5605cd652810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.181+0000 7fe7faffd640 1 -- 172.21.15.12:0/3864704401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605cd652e00 con 0x5605cd63e8c0 2024-05-09T09:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.181+0000 7fe7faffd640 1 --2- 172.21.15.12:0/3864704401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd63e8c0 0x5605cd6512c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5605cd65bbd0 tx=0x7fe7ec004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd28a41a39bc7281 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.181+0000 7fe7deff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.181+0000 7fe7deff5640 1 -- 172.21.15.12:0/3864704401 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec031030 con 0x5605cd63e8c0 2024-05-09T09:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.181+0000 7fe7deff5640 1 -- 172.21.15.12:0/3864704401 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe7ec018030 con 0x5605cd63e8c0 2024-05-09T09:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.181+0000 7fe804223740 1 -- 172.21.15.12:0/3864704401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f00bc520 con 0x5605cd63e8c0 2024-05-09T09:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.181+0000 7fe7deff5640 1 -- 172.21.15.12:0/3864704401 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe7ec03d050 con 0x5605cd63e8c0 2024-05-09T09:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.181+0000 7fe804223740 10 client.26063 fetch_fsmap learned FSMap version 34 2024-05-09T09:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.181+0000 7fe804223740 1 -- 172.21.15.12:0/3864704401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5605cd654180 con 0x5605cd63e8c0 2024-05-09T09:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.182+0000 7fe7deff5640 1 -- 172.21.15.12:0/3864704401 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe7f00bc520 con 0x5605cd63e8c0 2024-05-09T09:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.182+0000 7fe7deff5640 1 -- 172.21.15.12:0/3864704401 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fe7ec004830 con 0x5605cd63e8c0 2024-05-09T09:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.182+0000 7fe804223740 10 client.26063 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T09:28:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.182+0000 7fe804223740 10 client.26063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.182+0000 7fe804223740 20 client.26063 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.182+0000 7fe804223740 1 --2- 172.21.15.12:0/3864704401 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5605cd648db0 0x5605cd6c1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.182+0000 7fe804223740 4 client.26063 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:28:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.182+0000 7fe804223740 1 -- 172.21.15.12:0/3864704401 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605cd5aedc0 con 0x5605cd648db0 2024-05-09T09:28:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.182+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/3864704401 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5605cd648db0 0x5605cd6c1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.182+0000 7fe7fa7fc640 1 --2- 172.21.15.12:0/3864704401 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5605cd648db0 0x5605cd6c1c60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a526df8c21a3f910 server_cookie=96f53c5420c4d2ab in_seq=0 out_seq=0 2024-05-09T09:28:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.182+0000 7fe7deff5640 10 client.26063 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.200+0000 7fe7deff5640 1 -- 172.21.15.12:0/3864704401 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605cd5aedc0 con 0x5605cd648db0 2024-05-09T09:28:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.200+0000 7fe7deff5640 10 client.26063 handle_command_reply: tid=0 2024-05-09T09:28:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe804223740 1 client.26063 shutdown 2024-05-09T09:28:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe804223740 1 -- 172.21.15.12:0/3864704401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd63e8c0 msgr2=0x5605cd6512c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe804223740 1 --2- 172.21.15.12:0/3864704401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd63e8c0 0x5605cd6512c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5605cd65bbd0 tx=0x7fe7ec004730 comp rx=0 tx=0).stop 2024-05-09T09:28:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe804223740 1 -- 172.21.15.12:0/3864704401 shutdown_connections 2024-05-09T09:28:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe804223740 1 --2- 172.21.15.12:0/3864704401 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5605cd648db0 0x5605cd6c1c60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe804223740 1 --2- 172.21.15.12:0/3864704401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605cd643c40 0x5605cd652810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe804223740 1 --2- 172.21.15.12:0/3864704401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605cd643250 0x5605cd651800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe804223740 1 --2- 172.21.15.12:0/3864704401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605cd63e8c0 0x5605cd6512c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe7deff5640 0 client.26063 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe804223740 1 -- 172.21.15.12:0/3864704401 >> 172.21.15.12:0/3864704401 conn(0x5605cd6399b0 msgr2=0x5605cd648050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe804223740 1 -- 172.21.15.12:0/3864704401 shutdown_connections 2024-05-09T09:28:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe804223740 1 -- 172.21.15.12:0/3864704401 wait complete. 2024-05-09T09:28:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe804223740 20 client.26063 trim_cache size 0 max 16384 2024-05-09T09:28:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:28:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:28:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:28:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:28:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe801514640 1 -- 172.21.15.12:0/3931858309 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe7e40a2c30 msgr2=0x7fe7e40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe801514640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe7e40a2c30 0x7fe7e40a50f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5605cd644040 tx=0x7fe7ec005f90 comp rx=0 tx=0).stop 2024-05-09T09:28:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe801514640 1 -- 172.21.15.12:0/3931858309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7fc102ab0 msgr2=0x7fe7fc19a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.201+0000 7fe801514640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7fc102ab0 0x7fe7fc19a290 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009230 tx=0x7fe7f00042f0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.202+0000 7fe801514640 1 -- 172.21.15.12:0/3931858309 shutdown_connections 2024-05-09T09:28:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.202+0000 7fe801514640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe7e40a2c30 0x7fe7e40a50f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.202+0000 7fe801514640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7fc103470 0x7fe7fc1a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.202+0000 7fe801514640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7fc102ab0 0x7fe7fc19a290 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.202+0000 7fe801514640 1 --2- 172.21.15.12:0/3931858309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc1018c0 0x7fe7fc199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.202+0000 7fe801514640 1 -- 172.21.15.12:0/3931858309 >> 172.21.15.12:0/3931858309 conn(0x7fe7fc0fce50 msgr2=0x7fe7fc073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.202+0000 7fe801514640 1 -- 172.21.15.12:0/3931858309 shutdown_connections 2024-05-09T09:28:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:07.202+0000 7fe801514640 1 -- 172.21.15.12:0/3931858309 wait complete. 2024-05-09T09:28:07.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:07 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:28:07.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:07 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:28:07.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:07 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:28:07.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:07 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:28:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:07 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:28:07.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:07 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:28:07.534 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm -rf linux-4.0.5 2024-05-09T09:28:08.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:08 smithi012 ceph-mon[30140]: pgmap v445: 129 pgs: 129 active+clean; 148 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.6 MiB/s wr, 443 op/s 2024-05-09T09:28:08.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:08 smithi170 ceph-mon[33632]: pgmap v445: 129 pgs: 129 active+clean; 148 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.6 MiB/s wr, 443 op/s 2024-05-09T09:28:08.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:08 smithi016 ceph-mon[33335]: pgmap v445: 129 pgs: 129 active+clean; 148 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.6 MiB/s wr, 443 op/s 2024-05-09T09:28:10.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:10 smithi012 ceph-mon[30140]: pgmap v446: 129 pgs: 129 active+clean; 148 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.3 MiB/s wr, 327 op/s 2024-05-09T09:28:10.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:10 smithi170 ceph-mon[33632]: pgmap v446: 129 pgs: 129 active+clean; 148 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.3 MiB/s wr, 327 op/s 2024-05-09T09:28:10.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:10 smithi016 ceph-mon[33335]: pgmap v446: 129 pgs: 129 active+clean; 148 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.3 MiB/s wr, 327 op/s 2024-05-09T09:28:11.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:11 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:11.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:11 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:11.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:11 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:12.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:12 smithi012 ceph-mon[30140]: pgmap v447: 129 pgs: 129 active+clean; 148 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 1.2 MiB/s wr, 273 op/s 2024-05-09T09:28:12.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:12 smithi170 ceph-mon[33632]: pgmap v447: 129 pgs: 129 active+clean; 148 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 1.2 MiB/s wr, 273 op/s 2024-05-09T09:28:12.513 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:12 smithi016 ceph-mon[33335]: pgmap v447: 129 pgs: 129 active+clean; 148 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 1.2 MiB/s wr, 273 op/s 2024-05-09T09:28:13.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:13 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:13.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:13 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:13.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:13 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:14.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:14 smithi012 ceph-mon[30140]: pgmap v448: 129 pgs: 129 active+clean; 151 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 998 KiB/s wr, 230 op/s 2024-05-09T09:28:14.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:14 smithi170 ceph-mon[33632]: pgmap v448: 129 pgs: 129 active+clean; 151 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 998 KiB/s wr, 230 op/s 2024-05-09T09:28:14.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:14 smithi016 ceph-mon[33335]: pgmap v448: 129 pgs: 129 active+clean; 151 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 998 KiB/s wr, 230 op/s 2024-05-09T09:28:16.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:16 smithi012 ceph-mon[30140]: pgmap v449: 129 pgs: 129 active+clean; 151 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 831 KiB/s wr, 240 op/s 2024-05-09T09:28:16.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:16 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:16.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:16 smithi170 ceph-mon[33632]: pgmap v449: 129 pgs: 129 active+clean; 151 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 831 KiB/s wr, 240 op/s 2024-05-09T09:28:16.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:16 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:16.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:16 smithi016 ceph-mon[33335]: pgmap v449: 129 pgs: 129 active+clean; 151 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 831 KiB/s wr, 240 op/s 2024-05-09T09:28:16.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:16 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:17.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:17 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:17.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:17 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:17.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:17 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:18.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:18 smithi170 ceph-mon[33632]: pgmap v450: 129 pgs: 129 active+clean; 151 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 545 KiB/s wr, 241 op/s 2024-05-09T09:28:18.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:18 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:18.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:18 smithi012 ceph-mon[30140]: pgmap v450: 129 pgs: 129 active+clean; 151 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 545 KiB/s wr, 241 op/s 2024-05-09T09:28:18.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:18 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:18.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:18 smithi016 ceph-mon[33335]: pgmap v450: 129 pgs: 129 active+clean; 151 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 545 KiB/s wr, 241 op/s 2024-05-09T09:28:18.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:18 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.296+0000 7f8fc4129640 1 -- 172.21.15.12:0/2443312995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fbc100790 msgr2=0x7f8fbc100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.296+0000 7f8fc4129640 1 --2- 172.21.15.12:0/2443312995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fbc100790 0x7f8fbc100bb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009900 tx=0x7f8fb802a410 comp rx=0 tx=0).stop 2024-05-09T09:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.296+0000 7f8fc4129640 1 -- 172.21.15.12:0/2443312995 shutdown_connections 2024-05-09T09:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.296+0000 7f8fc4129640 1 --2- 172.21.15.12:0/2443312995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fbc102340 0x7f8fbc108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.296+0000 7f8fc4129640 1 --2- 172.21.15.12:0/2443312995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fbc101980 0x7f8fbc101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.296+0000 7f8fc4129640 1 --2- 172.21.15.12:0/2443312995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fbc100790 0x7f8fbc100bb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.296+0000 7f8fc4129640 1 -- 172.21.15.12:0/2443312995 >> 172.21.15.12:0/2443312995 conn(0x7f8fbc0fbd00 msgr2=0x7f8fbc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.296+0000 7f8fc4129640 1 -- 172.21.15.12:0/2443312995 shutdown_connections 2024-05-09T09:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.296+0000 7f8fc4129640 1 -- 172.21.15.12:0/2443312995 wait complete. 2024-05-09T09:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.297+0000 7f8fc4129640 1 Processor -- start 2024-05-09T09:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.297+0000 7f8fc4129640 1 -- start start 2024-05-09T09:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.297+0000 7f8fc4129640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fbc100790 0x7f8fbc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.297+0000 7f8fc4129640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fbc101980 0x7f8fbc198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.297+0000 7f8fc4129640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fbc102340 0x7f8fbc19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.297+0000 7f8fc4129640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc10d520 con 0x7f8fbc102340 2024-05-09T09:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.297+0000 7f8fc4129640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc10d3a0 con 0x7f8fbc100790 2024-05-09T09:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.297+0000 7f8fc4129640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc10d6a0 con 0x7f8fbc101980 2024-05-09T09:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.297+0000 7f8fc1ec5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fbc100790 0x7f8fbc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.297+0000 7f8fc26c6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fbc102340 0x7f8fbc19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.297+0000 7f8fc1ec5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fbc100790 0x7f8fbc197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:44552/0 (socket says 172.21.15.12:44552) 2024-05-09T09:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.297+0000 7f8fc1ec5640 1 -- 172.21.15.12:0/3476697902 learned_addr learned my addr 172.21.15.12:0/3476697902 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.297+0000 7f8fc16c4640 1 --2- 172.21.15.12:0/3476697902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fbc101980 0x7f8fbc198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.298+0000 7f8fc1ec5640 1 -- 172.21.15.12:0/3476697902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fbc101980 msgr2=0x7f8fbc198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.298+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/3476697902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fbc101980 0x7f8fbc198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.298+0000 7f8fc1ec5640 1 -- 172.21.15.12:0/3476697902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fbc102340 msgr2=0x7f8fbc19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.298+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/3476697902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fbc102340 0x7f8fbc19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.298+0000 7f8fc1ec5640 1 -- 172.21.15.12:0/3476697902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc19f750 con 0x7f8fbc100790 2024-05-09T09:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.298+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/3476697902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fbc100790 0x7f8fbc197b30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fb802a930 tx=0x7f8fb8004f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.298+0000 7f8fb2ffd640 1 -- 172.21.15.12:0/3476697902 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb803f070 con 0x7f8fbc100790 2024-05-09T09:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.298+0000 7f8fb2ffd640 1 -- 172.21.15.12:0/3476697902 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8fb8005430 con 0x7f8fbc100790 2024-05-09T09:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.298+0000 7f8fc4129640 1 -- 172.21.15.12:0/3476697902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fbc19f9e0 con 0x7f8fbc100790 2024-05-09T09:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.298+0000 7f8fb2ffd640 1 -- 172.21.15.12:0/3476697902 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8035030 con 0x7f8fbc100790 2024-05-09T09:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.298+0000 7f8fc4129640 1 -- 172.21.15.12:0/3476697902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fbc19ff20 con 0x7f8fbc100790 2024-05-09T09:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.300+0000 7f8fb2ffd640 1 -- 172.21.15.12:0/3476697902 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8fb8042070 con 0x7f8fbc100790 2024-05-09T09:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fc6e38740 1 Processor -- start 2024-05-09T09:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fc6e38740 1 -- start start 2024-05-09T09:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fb2ffd640 1 --2- 172.21.15.12:0/3476697902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8fa80a2cb0 0x7f8fa80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fc6e38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x55903577ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fc6e38740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x559035781b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fc6e38740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 0x559035784650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fc6e38740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5590356dc340 con 0x55903577d050 2024-05-09T09:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fc6e38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559035684290 con 0x55903577c660 2024-05-09T09:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fc6e38740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5590356d8300 con 0x5590357821c0 2024-05-09T09:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fc26c6640 1 --2- 172.21.15.12:0/3476697902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8fa80a2cb0 0x7f8fa80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fc1ec5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x55903577ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fc16c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x559035781b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fc1ec5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x55903577ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:44560/0 (socket says 172.21.15.12:44560) 2024-05-09T09:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fc1ec5640 1 -- 172.21.15.12:0/3782899099 learned_addr learned my addr 172.21.15.12:0/3782899099 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.301+0000 7f8fb2ffd640 1 -- 172.21.15.12:0/3476697902 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8fb80efc00 con 0x7f8fbc100790 2024-05-09T09:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.302+0000 7f8fc26c6640 1 --2- 172.21.15.12:0/3782899099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 0x559035784650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.302+0000 7f8fc1ec5640 1 -- 172.21.15.12:0/3782899099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 msgr2=0x559035784650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.302+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/3782899099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 0x559035784650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.302+0000 7f8fc1ec5640 1 -- 172.21.15.12:0/3782899099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 msgr2=0x559035781b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.302+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/3782899099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x559035781b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.302+0000 7f8fc1ec5640 1 -- 172.21.15.12:0/3782899099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590356a88e0 con 0x55903577c660 2024-05-09T09:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.302+0000 7f8fc26c6640 1 --2- 172.21.15.12:0/3782899099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 0x559035784650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.302+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/3782899099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x55903577ca80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80b8490 tx=0x7f8fb8047000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a22c836bab9f6b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.302+0000 7f8fc26c6640 1 --2- 172.21.15.12:0/3476697902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8fa80a2cb0 0x7f8fa80a5170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005fd0 tx=0x7f8fb4009dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.302+0000 7f8fa7fff640 1 -- 172.21.15.12:0/3782899099 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8049070 con 0x55903577c660 2024-05-09T09:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.302+0000 7f8fa7fff640 1 -- 172.21.15.12:0/3782899099 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8fb80055e0 con 0x55903577c660 2024-05-09T09:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.302+0000 7f8fa7fff640 1 -- 172.21.15.12:0/3782899099 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80c03e0 con 0x55903577c660 2024-05-09T09:28:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.303+0000 7f8fc6e38740 1 -- 172.21.15.12:0/3782899099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 msgr2=0x55903577ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.303+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/3782899099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x55903577ca80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80b8490 tx=0x7f8fb8047000 comp rx=0 tx=0).stop 2024-05-09T09:28:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.303+0000 7f8fc6e38740 1 -- 172.21.15.12:0/3782899099 shutdown_connections 2024-05-09T09:28:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.303+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/3782899099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 0x559035784650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.303+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/3782899099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x559035781b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.303+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/3782899099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x55903577ca80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.303+0000 7f8fc6e38740 1 -- 172.21.15.12:0/3782899099 >> 172.21.15.12:0/3782899099 conn(0x5590357738c0 msgr2=0x559035773cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.303+0000 7f8fc6e38740 1 -- 172.21.15.12:0/3782899099 shutdown_connections 2024-05-09T09:28:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.303+0000 7f8fc6e38740 1 -- 172.21.15.12:0/3782899099 wait complete. 2024-05-09T09:28:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.303+0000 7f8fc6e38740 1 -- start start 2024-05-09T09:28:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.303+0000 7f8fc6e38740 1 Processor -- start 2024-05-09T09:28:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc6e38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x559035786570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc6e38740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x559035786ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc6e38740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 0x55903578daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc6e38740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559035784f30 con 0x55903577d050 2024-05-09T09:28:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc6e38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559035697790 con 0x55903577c660 2024-05-09T09:28:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc6e38740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5590357850b0 con 0x5590357821c0 2024-05-09T09:28:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc6e38740 10 client.0 _ll_register_callbacks cb 0x5590356d2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc16c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x559035786ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc16c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x559035786ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57958/0 (socket says 172.21.15.12:57958) 2024-05-09T09:28:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc16c4640 1 -- 172.21.15.12:0/1122649534 learned_addr learned my addr 172.21.15.12:0/1122649534 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc26c6640 1 --2- 172.21.15.12:0/1122649534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 0x55903578daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/1122649534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x559035786570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc16c4640 1 -- 172.21.15.12:0/1122649534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 msgr2=0x55903578daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc16c4640 1 --2- 172.21.15.12:0/1122649534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 0x55903578daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc16c4640 1 -- 172.21.15.12:0/1122649534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 msgr2=0x559035786570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc16c4640 1 --2- 172.21.15.12:0/1122649534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x559035786570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc16c4640 1 -- 172.21.15.12:0/1122649534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590353f0210 con 0x55903577d050 2024-05-09T09:28:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fc16c4640 1 --2- 172.21.15.12:0/1122649534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x559035786ab0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8fac012700 tx=0x7f8fac012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3202157f059aea4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.304+0000 7f8fa57fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fa57fa640 1 -- 172.21.15.12:0/1122649534 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac00f5e0 con 0x55903577d050 2024-05-09T09:28:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fa57fa640 1 -- 172.21.15.12:0/1122649534 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8fac00f780 con 0x55903577d050 2024-05-09T09:28:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fc6e38740 1 -- 172.21.15.12:0/1122649534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5590356e9a70 con 0x55903577d050 2024-05-09T09:28:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fa57fa640 1 -- 172.21.15.12:0/1122649534 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8fac00f9a0 con 0x55903577d050 2024-05-09T09:28:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fc6e38740 10 client.16632 fetch_fsmap learned FSMap version 40 2024-05-09T09:28:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fc6e38740 1 -- 172.21.15.12:0/1122649534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5590354bc620 con 0x55903577d050 2024-05-09T09:28:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fa57fa640 1 -- 172.21.15.12:0/1122649534 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x5590356e9a70 con 0x55903577d050 2024-05-09T09:28:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fa57fa640 1 -- 172.21.15.12:0/1122649534 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) ==== 8434+0+0 (secure 0 0 0) 0x7f8fac010c60 con 0x55903577d050 2024-05-09T09:28:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fc6e38740 10 client.16632 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:28:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fc6e38740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fc6e38740 20 client.16632 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/1122649534 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559035792140 0x5590357945a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fc6e38740 4 client.16632 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fc6e38740 1 -- 172.21.15.12:0/1122649534 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590356e3dc0 con 0x559035792140 2024-05-09T09:28:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.305+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/1122649534 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559035792140 0x5590357945a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.306+0000 7f8fa57fa640 10 client.16632 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.306+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/1122649534 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559035792140 0x5590357945a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1265839b4651d8 server_cookie=df07fbdc8e345a26 in_seq=0 out_seq=0 2024-05-09T09:28:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.306+0000 7f8fa57fa640 1 -- 172.21.15.12:0/1122649534 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590356e3dc0 con 0x559035792140 2024-05-09T09:28:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.306+0000 7f8fa57fa640 10 client.16632 handle_command_reply: tid=0 2024-05-09T09:28:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.306+0000 7f8fc6e38740 1 client.16632 shutdown 2024-05-09T09:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.307+0000 7f8fc6e38740 1 -- 172.21.15.12:0/1122649534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 msgr2=0x559035786ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.307+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/1122649534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x559035786ab0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8fac012700 tx=0x7f8fac012b00 comp rx=0 tx=0).stop 2024-05-09T09:28:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.307+0000 7f8fc6e38740 1 -- 172.21.15.12:0/1122649534 shutdown_connections 2024-05-09T09:28:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.307+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/1122649534 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x559035792140 0x5590357945a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.307+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/1122649534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 0x55903578daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.307+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/1122649534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x559035786ab0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.307+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/1122649534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x559035786570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.307+0000 7f8fa57fa640 0 client.16632 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.307+0000 7f8fc6e38740 1 -- 172.21.15.12:0/1122649534 >> 172.21.15.12:0/1122649534 conn(0x5590357738c0 msgr2=0x559035772a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.307+0000 7f8fc6e38740 1 -- 172.21.15.12:0/1122649534 shutdown_connections 2024-05-09T09:28:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.307+0000 7f8fc6e38740 1 -- 172.21.15.12:0/1122649534 wait complete. 2024-05-09T09:28:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.307+0000 7f8fc6e38740 20 client.16632 trim_cache size 0 max 16384 2024-05-09T09:28:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.322+0000 7f8fc6e38740 1 Processor -- start 2024-05-09T09:28:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.322+0000 7f8fc6e38740 1 -- start start 2024-05-09T09:28:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.322+0000 7f8fc6e38740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55903577c660 0x559035773ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.322+0000 7f8fc6e38740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x55903578dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.322+0000 7f8fc6e38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590357821c0 0x5590357768c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.322+0000 7f8fc6e38740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1a83a0 con 0x55903577d050 2024-05-09T09:28:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.322+0000 7f8fc6e38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1aa0f0 con 0x5590357821c0 2024-05-09T09:28:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc6e38740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1a8b00 con 0x55903577c660 2024-05-09T09:28:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc16c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x55903578dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc16c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x55903578dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57964/0 (socket says 172.21.15.12:57964) 2024-05-09T09:28:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc16c4640 1 -- 172.21.15.12:0/2827252371 learned_addr learned my addr 172.21.15.12:0/2827252371 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/2827252371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55903577c660 0x559035773ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc26c6640 1 --2- 172.21.15.12:0/2827252371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590357821c0 0x5590357768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc16c4640 1 -- 172.21.15.12:0/2827252371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55903577c660 msgr2=0x559035773ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc16c4640 1 --2- 172.21.15.12:0/2827252371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55903577c660 0x559035773ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc16c4640 1 -- 172.21.15.12:0/2827252371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590357821c0 msgr2=0x5590357768c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc16c4640 1 --2- 172.21.15.12:0/2827252371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590357821c0 0x5590357768c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc16c4640 1 -- 172.21.15.12:0/2827252371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559035774630 con 0x55903577d050 2024-05-09T09:28:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc26c6640 1 --2- 172.21.15.12:0/2827252371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590357821c0 0x5590357768c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:28:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/2827252371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55903577c660 0x559035773ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fc16c4640 1 --2- 172.21.15.12:0/2827252371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x55903578dab0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0059f0 tx=0x7f8fac01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e2ee417f8718bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fa77fe640 1 -- 172.21.15.12:0/2827252371 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac00f5e0 con 0x55903577d050 2024-05-09T09:28:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fa77fe640 1 -- 172.21.15.12:0/2827252371 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8fac00f780 con 0x55903577d050 2024-05-09T09:28:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.323+0000 7f8fa77fe640 1 -- 172.21.15.12:0/2827252371 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac005560 con 0x55903577d050 2024-05-09T09:28:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.324+0000 7f8fc6e38740 1 -- 172.21.15.12:0/2827252371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 msgr2=0x55903578dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.324+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/2827252371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x55903578dab0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0059f0 tx=0x7f8fac01c970 comp rx=0 tx=0).stop 2024-05-09T09:28:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.324+0000 7f8fc6e38740 1 -- 172.21.15.12:0/2827252371 shutdown_connections 2024-05-09T09:28:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.324+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/2827252371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590357821c0 0x5590357768c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.324+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/2827252371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x55903578dab0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.324+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/2827252371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55903577c660 0x559035773ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.324+0000 7f8fc6e38740 1 -- 172.21.15.12:0/2827252371 >> 172.21.15.12:0/2827252371 conn(0x5590357738c0 msgr2=0x559035781940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.324+0000 7f8fc6e38740 1 -- 172.21.15.12:0/2827252371 shutdown_connections 2024-05-09T09:28:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.324+0000 7f8fc6e38740 1 -- 172.21.15.12:0/2827252371 wait complete. 2024-05-09T09:28:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.324+0000 7f8fc6e38740 1 -- start start 2024-05-09T09:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.324+0000 7f8fc6e38740 1 Processor -- start 2024-05-09T09:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc6e38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x55903578b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc6e38740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x55903578b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc6e38740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 0x55903578c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc6e38740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1a9a20 con 0x55903577d050 2024-05-09T09:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc6e38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1a9290 con 0x55903577c660 2024-05-09T09:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc6e38740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1a6380 con 0x5590357821c0 2024-05-09T09:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc6e38740 10 client.0 _ll_register_callbacks cb 0x7f8fac009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc16c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x55903578b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc1ec5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x55903578b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc16c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x55903578b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57980/0 (socket says 172.21.15.12:57980) 2024-05-09T09:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc16c4640 1 -- 172.21.15.12:0/2539387997 learned_addr learned my addr 172.21.15.12:0/2539387997 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc26c6640 1 --2- 172.21.15.12:0/2539387997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 0x55903578c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc16c4640 1 -- 172.21.15.12:0/2539387997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 msgr2=0x55903578c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc16c4640 1 --2- 172.21.15.12:0/2539387997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 0x55903578c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc16c4640 1 -- 172.21.15.12:0/2539387997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 msgr2=0x55903578b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc16c4640 1 --2- 172.21.15.12:0/2539387997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x55903578b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc16c4640 1 -- 172.21.15.12:0/2539387997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55903578cb40 con 0x55903577d050 2024-05-09T09:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/2539387997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x55903578b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc16c4640 1 --2- 172.21.15.12:0/2539387997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x55903578b540 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00ffa0 tx=0x7f8fac01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb4c0288096b077 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fa57fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fa57fa640 1 -- 172.21.15.12:0/2539387997 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac00f5e0 con 0x55903577d050 2024-05-09T09:28:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fa57fa640 1 -- 172.21.15.12:0/2539387997 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8fac0054f0 con 0x55903577d050 2024-05-09T09:28:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.325+0000 7f8fc6e38740 1 -- 172.21.15.12:0/2539387997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fac00f780 con 0x55903577d050 2024-05-09T09:28:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.326+0000 7f8fa57fa640 1 -- 172.21.15.12:0/2539387997 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8fac01c9e0 con 0x55903577d050 2024-05-09T09:28:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.326+0000 7f8fc6e38740 10 client.16638 fetch_fsmap learned FSMap version 40 2024-05-09T09:28:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.326+0000 7f8fc6e38740 1 -- 172.21.15.12:0/2539387997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5590357f8720 con 0x55903577d050 2024-05-09T09:28:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.326+0000 7f8fa57fa640 1 -- 172.21.15.12:0/2539387997 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8fac00f780 con 0x55903577d050 2024-05-09T09:28:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.326+0000 7f8fa57fa640 1 -- 172.21.15.12:0/2539387997 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) ==== 8434+0+0 (secure 0 0 0) 0x7f8fac029070 con 0x55903577d050 2024-05-09T09:28:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.326+0000 7f8fc6e38740 10 client.16638 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:28:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.326+0000 7f8fc6e38740 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.326+0000 7f8fc6e38740 20 client.16638 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.326+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/2539387997 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5590357f8d00 0x5590357fb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.326+0000 7f8fc6e38740 4 client.16638 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:28:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.326+0000 7f8fc6e38740 1 -- 172.21.15.12:0/2539387997 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590356618e0 con 0x5590357f8d00 2024-05-09T09:28:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.326+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/2539387997 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5590357f8d00 0x5590357fb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.327+0000 7f8fa57fa640 10 client.16638 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.327+0000 7f8fc1ec5640 1 --2- 172.21.15.12:0/2539387997 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5590357f8d00 0x5590357fb120 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecf8aee6416605b1 server_cookie=62146210c814771e in_seq=0 out_seq=0 2024-05-09T09:28:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.366+0000 7f8fa57fa640 1 -- 172.21.15.12:0/2539387997 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590356618e0 con 0x5590357f8d00 2024-05-09T09:28:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.366+0000 7f8fa57fa640 10 client.16638 handle_command_reply: tid=0 2024-05-09T09:28:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.366+0000 7f8fc6e38740 1 client.16638 shutdown 2024-05-09T09:28:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.366+0000 7f8fc6e38740 1 -- 172.21.15.12:0/2539387997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 msgr2=0x55903578b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.366+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/2539387997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x55903578b540 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00ffa0 tx=0x7f8fac01c4a0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.366+0000 7f8fc6e38740 1 -- 172.21.15.12:0/2539387997 shutdown_connections 2024-05-09T09:28:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.366+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/2539387997 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5590357f8d00 0x5590357fb120 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.366+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/2539387997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590357821c0 0x55903578c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.366+0000 7f8fa57fa640 0 client.16638 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.366+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/2539387997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55903577d050 0x55903578b540 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.366+0000 7f8fc6e38740 1 --2- 172.21.15.12:0/2539387997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55903577c660 0x55903578b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.366+0000 7f8fc6e38740 1 -- 172.21.15.12:0/2539387997 >> 172.21.15.12:0/2539387997 conn(0x5590357738c0 msgr2=0x559035773ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc6e38740 1 -- 172.21.15.12:0/2539387997 shutdown_connections 2024-05-09T09:28:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc6e38740 1 -- 172.21.15.12:0/2539387997 wait complete. 2024-05-09T09:28:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc6e38740 20 client.16638 trim_cache size 0 max 16384 2024-05-09T09:28:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:28:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:28:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:28:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:28:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc4129640 1 -- 172.21.15.12:0/3476697902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8fa80a2cb0 msgr2=0x7f8fa80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc4129640 1 --2- 172.21.15.12:0/3476697902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8fa80a2cb0 0x7f8fa80a5170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005fd0 tx=0x7f8fb4009dd0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc4129640 1 -- 172.21.15.12:0/3476697902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fbc100790 msgr2=0x7f8fbc197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc4129640 1 --2- 172.21.15.12:0/3476697902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fbc100790 0x7f8fbc197b30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fb802a930 tx=0x7f8fb8004f70 comp rx=0 tx=0).stop 2024-05-09T09:28:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc4129640 1 -- 172.21.15.12:0/3476697902 shutdown_connections 2024-05-09T09:28:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc4129640 1 --2- 172.21.15.12:0/3476697902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8fa80a2cb0 0x7f8fa80a5170 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc4129640 1 --2- 172.21.15.12:0/3476697902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fbc102340 0x7f8fbc19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc4129640 1 --2- 172.21.15.12:0/3476697902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fbc101980 0x7f8fbc198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc4129640 1 --2- 172.21.15.12:0/3476697902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fbc100790 0x7f8fbc197b30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc4129640 1 -- 172.21.15.12:0/3476697902 >> 172.21.15.12:0/3476697902 conn(0x7f8fbc0fbd00 msgr2=0x7f8fbc104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc4129640 1 -- 172.21.15.12:0/3476697902 shutdown_connections 2024-05-09T09:28:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:20.367+0000 7f8fc4129640 1 -- 172.21.15.12:0/3476697902 wait complete. 2024-05-09T09:28:20.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:20 smithi012 ceph-mon[30140]: pgmap v451: 129 pgs: 129 active+clean; 153 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 690 KiB/s wr, 246 op/s 2024-05-09T09:28:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:20 smithi170 ceph-mon[33632]: pgmap v451: 129 pgs: 129 active+clean; 153 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 690 KiB/s wr, 246 op/s 2024-05-09T09:28:20.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:20 smithi016 ceph-mon[33335]: pgmap v451: 129 pgs: 129 active+clean; 153 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 690 KiB/s wr, 246 op/s 2024-05-09T09:28:21.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:21 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:21.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:21 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:21.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:21 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:22.404 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:22 smithi016 ceph-mon[33335]: pgmap v452: 129 pgs: 129 active+clean; 151 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 762 KiB/s wr, 311 op/s 2024-05-09T09:28:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:22 smithi012 ceph-mon[30140]: pgmap v452: 129 pgs: 129 active+clean; 151 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 762 KiB/s wr, 311 op/s 2024-05-09T09:28:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:22 smithi170 ceph-mon[33632]: pgmap v452: 129 pgs: 129 active+clean; 151 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 762 KiB/s wr, 311 op/s 2024-05-09T09:28:23.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:23 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:23.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:23 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:23.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:23 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:23.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:28:23.701 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:28:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.818+0000 7f81925c8640 1 -- 172.21.15.12:0/2960005935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818c101960 msgr2=0x7f818c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.818+0000 7f81925c8640 1 --2- 172.21.15.12:0/2960005935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818c101960 0x7f818c101de0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8180009900 tx=0x7f818002a410 comp rx=0 tx=0).stop 2024-05-09T09:28:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.818+0000 7f81925c8640 1 -- 172.21.15.12:0/2960005935 shutdown_connections 2024-05-09T09:28:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.818+0000 7f81925c8640 1 --2- 172.21.15.12:0/2960005935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f818c102320 0x7f818c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.818+0000 7f81925c8640 1 --2- 172.21.15.12:0/2960005935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818c101960 0x7f818c101de0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.818+0000 7f81925c8640 1 --2- 172.21.15.12:0/2960005935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c100770 0x7f818c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.818+0000 7f81925c8640 1 -- 172.21.15.12:0/2960005935 >> 172.21.15.12:0/2960005935 conn(0x7f818c0fbce0 msgr2=0x7f818c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.818+0000 7f81925c8640 1 -- 172.21.15.12:0/2960005935 shutdown_connections 2024-05-09T09:28:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.818+0000 7f81925c8640 1 -- 172.21.15.12:0/2960005935 wait complete. 2024-05-09T09:28:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.818+0000 7f81925c8640 1 Processor -- start 2024-05-09T09:28:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.818+0000 7f81925c8640 1 -- start start 2024-05-09T09:28:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f81925c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c100770 0x7f818c199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f81925c8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818c101960 0x7f818c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f81925c8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f818c102320 0x7f818c1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f81925c8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f818c10d4e0 con 0x7f818c101960 2024-05-09T09:28:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f81925c8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f818c10d360 con 0x7f818c100770 2024-05-09T09:28:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f81925c8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f818c10d660 con 0x7f818c102320 2024-05-09T09:28:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f818b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818c101960 0x7f818c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f818bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c100770 0x7f818c199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f818bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c100770 0x7f818c199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:42966/0 (socket says 172.21.15.12:42966) 2024-05-09T09:28:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f818bfff640 1 -- 172.21.15.12:0/268865867 learned_addr learned my addr 172.21.15.12:0/268865867 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f8190b65640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f818c102320 0x7f818c1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f818b7fe640 1 -- 172.21.15.12:0/268865867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f818c102320 msgr2=0x7f818c1a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:23.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f818b7fe640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f818c102320 0x7f818c1a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f818b7fe640 1 -- 172.21.15.12:0/268865867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c100770 msgr2=0x7f818c199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:23.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f818b7fe640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c100770 0x7f818c199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f818b7fe640 1 -- 172.21.15.12:0/268865867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c1a1990 con 0x7f818c101960 2024-05-09T09:28:23.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f8190b65640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f818c102320 0x7f818c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:28:23.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.819+0000 7f818b7fe640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818c101960 0x7f818c19a2b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8180005ab0 tx=0x7f81800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:23.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.820+0000 7f81897fa640 1 -- 172.21.15.12:0/268865867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818003f030 con 0x7f818c101960 2024-05-09T09:28:23.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.820+0000 7f81897fa640 1 -- 172.21.15.12:0/268865867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8180003ac0 con 0x7f818c101960 2024-05-09T09:28:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.820+0000 7f81925c8640 1 -- 172.21.15.12:0/268865867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818c1a1c20 con 0x7f818c101960 2024-05-09T09:28:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.820+0000 7f81897fa640 1 -- 172.21.15.12:0/268865867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180035070 con 0x7f818c101960 2024-05-09T09:28:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.820+0000 7f81925c8640 1 -- 172.21.15.12:0/268865867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f818c1a2160 con 0x7f818c101960 2024-05-09T09:28:23.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.822+0000 7f81897fa640 1 -- 172.21.15.12:0/268865867 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8180042030 con 0x7f818c101960 2024-05-09T09:28:23.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f81953b1740 1 Processor -- start 2024-05-09T09:28:23.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f81953b1740 1 -- start start 2024-05-09T09:28:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f81953b1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 0x560d76090350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f81953b1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d76095420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f81953b1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d76095a90 0x560d76097f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f81953b1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560d75ff4d00 con 0x560d7608ff30 2024-05-09T09:28:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f81953b1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d75f9c4c0 con 0x560d76095a90 2024-05-09T09:28:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f81953b1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560d75ff0cc0 con 0x560d76090920 2024-05-09T09:28:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f81897fa640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f81740a2c10 0x7f81740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f8190b65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 0x560d76090350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f8190b65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 0x560d76090350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40916/0 (socket says 172.21.15.12:40916) 2024-05-09T09:28:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f8190b65640 1 -- 172.21.15.12:0/2118484238 learned_addr learned my addr 172.21.15.12:0/2118484238 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f818b7fe640 1 --2- 172.21.15.12:0/2118484238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d76095a90 0x560d76097f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f8190b65640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f81740a2c10 0x7f81740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f818bfff640 1 --2- 172.21.15.12:0/2118484238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d76095420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f81897fa640 1 -- 172.21.15.12:0/268865867 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f81800ef8b0 con 0x7f818c101960 2024-05-09T09:28:23.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f8190b65640 1 -- 172.21.15.12:0/2118484238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 msgr2=0x560d76095420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:23.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f8190b65640 1 --2- 172.21.15.12:0/2118484238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d76095420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f8190b65640 1 -- 172.21.15.12:0/2118484238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d76095a90 msgr2=0x560d76097f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:23.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f8190b65640 1 --2- 172.21.15.12:0/2118484238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d76095a90 0x560d76097f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.823+0000 7f8190b65640 1 -- 172.21.15.12:0/2118484238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d75dd27d0 con 0x560d7608ff30 2024-05-09T09:28:23.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.824+0000 7f8190b65640 1 --2- 172.21.15.12:0/2118484238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 0x560d76090350 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8184013ef0 tx=0x7f81840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee1705769a7f32e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:23.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.824+0000 7f8190b65640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f81740a2c10 0x7f81740a50d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8184007310 tx=0x7f8184015000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:23.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.824+0000 7f81727fc640 1 -- 172.21.15.12:0/2118484238 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184021070 con 0x560d7608ff30 2024-05-09T09:28:23.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.824+0000 7f81727fc640 1 -- 172.21.15.12:0/2118484238 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8184007c90 con 0x560d7608ff30 2024-05-09T09:28:23.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.824+0000 7f81727fc640 1 -- 172.21.15.12:0/2118484238 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184005440 con 0x560d7608ff30 2024-05-09T09:28:23.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 -- 172.21.15.12:0/2118484238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 msgr2=0x560d76090350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:23.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 --2- 172.21.15.12:0/2118484238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 0x560d76090350 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8184013ef0 tx=0x7f81840107a0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 -- 172.21.15.12:0/2118484238 shutdown_connections 2024-05-09T09:28:23.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 --2- 172.21.15.12:0/2118484238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d76095a90 0x560d76097f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 --2- 172.21.15.12:0/2118484238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d76095420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 --2- 172.21.15.12:0/2118484238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 0x560d76090350 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 -- 172.21.15.12:0/2118484238 >> 172.21.15.12:0/2118484238 conn(0x560d7608b5a0 msgr2=0x560d7608b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:23.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 -- 172.21.15.12:0/2118484238 shutdown_connections 2024-05-09T09:28:23.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 -- 172.21.15.12:0/2118484238 wait complete. 2024-05-09T09:28:23.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 -- start start 2024-05-09T09:28:23.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 Processor -- start 2024-05-09T09:28:23.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d7608ff30 0x560d7609a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d7609a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 0x560d760a1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560d760989e0 con 0x560d76095a90 2024-05-09T09:28:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d75fafb90 con 0x560d7608ff30 2024-05-09T09:28:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560d76098b60 con 0x560d76090920 2024-05-09T09:28:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.825+0000 7f81953b1740 10 client.0 _ll_register_callbacks cb 0x560d75febda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f8190b65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 0x560d760a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f818bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d7608ff30 0x560d7609a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f8190b65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 0x560d760a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40928/0 (socket says 172.21.15.12:40928) 2024-05-09T09:28:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f818b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d7609a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f8190b65640 1 -- 172.21.15.12:0/4192475400 learned_addr learned my addr 172.21.15.12:0/4192475400 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f8190b65640 1 -- 172.21.15.12:0/4192475400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 msgr2=0x560d7609a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f8190b65640 1 --2- 172.21.15.12:0/4192475400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d7609a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f8190b65640 1 -- 172.21.15.12:0/4192475400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d7608ff30 msgr2=0x560d7609a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f8190b65640 1 --2- 172.21.15.12:0/4192475400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d7608ff30 0x560d7609a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f8190b65640 1 -- 172.21.15.12:0/4192475400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d75d083f0 con 0x560d76095a90 2024-05-09T09:28:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f818bfff640 1 --2- 172.21.15.12:0/4192475400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d7608ff30 0x560d7609a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f818b7fe640 1 --2- 172.21.15.12:0/4192475400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d7609a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f8190b65640 1 --2- 172.21.15.12:0/4192475400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 0x560d760a1930 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8184007780 tx=0x7f8184004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7421f5667266227f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f816b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f816b7fe640 1 -- 172.21.15.12:0/4192475400 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184032060 con 0x560d76095a90 2024-05-09T09:28:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f81953b1740 1 -- 172.21.15.12:0/4192475400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560d76002430 con 0x560d76095a90 2024-05-09T09:28:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f816b7fe640 1 -- 172.21.15.12:0/4192475400 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f81840053d0 con 0x560d76095a90 2024-05-09T09:28:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f816b7fe640 1 -- 172.21.15.12:0/4192475400 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8184021040 con 0x560d76095a90 2024-05-09T09:28:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.826+0000 7f81953b1740 10 client.16650 fetch_fsmap learned FSMap version 41 2024-05-09T09:28:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.827+0000 7f81953b1740 1 -- 172.21.15.12:0/4192475400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560d75cc80f0 con 0x560d76095a90 2024-05-09T09:28:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.827+0000 7f816b7fe640 1 -- 172.21.15.12:0/4192475400 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x560d76002430 con 0x560d76095a90 2024-05-09T09:28:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.827+0000 7f816b7fe640 1 -- 172.21.15.12:0/4192475400 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8438+0+0 (secure 0 0 0) 0x7f818402c030 con 0x560d76095a90 2024-05-09T09:28:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.827+0000 7f81953b1740 10 client.16650 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T09:28:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.827+0000 7f81953b1740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.827+0000 7f81953b1740 20 client.16650 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.827+0000 7f81953b1740 1 --2- 172.21.15.12:0/4192475400 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d760a6170 0x560d760a85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.827+0000 7f81953b1740 4 client.16650 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.827+0000 7f81953b1740 1 -- 172.21.15.12:0/4192475400 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d75ffc780 con 0x560d760a6170 2024-05-09T09:28:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.827+0000 7f818bfff640 1 --2- 172.21.15.12:0/4192475400 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d760a6170 0x560d760a85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.828+0000 7f816b7fe640 10 client.16650 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.828+0000 7f818bfff640 1 --2- 172.21.15.12:0/4192475400 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d760a6170 0x560d760a85a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2257b0dc4cda5a1e server_cookie=dadced38d1442e77 in_seq=0 out_seq=0 2024-05-09T09:28:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.828+0000 7f816b7fe640 1 -- 172.21.15.12:0/4192475400 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d75ffc780 con 0x560d760a6170 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.828+0000 7f816b7fe640 10 client.16650 handle_command_reply: tid=0 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.828+0000 7f81953b1740 1 client.16650 shutdown 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.829+0000 7f81953b1740 1 -- 172.21.15.12:0/4192475400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 msgr2=0x560d760a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.829+0000 7f81953b1740 1 --2- 172.21.15.12:0/4192475400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 0x560d760a1930 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8184007780 tx=0x7f8184004c10 comp rx=0 tx=0).stop 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.829+0000 7f81953b1740 1 -- 172.21.15.12:0/4192475400 shutdown_connections 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.829+0000 7f81953b1740 1 --2- 172.21.15.12:0/4192475400 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d760a6170 0x560d760a85a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.829+0000 7f81953b1740 1 --2- 172.21.15.12:0/4192475400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 0x560d760a1930 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.829+0000 7f81953b1740 1 --2- 172.21.15.12:0/4192475400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d7609a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.829+0000 7f81953b1740 1 --2- 172.21.15.12:0/4192475400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d7608ff30 0x560d7609a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.829+0000 7f816b7fe640 0 client.16650 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.829+0000 7f81953b1740 1 -- 172.21.15.12:0/4192475400 >> 172.21.15.12:0/4192475400 conn(0x560d7608b5a0 msgr2=0x560d7608a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.829+0000 7f81953b1740 1 -- 172.21.15.12:0/4192475400 shutdown_connections 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.829+0000 7f81953b1740 1 -- 172.21.15.12:0/4192475400 wait complete. 2024-05-09T09:28:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.829+0000 7f81953b1740 20 client.16650 trim_cache size 0 max 16384 2024-05-09T09:28:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f81953b1740 1 Processor -- start 2024-05-09T09:28:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f81953b1740 1 -- start start 2024-05-09T09:28:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f81953b1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d7608ff30 0x560d7608e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f81953b1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d760a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f81953b1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 0x560d7608ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f81953b1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1ac780 con 0x560d76095a90 2024-05-09T09:28:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f81953b1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1ae500 con 0x560d7608ff30 2024-05-09T09:28:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f81953b1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1acf10 con 0x560d76090920 2024-05-09T09:28:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f8190b65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 0x560d7608ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f8190b65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 0x560d7608ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40930/0 (socket says 172.21.15.12:40930) 2024-05-09T09:28:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f8190b65640 1 -- 172.21.15.12:0/2887263722 learned_addr learned my addr 172.21.15.12:0/2887263722 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f818bfff640 1 --2- 172.21.15.12:0/2887263722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d7608ff30 0x560d7608e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f8190b65640 1 -- 172.21.15.12:0/2887263722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 msgr2=0x560d760a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f8190b65640 1 --2- 172.21.15.12:0/2887263722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d760a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f8190b65640 1 -- 172.21.15.12:0/2887263722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d7608ff30 msgr2=0x560d7608e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f8190b65640 1 --2- 172.21.15.12:0/2887263722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d7608ff30 0x560d7608e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f8190b65640 1 -- 172.21.15.12:0/2887263722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d760a8c20 con 0x560d76095a90 2024-05-09T09:28:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.847+0000 7f818bfff640 1 --2- 172.21.15.12:0/2887263722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d7608ff30 0x560d7608e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:28:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.848+0000 7f8190b65640 1 --2- 172.21.15.12:0/2887263722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 0x560d7608ead0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8184013a70 tx=0x7f81840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb66c9c844fabae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.848+0000 7f8171ffb640 1 -- 172.21.15.12:0/2887263722 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184032070 con 0x560d76095a90 2024-05-09T09:28:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.848+0000 7f8171ffb640 1 -- 172.21.15.12:0/2887263722 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8184004e80 con 0x560d76095a90 2024-05-09T09:28:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.848+0000 7f8171ffb640 1 -- 172.21.15.12:0/2887263722 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81840058f0 con 0x560d76095a90 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.848+0000 7f81953b1740 1 -- 172.21.15.12:0/2887263722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 msgr2=0x560d7608ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.848+0000 7f81953b1740 1 --2- 172.21.15.12:0/2887263722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 0x560d7608ead0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8184013a70 tx=0x7f81840107a0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.848+0000 7f81953b1740 1 -- 172.21.15.12:0/2887263722 shutdown_connections 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.848+0000 7f81953b1740 1 --2- 172.21.15.12:0/2887263722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d76095a90 0x560d7608ead0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.848+0000 7f81953b1740 1 --2- 172.21.15.12:0/2887263722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d760a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.848+0000 7f81953b1740 1 --2- 172.21.15.12:0/2887263722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d7608ff30 0x560d7608e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.848+0000 7f81953b1740 1 -- 172.21.15.12:0/2887263722 >> 172.21.15.12:0/2887263722 conn(0x560d7608b5a0 msgr2=0x560d76091e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.848+0000 7f81953b1740 1 -- 172.21.15.12:0/2887263722 shutdown_connections 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.848+0000 7f81953b1740 1 -- 172.21.15.12:0/2887263722 wait complete. 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f81953b1740 1 -- start start 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f81953b1740 1 Processor -- start 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f81953b1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 0x560d7609e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f81953b1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d7609eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f81953b1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d76095a90 0x560d7610a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f81953b1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1ade30 con 0x560d7608ff30 2024-05-09T09:28:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f81953b1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1ad6a0 con 0x560d76095a90 2024-05-09T09:28:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f81953b1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1aa760 con 0x560d76090920 2024-05-09T09:28:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f81953b1740 10 client.0 _ll_register_callbacks cb 0x7f81800319d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f818bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 0x560d7609e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f818bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 0x560d7609e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40946/0 (socket says 172.21.15.12:40946) 2024-05-09T09:28:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f818bfff640 1 -- 172.21.15.12:0/3558433834 learned_addr learned my addr 172.21.15.12:0/3558433834 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f8190b65640 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d76095a90 0x560d7610a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f818b7fe640 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d7609eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f818bfff640 1 -- 172.21.15.12:0/3558433834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 msgr2=0x560d7609eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f818bfff640 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d7609eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f818bfff640 1 -- 172.21.15.12:0/3558433834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d76095a90 msgr2=0x560d7610a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f818bfff640 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d76095a90 0x560d7610a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.849+0000 7f818bfff640 1 -- 172.21.15.12:0/3558433834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7610ad00 con 0x560d7608ff30 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.850+0000 7f8190b65640 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d76095a90 0x560d7610a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.850+0000 7f818b7fe640 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d7609eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.850+0000 7f818bfff640 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 0x560d7609e860 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f817c000c00 tx=0x7f817c012b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d164269cc74469 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.850+0000 7f816b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.850+0000 7f816b7fe640 1 -- 172.21.15.12:0/3558433834 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c004230 con 0x560d7608ff30 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.850+0000 7f816b7fe640 1 -- 172.21.15.12:0/3558433834 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f817c0043d0 con 0x560d7608ff30 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.850+0000 7f81953b1740 1 -- 172.21.15.12:0/3558433834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8184004e80 con 0x560d7608ff30 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.850+0000 7f816b7fe640 1 -- 172.21.15.12:0/3558433834 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f817c02f050 con 0x560d7608ff30 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.850+0000 7f81953b1740 10 client.16656 fetch_fsmap learned FSMap version 41 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.850+0000 7f81953b1740 1 -- 172.21.15.12:0/3558433834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560d7610c080 con 0x560d7608ff30 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.850+0000 7f816b7fe640 1 -- 172.21.15.12:0/3558433834 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8184004e80 con 0x560d7608ff30 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.850+0000 7f816b7fe640 1 -- 172.21.15.12:0/3558433834 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8438+0+0 (secure 0 0 0) 0x7f817c0035f0 con 0x560d7608ff30 2024-05-09T09:28:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 10 client.16656 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 20 client.16656 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d7610caf0 0x560d7610ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 4 client.16656 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 1 -- 172.21.15.12:0/3558433834 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d75f796f0 con 0x560d7610caf0 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f818b7fe640 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d7610caf0 0x560d7610ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f816b7fe640 10 client.16656 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f818b7fe640 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d7610caf0 0x560d7610ef10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=131d553c0918ac9a server_cookie=b15c82df9bb8f639 in_seq=0 out_seq=0 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f816b7fe640 1 -- 172.21.15.12:0/3558433834 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560d75f796f0 con 0x560d7610caf0 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f816b7fe640 10 client.16656 handle_command_reply: tid=0 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 1 client.16656 shutdown 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 1 -- 172.21.15.12:0/3558433834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 msgr2=0x560d7609e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 0x560d7609e860 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f817c000c00 tx=0x7f817c012b40 comp rx=0 tx=0).stop 2024-05-09T09:28:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 1 -- 172.21.15.12:0/3558433834 shutdown_connections 2024-05-09T09:28:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x560d7610caf0 0x560d7610ef10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d76095a90 0x560d7610a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f816b7fe640 0 client.16656 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d76090920 0x560d7609eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.851+0000 7f81953b1740 1 --2- 172.21.15.12:0/3558433834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d7608ff30 0x560d7609e860 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81953b1740 1 -- 172.21.15.12:0/3558433834 >> 172.21.15.12:0/3558433834 conn(0x560d7608b5a0 msgr2=0x560d7608e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81953b1740 1 -- 172.21.15.12:0/3558433834 shutdown_connections 2024-05-09T09:28:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81953b1740 1 -- 172.21.15.12:0/3558433834 wait complete. 2024-05-09T09:28:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81953b1740 20 client.16656 trim_cache size 0 max 16384 2024-05-09T09:28:23.859 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:28:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T09:28:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T09:28:23.860 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:28:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81925c8640 1 -- 172.21.15.12:0/268865867 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f81740a2c10 msgr2=0x7f81740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81925c8640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f81740a2c10 0x7f81740a50d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8184007310 tx=0x7f8184015000 comp rx=0 tx=0).stop 2024-05-09T09:28:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81925c8640 1 -- 172.21.15.12:0/268865867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818c101960 msgr2=0x7f818c19a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81925c8640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818c101960 0x7f818c19a2b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8180005ab0 tx=0x7f81800042f0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81925c8640 1 -- 172.21.15.12:0/268865867 shutdown_connections 2024-05-09T09:28:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81925c8640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f81740a2c10 0x7f81740a50d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81925c8640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f818c102320 0x7f818c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81925c8640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818c101960 0x7f818c19a2b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81925c8640 1 --2- 172.21.15.12:0/268865867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818c100770 0x7f818c199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81925c8640 1 -- 172.21.15.12:0/268865867 >> 172.21.15.12:0/268865867 conn(0x7f818c0fbce0 msgr2=0x7f818c109570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81925c8640 1 -- 172.21.15.12:0/268865867 shutdown_connections 2024-05-09T09:28:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.852+0000 7f81925c8640 1 -- 172.21.15.12:0/268865867 wait complete. 2024-05-09T09:28:23.864 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:28:23.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:28:23.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.972+0000 7f1ea63c1640 1 -- 172.21.15.12:0/3131538839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ea0100a10 msgr2=0x7f1ea0100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:23.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.972+0000 7f1ea63c1640 1 --2- 172.21.15.12:0/3131538839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ea0100a10 0x7f1ea0100e90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009930 tx=0x7f1e9402a420 comp rx=0 tx=0).stop 2024-05-09T09:28:23.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.972+0000 7f1ea63c1640 1 -- 172.21.15.12:0/3131538839 shutdown_connections 2024-05-09T09:28:23.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.972+0000 7f1ea63c1640 1 --2- 172.21.15.12:0/3131538839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea01013d0 0x7f1ea0108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.972+0000 7f1ea63c1640 1 --2- 172.21.15.12:0/3131538839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ea0100a10 0x7f1ea0100e90 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.972+0000 7f1ea63c1640 1 --2- 172.21.15.12:0/3131538839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ea0101d70 0x7f1ea0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.972+0000 7f1ea63c1640 1 -- 172.21.15.12:0/3131538839 >> 172.21.15.12:0/3131538839 conn(0x7f1ea00fb460 msgr2=0x7f1ea00fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:23.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.972+0000 7f1ea63c1640 1 -- 172.21.15.12:0/3131538839 shutdown_connections 2024-05-09T09:28:23.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.973+0000 7f1ea63c1640 1 -- 172.21.15.12:0/3131538839 wait complete. 2024-05-09T09:28:23.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.973+0000 7f1ea63c1640 1 Processor -- start 2024-05-09T09:28:23.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.973+0000 7f1ea63c1640 1 -- start start 2024-05-09T09:28:23.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.973+0000 7f1ea63c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ea0100a10 0x7f1ea0197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.973+0000 7f1ea63c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea01013d0 0x7f1ea0198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.973+0000 7f1ea63c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ea0101d70 0x7f1ea019f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.973+0000 7f1ea63c1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea010c410 con 0x7f1ea0100a10 2024-05-09T09:28:23.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.973+0000 7f1ea63c1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea010c290 con 0x7f1ea01013d0 2024-05-09T09:28:23.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.973+0000 7f1ea63c1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea010c590 con 0x7f1ea0101d70 2024-05-09T09:28:23.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ea0100a10 0x7f1ea0197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea01013d0 0x7f1ea0198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ea0100a10 0x7f1ea0197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40962/0 (socket says 172.21.15.12:40962) 2024-05-09T09:28:23.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9ffff640 1 -- 172.21.15.12:0/2881045645 learned_addr learned my addr 172.21.15.12:0/2881045645 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:23.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1ea495e640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ea0101d70 0x7f1ea019f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9ffff640 1 -- 172.21.15.12:0/2881045645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ea0101d70 msgr2=0x7f1ea019f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:23.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9ffff640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ea0101d70 0x7f1ea019f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9ffff640 1 -- 172.21.15.12:0/2881045645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea01013d0 msgr2=0x7f1ea0198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:23.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9ffff640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea01013d0 0x7f1ea0198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:23.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9ffff640 1 -- 172.21.15.12:0/2881045645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea019f860 con 0x7f1ea0100a10 2024-05-09T09:28:23.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1ea495e640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ea0101d70 0x7f1ea019f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:23.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9f7fe640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea01013d0 0x7f1ea0198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:23.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9ffff640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ea0100a10 0x7f1ea0197c50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000e9c0 tx=0x7f1e9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:23.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9d7fa640 1 -- 172.21.15.12:0/2881045645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000cd80 con 0x7f1ea0100a10 2024-05-09T09:28:23.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1ea63c1640 1 -- 172.21.15.12:0/2881045645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea019fb50 con 0x7f1ea0100a10 2024-05-09T09:28:23.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9d7fa640 1 -- 172.21.15.12:0/2881045645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1e900035f0 con 0x7f1ea0100a10 2024-05-09T09:28:23.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1e9d7fa640 1 -- 172.21.15.12:0/2881045645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90010690 con 0x7f1ea0100a10 2024-05-09T09:28:23.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.974+0000 7f1ea63c1640 1 -- 172.21.15.12:0/2881045645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea01a00a0 con 0x7f1ea0100a10 2024-05-09T09:28:23.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.975+0000 7f1ea63c1640 1 -- 172.21.15.12:0/2881045645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea01a0270 con 0x7f1ea0100a10 2024-05-09T09:28:23.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.977+0000 7f1e9d7fa640 1 -- 172.21.15.12:0/2881045645 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1e900037a0 con 0x7f1ea0100a10 2024-05-09T09:28:23.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.978+0000 7f1e9d7fa640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1e880a2b90 0x7f1e880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:23.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.978+0000 7f1e9f7fe640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1e880a2b90 0x7f1e880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:23.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.978+0000 7f1e9d7fa640 1 -- 172.21.15.12:0/2881045645 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f1e900cf880 con 0x7f1ea0100a10 2024-05-09T09:28:23.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.978+0000 7f1e9f7fe640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1e880a2b90 0x7f1e880a5050 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1e94005ab0 tx=0x7f1e94000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:23.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:23.986+0000 7f1e9d7fa640 1 -- 172.21.15.12:0/2881045645 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1e9009c410 con 0x7f1ea0100a10 2024-05-09T09:28:24.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.348+0000 7f1ea63c1640 1 -- 172.21.15.12:0/2881045645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ea01a0440 con 0x7f1ea0100a10 2024-05-09T09:28:24.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.348+0000 7f1e9d7fa640 1 -- 172.21.15.12:0/2881045645 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11964 (secure 0 0 0) 0x7f1e9009bd10 con 0x7f1ea0100a10 2024-05-09T09:28:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:28:24.349 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:28:21.638403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25883":{"gid":25883,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3067829157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3067829157},{"type":"v1","addr":"172.21.15.16:6837","nonce":3067829157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[14628,24484,24515,24523,24502]},"id":1}]} 2024-05-09T09:28:24.350 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 41 2024-05-09T09:28:24.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.350+0000 7f1ea63c1640 1 -- 172.21.15.12:0/2881045645 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1e880a2b90 msgr2=0x7f1e880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.350+0000 7f1ea63c1640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1e880a2b90 0x7f1e880a5050 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1e94005ab0 tx=0x7f1e94000bc0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.350+0000 7f1ea63c1640 1 -- 172.21.15.12:0/2881045645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ea0100a10 msgr2=0x7f1ea0197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.350+0000 7f1ea63c1640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ea0100a10 0x7f1ea0197c50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000e9c0 tx=0x7f1e9000ee80 comp rx=0 tx=0).stop 2024-05-09T09:28:24.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.351+0000 7f1ea63c1640 1 -- 172.21.15.12:0/2881045645 shutdown_connections 2024-05-09T09:28:24.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.351+0000 7f1ea63c1640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1e880a2b90 0x7f1e880a5050 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.351+0000 7f1ea63c1640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ea0101d70 0x7f1ea019f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.351+0000 7f1ea63c1640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea01013d0 0x7f1ea0198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.351+0000 7f1ea63c1640 1 --2- 172.21.15.12:0/2881045645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ea0100a10 0x7f1ea0197c50 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.351+0000 7f1ea63c1640 1 -- 172.21.15.12:0/2881045645 >> 172.21.15.12:0/2881045645 conn(0x7f1ea00fb460 msgr2=0x7f1ea00fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.351+0000 7f1ea63c1640 1 -- 172.21.15.12:0/2881045645 shutdown_connections 2024-05-09T09:28:24.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.351+0000 7f1ea63c1640 1 -- 172.21.15.12:0/2881045645 wait complete. 2024-05-09T09:28:24.364 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-09T09:28:24.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:24 smithi012 ceph-mon[30140]: pgmap v453: 129 pgs: 129 active+clean; 147 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 879 KiB/s wr, 337 op/s 2024-05-09T09:28:24.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.480+0000 7fb93cf30640 1 -- 172.21.15.12:0/1723576465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb938100770 msgr2=0x7fb938100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.480+0000 7fb93cf30640 1 --2- 172.21.15.12:0/1723576465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb938100770 0x7fb938100b90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb928009900 tx=0x7fb92802a410 comp rx=0 tx=0).stop 2024-05-09T09:28:24.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.480+0000 7fb93cf30640 1 -- 172.21.15.12:0/1723576465 shutdown_connections 2024-05-09T09:28:24.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.480+0000 7fb93cf30640 1 --2- 172.21.15.12:0/1723576465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb938102320 0x7fb938108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.480+0000 7fb93cf30640 1 --2- 172.21.15.12:0/1723576465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb938101960 0x7fb938101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.480+0000 7fb93cf30640 1 --2- 172.21.15.12:0/1723576465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb938100770 0x7fb938100b90 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.480+0000 7fb93cf30640 1 -- 172.21.15.12:0/1723576465 >> 172.21.15.12:0/1723576465 conn(0x7fb9380fbce0 msgr2=0x7fb9380fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.480+0000 7fb93cf30640 1 -- 172.21.15.12:0/1723576465 shutdown_connections 2024-05-09T09:28:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.480+0000 7fb93cf30640 1 -- 172.21.15.12:0/1723576465 wait complete. 2024-05-09T09:28:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.480+0000 7fb93cf30640 1 Processor -- start 2024-05-09T09:28:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.480+0000 7fb93cf30640 1 -- start start 2024-05-09T09:28:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93cf30640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb938100770 0x7fb9381959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93cf30640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb938101960 0x7fb938195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93cf30640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb938102320 0x7fb93819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93cf30640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb93810b2a0 con 0x7fb938100770 2024-05-09T09:28:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93cf30640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb93810b120 con 0x7fb938102320 2024-05-09T09:28:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93cf30640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb93810b420 con 0x7fb938101960 2024-05-09T09:28:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb938100770 0x7fb9381959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb938100770 0x7fb9381959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40984/0 (socket says 172.21.15.12:40984) 2024-05-09T09:28:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93659c640 1 -- 172.21.15.12:0/3978090773 learned_addr learned my addr 172.21.15.12:0/3978090773 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb92ffff640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb938101960 0x7fb938195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb936d9d640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb938102320 0x7fb93819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93659c640 1 -- 172.21.15.12:0/3978090773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb938101960 msgr2=0x7fb938195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93659c640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb938101960 0x7fb938195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93659c640 1 -- 172.21.15.12:0/3978090773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb938102320 msgr2=0x7fb93819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93659c640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb938102320 0x7fb93819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93659c640 1 -- 172.21.15.12:0/3978090773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93819d5c0 con 0x7fb938100770 2024-05-09T09:28:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.481+0000 7fb93659c640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb938100770 0x7fb9381959a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb928005ab0 tx=0x7fb9280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.482+0000 7fb92f7fe640 1 -- 172.21.15.12:0/3978090773 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92803f030 con 0x7fb938100770 2024-05-09T09:28:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.482+0000 7fb93cf30640 1 -- 172.21.15.12:0/3978090773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93819d850 con 0x7fb938100770 2024-05-09T09:28:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.482+0000 7fb92f7fe640 1 -- 172.21.15.12:0/3978090773 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb928031980 con 0x7fb938100770 2024-05-09T09:28:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.482+0000 7fb92f7fe640 1 -- 172.21.15.12:0/3978090773 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928035070 con 0x7fb938100770 2024-05-09T09:28:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.482+0000 7fb93cf30640 1 -- 172.21.15.12:0/3978090773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93819dd90 con 0x7fb938100770 2024-05-09T09:28:24.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.484+0000 7fb92f7fe640 1 -- 172.21.15.12:0/3978090773 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb928042030 con 0x7fb938100770 2024-05-09T09:28:24.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.484+0000 7fb93fc3f740 1 Processor -- start 2024-05-09T09:28:24.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb93fc3f740 1 -- start start 2024-05-09T09:28:24.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb93fc3f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 0x5624cbbee360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb93fc3f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbee930 0x5624cbbf3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb93fc3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbf3aa0 0x5624cbbf5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb93fc3f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5624cbb52c40 con 0x5624cbbedf40 2024-05-09T09:28:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb93fc3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5624cbafa580 con 0x5624cbbf3aa0 2024-05-09T09:28:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb93fc3f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5624cbb4ec00 con 0x5624cbbee930 2024-05-09T09:28:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 0x5624cbbee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb93659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbee930 0x5624cbbf3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 0x5624cbbee360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40988/0 (socket says 172.21.15.12:40988) 2024-05-09T09:28:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb936d9d640 1 -- 172.21.15.12:0/3742828306 learned_addr learned my addr 172.21.15.12:0/3742828306 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb92ffff640 1 --2- 172.21.15.12:0/3742828306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbf3aa0 0x5624cbbf5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb93659c640 1 -- 172.21.15.12:0/3742828306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbf3aa0 msgr2=0x5624cbbf5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb93659c640 1 --2- 172.21.15.12:0/3742828306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbf3aa0 0x5624cbbf5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb93659c640 1 -- 172.21.15.12:0/3742828306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 msgr2=0x5624cbbee360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb93659c640 1 --2- 172.21.15.12:0/3742828306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 0x5624cbbee360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb93659c640 1 -- 172.21.15.12:0/3742828306 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624cb8663f0 con 0x5624cbbee930 2024-05-09T09:28:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.485+0000 7fb936d9d640 1 --2- 172.21.15.12:0/3742828306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 0x5624cbbee360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.486+0000 7fb92f7fe640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb9240a2b90 0x7fb9240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.486+0000 7fb93659c640 1 --2- 172.21.15.12:0/3742828306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbee930 0x5624cbbf3430 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb9280ecfe0 tx=0x7fb9280f5cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b60e071dcdb5309b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.486+0000 7fb936d9d640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb9240a2b90 0x7fb9240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.486+0000 7fb92f7fe640 1 -- 172.21.15.12:0/3978090773 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb9280ef8e0 con 0x7fb938100770 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.486+0000 7fb92cff9640 1 -- 172.21.15.12:0/3742828306 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9280b8010 con 0x5624cbbee930 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.486+0000 7fb92cff9640 1 -- 172.21.15.12:0/3742828306 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb9280bc470 con 0x5624cbbee930 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.486+0000 7fb92cff9640 1 -- 172.21.15.12:0/3742828306 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9280bb6d0 con 0x5624cbbee930 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.487+0000 7fb93fc3f740 1 -- 172.21.15.12:0/3742828306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbee930 msgr2=0x5624cbbf3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.487+0000 7fb936d9d640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb9240a2b90 0x7fb9240a5050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x5624cbbef190 tx=0x7fb930009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.487+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/3742828306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbee930 0x5624cbbf3430 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb9280ecfe0 tx=0x7fb9280f5cd0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.487+0000 7fb93fc3f740 1 -- 172.21.15.12:0/3742828306 shutdown_connections 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.487+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/3742828306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbf3aa0 0x5624cbbf5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.487+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/3742828306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbee930 0x5624cbbf3430 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.487+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/3742828306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 0x5624cbbee360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.487+0000 7fb93fc3f740 1 -- 172.21.15.12:0/3742828306 >> 172.21.15.12:0/3742828306 conn(0x5624cbbe95b0 msgr2=0x5624cbbe99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.487+0000 7fb93fc3f740 1 -- 172.21.15.12:0/3742828306 shutdown_connections 2024-05-09T09:28:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.487+0000 7fb93fc3f740 1 -- 172.21.15.12:0/3742828306 wait complete. 2024-05-09T09:28:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.487+0000 7fb93fc3f740 1 -- start start 2024-05-09T09:28:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.487+0000 7fb93fc3f740 1 Processor -- start 2024-05-09T09:28:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93fc3f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 0x5624cbbfc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93fc3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 0x5624cbbfc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93fc3f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbf3aa0 0x5624cbc03990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93fc3f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5624cbbfac20 con 0x5624cbbedf40 2024-05-09T09:28:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93fc3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5624cbb0dc40 con 0x5624cbbee930 2024-05-09T09:28:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93fc3f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5624cbbfada0 con 0x5624cbbf3aa0 2024-05-09T09:28:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 0x5624cbbfc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93fc3f740 10 client.0 _ll_register_callbacks cb 0x5624cb865be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 0x5624cbbfc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41002/0 (socket says 172.21.15.12:41002) 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93659c640 1 -- 172.21.15.12:0/1641025124 learned_addr learned my addr 172.21.15.12:0/1641025124 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb92ffff640 1 --2- 172.21.15.12:0/1641025124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 0x5624cbbfc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb936d9d640 1 --2- 172.21.15.12:0/1641025124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbf3aa0 0x5624cbc03990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93659c640 1 -- 172.21.15.12:0/1641025124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbf3aa0 msgr2=0x5624cbc03990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93659c640 1 --2- 172.21.15.12:0/1641025124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbf3aa0 0x5624cbc03990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93659c640 1 -- 172.21.15.12:0/1641025124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 msgr2=0x5624cbbfc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93659c640 1 --2- 172.21.15.12:0/1641025124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 0x5624cbbfc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93659c640 1 -- 172.21.15.12:0/1641025124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624cb9307d0 con 0x5624cbbedf40 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93659c640 1 --2- 172.21.15.12:0/1641025124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 0x5624cbbfc410 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb9280c4ba0 tx=0x7fb92802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92c75d1a726fdc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb91dffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb91dffb640 1 -- 172.21.15.12:0/1641025124 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928049040 con 0x5624cbbedf40 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb91dffb640 1 -- 172.21.15.12:0/1641025124 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb9280b83e0 con 0x5624cbbedf40 2024-05-09T09:28:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.488+0000 7fb93fc3f740 1 -- 172.21.15.12:0/1641025124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9280bc470 con 0x5624cbbedf40 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb91dffb640 1 -- 172.21.15.12:0/1641025124 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb9280bb8b0 con 0x5624cbbedf40 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb93fc3f740 10 client.16674 fetch_fsmap learned FSMap version 41 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb93fc3f740 1 -- 172.21.15.12:0/1641025124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5624cbbfac20 con 0x5624cbbedf40 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb91dffb640 1 -- 172.21.15.12:0/1641025124 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb9280bc470 con 0x5624cbbedf40 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb91dffb640 1 -- 172.21.15.12:0/1641025124 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8438+0+0 (secure 0 0 0) 0x7fb9280bc060 con 0x5624cbbedf40 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb93fc3f740 10 client.16674 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb93fc3f740 10 client.16674 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb93fc3f740 20 client.16674 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/1641025124 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5624cbc08220 0x5624cbc0a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb93fc3f740 4 client.16674 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb93fc3f740 1 -- 172.21.15.12:0/1641025124 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624cbb60370 con 0x5624cbc08220 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb92ffff640 1 --2- 172.21.15.12:0/1641025124 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5624cbc08220 0x5624cbc0a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb92ffff640 1 --2- 172.21.15.12:0/1641025124 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5624cbc08220 0x5624cbc0a6a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cd7ac5e938c7da4 server_cookie=921a060d0ca0c82a in_seq=0 out_seq=0 2024-05-09T09:28:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.489+0000 7fb91dffb640 10 client.16674 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.490+0000 7fb91dffb640 1 -- 172.21.15.12:0/1641025124 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5624cbb60370 con 0x5624cbc08220 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.490+0000 7fb91dffb640 10 client.16674 handle_command_reply: tid=0 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.490+0000 7fb93fc3f740 1 client.16674 shutdown 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.490+0000 7fb93fc3f740 1 -- 172.21.15.12:0/1641025124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 msgr2=0x5624cbbfc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.490+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/1641025124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 0x5624cbbfc410 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb9280c4ba0 tx=0x7fb92802a410 comp rx=0 tx=0).stop 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.490+0000 7fb93fc3f740 1 -- 172.21.15.12:0/1641025124 shutdown_connections 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.490+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/1641025124 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5624cbc08220 0x5624cbc0a6a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.490+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/1641025124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbf3aa0 0x5624cbc03990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.490+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/1641025124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 0x5624cbbfc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.490+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/1641025124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbedf40 0x5624cbbfc410 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.490+0000 7fb91dffb640 0 client.16674 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.490+0000 7fb93fc3f740 1 -- 172.21.15.12:0/1641025124 >> 172.21.15.12:0/1641025124 conn(0x5624cbbe95b0 msgr2=0x5624cbbe87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.491+0000 7fb93fc3f740 1 -- 172.21.15.12:0/1641025124 shutdown_connections 2024-05-09T09:28:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.491+0000 7fb93fc3f740 1 -- 172.21.15.12:0/1641025124 wait complete. 2024-05-09T09:28:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.491+0000 7fb93fc3f740 20 client.16674 trim_cache size 0 max 16384 2024-05-09T09:28:24.499 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:24 smithi170 ceph-mon[33632]: pgmap v453: 129 pgs: 129 active+clean; 147 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 879 KiB/s wr, 337 op/s 2024-05-09T09:28:24.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:24 smithi016 ceph-mon[33335]: pgmap v453: 129 pgs: 129 active+clean; 147 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 879 KiB/s wr, 337 op/s 2024-05-09T09:28:24.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb93fc3f740 1 Processor -- start 2024-05-09T09:28:24.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb93fc3f740 1 -- start start 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb93fc3f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbedf40 0x5624cbbfc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb93fc3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 0x5624cbc03950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb93fc3f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbf3aa0 0x5624cbbec560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb93fc3f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381a8420 con 0x5624cbbf3aa0 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb93fc3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381aa1a0 con 0x5624cbbee930 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb93fc3f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381a8bb0 con 0x5624cbbedf40 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbf3aa0 0x5624cbbec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbf3aa0 0x5624cbbec560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41004/0 (socket says 172.21.15.12:41004) 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb936d9d640 1 -- 172.21.15.12:0/83652455 learned_addr learned my addr 172.21.15.12:0/83652455 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb93659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbedf40 0x5624cbbfc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb92ffff640 1 --2- 172.21.15.12:0/83652455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 0x5624cbc03950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb936d9d640 1 -- 172.21.15.12:0/83652455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbedf40 msgr2=0x5624cbbfc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb936d9d640 1 --2- 172.21.15.12:0/83652455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbedf40 0x5624cbbfc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb936d9d640 1 -- 172.21.15.12:0/83652455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 msgr2=0x5624cbc03950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb936d9d640 1 --2- 172.21.15.12:0/83652455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 0x5624cbc03950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb936d9d640 1 -- 172.21.15.12:0/83652455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624cbc0ad20 con 0x5624cbbf3aa0 2024-05-09T09:28:24.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb92ffff640 1 --2- 172.21.15.12:0/83652455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 0x5624cbc03950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:28:24.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.506+0000 7fb93659c640 1 --2- 172.21.15.12:0/83652455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbedf40 0x5624cbbfc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:24.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.507+0000 7fb936d9d640 1 --2- 172.21.15.12:0/83652455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbf3aa0 0x5624cbbec560 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb930011e80 tx=0x7fb930016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197f8e9424d5903c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.507+0000 7fb91ffff640 1 -- 172.21.15.12:0/83652455 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930010d80 con 0x5624cbbf3aa0 2024-05-09T09:28:24.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.507+0000 7fb91ffff640 1 -- 172.21.15.12:0/83652455 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb93001f930 con 0x5624cbbf3aa0 2024-05-09T09:28:24.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.507+0000 7fb91ffff640 1 -- 172.21.15.12:0/83652455 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9300053b0 con 0x5624cbbf3aa0 2024-05-09T09:28:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.507+0000 7fb93fc3f740 1 -- 172.21.15.12:0/83652455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbf3aa0 msgr2=0x5624cbbec560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.507+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/83652455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbf3aa0 0x5624cbbec560 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb930011e80 tx=0x7fb930016000 comp rx=0 tx=0).stop 2024-05-09T09:28:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.507+0000 7fb93fc3f740 1 -- 172.21.15.12:0/83652455 shutdown_connections 2024-05-09T09:28:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.507+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/83652455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbf3aa0 0x5624cbbec560 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.507+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/83652455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 0x5624cbc03950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.507+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/83652455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbedf40 0x5624cbbfc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.507+0000 7fb93fc3f740 1 -- 172.21.15.12:0/83652455 >> 172.21.15.12:0/83652455 conn(0x5624cbbe95b0 msgr2=0x5624cbbfc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.507+0000 7fb93fc3f740 1 -- 172.21.15.12:0/83652455 shutdown_connections 2024-05-09T09:28:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.507+0000 7fb93fc3f740 1 -- 172.21.15.12:0/83652455 wait complete. 2024-05-09T09:28:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93fc3f740 1 -- start start 2024-05-09T09:28:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93fc3f740 1 Processor -- start 2024-05-09T09:28:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93fc3f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbedf40 0x5624cbc010e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93fc3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 0x5624cbc01620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93fc3f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbf3aa0 0x5624cbc02630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93fc3f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381a9ad0 con 0x5624cbbf3aa0 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93fc3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381a9340 con 0x5624cbbee930 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93fc3f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381a6400 con 0x5624cbbedf40 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93fc3f740 10 client.0 _ll_register_callbacks cb 0x7fb930009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbedf40 0x5624cbc010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbedf40 0x5624cbc010e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:37700/0 (socket says 172.21.15.12:37700) 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93659c640 1 -- 172.21.15.12:0/981561510 learned_addr learned my addr 172.21.15.12:0/981561510 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb936d9d640 1 --2- 172.21.15.12:0/981561510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbf3aa0 0x5624cbc02630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb92ffff640 1 --2- 172.21.15.12:0/981561510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 0x5624cbc01620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93659c640 1 -- 172.21.15.12:0/981561510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 msgr2=0x5624cbc01620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93659c640 1 --2- 172.21.15.12:0/981561510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 0x5624cbc01620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93659c640 1 -- 172.21.15.12:0/981561510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbf3aa0 msgr2=0x5624cbc02630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93659c640 1 --2- 172.21.15.12:0/981561510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbf3aa0 0x5624cbc02630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.508+0000 7fb93659c640 1 -- 172.21.15.12:0/981561510 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624cbc02c20 con 0x5624cbbedf40 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb93659c640 1 --2- 172.21.15.12:0/981561510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbedf40 0x5624cbc010e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb9280b81d0 tx=0x7fb9280c4a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7169d2b3efa1e31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb91dffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb91dffb640 1 -- 172.21.15.12:0/981561510 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928049040 con 0x5624cbbedf40 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb93fc3f740 1 -- 172.21.15.12:0/981561510 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb93001f930 con 0x5624cbbedf40 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb91dffb640 1 -- 172.21.15.12:0/981561510 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb9280bbfd0 con 0x5624cbbedf40 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb91dffb640 1 -- 172.21.15.12:0/981561510 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb9280c9050 con 0x5624cbbedf40 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb93fc3f740 10 client.26227 fetch_fsmap learned FSMap version 41 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb93fc3f740 1 -- 172.21.15.12:0/981561510 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5624cbc02c20 con 0x5624cbbedf40 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb91dffb640 1 -- 172.21.15.12:0/981561510 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb93001f930 con 0x5624cbbedf40 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb91dffb640 1 -- 172.21.15.12:0/981561510 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) ==== 8438+0+0 (secure 0 0 0) 0x7fb9280bb290 con 0x5624cbbedf40 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb93fc3f740 10 client.26227 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb93fc3f740 10 client.26227 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:28:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb93fc3f740 20 client.26227 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/981561510 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5624cbc6ef10 0x5624cbc71330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb93fc3f740 4 client.26227 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.509+0000 7fb93fc3f740 1 -- 172.21.15.12:0/981561510 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624cbb5a6c0 con 0x5624cbc6ef10 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb92ffff640 1 --2- 172.21.15.12:0/981561510 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5624cbc6ef10 0x5624cbc71330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb92ffff640 1 --2- 172.21.15.12:0/981561510 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5624cbc6ef10 0x5624cbc71330 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b2c401c47992d4dc server_cookie=8852e504d329d82d in_seq=0 out_seq=0 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb91dffb640 10 client.26227 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb91dffb640 1 -- 172.21.15.12:0/981561510 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624cbb5a6c0 con 0x5624cbc6ef10 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb91dffb640 10 client.26227 handle_command_reply: tid=0 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb93fc3f740 1 client.26227 shutdown 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb93fc3f740 1 -- 172.21.15.12:0/981561510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbedf40 msgr2=0x5624cbc010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/981561510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbedf40 0x5624cbc010e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb9280b81d0 tx=0x7fb9280c4a50 comp rx=0 tx=0).stop 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb93fc3f740 1 -- 172.21.15.12:0/981561510 shutdown_connections 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/981561510 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5624cbc6ef10 0x5624cbc71330 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/981561510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624cbbf3aa0 0x5624cbc02630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb91dffb640 0 client.26227 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:28:24.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/981561510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624cbbee930 0x5624cbc01620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb93fc3f740 1 --2- 172.21.15.12:0/981561510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624cbbedf40 0x5624cbc010e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb93fc3f740 1 -- 172.21.15.12:0/981561510 >> 172.21.15.12:0/981561510 conn(0x5624cbbe95b0 msgr2=0x5624cbbec560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb93fc3f740 1 -- 172.21.15.12:0/981561510 shutdown_connections 2024-05-09T09:28:24.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.510+0000 7fb93fc3f740 1 -- 172.21.15.12:0/981561510 wait complete. 2024-05-09T09:28:24.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.511+0000 7fb93fc3f740 20 client.26227 trim_cache size 0 max 16384 2024-05-09T09:28:24.518 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:28:24.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.511+0000 7fb93cf30640 1 -- 172.21.15.12:0/3978090773 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb9240a2b90 msgr2=0x7fb9240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.511+0000 7fb93cf30640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb9240a2b90 0x7fb9240a5050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x5624cbbef190 tx=0x7fb930009500 comp rx=0 tx=0).stop 2024-05-09T09:28:24.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.511+0000 7fb93cf30640 1 -- 172.21.15.12:0/3978090773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb938100770 msgr2=0x7fb9381959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.511+0000 7fb93cf30640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb938100770 0x7fb9381959a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb928005ab0 tx=0x7fb9280042f0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.511+0000 7fb93cf30640 1 -- 172.21.15.12:0/3978090773 shutdown_connections 2024-05-09T09:28:24.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.511+0000 7fb93cf30640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb9240a2b90 0x7fb9240a5050 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.511+0000 7fb93cf30640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb938102320 0x7fb93819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.511+0000 7fb93cf30640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb938101960 0x7fb938195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.511+0000 7fb93cf30640 1 --2- 172.21.15.12:0/3978090773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb938100770 0x7fb9381959a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.511+0000 7fb93cf30640 1 -- 172.21.15.12:0/3978090773 >> 172.21.15.12:0/3978090773 conn(0x7fb9380fbce0 msgr2=0x7fb938104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.511+0000 7fb93cf30640 1 -- 172.21.15.12:0/3978090773 shutdown_connections 2024-05-09T09:28:24.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.511+0000 7fb93cf30640 1 -- 172.21.15.12:0/3978090773 wait complete. 2024-05-09T09:28:24.523 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-09T09:28:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.675+0000 7f9472df9640 1 -- 172.21.15.12:0/1537750446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f946c101960 msgr2=0x7f946c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.675+0000 7f9472df9640 1 --2- 172.21.15.12:0/1537750446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f946c101960 0x7f946c101de0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9458009900 tx=0x7f945802a410 comp rx=0 tx=0).stop 2024-05-09T09:28:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.675+0000 7f9472df9640 1 -- 172.21.15.12:0/1537750446 shutdown_connections 2024-05-09T09:28:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.675+0000 7f9472df9640 1 --2- 172.21.15.12:0/1537750446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f946c102320 0x7f946c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.675+0000 7f9472df9640 1 --2- 172.21.15.12:0/1537750446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f946c101960 0x7f946c101de0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.675+0000 7f9472df9640 1 --2- 172.21.15.12:0/1537750446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946c100770 0x7f946c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.675+0000 7f9472df9640 1 -- 172.21.15.12:0/1537750446 >> 172.21.15.12:0/1537750446 conn(0x7f946c0fbce0 msgr2=0x7f946c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.675+0000 7f9472df9640 1 -- 172.21.15.12:0/1537750446 shutdown_connections 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.675+0000 7f9472df9640 1 -- 172.21.15.12:0/1537750446 wait complete. 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9472df9640 1 Processor -- start 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9472df9640 1 -- start start 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9472df9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f946c100770 0x7f946c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9472df9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946c101960 0x7f946c19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9472df9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f946c102320 0x7f946c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9472df9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f946c10c3d0 con 0x7f946c100770 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9472df9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f946c10c250 con 0x7f946c101960 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9472df9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f946c10c550 con 0x7f946c102320 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9470b95640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f946c100770 0x7f946c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9470b95640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f946c100770 0x7f946c069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41026/0 (socket says 172.21.15.12:41026) 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9470b95640 1 -- 172.21.15.12:0/3638213974 learned_addr learned my addr 172.21.15.12:0/3638213974 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9471396640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f946c102320 0x7f946c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f946bfff640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946c101960 0x7f946c19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9470b95640 1 -- 172.21.15.12:0/3638213974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f946c102320 msgr2=0x7f946c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9470b95640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f946c102320 0x7f946c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9470b95640 1 -- 172.21.15.12:0/3638213974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946c101960 msgr2=0x7f946c19c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9470b95640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946c101960 0x7f946c19c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.676+0000 7f9470b95640 1 -- 172.21.15.12:0/3638213974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946c19f7e0 con 0x7f946c100770 2024-05-09T09:28:24.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.677+0000 7f9470b95640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f946c100770 0x7f946c069590 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f946000e9c0 tx=0x7f946000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.677+0000 7f9469ffb640 1 -- 172.21.15.12:0/3638213974 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946000cd80 con 0x7f946c100770 2024-05-09T09:28:24.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.677+0000 7f9469ffb640 1 -- 172.21.15.12:0/3638213974 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f94600035f0 con 0x7f946c100770 2024-05-09T09:28:24.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.677+0000 7f9472df9640 1 -- 172.21.15.12:0/3638213974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946c19fad0 con 0x7f946c100770 2024-05-09T09:28:24.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.677+0000 7f9469ffb640 1 -- 172.21.15.12:0/3638213974 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460010690 con 0x7f946c100770 2024-05-09T09:28:24.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.677+0000 7f9472df9640 1 -- 172.21.15.12:0/3638213974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946c19fff0 con 0x7f946c100770 2024-05-09T09:28:24.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.679+0000 7f9469ffb640 1 -- 172.21.15.12:0/3638213974 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f94600037a0 con 0x7f946c100770 2024-05-09T09:28:24.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f9475be9740 1 Processor -- start 2024-05-09T09:28:24.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f9475be9740 1 -- start start 2024-05-09T09:28:24.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f9475be9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 0x55c23b6e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f9475be9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6e5d40 0x55c23b6ea840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f9475be9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6eaeb0 0x55c23b6ed340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f9475be9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c23b645c40 con 0x55c23b6eaeb0 2024-05-09T09:28:24.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f9475be9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c23b5ed580 con 0x55c23b6e5350 2024-05-09T09:28:24.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f9475be9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c23b641c00 con 0x55c23b6e5d40 2024-05-09T09:28:24.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f9470b95640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6e5d40 0x55c23b6ea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f946bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6eaeb0 0x55c23b6ed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f9471396640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 0x55c23b6e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f946bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6eaeb0 0x55c23b6ed340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41030/0 (socket says 172.21.15.12:41030) 2024-05-09T09:28:24.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f9471396640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 0x55c23b6e5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:43114/0 (socket says 172.21.15.12:43114) 2024-05-09T09:28:24.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f946bfff640 1 -- 172.21.15.12:0/3727917524 learned_addr learned my addr 172.21.15.12:0/3727917524 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.680+0000 7f9469ffb640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f945c0a2b90 0x7f945c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.681+0000 7f9470b95640 1 -- 172.21.15.12:0/3727917524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 msgr2=0x55c23b6e5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.681+0000 7f9470b95640 1 --2- 172.21.15.12:0/3727917524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 0x55c23b6e5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.681+0000 7f9470b95640 1 -- 172.21.15.12:0/3727917524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6eaeb0 msgr2=0x55c23b6ed340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.681+0000 7f9470b95640 1 --2- 172.21.15.12:0/3727917524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6eaeb0 0x55c23b6ed340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.681+0000 7f9471396640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f945c0a2b90 0x7f945c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.681+0000 7f9470b95640 1 -- 172.21.15.12:0/3727917524 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c23b3593f0 con 0x55c23b6e5d40 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.681+0000 7f9469ffb640 1 -- 172.21.15.12:0/3638213974 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f94600cf840 con 0x7f946c100770 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.681+0000 7f9470b95640 1 --2- 172.21.15.12:0/3727917524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6e5d40 0x55c23b6ea840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9460098640 tx=0x7f94600867f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e86fafc257485ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.681+0000 7f9471396640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f945c0a2b90 0x7f945c0a5050 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9464009fd0 tx=0x7f9464004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.681+0000 7f9456ffd640 1 -- 172.21.15.12:0/3727917524 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946009b5d0 con 0x55c23b6e5d40 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.681+0000 7f9456ffd640 1 -- 172.21.15.12:0/3727917524 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9460003ba0 con 0x55c23b6e5d40 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.681+0000 7f9456ffd640 1 -- 172.21.15.12:0/3727917524 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94600982e0 con 0x55c23b6e5d40 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.682+0000 7f9475be9740 1 -- 172.21.15.12:0/3727917524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6e5d40 msgr2=0x55c23b6ea840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.682+0000 7f9475be9740 1 --2- 172.21.15.12:0/3727917524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6e5d40 0x55c23b6ea840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9460098640 tx=0x7f94600867f0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.682+0000 7f9475be9740 1 -- 172.21.15.12:0/3727917524 shutdown_connections 2024-05-09T09:28:24.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.682+0000 7f9475be9740 1 --2- 172.21.15.12:0/3727917524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6eaeb0 0x55c23b6ed340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.682+0000 7f9475be9740 1 --2- 172.21.15.12:0/3727917524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6e5d40 0x55c23b6ea840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.682+0000 7f9475be9740 1 --2- 172.21.15.12:0/3727917524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 0x55c23b6e5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.682+0000 7f9475be9740 1 -- 172.21.15.12:0/3727917524 >> 172.21.15.12:0/3727917524 conn(0x55c23b6dc5b0 msgr2=0x55c23b6dc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.682+0000 7f9475be9740 1 -- 172.21.15.12:0/3727917524 shutdown_connections 2024-05-09T09:28:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.682+0000 7f9475be9740 1 -- 172.21.15.12:0/3727917524 wait complete. 2024-05-09T09:28:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.682+0000 7f9475be9740 1 -- start start 2024-05-09T09:28:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.682+0000 7f9475be9740 1 Processor -- start 2024-05-09T09:28:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9475be9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 0x55c23b6ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9475be9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6e5d40 0x55c23b6efc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9475be9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9475be9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c23b6ede00 con 0x55c23b6eaeb0 2024-05-09T09:28:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9475be9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c23b600c40 con 0x55c23b6e5350 2024-05-09T09:28:24.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9475be9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c23b6edf80 con 0x55c23b6e5d40 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9475be9740 10 client.0 _ll_register_callbacks cb 0x55c23b358be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9471396640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9471396640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41038/0 (socket says 172.21.15.12:41038) 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9471396640 1 -- 172.21.15.12:0/1656282521 learned_addr learned my addr 172.21.15.12:0/1656282521 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9470b95640 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 0x55c23b6ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f946bfff640 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6e5d40 0x55c23b6efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9471396640 1 -- 172.21.15.12:0/1656282521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6e5d40 msgr2=0x55c23b6efc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9471396640 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6e5d40 0x55c23b6efc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9471396640 1 -- 172.21.15.12:0/1656282521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 msgr2=0x55c23b6ef6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9471396640 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 0x55c23b6ef6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9471396640 1 -- 172.21.15.12:0/1656282521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c23b4237d0 con 0x55c23b6eaeb0 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f946bfff640 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6e5d40 0x55c23b6efc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f9471396640 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f6c60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f946400fb90 tx=0x7f9464004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c03dec0941f1c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.683+0000 7f944bfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f944bfff640 1 -- 172.21.15.12:0/1656282521 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946401eac0 con 0x55c23b6eaeb0 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f9475be9740 1 -- 172.21.15.12:0/1656282521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c23b653370 con 0x55c23b6eaeb0 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f944bfff640 1 -- 172.21.15.12:0/1656282521 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f946401ec60 con 0x55c23b6eaeb0 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f944bfff640 1 -- 172.21.15.12:0/1656282521 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f946401ee60 con 0x55c23b6eaeb0 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f9475be9740 10 client.16692 fetch_fsmap learned FSMap version 41 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f9475be9740 1 -- 172.21.15.12:0/1656282521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c23b6fa090 con 0x55c23b6eaeb0 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f944bfff640 1 -- 172.21.15.12:0/1656282521 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55c23b653370 con 0x55c23b6eaeb0 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f944bfff640 1 -- 172.21.15.12:0/1656282521 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8438+0+0 (secure 0 0 0) 0x7f94640284d0 con 0x55c23b6eaeb0 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f9475be9740 10 client.16692 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f9475be9740 10 client.16692 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f9475be9740 20 client.16692 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f9475be9740 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55c23b6fb7e0 0x55c23b6fdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f9475be9740 4 client.16692 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:24.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f9475be9740 1 -- 172.21.15.12:0/1656282521 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c23b64d6c0 con 0x55c23b6fb7e0 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.684+0000 7f9470b95640 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55c23b6fb7e0 0x55c23b6fdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.685+0000 7f9470b95640 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55c23b6fb7e0 0x55c23b6fdc30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6fbd56fe304098b server_cookie=92b24468208b6014 in_seq=0 out_seq=0 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.685+0000 7f944bfff640 10 client.16692 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f944bfff640 1 -- 172.21.15.12:0/1656282521 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c23b64d6c0 con 0x55c23b6fb7e0 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f944bfff640 10 client.16692 handle_command_reply: tid=0 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f9475be9740 1 client.16692 shutdown 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f9475be9740 1 -- 172.21.15.12:0/1656282521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6eaeb0 msgr2=0x55c23b6f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f9475be9740 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f6c60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f946400fb90 tx=0x7f9464004ab0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f9475be9740 1 -- 172.21.15.12:0/1656282521 shutdown_connections 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f9475be9740 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55c23b6fb7e0 0x55c23b6fdc30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f9475be9740 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f6c60 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f9475be9740 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6e5d40 0x55c23b6efc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f944bfff640 0 client.16692 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:28:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f9475be9740 1 --2- 172.21.15.12:0/1656282521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 0x55c23b6ef6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f9475be9740 1 -- 172.21.15.12:0/1656282521 >> 172.21.15.12:0/1656282521 conn(0x55c23b6dc5b0 msgr2=0x55c23b6db7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f9475be9740 1 -- 172.21.15.12:0/1656282521 shutdown_connections 2024-05-09T09:28:24.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f9475be9740 1 -- 172.21.15.12:0/1656282521 wait complete. 2024-05-09T09:28:24.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.686+0000 7f9475be9740 20 client.16692 trim_cache size 0 max 16384 2024-05-09T09:28:24.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.700+0000 7f9475be9740 1 Processor -- start 2024-05-09T09:28:24.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.700+0000 7f9475be9740 1 -- start start 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f9475be9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 0x55c23b6ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f9475be9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5d40 0x55c23b6efc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f9475be9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f9475be9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1a8400 con 0x55c23b6e5d40 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f9475be9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1aa180 con 0x55c23b6e5350 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f9475be9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1a8b90 con 0x55c23b6eaeb0 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f946bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5d40 0x55c23b6efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f946bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5d40 0x55c23b6efc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41044/0 (socket says 172.21.15.12:41044) 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f946bfff640 1 -- 172.21.15.12:0/270065378 learned_addr learned my addr 172.21.15.12:0/270065378 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f9471396640 1 --2- 172.21.15.12:0/270065378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f9470b95640 1 --2- 172.21.15.12:0/270065378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 0x55c23b6ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f946bfff640 1 -- 172.21.15.12:0/270065378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6eaeb0 msgr2=0x55c23b6f6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f946bfff640 1 --2- 172.21.15.12:0/270065378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f946bfff640 1 -- 172.21.15.12:0/270065378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 msgr2=0x55c23b6ef6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f946bfff640 1 --2- 172.21.15.12:0/270065378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 0x55c23b6ef6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f946bfff640 1 -- 172.21.15.12:0/270065378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c23b6dc0f0 con 0x55c23b6e5d40 2024-05-09T09:28:24.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f946bfff640 1 --2- 172.21.15.12:0/270065378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5d40 0x55c23b6efc20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f945802a930 tx=0x7f9458032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a1ba28d7048dcf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.701+0000 7f94567fc640 1 -- 172.21.15.12:0/270065378 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458035040 con 0x55c23b6e5d40 2024-05-09T09:28:24.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.702+0000 7f94567fc640 1 -- 172.21.15.12:0/270065378 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9458031810 con 0x55c23b6e5d40 2024-05-09T09:28:24.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.702+0000 7f94567fc640 1 -- 172.21.15.12:0/270065378 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458004780 con 0x55c23b6e5d40 2024-05-09T09:28:24.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.702+0000 7f9475be9740 1 -- 172.21.15.12:0/270065378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5d40 msgr2=0x55c23b6efc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.702+0000 7f9475be9740 1 --2- 172.21.15.12:0/270065378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5d40 0x55c23b6efc20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f945802a930 tx=0x7f9458032e60 comp rx=0 tx=0).stop 2024-05-09T09:28:24.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.702+0000 7f9475be9740 1 -- 172.21.15.12:0/270065378 shutdown_connections 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.702+0000 7f9475be9740 1 --2- 172.21.15.12:0/270065378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.702+0000 7f9475be9740 1 --2- 172.21.15.12:0/270065378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5d40 0x55c23b6efc20 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.702+0000 7f9475be9740 1 --2- 172.21.15.12:0/270065378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5350 0x55c23b6ef6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.702+0000 7f9475be9740 1 -- 172.21.15.12:0/270065378 >> 172.21.15.12:0/270065378 conn(0x55c23b6dc5b0 msgr2=0x55c23b6dc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.702+0000 7f9475be9740 1 -- 172.21.15.12:0/270065378 shutdown_connections 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.703+0000 7f9475be9740 1 -- 172.21.15.12:0/270065378 wait complete. 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.703+0000 7f9475be9740 1 -- start start 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.703+0000 7f9475be9740 1 Processor -- start 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.703+0000 7f9475be9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5350 0x55c23b6f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.703+0000 7f9475be9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5d40 0x55c23b6f3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.703+0000 7f9475be9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.703+0000 7f9475be9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1a9ab0 con 0x55c23b6e5350 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.703+0000 7f9475be9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1a9320 con 0x55c23b6e5d40 2024-05-09T09:28:24.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.703+0000 7f9475be9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1a63e0 con 0x55c23b6eaeb0 2024-05-09T09:28:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.703+0000 7f9475be9740 10 client.0 _ll_register_callbacks cb 0x7f9464009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.703+0000 7f9470b95640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5350 0x55c23b6f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f9471396640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f9470b95640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5350 0x55c23b6f6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41058/0 (socket says 172.21.15.12:41058) 2024-05-09T09:28:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f9470b95640 1 -- 172.21.15.12:0/337080933 learned_addr learned my addr 172.21.15.12:0/337080933 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f946bfff640 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5d40 0x55c23b6f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f9471396640 1 -- 172.21.15.12:0/337080933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5d40 msgr2=0x55c23b6f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f9471396640 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5d40 0x55c23b6f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f9471396640 1 -- 172.21.15.12:0/337080933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5350 msgr2=0x55c23b6f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f9471396640 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5350 0x55c23b6f6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f9471396640 1 -- 172.21.15.12:0/337080933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c23b75fe20 con 0x55c23b6eaeb0 2024-05-09T09:28:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f946bfff640 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5d40 0x55c23b6f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:24.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f9470b95640 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5350 0x55c23b6f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:24.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f9471396640 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f3d50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f946400ffd0 tx=0x7f9464028e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b1437130c07f81a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f944bfff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:28:24.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f944bfff640 1 -- 172.21.15.12:0/337080933 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946403c070 con 0x55c23b6eaeb0 2024-05-09T09:28:24.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f944bfff640 1 -- 172.21.15.12:0/337080933 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f9464028630 con 0x55c23b6eaeb0 2024-05-09T09:28:24.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f9475be9740 1 -- 172.21.15.12:0/337080933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9458031810 con 0x55c23b6eaeb0 2024-05-09T09:28:24.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f944bfff640 1 -- 172.21.15.12:0/337080933 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f946400f950 con 0x55c23b6eaeb0 2024-05-09T09:28:24.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.704+0000 7f9475be9740 10 client.26245 fetch_fsmap learned FSMap version 41 2024-05-09T09:28:24.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.705+0000 7f9475be9740 1 -- 172.21.15.12:0/337080933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c23b761120 con 0x55c23b6eaeb0 2024-05-09T09:28:24.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.705+0000 7f944bfff640 1 -- 172.21.15.12:0/337080933 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9458031810 con 0x55c23b6eaeb0 2024-05-09T09:28:24.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.705+0000 7f944bfff640 1 -- 172.21.15.12:0/337080933 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) ==== 8438+0+0 (secure 0 0 0) 0x7f9464036420 con 0x55c23b6eaeb0 2024-05-09T09:28:24.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.705+0000 7f9475be9740 10 client.26245 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T09:28:24.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.705+0000 7f9475be9740 10 client.26245 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:28:24.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.705+0000 7f9475be9740 20 client.26245 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.705+0000 7f9475be9740 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55c23b761b60 0x55c23b763f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.705+0000 7f9475be9740 4 client.26245 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.705+0000 7f9475be9740 1 -- 172.21.15.12:0/337080933 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c23b5ca820 con 0x55c23b761b60 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.705+0000 7f9470b95640 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55c23b761b60 0x55c23b763f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f944bfff640 10 client.26245 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f9470b95640 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55c23b761b60 0x55c23b763f80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eaa795e7162a4c9b server_cookie=e22ac5f7c3ae9a2b in_seq=0 out_seq=0 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f944bfff640 1 -- 172.21.15.12:0/337080933 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c23b5ca820 con 0x55c23b761b60 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f944bfff640 10 client.26245 handle_command_reply: tid=0 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f9475be9740 1 client.26245 shutdown 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f9475be9740 1 -- 172.21.15.12:0/337080933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6eaeb0 msgr2=0x55c23b6f3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f9475be9740 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f3d50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f946400ffd0 tx=0x7f9464028e00 comp rx=0 tx=0).stop 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f9475be9740 1 -- 172.21.15.12:0/337080933 shutdown_connections 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f9475be9740 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55c23b761b60 0x55c23b763f80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f9475be9740 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c23b6eaeb0 0x55c23b6f3d50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f9475be9740 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c23b6e5d40 0x55c23b6f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f9475be9740 1 --2- 172.21.15.12:0/337080933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c23b6e5350 0x55c23b6f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f944bfff640 0 client.26245 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:28:24.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f9475be9740 1 -- 172.21.15.12:0/337080933 >> 172.21.15.12:0/337080933 conn(0x55c23b6dc5b0 msgr2=0x55c23b6ef6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f9475be9740 1 -- 172.21.15.12:0/337080933 shutdown_connections 2024-05-09T09:28:24.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f9475be9740 1 -- 172.21.15.12:0/337080933 wait complete. 2024-05-09T09:28:24.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.706+0000 7f9475be9740 20 client.26245 trim_cache size 0 max 16384 2024-05-09T09:28:24.713 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:28:24.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.707+0000 7f9472df9640 1 -- 172.21.15.12:0/3638213974 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f945c0a2b90 msgr2=0x7f945c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.707+0000 7f9472df9640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f945c0a2b90 0x7f945c0a5050 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9464009fd0 tx=0x7f9464004a40 comp rx=0 tx=0).stop 2024-05-09T09:28:24.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.707+0000 7f9472df9640 1 -- 172.21.15.12:0/3638213974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f946c100770 msgr2=0x7f946c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.707+0000 7f9472df9640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f946c100770 0x7f946c069590 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f946000e9c0 tx=0x7f946000ee80 comp rx=0 tx=0).stop 2024-05-09T09:28:24.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.707+0000 7f9472df9640 1 -- 172.21.15.12:0/3638213974 shutdown_connections 2024-05-09T09:28:24.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.707+0000 7f9472df9640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f945c0a2b90 0x7f945c0a5050 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.707+0000 7f9472df9640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f946c102320 0x7f946c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.707+0000 7f9472df9640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946c101960 0x7f946c19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.707+0000 7f9472df9640 1 --2- 172.21.15.12:0/3638213974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f946c100770 0x7f946c069590 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.707+0000 7f9472df9640 1 -- 172.21.15.12:0/3638213974 >> 172.21.15.12:0/3638213974 conn(0x7f946c0fbce0 msgr2=0x7f946c104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.707+0000 7f9472df9640 1 -- 172.21.15.12:0/3638213974 shutdown_connections 2024-05-09T09:28:24.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.707+0000 7f9472df9640 1 -- 172.21.15.12:0/3638213974 wait complete. 2024-05-09T09:28:24.719 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:28:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.830+0000 7fdc1c55a640 1 -- 172.21.15.12:0/3209868597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc14102320 msgr2=0x7fdc14108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.830+0000 7fdc1c55a640 1 --2- 172.21.15.12:0/3209868597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc14102320 0x7fdc14108bb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009900 tx=0x7fdc0c02a410 comp rx=0 tx=0).stop 2024-05-09T09:28:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.830+0000 7fdc1c55a640 1 -- 172.21.15.12:0/3209868597 shutdown_connections 2024-05-09T09:28:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.830+0000 7fdc1c55a640 1 --2- 172.21.15.12:0/3209868597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc14102320 0x7fdc14108bb0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.830+0000 7fdc1c55a640 1 --2- 172.21.15.12:0/3209868597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc14101960 0x7fdc14101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.830+0000 7fdc1c55a640 1 --2- 172.21.15.12:0/3209868597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc14100770 0x7fdc14100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.830+0000 7fdc1c55a640 1 -- 172.21.15.12:0/3209868597 >> 172.21.15.12:0/3209868597 conn(0x7fdc140fbce0 msgr2=0x7fdc140fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.830+0000 7fdc1c55a640 1 -- 172.21.15.12:0/3209868597 shutdown_connections 2024-05-09T09:28:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.830+0000 7fdc1c55a640 1 -- 172.21.15.12:0/3209868597 wait complete. 2024-05-09T09:28:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.830+0000 7fdc1c55a640 1 Processor -- start 2024-05-09T09:28:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.830+0000 7fdc1c55a640 1 -- start start 2024-05-09T09:28:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc1c55a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc14100770 0x7fdc14197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc1c55a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc14101960 0x7fdc141980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc1c55a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc14102320 0x7fdc1419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc1c55a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1410c3d0 con 0x7fdc14101960 2024-05-09T09:28:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc1c55a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1410c250 con 0x7fdc14102320 2024-05-09T09:28:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc1c55a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1410c550 con 0x7fdc14100770 2024-05-09T09:28:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc19af5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc14101960 0x7fdc141980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc1a2f6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc14100770 0x7fdc14197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc19af5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc14101960 0x7fdc141980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41068/0 (socket says 172.21.15.12:41068) 2024-05-09T09:28:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc1a2f6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc14100770 0x7fdc14197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:37790/0 (socket says 172.21.15.12:37790) 2024-05-09T09:28:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc19af5640 1 -- 172.21.15.12:0/2374739891 learned_addr learned my addr 172.21.15.12:0/2374739891 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc1aaf7640 1 --2- 172.21.15.12:0/2374739891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc14102320 0x7fdc1419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc19af5640 1 -- 172.21.15.12:0/2374739891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc14100770 msgr2=0x7fdc14197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc19af5640 1 --2- 172.21.15.12:0/2374739891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc14100770 0x7fdc14197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc19af5640 1 -- 172.21.15.12:0/2374739891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc14102320 msgr2=0x7fdc1419f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc19af5640 1 --2- 172.21.15.12:0/2374739891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc14102320 0x7fdc1419f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc19af5640 1 -- 172.21.15.12:0/2374739891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1419f7c0 con 0x7fdc14101960 2024-05-09T09:28:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc19af5640 1 --2- 172.21.15.12:0/2374739891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc14101960 0x7fdc141980e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800e9f0 tx=0x7fdc0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.831+0000 7fdc137fe640 1 -- 172.21.15.12:0/2374739891 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0800cd80 con 0x7fdc14101960 2024-05-09T09:28:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.832+0000 7fdc1c55a640 1 -- 172.21.15.12:0/2374739891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1419fab0 con 0x7fdc14101960 2024-05-09T09:28:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.832+0000 7fdc137fe640 1 -- 172.21.15.12:0/2374739891 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdc08010900 con 0x7fdc14101960 2024-05-09T09:28:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.832+0000 7fdc1c55a640 1 -- 172.21.15.12:0/2374739891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc141a0050 con 0x7fdc14101960 2024-05-09T09:28:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.832+0000 7fdc137fe640 1 -- 172.21.15.12:0/2374739891 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc080189e0 con 0x7fdc14101960 2024-05-09T09:28:24.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.834+0000 7fdc1f268740 1 Processor -- start 2024-05-09T09:28:24.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.834+0000 7fdc137fe640 1 -- 172.21.15.12:0/2374739891 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdc08010ab0 con 0x7fdc14101960 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1f268740 1 -- start start 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1f268740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 0x55eb7de27770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1f268740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27d40 0x55eb7de2c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1f268740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de2ceb0 0x55eb7de2f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1f268740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55eb7dd87c40 con 0x55eb7de27350 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1f268740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eb7dd2f580 con 0x55eb7de2ceb0 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1f268740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55eb7dd83c00 con 0x55eb7de27d40 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1aaf7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 0x55eb7de27770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1a2f6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27d40 0x55eb7de2c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1aaf7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 0x55eb7de27770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41074/0 (socket says 172.21.15.12:41074) 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1aaf7640 1 -- 172.21.15.12:0/1873225968 learned_addr learned my addr 172.21.15.12:0/1873225968 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc19af5640 1 --2- 172.21.15.12:0/1873225968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de2ceb0 0x55eb7de2f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1a2f6640 1 -- 172.21.15.12:0/1873225968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de2ceb0 msgr2=0x55eb7de2f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1a2f6640 1 --2- 172.21.15.12:0/1873225968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de2ceb0 0x55eb7de2f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1a2f6640 1 -- 172.21.15.12:0/1873225968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 msgr2=0x55eb7de27770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1a2f6640 1 --2- 172.21.15.12:0/1873225968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 0x55eb7de27770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1a2f6640 1 -- 172.21.15.12:0/1873225968 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb7da9b3f0 con 0x55eb7de27d40 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.835+0000 7fdc1aaf7640 1 --2- 172.21.15.12:0/1873225968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 0x55eb7de27770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.836+0000 7fdc137fe640 1 --2- 172.21.15.12:0/2374739891 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdbfc0a2b90 0x7fdbfc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.836+0000 7fdc1a2f6640 1 --2- 172.21.15.12:0/1873225968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27d40 0x55eb7de2c840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc04005fd0 tx=0x7fdc040118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a48bc2d6951ccb7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.836+0000 7fdc1aaf7640 1 --2- 172.21.15.12:0/2374739891 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdbfc0a2b90 0x7fdbfc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.836+0000 7fdc137fe640 1 -- 172.21.15.12:0/2374739891 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fdc080cf770 con 0x7fdc14101960 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.836+0000 7fdc10bfe640 1 -- 172.21.15.12:0/1873225968 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04004b20 con 0x55eb7de27d40 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.836+0000 7fdc10bfe640 1 -- 172.21.15.12:0/1873225968 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdc04003e40 con 0x55eb7de27d40 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.836+0000 7fdc10bfe640 1 -- 172.21.15.12:0/1873225968 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04004860 con 0x55eb7de27d40 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1aaf7640 1 --2- 172.21.15.12:0/2374739891 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdbfc0a2b90 0x7fdbfc0a5050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c031570 tx=0x7fdc0c005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 -- 172.21.15.12:0/1873225968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27d40 msgr2=0x55eb7de2c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 --2- 172.21.15.12:0/1873225968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27d40 0x55eb7de2c840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc04005fd0 tx=0x7fdc040118d0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 -- 172.21.15.12:0/1873225968 shutdown_connections 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 --2- 172.21.15.12:0/1873225968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de2ceb0 0x55eb7de2f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 --2- 172.21.15.12:0/1873225968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27d40 0x55eb7de2c840 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 --2- 172.21.15.12:0/1873225968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 0x55eb7de27770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 -- 172.21.15.12:0/1873225968 >> 172.21.15.12:0/1873225968 conn(0x55eb7de1e5b0 msgr2=0x55eb7de1e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 -- 172.21.15.12:0/1873225968 shutdown_connections 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 -- 172.21.15.12:0/1873225968 wait complete. 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 -- start start 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 Processor -- start 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27350 0x55eb7de31720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27d40 0x55eb7de31c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 0x55eb7de38ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55eb7de2fe60 con 0x55eb7de2ceb0 2024-05-09T09:28:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eb7dd42c40 con 0x55eb7de27d40 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.837+0000 7fdc1f268740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55eb7de2ffe0 con 0x55eb7de27350 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdc1f268740 10 client.0 _ll_register_callbacks cb 0x55eb7da9abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdc1aaf7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 0x55eb7de38ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdc1aaf7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 0x55eb7de38ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41080/0 (socket says 172.21.15.12:41080) 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdc1aaf7640 1 -- 172.21.15.12:0/3400609853 learned_addr learned my addr 172.21.15.12:0/3400609853 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdc1a2f6640 1 --2- 172.21.15.12:0/3400609853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27350 0x55eb7de31720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdc19af5640 1 --2- 172.21.15.12:0/3400609853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27d40 0x55eb7de31c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdc1aaf7640 1 -- 172.21.15.12:0/3400609853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27350 msgr2=0x55eb7de31720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdc1aaf7640 1 --2- 172.21.15.12:0/3400609853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27350 0x55eb7de31720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdc1aaf7640 1 -- 172.21.15.12:0/3400609853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27d40 msgr2=0x55eb7de31c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdc1aaf7640 1 --2- 172.21.15.12:0/3400609853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27d40 0x55eb7de31c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdc1aaf7640 1 -- 172.21.15.12:0/3400609853 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb7db657d0 con 0x55eb7de2ceb0 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdc1aaf7640 1 --2- 172.21.15.12:0/3400609853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 0x55eb7de38ca0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c02a930 tx=0x7fdc0c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18f2234f1e47a2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdbf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdbf9ffb640 1 -- 172.21.15.12:0/3400609853 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c04c020 con 0x55eb7de2ceb0 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdbf9ffb640 1 -- 172.21.15.12:0/3400609853 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fdc0c004250 con 0x55eb7de2ceb0 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.838+0000 7fdc1f268740 1 -- 172.21.15.12:0/3400609853 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb7dd95370 con 0x55eb7de2ceb0 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.839+0000 7fdbf9ffb640 1 -- 172.21.15.12:0/3400609853 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdc0c057050 con 0x55eb7de2ceb0 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.839+0000 7fdc1f268740 10 client.16710 fetch_fsmap learned FSMap version 41 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.839+0000 7fdc1f268740 1 -- 172.21.15.12:0/3400609853 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eb7de3c080 con 0x55eb7de2ceb0 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.839+0000 7fdbf9ffb640 1 -- 172.21.15.12:0/3400609853 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55eb7dd95370 con 0x55eb7de2ceb0 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.839+0000 7fdbf9ffb640 1 -- 172.21.15.12:0/3400609853 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8438+0+0 (secure 0 0 0) 0x7fdc0c048500 con 0x55eb7de2ceb0 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.839+0000 7fdc1f268740 10 client.16710 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.839+0000 7fdc1f268740 10 client.16710 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.839+0000 7fdc1f268740 20 client.16710 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.839+0000 7fdc1f268740 1 --2- 172.21.15.12:0/3400609853 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55eb7de3d890 0x55eb7de3fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.839+0000 7fdc1f268740 4 client.16710 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.839+0000 7fdc1f268740 1 -- 172.21.15.12:0/3400609853 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb7dd8f6c0 con 0x55eb7de3d890 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.839+0000 7fdc1a2f6640 1 --2- 172.21.15.12:0/3400609853 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55eb7de3d890 0x55eb7de3fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.840+0000 7fdbf9ffb640 10 client.16710 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.840+0000 7fdc1a2f6640 1 --2- 172.21.15.12:0/3400609853 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55eb7de3d890 0x55eb7de3fd30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac44ae1af0f4e2b3 server_cookie=38cd8f1a1ba66914 in_seq=0 out_seq=0 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.841+0000 7fdbf9ffb640 1 -- 172.21.15.12:0/3400609853 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb7dd8f6c0 con 0x55eb7de3d890 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.841+0000 7fdbf9ffb640 10 client.16710 handle_command_reply: tid=0 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.841+0000 7fdc1f268740 1 client.16710 shutdown 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.841+0000 7fdc1f268740 1 -- 172.21.15.12:0/3400609853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 msgr2=0x55eb7de38ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.841+0000 7fdc1f268740 1 --2- 172.21.15.12:0/3400609853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 0x55eb7de38ca0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c02a930 tx=0x7fdc0c02a410 comp rx=0 tx=0).stop 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.841+0000 7fdc1f268740 1 -- 172.21.15.12:0/3400609853 shutdown_connections 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.841+0000 7fdc1f268740 1 --2- 172.21.15.12:0/3400609853 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55eb7de3d890 0x55eb7de3fd30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.841+0000 7fdc1f268740 1 --2- 172.21.15.12:0/3400609853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 0x55eb7de38ca0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.841+0000 7fdc1f268740 1 --2- 172.21.15.12:0/3400609853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27d40 0x55eb7de31c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.841+0000 7fdbf9ffb640 0 client.16710 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:28:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.841+0000 7fdc1f268740 1 --2- 172.21.15.12:0/3400609853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27350 0x55eb7de31720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.841+0000 7fdc1f268740 1 -- 172.21.15.12:0/3400609853 >> 172.21.15.12:0/3400609853 conn(0x55eb7de1e5b0 msgr2=0x55eb7de1d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.842+0000 7fdc1f268740 1 -- 172.21.15.12:0/3400609853 shutdown_connections 2024-05-09T09:28:24.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.842+0000 7fdc1f268740 1 -- 172.21.15.12:0/3400609853 wait complete. 2024-05-09T09:28:24.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.842+0000 7fdc1f268740 20 client.16710 trim_cache size 0 max 16384 2024-05-09T09:28:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.856+0000 7fdc1f268740 1 Processor -- start 2024-05-09T09:28:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.856+0000 7fdc1f268740 1 -- start start 2024-05-09T09:28:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.856+0000 7fdc1f268740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27350 0x55eb7de29100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.856+0000 7fdc1f268740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27d40 0x55eb7de29640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.856+0000 7fdc1f268740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 0x55eb7de38bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.856+0000 7fdc1f268740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141a8410 con 0x55eb7de2ceb0 2024-05-09T09:28:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.856+0000 7fdc1f268740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141aa190 con 0x55eb7de27350 2024-05-09T09:28:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.856+0000 7fdc1f268740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141a8ba0 con 0x55eb7de27d40 2024-05-09T09:28:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdc1a2f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27350 0x55eb7de29100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdc1aaf7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 0x55eb7de38bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdc1a2f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27350 0x55eb7de29100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:43196/0 (socket says 172.21.15.12:43196) 2024-05-09T09:28:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdc1a2f6640 1 -- 172.21.15.12:0/695808402 learned_addr learned my addr 172.21.15.12:0/695808402 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdc1aaf7640 1 --2- 172.21.15.12:0/695808402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 0x55eb7de38bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41094/0 (socket says 172.21.15.12:41094) 2024-05-09T09:28:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdc19af5640 1 --2- 172.21.15.12:0/695808402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27d40 0x55eb7de29640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdc1a2f6640 1 -- 172.21.15.12:0/695808402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27d40 msgr2=0x55eb7de29640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdc1a2f6640 1 --2- 172.21.15.12:0/695808402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27d40 0x55eb7de29640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdc1a2f6640 1 -- 172.21.15.12:0/695808402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 msgr2=0x55eb7de38bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdc1a2f6640 1 --2- 172.21.15.12:0/695808402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 0x55eb7de38bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdc1a2f6640 1 -- 172.21.15.12:0/695808402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb7de1f320 con 0x55eb7de27350 2024-05-09T09:28:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdc1aaf7640 1 --2- 172.21.15.12:0/695808402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 0x55eb7de38bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdc1a2f6640 1 --2- 172.21.15.12:0/695808402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27350 0x55eb7de29100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdc04004660 tx=0x7fdc04004520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f13f0c550267f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdbfbfff640 1 -- 172.21.15.12:0/695808402 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04009290 con 0x55eb7de27350 2024-05-09T09:28:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdbfbfff640 1 -- 172.21.15.12:0/695808402 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdc04009b00 con 0x55eb7de27350 2024-05-09T09:28:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.857+0000 7fdbfbfff640 1 -- 172.21.15.12:0/695808402 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0402b170 con 0x55eb7de27350 2024-05-09T09:28:24.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.858+0000 7fdc1f268740 1 -- 172.21.15.12:0/695808402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27350 msgr2=0x55eb7de29100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.858+0000 7fdc1f268740 1 --2- 172.21.15.12:0/695808402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27350 0x55eb7de29100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdc04004660 tx=0x7fdc04004520 comp rx=0 tx=0).stop 2024-05-09T09:28:24.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.858+0000 7fdc1f268740 1 -- 172.21.15.12:0/695808402 shutdown_connections 2024-05-09T09:28:24.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.858+0000 7fdc1f268740 1 --2- 172.21.15.12:0/695808402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de2ceb0 0x55eb7de38bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.858+0000 7fdc1f268740 1 --2- 172.21.15.12:0/695808402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de27d40 0x55eb7de29640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.858+0000 7fdc1f268740 1 --2- 172.21.15.12:0/695808402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27350 0x55eb7de29100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.858+0000 7fdc1f268740 1 -- 172.21.15.12:0/695808402 >> 172.21.15.12:0/695808402 conn(0x55eb7de1e5b0 msgr2=0x55eb7de31720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1f268740 1 -- 172.21.15.12:0/695808402 shutdown_connections 2024-05-09T09:28:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1f268740 1 -- 172.21.15.12:0/695808402 wait complete. 2024-05-09T09:28:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1f268740 1 -- start start 2024-05-09T09:28:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1f268740 1 Processor -- start 2024-05-09T09:28:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1f268740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 0x55eb7de362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1f268740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27d40 0x55eb7de36820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1f268740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de2ceb0 0x55eb7de37830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1f268740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141a9ac0 con 0x55eb7de27350 2024-05-09T09:28:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1f268740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141a9330 con 0x55eb7de27d40 2024-05-09T09:28:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1f268740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141a63f0 con 0x55eb7de2ceb0 2024-05-09T09:28:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1f268740 10 client.0 _ll_register_callbacks cb 0x7fdc0c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1a2f6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 0x55eb7de362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc19af5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27d40 0x55eb7de36820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc19af5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27d40 0x55eb7de36820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:43208/0 (socket says 172.21.15.12:43208) 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1a2f6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 0x55eb7de362e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41106/0 (socket says 172.21.15.12:41106) 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc19af5640 1 -- 172.21.15.12:0/2198198374 learned_addr learned my addr 172.21.15.12:0/2198198374 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.859+0000 7fdc1aaf7640 1 --2- 172.21.15.12:0/2198198374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de2ceb0 0x55eb7de37830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdc19af5640 1 -- 172.21.15.12:0/2198198374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de2ceb0 msgr2=0x55eb7de37830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdc19af5640 1 --2- 172.21.15.12:0/2198198374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de2ceb0 0x55eb7de37830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdc19af5640 1 -- 172.21.15.12:0/2198198374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 msgr2=0x55eb7de362e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdc19af5640 1 --2- 172.21.15.12:0/2198198374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 0x55eb7de362e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdc19af5640 1 -- 172.21.15.12:0/2198198374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb7de37e20 con 0x55eb7de27d40 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdc1a2f6640 1 --2- 172.21.15.12:0/2198198374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 0x55eb7de362e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdc19af5640 1 --2- 172.21.15.12:0/2198198374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27d40 0x55eb7de36820 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdc0809b900 tx=0x7fdc0809b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b903e592799de3d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdbf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdbf9ffb640 1 -- 172.21.15.12:0/2198198374 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08018d80 con 0x55eb7de27d40 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdbf9ffb640 1 -- 172.21.15.12:0/2198198374 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fdc0809b050 con 0x55eb7de27d40 2024-05-09T09:28:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdc1f268740 1 -- 172.21.15.12:0/2198198374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc04009b00 con 0x55eb7de27d40 2024-05-09T09:28:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdbf9ffb640 1 -- 172.21.15.12:0/2198198374 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdc0809ffb0 con 0x55eb7de27d40 2024-05-09T09:28:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdc1f268740 10 client.26153 fetch_fsmap learned FSMap version 41 2024-05-09T09:28:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdc1f268740 1 -- 172.21.15.12:0/2198198374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eb7dea3eb0 con 0x55eb7de27d40 2024-05-09T09:28:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.860+0000 7fdbf9ffb640 1 -- 172.21.15.12:0/2198198374 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdc04009b00 con 0x55eb7de27d40 2024-05-09T09:28:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.861+0000 7fdbf9ffb640 1 -- 172.21.15.12:0/2198198374 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 41) ==== 8438+0+0 (secure 0 0 0) 0x7fdc08097b50 con 0x55eb7de27d40 2024-05-09T09:28:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.861+0000 7fdc1f268740 10 client.26153 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T09:28:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.861+0000 7fdc1f268740 10 client.26153 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:28:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.861+0000 7fdc1f268740 20 client.26153 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.861+0000 7fdc1f268740 1 --2- 172.21.15.12:0/2198198374 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55eb7dea48d0 0x55eb7dea6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.861+0000 7fdc1f268740 4 client.26153 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:28:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.861+0000 7fdc1f268740 1 -- 172.21.15.12:0/2198198374 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb7dd0c820 con 0x55eb7dea48d0 2024-05-09T09:28:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.861+0000 7fdc1a2f6640 1 --2- 172.21.15.12:0/2198198374 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55eb7dea48d0 0x55eb7dea6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.861+0000 7fdc1a2f6640 1 --2- 172.21.15.12:0/2198198374 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55eb7dea48d0 0x55eb7dea6cf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=489cedd54b120fa6 server_cookie=999d1fc9ceb1f781 in_seq=0 out_seq=0 2024-05-09T09:28:24.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.861+0000 7fdbf9ffb640 10 client.26153 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:28:24.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdbf9ffb640 1 -- 172.21.15.12:0/2198198374 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb7dd0c820 con 0x55eb7dea48d0 2024-05-09T09:28:24.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdbf9ffb640 10 client.26153 handle_command_reply: tid=0 2024-05-09T09:28:24.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1f268740 1 client.26153 shutdown 2024-05-09T09:28:24.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1f268740 1 -- 172.21.15.12:0/2198198374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27d40 msgr2=0x55eb7de36820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1f268740 1 --2- 172.21.15.12:0/2198198374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27d40 0x55eb7de36820 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdc0809b900 tx=0x7fdc0809b800 comp rx=0 tx=0).stop 2024-05-09T09:28:24.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1f268740 1 -- 172.21.15.12:0/2198198374 shutdown_connections 2024-05-09T09:28:24.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1f268740 1 --2- 172.21.15.12:0/2198198374 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x55eb7dea48d0 0x55eb7dea6cf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1f268740 1 --2- 172.21.15.12:0/2198198374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb7de2ceb0 0x55eb7de37830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1f268740 1 --2- 172.21.15.12:0/2198198374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb7de27d40 0x55eb7de36820 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1f268740 1 --2- 172.21.15.12:0/2198198374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb7de27350 0x55eb7de362e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdbf9ffb640 0 client.26153 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:28:24.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1f268740 1 -- 172.21.15.12:0/2198198374 >> 172.21.15.12:0/2198198374 conn(0x55eb7de1e5b0 msgr2=0x55eb7de316e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1f268740 1 -- 172.21.15.12:0/2198198374 shutdown_connections 2024-05-09T09:28:24.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1f268740 1 -- 172.21.15.12:0/2198198374 wait complete. 2024-05-09T09:28:24.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1f268740 20 client.26153 trim_cache size 0 max 16384 2024-05-09T09:28:24.870 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:28:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1c55a640 1 -- 172.21.15.12:0/2374739891 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdbfc0a2b90 msgr2=0x7fdbfc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1c55a640 1 --2- 172.21.15.12:0/2374739891 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdbfc0a2b90 0x7fdbfc0a5050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c031570 tx=0x7fdc0c005a40 comp rx=0 tx=0).stop 2024-05-09T09:28:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1c55a640 1 -- 172.21.15.12:0/2374739891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc14101960 msgr2=0x7fdc141980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.863+0000 7fdc1c55a640 1 --2- 172.21.15.12:0/2374739891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc14101960 0x7fdc141980e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800e9f0 tx=0x7fdc0800eeb0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.864+0000 7fdc1c55a640 1 -- 172.21.15.12:0/2374739891 shutdown_connections 2024-05-09T09:28:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.864+0000 7fdc1c55a640 1 --2- 172.21.15.12:0/2374739891 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fdbfc0a2b90 0x7fdbfc0a5050 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.864+0000 7fdc1c55a640 1 --2- 172.21.15.12:0/2374739891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc14102320 0x7fdc1419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.864+0000 7fdc1c55a640 1 --2- 172.21.15.12:0/2374739891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc14101960 0x7fdc141980e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.864+0000 7fdc1c55a640 1 --2- 172.21.15.12:0/2374739891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc14100770 0x7fdc14197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:24.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.864+0000 7fdc1c55a640 1 -- 172.21.15.12:0/2374739891 >> 172.21.15.12:0/2374739891 conn(0x7fdc140fbce0 msgr2=0x7fdc140fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:24.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.864+0000 7fdc1c55a640 1 -- 172.21.15.12:0/2374739891 shutdown_connections 2024-05-09T09:28:24.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:24.864+0000 7fdc1c55a640 1 -- 172.21.15.12:0/2374739891 wait complete. 2024-05-09T09:28:24.877 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:28:25.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.056+0000 7fc532b77640 1 -- 172.21.15.12:0/3734456169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc52c1000e0 msgr2=0x7fc52c102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.056+0000 7fc532b77640 1 --2- 172.21.15.12:0/3734456169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc52c1000e0 0x7fc52c102510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc520009900 tx=0x7fc52002a3f0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.056+0000 7fc532b77640 1 -- 172.21.15.12:0/3734456169 shutdown_connections 2024-05-09T09:28:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.056+0000 7fc532b77640 1 --2- 172.21.15.12:0/3734456169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc52c105400 0x7fc52c105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.056+0000 7fc532b77640 1 --2- 172.21.15.12:0/3734456169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc52c102a50 0x7fc52c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.056+0000 7fc532b77640 1 --2- 172.21.15.12:0/3734456169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc52c1000e0 0x7fc52c102510 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.056+0000 7fc532b77640 1 -- 172.21.15.12:0/3734456169 >> 172.21.15.12:0/3734456169 conn(0x7fc52c0fbce0 msgr2=0x7fc52c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.056+0000 7fc532b77640 1 -- 172.21.15.12:0/3734456169 shutdown_connections 2024-05-09T09:28:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.056+0000 7fc532b77640 1 -- 172.21.15.12:0/3734456169 wait complete. 2024-05-09T09:28:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.057+0000 7fc532b77640 1 Processor -- start 2024-05-09T09:28:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.057+0000 7fc532b77640 1 -- start start 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.058+0000 7fc532b77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc52c1000e0 0x7fc52c19e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.058+0000 7fc532b77640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc52c102a50 0x7fc52c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.058+0000 7fc532b77640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc52c105400 0x7fc52c1a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.058+0000 7fc532b77640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c113b00 con 0x7fc52c105400 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.058+0000 7fc532b77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c113980 con 0x7fc52c1000e0 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.058+0000 7fc532b77640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c113c80 con 0x7fc52c102a50 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.058+0000 7fc531114640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc52c105400 0x7fc52c1a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.058+0000 7fc531114640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc52c105400 0x7fc52c1a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41116/0 (socket says 172.21.15.12:41116) 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.058+0000 7fc531114640 1 -- 172.21.15.12:0/1526408411 learned_addr learned my addr 172.21.15.12:0/1526408411 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.058+0000 7fc530913640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc52c1000e0 0x7fc52c19e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.058+0000 7fc52bfff640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc52c102a50 0x7fc52c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.059+0000 7fc531114640 1 -- 172.21.15.12:0/1526408411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc52c102a50 msgr2=0x7fc52c19e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.059+0000 7fc531114640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc52c102a50 0x7fc52c19e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.059+0000 7fc531114640 1 -- 172.21.15.12:0/1526408411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc52c1000e0 msgr2=0x7fc52c19e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.059+0000 7fc531114640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc52c1000e0 0x7fc52c19e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.059+0000 7fc531114640 1 -- 172.21.15.12:0/1526408411 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c1a5df0 con 0x7fc52c105400 2024-05-09T09:28:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.059+0000 7fc531114640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc52c105400 0x7fc52c1a5780 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc52400eb10 tx=0x7fc52400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.060+0000 7fc529ffb640 1 -- 172.21.15.12:0/1526408411 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524004b20 con 0x7fc52c105400 2024-05-09T09:28:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.060+0000 7fc529ffb640 1 -- 172.21.15.12:0/1526408411 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc524004cc0 con 0x7fc52c105400 2024-05-09T09:28:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.060+0000 7fc529ffb640 1 -- 172.21.15.12:0/1526408411 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524010700 con 0x7fc52c105400 2024-05-09T09:28:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.060+0000 7fc532b77640 1 -- 172.21.15.12:0/1526408411 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52c1a60e0 con 0x7fc52c105400 2024-05-09T09:28:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.060+0000 7fc532b77640 1 -- 172.21.15.12:0/1526408411 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52c1a65d0 con 0x7fc52c105400 2024-05-09T09:28:25.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.065+0000 7fc529ffb640 1 -- 172.21.15.12:0/1526408411 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc524004e70 con 0x7fc52c105400 2024-05-09T09:28:25.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.065+0000 7fc535965740 1 Processor -- start 2024-05-09T09:28:25.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.066+0000 7fc535965740 1 -- start start 2024-05-09T09:28:25.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.066+0000 7fc535965740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 0x55983e826360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.066+0000 7fc535965740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 0x55983e82b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.066+0000 7fc535965740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 0x55983e82df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.066+0000 7fc535965740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55983e78ac40 con 0x55983e826930 2024-05-09T09:28:25.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.066+0000 7fc535965740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55983e732580 con 0x55983e825f40 2024-05-09T09:28:25.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.066+0000 7fc535965740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55983e786c00 con 0x55983e82baa0 2024-05-09T09:28:25.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.066+0000 7fc529ffb640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc51c0a2c90 0x7fc51c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.066+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 0x55983e82df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.066+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 0x55983e82df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:37856/0 (socket says 172.21.15.12:37856) 2024-05-09T09:28:25.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.066+0000 7fc52bfff640 1 -- 172.21.15.12:0/2798864089 learned_addr learned my addr 172.21.15.12:0/2798864089 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.067+0000 7fc52bfff640 1 --2- 172.21.15.12:0/2798864089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 0x55983e826360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.067+0000 7fc529ffb640 1 -- 172.21.15.12:0/1526408411 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc5240cf9a0 con 0x7fc52c105400 2024-05-09T09:28:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.067+0000 7fc52bfff640 1 -- 172.21.15.12:0/2798864089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 msgr2=0x55983e826360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.067+0000 7fc52bfff640 1 --2- 172.21.15.12:0/2798864089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 0x55983e826360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.067+0000 7fc52bfff640 1 -- 172.21.15.12:0/2798864089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 msgr2=0x55983e82b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.067+0000 7fc52bfff640 1 --2- 172.21.15.12:0/2798864089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 0x55983e82b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.067+0000 7fc52bfff640 1 -- 172.21.15.12:0/2798864089 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55983e49e3f0 con 0x55983e82baa0 2024-05-09T09:28:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.067+0000 7fc52bfff640 1 --2- 172.21.15.12:0/2798864089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 0x55983e82df30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc518009fd0 tx=0x7fc518005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaacd5aec750cc33 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:25.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.067+0000 7fc516ffd640 1 -- 172.21.15.12:0/2798864089 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518027070 con 0x55983e82baa0 2024-05-09T09:28:25.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.068+0000 7fc516ffd640 1 -- 172.21.15.12:0/2798864089 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc5180054f0 con 0x55983e82baa0 2024-05-09T09:28:25.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.068+0000 7fc516ffd640 1 -- 172.21.15.12:0/2798864089 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51802a040 con 0x55983e82baa0 2024-05-09T09:28:25.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.068+0000 7fc531114640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc51c0a2c90 0x7fc51c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.069+0000 7fc535965740 1 -- 172.21.15.12:0/2798864089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 msgr2=0x55983e82df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.069+0000 7fc535965740 1 --2- 172.21.15.12:0/2798864089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 0x55983e82df30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc518009fd0 tx=0x7fc518005850 comp rx=0 tx=0).stop 2024-05-09T09:28:25.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.069+0000 7fc531114640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc51c0a2c90 0x7fc51c0a5150 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc524098230 tx=0x7fc524009a70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:25.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.069+0000 7fc535965740 1 -- 172.21.15.12:0/2798864089 shutdown_connections 2024-05-09T09:28:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.069+0000 7fc535965740 1 --2- 172.21.15.12:0/2798864089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 0x55983e82df30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.069+0000 7fc535965740 1 --2- 172.21.15.12:0/2798864089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 0x55983e82b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.069+0000 7fc535965740 1 --2- 172.21.15.12:0/2798864089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 0x55983e826360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.069+0000 7fc535965740 1 -- 172.21.15.12:0/2798864089 >> 172.21.15.12:0/2798864089 conn(0x55983e8215b0 msgr2=0x55983e8219b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.070+0000 7fc535965740 1 -- 172.21.15.12:0/2798864089 shutdown_connections 2024-05-09T09:28:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.070+0000 7fc535965740 1 -- 172.21.15.12:0/2798864089 wait complete. 2024-05-09T09:28:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.070+0000 7fc535965740 1 -- start start 2024-05-09T09:28:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.070+0000 7fc535965740 1 Processor -- start 2024-05-09T09:28:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.071+0000 7fc535965740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 0x55983e830360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.071+0000 7fc535965740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 0x55983e8308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.071+0000 7fc535965740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 0x55983e8378e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.071+0000 7fc535965740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55983e82ea50 con 0x55983e826930 2024-05-09T09:28:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.071+0000 7fc535965740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55983e745c40 con 0x55983e825f40 2024-05-09T09:28:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.071+0000 7fc535965740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55983e82ebd0 con 0x55983e82baa0 2024-05-09T09:28:25.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.071+0000 7fc535965740 10 client.0 _ll_register_callbacks cb 0x55983e49dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:25.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.071+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 0x55983e830360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.071+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 0x55983e830360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:43234/0 (socket says 172.21.15.12:43234) 2024-05-09T09:28:25.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.071+0000 7fc52bfff640 1 -- 172.21.15.12:0/1035381739 learned_addr learned my addr 172.21.15.12:0/1035381739 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:25.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.072+0000 7fc530913640 1 --2- 172.21.15.12:0/1035381739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 0x55983e8308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.072+0000 7fc52bfff640 1 --2- 172.21.15.12:0/1035381739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 0x55983e8378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.072+0000 7fc52bfff640 1 -- 172.21.15.12:0/1035381739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 msgr2=0x55983e8378e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.072+0000 7fc52bfff640 1 --2- 172.21.15.12:0/1035381739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 0x55983e8378e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.072+0000 7fc52bfff640 1 -- 172.21.15.12:0/1035381739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 msgr2=0x55983e8308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.072+0000 7fc52bfff640 1 --2- 172.21.15.12:0/1035381739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 0x55983e8308a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.072+0000 7fc52bfff640 1 -- 172.21.15.12:0/1035381739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55983e5687d0 con 0x55983e825f40 2024-05-09T09:28:25.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.073+0000 7fc52bfff640 1 --2- 172.21.15.12:0/1035381739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 0x55983e830360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc518009d50 tx=0x7fc518005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e8953b75989dcfd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:25.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.073+0000 7fc50bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:28:25.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.073+0000 7fc50bfff640 1 -- 172.21.15.12:0/1035381739 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518027050 con 0x55983e825f40 2024-05-09T09:28:25.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.073+0000 7fc50bfff640 1 -- 172.21.15.12:0/1035381739 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc518004a70 con 0x55983e825f40 2024-05-09T09:28:25.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.073+0000 7fc535965740 1 -- 172.21.15.12:0/1035381739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5180054f0 con 0x55983e825f40 2024-05-09T09:28:25.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.074+0000 7fc50bfff640 1 -- 172.21.15.12:0/1035381739 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc518044050 con 0x55983e825f40 2024-05-09T09:28:25.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.074+0000 7fc535965740 10 client.26165 fetch_fsmap learned FSMap version 41 2024-05-09T09:28:25.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.074+0000 7fc535965740 1 -- 172.21.15.12:0/1035381739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55983e83ac50 con 0x55983e825f40 2024-05-09T09:28:25.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.074+0000 7fc50bfff640 1 -- 172.21.15.12:0/1035381739 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc5180054f0 con 0x55983e825f40 2024-05-09T09:28:25.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.074+0000 7fc50bfff640 1 -- 172.21.15.12:0/1035381739 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 41) ==== 8438+0+0 (secure 0 0 0) 0x7fc518030570 con 0x55983e825f40 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.074+0000 7fc535965740 10 client.26165 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.075+0000 7fc535965740 10 client.26165 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.075+0000 7fc535965740 20 client.26165 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.075+0000 7fc535965740 1 --2- 172.21.15.12:0/1035381739 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55983e83bff0 0x55983e83e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.075+0000 7fc535965740 4 client.26165 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.075+0000 7fc535965740 1 -- 172.21.15.12:0/1035381739 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55983e798370 con 0x55983e83bff0 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.075+0000 7fc530913640 1 --2- 172.21.15.12:0/1035381739 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55983e83bff0 0x55983e83e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.076+0000 7fc50bfff640 10 client.26165 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.076+0000 7fc530913640 1 --2- 172.21.15.12:0/1035381739 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55983e83bff0 0x55983e83e410 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e0e2d9ff1747deb server_cookie=f283650056ae3e8a in_seq=0 out_seq=0 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.076+0000 7fc50bfff640 1 -- 172.21.15.12:0/1035381739 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55983e798370 con 0x55983e83bff0 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.076+0000 7fc50bfff640 10 client.26165 handle_command_reply: tid=0 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.077+0000 7fc535965740 1 client.26165 shutdown 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.077+0000 7fc535965740 1 -- 172.21.15.12:0/1035381739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 msgr2=0x55983e830360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.077+0000 7fc535965740 1 --2- 172.21.15.12:0/1035381739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 0x55983e830360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc518009d50 tx=0x7fc518005f90 comp rx=0 tx=0).stop 2024-05-09T09:28:25.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.077+0000 7fc535965740 1 -- 172.21.15.12:0/1035381739 shutdown_connections 2024-05-09T09:28:25.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.077+0000 7fc535965740 1 --2- 172.21.15.12:0/1035381739 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55983e83bff0 0x55983e83e410 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.077+0000 7fc535965740 1 --2- 172.21.15.12:0/1035381739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 0x55983e8378e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.077+0000 7fc535965740 1 --2- 172.21.15.12:0/1035381739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 0x55983e8308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.077+0000 7fc535965740 1 --2- 172.21.15.12:0/1035381739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 0x55983e830360 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.077+0000 7fc535965740 1 -- 172.21.15.12:0/1035381739 >> 172.21.15.12:0/1035381739 conn(0x55983e8215b0 msgr2=0x55983e8207b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:25.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.077+0000 7fc50bfff640 0 client.26165 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:28:25.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.078+0000 7fc535965740 1 -- 172.21.15.12:0/1035381739 shutdown_connections 2024-05-09T09:28:25.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.078+0000 7fc535965740 1 -- 172.21.15.12:0/1035381739 wait complete. 2024-05-09T09:28:25.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.078+0000 7fc535965740 20 client.26165 trim_cache size 0 max 16384 2024-05-09T09:28:25.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.105+0000 7fc535965740 1 Processor -- start 2024-05-09T09:28:25.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.106+0000 7fc535965740 1 -- start start 2024-05-09T09:28:25.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.106+0000 7fc535965740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 0x55983e830800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.106+0000 7fc535965740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 0x55983e8378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.106+0000 7fc535965740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 0x55983e82f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.106+0000 7fc535965740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1a84a0 con 0x55983e826930 2024-05-09T09:28:25.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.106+0000 7fc535965740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1aa220 con 0x55983e825f40 2024-05-09T09:28:25.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.106+0000 7fc535965740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1a8c30 con 0x55983e82baa0 2024-05-09T09:28:25.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.106+0000 7fc530913640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 0x55983e8378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.106+0000 7fc530913640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 0x55983e8378a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41138/0 (socket says 172.21.15.12:41138) 2024-05-09T09:28:25.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.106+0000 7fc530913640 1 -- 172.21.15.12:0/4109154752 learned_addr learned my addr 172.21.15.12:0/4109154752 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:25.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.107+0000 7fc530913640 1 -- 172.21.15.12:0/4109154752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 msgr2=0x55983e82f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.107+0000 7fc530913640 1 --2- 172.21.15.12:0/4109154752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 0x55983e82f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.107+0000 7fc530913640 1 -- 172.21.15.12:0/4109154752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 msgr2=0x55983e830800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.107+0000 7fc530913640 1 --2- 172.21.15.12:0/4109154752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 0x55983e830800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.107+0000 7fc530913640 1 -- 172.21.15.12:0/4109154752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55983e826d10 con 0x55983e826930 2024-05-09T09:28:25.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.107+0000 7fc530913640 1 --2- 172.21.15.12:0/4109154752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 0x55983e8378a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc520009900 tx=0x7fc520000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f3fe121f8d0a82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:25.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.107+0000 7fc5167fc640 1 -- 172.21.15.12:0/4109154752 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520035040 con 0x55983e826930 2024-05-09T09:28:25.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.108+0000 7fc5167fc640 1 -- 172.21.15.12:0/4109154752 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc520045c70 con 0x55983e826930 2024-05-09T09:28:25.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.108+0000 7fc5167fc640 1 -- 172.21.15.12:0/4109154752 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5200047a0 con 0x55983e826930 2024-05-09T09:28:25.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.109+0000 7fc535965740 1 -- 172.21.15.12:0/4109154752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 msgr2=0x55983e8378a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.109+0000 7fc535965740 1 --2- 172.21.15.12:0/4109154752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 0x55983e8378a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc520009900 tx=0x7fc520000bc0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.109+0000 7fc535965740 1 -- 172.21.15.12:0/4109154752 shutdown_connections 2024-05-09T09:28:25.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.109+0000 7fc535965740 1 --2- 172.21.15.12:0/4109154752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e82baa0 0x55983e82f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.109+0000 7fc535965740 1 --2- 172.21.15.12:0/4109154752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e826930 0x55983e8378a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.109+0000 7fc535965740 1 --2- 172.21.15.12:0/4109154752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e825f40 0x55983e830800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.109+0000 7fc535965740 1 -- 172.21.15.12:0/4109154752 >> 172.21.15.12:0/4109154752 conn(0x55983e8215b0 msgr2=0x55983e82ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:25.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.109+0000 7fc535965740 1 -- 172.21.15.12:0/4109154752 shutdown_connections 2024-05-09T09:28:25.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.109+0000 7fc535965740 1 -- 172.21.15.12:0/4109154752 wait complete. 2024-05-09T09:28:25.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.110+0000 7fc535965740 1 -- start start 2024-05-09T09:28:25.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.110+0000 7fc535965740 1 Processor -- start 2024-05-09T09:28:25.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.110+0000 7fc535965740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e825f40 0x55983e834c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.110+0000 7fc535965740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e826930 0x55983e8351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.110+0000 7fc535965740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e82baa0 0x55983e836170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.110+0000 7fc535965740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1a9b50 con 0x55983e825f40 2024-05-09T09:28:25.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.110+0000 7fc535965740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1a93c0 con 0x55983e82baa0 2024-05-09T09:28:25.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.110+0000 7fc535965740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c0735f0 con 0x55983e826930 2024-05-09T09:28:25.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.110+0000 7fc535965740 10 client.0 _ll_register_callbacks cb 0x7fc518009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:25.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.111+0000 7fc530913640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e826930 0x55983e8351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.111+0000 7fc530913640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e826930 0x55983e8351d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:37896/0 (socket says 172.21.15.12:37896) 2024-05-09T09:28:25.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.111+0000 7fc530913640 1 -- 172.21.15.12:0/2244065877 learned_addr learned my addr 172.21.15.12:0/2244065877 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.111+0000 7fc52bfff640 1 --2- 172.21.15.12:0/2244065877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e82baa0 0x55983e836170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.111+0000 7fc52bfff640 1 --2- 172.21.15.12:0/2244065877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e825f40 0x55983e834c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.111+0000 7fc530913640 1 -- 172.21.15.12:0/2244065877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e82baa0 msgr2=0x55983e836170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.111+0000 7fc530913640 1 --2- 172.21.15.12:0/2244065877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e82baa0 0x55983e836170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.111+0000 7fc530913640 1 -- 172.21.15.12:0/2244065877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e825f40 msgr2=0x55983e834c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.111+0000 7fc530913640 1 --2- 172.21.15.12:0/2244065877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e825f40 0x55983e834c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.111+0000 7fc530913640 1 -- 172.21.15.12:0/2244065877 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55983e836760 con 0x55983e826930 2024-05-09T09:28:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.112+0000 7fc530913640 1 --2- 172.21.15.12:0/2244065877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e826930 0x55983e8351d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc52002a9e0 tx=0x7fc520004350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6b107970611d2f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.112+0000 7fc50bfff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:28:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.113+0000 7fc50bfff640 1 -- 172.21.15.12:0/2244065877 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520035070 con 0x55983e826930 2024-05-09T09:28:25.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.113+0000 7fc50bfff640 1 -- 172.21.15.12:0/2244065877 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc520043040 con 0x55983e826930 2024-05-09T09:28:25.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.114+0000 7fc535965740 1 -- 172.21.15.12:0/2244065877 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc520045c70 con 0x55983e826930 2024-05-09T09:28:25.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.115+0000 7fc50bfff640 1 -- 172.21.15.12:0/2244065877 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc520045260 con 0x55983e826930 2024-05-09T09:28:25.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.115+0000 7fc535965740 10 client.26278 fetch_fsmap learned FSMap version 41 2024-05-09T09:28:25.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.115+0000 7fc535965740 1 -- 172.21.15.12:0/2244065877 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55983e837a90 con 0x55983e826930 2024-05-09T09:28:25.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.116+0000 7fc50bfff640 1 -- 172.21.15.12:0/2244065877 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc520045c70 con 0x55983e826930 2024-05-09T09:28:25.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.116+0000 7fc50bfff640 1 -- 172.21.15.12:0/2244065877 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) ==== 8438+0+0 (secure 0 0 0) 0x7fc520031b00 con 0x55983e826930 2024-05-09T09:28:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.116+0000 7fc535965740 10 client.26278 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T09:28:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.116+0000 7fc535965740 10 client.26278 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:28:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.116+0000 7fc535965740 20 client.26278 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.116+0000 7fc535965740 1 --2- 172.21.15.12:0/2244065877 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55983e8a2ba0 0x55983e8a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.116+0000 7fc535965740 4 client.26278 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:28:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.116+0000 7fc535965740 1 -- 172.21.15.12:0/2244065877 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55983e7926c0 con 0x55983e8a2ba0 2024-05-09T09:28:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.116+0000 7fc52bfff640 1 --2- 172.21.15.12:0/2244065877 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55983e8a2ba0 0x55983e8a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.117+0000 7fc52bfff640 1 --2- 172.21.15.12:0/2244065877 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55983e8a2ba0 0x55983e8a4fc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ecc28f59ffa7cd3 server_cookie=7c93531b661fe72 in_seq=0 out_seq=0 2024-05-09T09:28:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.117+0000 7fc50bfff640 10 client.26278 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:28:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc50bfff640 1 -- 172.21.15.12:0/2244065877 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55983e7926c0 con 0x55983e8a2ba0 2024-05-09T09:28:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc50bfff640 10 client.26278 handle_command_reply: tid=0 2024-05-09T09:28:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc535965740 1 client.26278 shutdown 2024-05-09T09:28:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc535965740 1 -- 172.21.15.12:0/2244065877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e826930 msgr2=0x55983e8351d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc535965740 1 --2- 172.21.15.12:0/2244065877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e826930 0x55983e8351d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc52002a9e0 tx=0x7fc520004350 comp rx=0 tx=0).stop 2024-05-09T09:28:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc535965740 1 -- 172.21.15.12:0/2244065877 shutdown_connections 2024-05-09T09:28:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc535965740 1 --2- 172.21.15.12:0/2244065877 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x55983e8a2ba0 0x55983e8a4fc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc535965740 1 --2- 172.21.15.12:0/2244065877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55983e82baa0 0x55983e836170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc535965740 1 --2- 172.21.15.12:0/2244065877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55983e826930 0x55983e8351d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc535965740 1 --2- 172.21.15.12:0/2244065877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55983e825f40 0x55983e834c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc50bfff640 0 client.26278 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:28:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc535965740 1 -- 172.21.15.12:0/2244065877 >> 172.21.15.12:0/2244065877 conn(0x55983e8215b0 msgr2=0x55983e82f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc535965740 1 -- 172.21.15.12:0/2244065877 shutdown_connections 2024-05-09T09:28:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc535965740 1 -- 172.21.15.12:0/2244065877 wait complete. 2024-05-09T09:28:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.118+0000 7fc535965740 20 client.26278 trim_cache size 0 max 16384 2024-05-09T09:28:25.120 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:28:25.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.119+0000 7fc532b77640 1 -- 172.21.15.12:0/1526408411 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc51c0a2c90 msgr2=0x7fc51c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.119+0000 7fc532b77640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc51c0a2c90 0x7fc51c0a5150 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc524098230 tx=0x7fc524009a70 comp rx=0 tx=0).stop 2024-05-09T09:28:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.119+0000 7fc532b77640 1 -- 172.21.15.12:0/1526408411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc52c105400 msgr2=0x7fc52c1a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.119+0000 7fc532b77640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc52c105400 0x7fc52c1a5780 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc52400eb10 tx=0x7fc52400c2d0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.119+0000 7fc532b77640 1 -- 172.21.15.12:0/1526408411 shutdown_connections 2024-05-09T09:28:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.119+0000 7fc532b77640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc51c0a2c90 0x7fc51c0a5150 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.119+0000 7fc532b77640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc52c105400 0x7fc52c1a5780 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.119+0000 7fc532b77640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc52c102a50 0x7fc52c19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.119+0000 7fc532b77640 1 --2- 172.21.15.12:0/1526408411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc52c1000e0 0x7fc52c19e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.119+0000 7fc532b77640 1 -- 172.21.15.12:0/1526408411 >> 172.21.15.12:0/1526408411 conn(0x7fc52c0fbce0 msgr2=0x7fc52c100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:25.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.119+0000 7fc532b77640 1 -- 172.21.15.12:0/1526408411 shutdown_connections 2024-05-09T09:28:25.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.119+0000 7fc532b77640 1 -- 172.21.15.12:0/1526408411 wait complete. 2024-05-09T09:28:25.131 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:28:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.252+0000 7fc11ff22640 1 -- 172.21.15.12:0/3236006748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc118104c80 msgr2=0x7fc118105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.252+0000 7fc11ff22640 1 --2- 172.21.15.12:0/3236006748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc118104c80 0x7fc118105100 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc108009900 tx=0x7fc10802a410 comp rx=0 tx=0).stop 2024-05-09T09:28:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.252+0000 7fc11ff22640 1 -- 172.21.15.12:0/3236006748 shutdown_connections 2024-05-09T09:28:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.252+0000 7fc11ff22640 1 --2- 172.21.15.12:0/3236006748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc118105640 0x7fc11810bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.252+0000 7fc11ff22640 1 --2- 172.21.15.12:0/3236006748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc118104c80 0x7fc118105100 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.252+0000 7fc11ff22640 1 --2- 172.21.15.12:0/3236006748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc118103a90 0x7fc118103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.252+0000 7fc11ff22640 1 -- 172.21.15.12:0/3236006748 >> 172.21.15.12:0/3236006748 conn(0x7fc1180ff020 msgr2=0x7fc118101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.252+0000 7fc11ff22640 1 -- 172.21.15.12:0/3236006748 shutdown_connections 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.252+0000 7fc11ff22640 1 -- 172.21.15.12:0/3236006748 wait complete. 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11ff22640 1 Processor -- start 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11ff22640 1 -- start start 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11ff22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc118103a90 0x7fc118199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11ff22640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc118104c80 0x7fc11819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11ff22640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc118105640 0x7fc1181a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11ff22640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc11810f6f0 con 0x7fc118104c80 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11ff22640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc11810f570 con 0x7fc118103a90 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11ff22640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc11810f870 con 0x7fc118105640 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11d4bd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc118104c80 0x7fc11819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11dcbe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc118103a90 0x7fc118199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11d4bd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc118104c80 0x7fc11819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41160/0 (socket says 172.21.15.12:41160) 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11dcbe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc118103a90 0x7fc118199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:43274/0 (socket says 172.21.15.12:43274) 2024-05-09T09:28:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11d4bd640 1 -- 172.21.15.12:0/817284673 learned_addr learned my addr 172.21.15.12:0/817284673 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.253+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/817284673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc118105640 0x7fc1181a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.254+0000 7fc11dcbe640 1 -- 172.21.15.12:0/817284673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc118105640 msgr2=0x7fc1181a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.254+0000 7fc11dcbe640 1 --2- 172.21.15.12:0/817284673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc118105640 0x7fc1181a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.254+0000 7fc11dcbe640 1 -- 172.21.15.12:0/817284673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc118104c80 msgr2=0x7fc11819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.254+0000 7fc11dcbe640 1 --2- 172.21.15.12:0/817284673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc118104c80 0x7fc11819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.254+0000 7fc11dcbe640 1 -- 172.21.15.12:0/817284673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1181a1990 con 0x7fc118103a90 2024-05-09T09:28:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.254+0000 7fc11dcbe640 1 --2- 172.21.15.12:0/817284673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc118103a90 0x7fc118199d70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc11400e9c0 tx=0x7fc11400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.254+0000 7fc10effd640 1 -- 172.21.15.12:0/817284673 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11400ccc0 con 0x7fc118103a90 2024-05-09T09:28:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.254+0000 7fc10effd640 1 -- 172.21.15.12:0/817284673 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc1140035f0 con 0x7fc118103a90 2024-05-09T09:28:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.254+0000 7fc11ff22640 1 -- 172.21.15.12:0/817284673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1181a1c80 con 0x7fc118103a90 2024-05-09T09:28:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.254+0000 7fc10effd640 1 -- 172.21.15.12:0/817284673 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11400ce60 con 0x7fc118103a90 2024-05-09T09:28:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.254+0000 7fc11ff22640 1 -- 172.21.15.12:0/817284673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1181a21d0 con 0x7fc118103a90 2024-05-09T09:28:25.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.256+0000 7fc10effd640 1 -- 172.21.15.12:0/817284673 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc1140037a0 con 0x7fc118103a90 2024-05-09T09:28:25.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.257+0000 7fc122c34740 1 Processor -- start 2024-05-09T09:28:25.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.257+0000 7fc122c34740 1 -- start start 2024-05-09T09:28:25.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.257+0000 7fc122c34740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 0x5600ae87e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.257+0000 7fc122c34740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87e930 0x5600ae883430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.257+0000 7fc122c34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae883aa0 0x5600ae885f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.257+0000 7fc122c34740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5600ae7e2c40 con 0x5600ae87df40 2024-05-09T09:28:25.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.257+0000 7fc122c34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5600ae78a580 con 0x5600ae883aa0 2024-05-09T09:28:25.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.257+0000 7fc122c34740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5600ae7dec00 con 0x5600ae87e930 2024-05-09T09:28:25.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.257+0000 7fc10effd640 1 --2- 172.21.15.12:0/817284673 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc1040a2b90 0x7fc1040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.257+0000 7fc11e4bf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 0x5600ae87e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11e4bf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 0x5600ae87e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41172/0 (socket says 172.21.15.12:41172) 2024-05-09T09:28:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11e4bf640 1 -- 172.21.15.12:0/3447595573 learned_addr learned my addr 172.21.15.12:0/3447595573 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11d4bd640 1 --2- 172.21.15.12:0/3447595573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae883aa0 0x5600ae885f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc10effd640 1 -- 172.21.15.12:0/817284673 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc1140cf820 con 0x7fc118103a90 2024-05-09T09:28:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11dcbe640 1 --2- 172.21.15.12:0/3447595573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87e930 0x5600ae883430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/817284673 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc1040a2b90 0x7fc1040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11e4bf640 1 -- 172.21.15.12:0/3447595573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87e930 msgr2=0x5600ae883430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/3447595573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87e930 0x5600ae883430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11e4bf640 1 -- 172.21.15.12:0/3447595573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae883aa0 msgr2=0x5600ae885f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/3447595573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae883aa0 0x5600ae885f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11e4bf640 1 -- 172.21.15.12:0/3447595573 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ae4f63f0 con 0x5600ae87df40 2024-05-09T09:28:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11d4bd640 1 --2- 172.21.15.12:0/3447595573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae883aa0 0x5600ae885f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11dcbe640 1 --2- 172.21.15.12:0/3447595573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87e930 0x5600ae883430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:28:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/3447595573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 0x5600ae87e360 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc110005c40 tx=0x7fc11000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbaf40b5aec94871 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc103fff640 1 -- 172.21.15.12:0/3447595573 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110022020 con 0x5600ae87df40 2024-05-09T09:28:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc103fff640 1 -- 172.21.15.12:0/3447595573 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc110013170 con 0x5600ae87df40 2024-05-09T09:28:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.258+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/817284673 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc1040a2b90 0x7fc1040a5050 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc110023000 tx=0x7fc110021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.259+0000 7fc103fff640 1 -- 172.21.15.12:0/3447595573 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100107e0 con 0x5600ae87df40 2024-05-09T09:28:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.259+0000 7fc122c34740 1 -- 172.21.15.12:0/3447595573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 msgr2=0x5600ae87e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.259+0000 7fc122c34740 1 --2- 172.21.15.12:0/3447595573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 0x5600ae87e360 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc110005c40 tx=0x7fc11000f3c0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.259+0000 7fc122c34740 1 -- 172.21.15.12:0/3447595573 shutdown_connections 2024-05-09T09:28:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.259+0000 7fc122c34740 1 --2- 172.21.15.12:0/3447595573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae883aa0 0x5600ae885f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.259+0000 7fc122c34740 1 --2- 172.21.15.12:0/3447595573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87e930 0x5600ae883430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.259+0000 7fc122c34740 1 --2- 172.21.15.12:0/3447595573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 0x5600ae87e360 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.259+0000 7fc122c34740 1 -- 172.21.15.12:0/3447595573 >> 172.21.15.12:0/3447595573 conn(0x5600ae8795b0 msgr2=0x5600ae8799b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.259+0000 7fc122c34740 1 -- 172.21.15.12:0/3447595573 shutdown_connections 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.259+0000 7fc122c34740 1 -- 172.21.15.12:0/3447595573 wait complete. 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.260+0000 7fc122c34740 1 -- start start 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.260+0000 7fc122c34740 1 Processor -- start 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.260+0000 7fc122c34740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 0x5600ae8883c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.260+0000 7fc122c34740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87e930 0x5600ae888900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.260+0000 7fc122c34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae883aa0 0x5600ae88f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.260+0000 7fc122c34740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5600ae886a50 con 0x5600ae87df40 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.260+0000 7fc122c34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5600ae79dc40 con 0x5600ae883aa0 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.260+0000 7fc122c34740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5600ae886bd0 con 0x5600ae87e930 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.260+0000 7fc122c34740 10 client.0 _ll_register_callbacks cb 0x5600ae4f5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.260+0000 7fc11dcbe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 0x5600ae8883c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.260+0000 7fc11dcbe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 0x5600ae8883c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41182/0 (socket says 172.21.15.12:41182) 2024-05-09T09:28:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.260+0000 7fc11dcbe640 1 -- 172.21.15.12:0/3152547296 learned_addr learned my addr 172.21.15.12:0/3152547296 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.260+0000 7fc11d4bd640 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87e930 0x5600ae888900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae883aa0 0x5600ae88f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc11dcbe640 1 -- 172.21.15.12:0/3152547296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87e930 msgr2=0x5600ae888900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc11dcbe640 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87e930 0x5600ae888900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc11dcbe640 1 -- 172.21.15.12:0/3152547296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae883aa0 msgr2=0x5600ae88f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc11dcbe640 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae883aa0 0x5600ae88f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc11dcbe640 1 -- 172.21.15.12:0/3152547296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600ae5c07d0 con 0x5600ae87df40 2024-05-09T09:28:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc11d4bd640 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87e930 0x5600ae888900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae883aa0 0x5600ae88f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:28:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc11dcbe640 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 0x5600ae8883c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc114009230 tx=0x7fc114003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19003b77587fb257 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc1017fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc1017fa640 1 -- 172.21.15.12:0/3152547296 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11409bcb0 con 0x5600ae87df40 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc1017fa640 1 -- 172.21.15.12:0/3152547296 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc11409b100 con 0x5600ae87df40 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc122c34740 1 -- 172.21.15.12:0/3152547296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc110013170 con 0x5600ae87df40 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc1017fa640 1 -- 172.21.15.12:0/3152547296 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc1140a0060 con 0x5600ae87df40 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc122c34740 10 client.16743 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.261+0000 7fc122c34740 1 -- 172.21.15.12:0/3152547296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5600ae892d20 con 0x5600ae87df40 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.262+0000 7fc1017fa640 1 -- 172.21.15.12:0/3152547296 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc110013170 con 0x5600ae87df40 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.262+0000 7fc1017fa640 1 -- 172.21.15.12:0/3152547296 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fc114097a00 con 0x5600ae87df40 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.262+0000 7fc122c34740 10 client.16743 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.262+0000 7fc122c34740 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.262+0000 7fc122c34740 20 client.16743 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.262+0000 7fc122c34740 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5600ae894500 0x5600ae896980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.262+0000 7fc122c34740 4 client.16743 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.262+0000 7fc122c34740 1 -- 172.21.15.12:0/3152547296 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600ae7f0370 con 0x5600ae894500 2024-05-09T09:28:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.262+0000 7fc11d4bd640 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5600ae894500 0x5600ae896980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.263+0000 7fc11d4bd640 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5600ae894500 0x5600ae896980 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6881a07a0a9884b5 server_cookie=e60030c01504158f in_seq=0 out_seq=0 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.263+0000 7fc1017fa640 10 client.16743 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.263+0000 7fc1017fa640 1 -- 172.21.15.12:0/3152547296 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600ae7f0370 con 0x5600ae894500 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.263+0000 7fc1017fa640 10 client.16743 handle_command_reply: tid=0 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.263+0000 7fc122c34740 1 client.16743 shutdown 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.264+0000 7fc122c34740 1 -- 172.21.15.12:0/3152547296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 msgr2=0x5600ae8883c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.264+0000 7fc122c34740 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 0x5600ae8883c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc114009230 tx=0x7fc114003b80 comp rx=0 tx=0).stop 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.264+0000 7fc122c34740 1 -- 172.21.15.12:0/3152547296 shutdown_connections 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.264+0000 7fc122c34740 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5600ae894500 0x5600ae896980 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.264+0000 7fc122c34740 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae883aa0 0x5600ae88f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.264+0000 7fc122c34740 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87e930 0x5600ae888900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.264+0000 7fc122c34740 1 --2- 172.21.15.12:0/3152547296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae87df40 0x5600ae8883c0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.264+0000 7fc122c34740 1 -- 172.21.15.12:0/3152547296 >> 172.21.15.12:0/3152547296 conn(0x5600ae8795b0 msgr2=0x5600ae8787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.264+0000 7fc1017fa640 0 client.16743 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.264+0000 7fc122c34740 1 -- 172.21.15.12:0/3152547296 shutdown_connections 2024-05-09T09:28:25.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.264+0000 7fc122c34740 1 -- 172.21.15.12:0/3152547296 wait complete. 2024-05-09T09:28:25.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.264+0000 7fc122c34740 20 client.16743 trim_cache size 0 max 16384 2024-05-09T09:28:25.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.279+0000 7fc122c34740 1 Processor -- start 2024-05-09T09:28:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.279+0000 7fc122c34740 1 -- start start 2024-05-09T09:28:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc122c34740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87df40 0x5600ae8886e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc122c34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae87e930 0x5600ae88f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc122c34740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 0x5600ae877f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc122c34740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc1181ac6d0 con 0x5600ae883aa0 2024-05-09T09:28:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc122c34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1181ae450 con 0x5600ae87e930 2024-05-09T09:28:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc122c34740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc1181ace60 con 0x5600ae87df40 2024-05-09T09:28:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc11e4bf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 0x5600ae877f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc11e4bf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 0x5600ae877f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41188/0 (socket says 172.21.15.12:41188) 2024-05-09T09:28:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc11e4bf640 1 -- 172.21.15.12:0/1083449935 learned_addr learned my addr 172.21.15.12:0/1083449935 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc11dcbe640 1 --2- 172.21.15.12:0/1083449935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87df40 0x5600ae8886e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc11d4bd640 1 --2- 172.21.15.12:0/1083449935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae87e930 0x5600ae88f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc11e4bf640 1 -- 172.21.15.12:0/1083449935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87df40 msgr2=0x5600ae8886e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/1083449935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87df40 0x5600ae8886e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc11e4bf640 1 -- 172.21.15.12:0/1083449935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae87e930 msgr2=0x5600ae88f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/1083449935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae87e930 0x5600ae88f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc11e4bf640 1 -- 172.21.15.12:0/1083449935 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ae86ad20 con 0x5600ae883aa0 2024-05-09T09:28:25.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc11dcbe640 1 --2- 172.21.15.12:0/1083449935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87df40 0x5600ae8886e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:25.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.280+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/1083449935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 0x5600ae877f20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc110009270 tx=0x7fc110005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8685e9f1696e89d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:25.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.281+0000 7fc1037fe640 1 -- 172.21.15.12:0/1083449935 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110013170 con 0x5600ae883aa0 2024-05-09T09:28:25.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.281+0000 7fc1037fe640 1 -- 172.21.15.12:0/1083449935 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc110013540 con 0x5600ae883aa0 2024-05-09T09:28:25.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.281+0000 7fc1037fe640 1 -- 172.21.15.12:0/1083449935 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100052b0 con 0x5600ae883aa0 2024-05-09T09:28:25.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.281+0000 7fc122c34740 1 -- 172.21.15.12:0/1083449935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 msgr2=0x5600ae877f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.281+0000 7fc122c34740 1 --2- 172.21.15.12:0/1083449935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 0x5600ae877f20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc110009270 tx=0x7fc110005f10 comp rx=0 tx=0).stop 2024-05-09T09:28:25.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.282+0000 7fc122c34740 1 -- 172.21.15.12:0/1083449935 shutdown_connections 2024-05-09T09:28:25.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.282+0000 7fc122c34740 1 --2- 172.21.15.12:0/1083449935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 0x5600ae877f20 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.282+0000 7fc122c34740 1 --2- 172.21.15.12:0/1083449935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae87e930 0x5600ae88f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.282+0000 7fc122c34740 1 --2- 172.21.15.12:0/1083449935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87df40 0x5600ae8886e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.282+0000 7fc122c34740 1 -- 172.21.15.12:0/1083449935 >> 172.21.15.12:0/1083449935 conn(0x5600ae8795b0 msgr2=0x5600ae887fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.282+0000 7fc122c34740 1 -- 172.21.15.12:0/1083449935 shutdown_connections 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.282+0000 7fc122c34740 1 -- 172.21.15.12:0/1083449935 wait complete. 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.282+0000 7fc122c34740 1 -- start start 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.282+0000 7fc122c34740 1 Processor -- start 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.282+0000 7fc122c34740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87df40 0x5600ae88c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc122c34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae87e930 0x5600ae88ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc122c34740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 0x5600ae8f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc122c34740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc1181add80 con 0x5600ae883aa0 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc122c34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1181ad5f0 con 0x5600ae87e930 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc122c34740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc1181aa6b0 con 0x5600ae87df40 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc122c34740 10 client.0 _ll_register_callbacks cb 0x7fc114003a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc11e4bf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 0x5600ae8f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc11e4bf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 0x5600ae8f8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41204/0 (socket says 172.21.15.12:41204) 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc11e4bf640 1 -- 172.21.15.12:0/3092024696 learned_addr learned my addr 172.21.15.12:0/3092024696 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc11d4bd640 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae87e930 0x5600ae88ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc11dcbe640 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87df40 0x5600ae88c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc11e4bf640 1 -- 172.21.15.12:0/3092024696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87df40 msgr2=0x5600ae88c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87df40 0x5600ae88c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc11e4bf640 1 -- 172.21.15.12:0/3092024696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae87e930 msgr2=0x5600ae88ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae87e930 0x5600ae88ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc11e4bf640 1 -- 172.21.15.12:0/3092024696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600ae8f9070 con 0x5600ae883aa0 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc11d4bd640 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae87e930 0x5600ae88ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc11e4bf640 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 0x5600ae8f8ab0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc110009ca0 tx=0x7fc110005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70be8c4a19132f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc1017fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.283+0000 7fc1017fa640 1 -- 172.21.15.12:0/3092024696 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110041060 con 0x5600ae883aa0 2024-05-09T09:28:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc122c34740 1 -- 172.21.15.12:0/3092024696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc110013540 con 0x5600ae883aa0 2024-05-09T09:28:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc1017fa640 1 -- 172.21.15.12:0/3092024696 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc110010e00 con 0x5600ae883aa0 2024-05-09T09:28:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc1017fa640 1 -- 172.21.15.12:0/3092024696 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc110010580 con 0x5600ae883aa0 2024-05-09T09:28:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc122c34740 10 client.16749 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc122c34740 1 -- 172.21.15.12:0/3092024696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5600ae8fa370 con 0x5600ae883aa0 2024-05-09T09:28:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc1017fa640 1 -- 172.21.15.12:0/3092024696 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc110013540 con 0x5600ae883aa0 2024-05-09T09:28:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc1017fa640 1 -- 172.21.15.12:0/3092024696 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fc1100339d0 con 0x5600ae883aa0 2024-05-09T09:28:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc122c34740 10 client.16749 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc122c34740 10 client.16749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc122c34740 20 client.16749 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc122c34740 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5600ae8fab40 0x5600ae8fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc122c34740 4 client.16749 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:28:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc122c34740 1 -- 172.21.15.12:0/3092024696 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600ae7ea6c0 con 0x5600ae8fab40 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.284+0000 7fc11dcbe640 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5600ae8fab40 0x5600ae8fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc11dcbe640 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5600ae8fab40 0x5600ae8fcf60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9de4003d50306f1 server_cookie=31e6b8d97d4971db in_seq=0 out_seq=0 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc1017fa640 10 client.16749 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc1017fa640 1 -- 172.21.15.12:0/3092024696 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600ae7ea6c0 con 0x5600ae8fab40 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc1017fa640 10 client.16749 handle_command_reply: tid=0 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc122c34740 1 client.16749 shutdown 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc122c34740 1 -- 172.21.15.12:0/3092024696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 msgr2=0x5600ae8f8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc122c34740 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 0x5600ae8f8ab0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc110009ca0 tx=0x7fc110005ce0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc122c34740 1 -- 172.21.15.12:0/3092024696 shutdown_connections 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc122c34740 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5600ae8fab40 0x5600ae8fcf60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc122c34740 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5600ae883aa0 0x5600ae8f8ab0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc122c34740 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600ae87e930 0x5600ae88ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc1017fa640 0 client.16749 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc122c34740 1 --2- 172.21.15.12:0/3092024696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600ae87df40 0x5600ae88c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc122c34740 1 -- 172.21.15.12:0/3092024696 >> 172.21.15.12:0/3092024696 conn(0x5600ae8795b0 msgr2=0x5600ae87c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc122c34740 1 -- 172.21.15.12:0/3092024696 shutdown_connections 2024-05-09T09:28:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.285+0000 7fc122c34740 1 -- 172.21.15.12:0/3092024696 wait complete. 2024-05-09T09:28:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.286+0000 7fc122c34740 20 client.16749 trim_cache size 0 max 16384 2024-05-09T09:28:25.293 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:28:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.286+0000 7fc11ff22640 1 -- 172.21.15.12:0/817284673 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc1040a2b90 msgr2=0x7fc1040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.286+0000 7fc11ff22640 1 --2- 172.21.15.12:0/817284673 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc1040a2b90 0x7fc1040a5050 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc110023000 tx=0x7fc110021000 comp rx=0 tx=0).stop 2024-05-09T09:28:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.286+0000 7fc11ff22640 1 -- 172.21.15.12:0/817284673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc118103a90 msgr2=0x7fc118199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.286+0000 7fc11ff22640 1 --2- 172.21.15.12:0/817284673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc118103a90 0x7fc118199d70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc11400e9c0 tx=0x7fc11400ee80 comp rx=0 tx=0).stop 2024-05-09T09:28:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.286+0000 7fc11ff22640 1 -- 172.21.15.12:0/817284673 shutdown_connections 2024-05-09T09:28:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.286+0000 7fc11ff22640 1 --2- 172.21.15.12:0/817284673 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc1040a2b90 0x7fc1040a5050 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.286+0000 7fc11ff22640 1 --2- 172.21.15.12:0/817284673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc118105640 0x7fc1181a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.286+0000 7fc11ff22640 1 --2- 172.21.15.12:0/817284673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc118104c80 0x7fc11819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.286+0000 7fc11ff22640 1 --2- 172.21.15.12:0/817284673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc118103a90 0x7fc118199d70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:25.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.286+0000 7fc11ff22640 1 -- 172.21.15.12:0/817284673 >> 172.21.15.12:0/817284673 conn(0x7fc1180ff020 msgr2=0x7fc118107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:25.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.286+0000 7fc11ff22640 1 -- 172.21.15.12:0/817284673 shutdown_connections 2024-05-09T09:28:25.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:25.286+0000 7fc11ff22640 1 -- 172.21.15.12:0/817284673 wait complete. 2024-05-09T09:28:25.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:25 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2881045645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:28:25.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:25 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2881045645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:28:25.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:25 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2881045645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:28:26.130 INFO:tasks.workunit.client.0.smithi012.stderr:+ tar xf doc.tar 2024-05-09T09:28:26.299 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:28:26.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:28:26.299 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:28:26.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.440+0000 7f73ed19b640 1 -- 172.21.15.12:0/3549212630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73e8101960 msgr2=0x7f73e8101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.440+0000 7f73ed19b640 1 --2- 172.21.15.12:0/3549212630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73e8101960 0x7f73e8101de0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009900 tx=0x7f73d002a410 comp rx=0 tx=0).stop 2024-05-09T09:28:26.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.440+0000 7f73ed19b640 1 -- 172.21.15.12:0/3549212630 shutdown_connections 2024-05-09T09:28:26.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.440+0000 7f73ed19b640 1 --2- 172.21.15.12:0/3549212630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73e8102320 0x7f73e8108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.440+0000 7f73ed19b640 1 --2- 172.21.15.12:0/3549212630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73e8101960 0x7f73e8101de0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.440+0000 7f73ed19b640 1 --2- 172.21.15.12:0/3549212630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8100770 0x7f73e8100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.440+0000 7f73ed19b640 1 -- 172.21.15.12:0/3549212630 >> 172.21.15.12:0/3549212630 conn(0x7f73e80fbce0 msgr2=0x7f73e80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:26.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.440+0000 7f73ed19b640 1 -- 172.21.15.12:0/3549212630 shutdown_connections 2024-05-09T09:28:26.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.440+0000 7f73ed19b640 1 -- 172.21.15.12:0/3549212630 wait complete. 2024-05-09T09:28:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.441+0000 7f73ed19b640 1 Processor -- start 2024-05-09T09:28:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.441+0000 7f73ed19b640 1 -- start start 2024-05-09T09:28:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.441+0000 7f73ed19b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73e8100770 0x7f73e8197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.441+0000 7f73ed19b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73e8101960 0x7f73e8198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.441+0000 7f73ed19b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8102320 0x7f73e819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73ed19b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73e810c3d0 con 0x7f73e8100770 2024-05-09T09:28:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73ed19b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73e810c250 con 0x7f73e8102320 2024-05-09T09:28:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73ed19b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f73e810c550 con 0x7f73e8101960 2024-05-09T09:28:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73e6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73e8100770 0x7f73e8197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73e759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8102320 0x7f73e819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73e6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73e8100770 0x7f73e8197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41226/0 (socket says 172.21.15.12:41226) 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73e6d9d640 1 -- 172.21.15.12:0/1112767424 learned_addr learned my addr 172.21.15.12:0/1112767424 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73e659c640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73e8101960 0x7f73e8198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73e6d9d640 1 -- 172.21.15.12:0/1112767424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73e8101960 msgr2=0x7f73e8198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73e6d9d640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73e8101960 0x7f73e8198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73e6d9d640 1 -- 172.21.15.12:0/1112767424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8102320 msgr2=0x7f73e819f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73e6d9d640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8102320 0x7f73e819f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73e6d9d640 1 -- 172.21.15.12:0/1112767424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e819f7f0 con 0x7f73e8100770 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73e659c640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73e8101960 0x7f73e8198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73e759e640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8102320 0x7f73e819f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.442+0000 7f73e6d9d640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73e8100770 0x7f73e8197bd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f73d800ebc0 tx=0x7f73d800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.443+0000 7f73dffff640 1 -- 172.21.15.12:0/1112767424 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8003b70 con 0x7f73e8100770 2024-05-09T09:28:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.443+0000 7f73ed19b640 1 -- 172.21.15.12:0/1112767424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e819fae0 con 0x7f73e8100770 2024-05-09T09:28:26.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.443+0000 7f73dffff640 1 -- 172.21.15.12:0/1112767424 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f73d8003d10 con 0x7f73e8100770 2024-05-09T09:28:26.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.443+0000 7f73dffff640 1 -- 172.21.15.12:0/1112767424 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8019860 con 0x7f73e8100770 2024-05-09T09:28:26.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.443+0000 7f73ed19b640 1 -- 172.21.15.12:0/1112767424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e81a0050 con 0x7f73e8100770 2024-05-09T09:28:26.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.445+0000 7f73dffff640 1 -- 172.21.15.12:0/1112767424 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f73d8019a00 con 0x7f73e8100770 2024-05-09T09:28:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.446+0000 7f73eff83740 1 Processor -- start 2024-05-09T09:28:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.446+0000 7f73eff83740 1 -- start start 2024-05-09T09:28:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73eff83740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 0x5640b179f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73dffff640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f73d40a2c10 0x7f73d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73eff83740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179feb0 0x5640b17a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73eff83740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b17a5020 0x5640b17a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73eff83740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5640b1703990 con 0x5640b179f4c0 2024-05-09T09:28:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73eff83740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5640b16ab5b0 con 0x5640b17a5020 2024-05-09T09:28:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73eff83740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5640b16ff950 con 0x5640b179feb0 2024-05-09T09:28:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e6d9d640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f73d40a2c10 0x7f73d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 0x5640b179f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 0x5640b179f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41240/0 (socket says 172.21.15.12:41240) 2024-05-09T09:28:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e759e640 1 -- 172.21.15.12:0/2511320921 learned_addr learned my addr 172.21.15.12:0/2511320921 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e659c640 1 --2- 172.21.15.12:0/2511320921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179feb0 0x5640b17a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e759e640 1 --2- 172.21.15.12:0/2511320921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b17a5020 0x5640b17a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73dffff640 1 -- 172.21.15.12:0/1112767424 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f73d80cfcc0 con 0x7f73e8100770 2024-05-09T09:28:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e659c640 1 -- 172.21.15.12:0/2511320921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b17a5020 msgr2=0x5640b17a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e659c640 1 --2- 172.21.15.12:0/2511320921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b17a5020 0x5640b17a74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e659c640 1 -- 172.21.15.12:0/2511320921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 msgr2=0x5640b179f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e659c640 1 --2- 172.21.15.12:0/2511320921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 0x5640b179f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e659c640 1 -- 172.21.15.12:0/2511320921 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640b14df7e0 con 0x5640b179feb0 2024-05-09T09:28:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e759e640 1 --2- 172.21.15.12:0/2511320921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 0x5640b179f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e6d9d640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f73d40a2c10 0x7f73d40a50d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f73d8086cf0 tx=0x7f73d800cf90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.447+0000 7f73e659c640 1 --2- 172.21.15.12:0/2511320921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179feb0 0x5640b17a49b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f73d0005ab0 tx=0x7f73d0005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3f7319f03dc8158 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.448+0000 7f73dd7fa640 1 -- 172.21.15.12:0/2511320921 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0035070 con 0x5640b179feb0 2024-05-09T09:28:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.448+0000 7f73dd7fa640 1 -- 172.21.15.12:0/2511320921 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f73d0031890 con 0x5640b179feb0 2024-05-09T09:28:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.448+0000 7f73dd7fa640 1 -- 172.21.15.12:0/2511320921 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0004a00 con 0x5640b179feb0 2024-05-09T09:28:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.449+0000 7f73eff83740 1 -- 172.21.15.12:0/2511320921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179feb0 msgr2=0x5640b17a49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.449+0000 7f73eff83740 1 --2- 172.21.15.12:0/2511320921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179feb0 0x5640b17a49b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f73d0005ab0 tx=0x7f73d0005d50 comp rx=0 tx=0).stop 2024-05-09T09:28:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.449+0000 7f73eff83740 1 -- 172.21.15.12:0/2511320921 shutdown_connections 2024-05-09T09:28:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.449+0000 7f73eff83740 1 --2- 172.21.15.12:0/2511320921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b17a5020 0x5640b17a74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.449+0000 7f73eff83740 1 --2- 172.21.15.12:0/2511320921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179feb0 0x5640b17a49b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.449+0000 7f73eff83740 1 --2- 172.21.15.12:0/2511320921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 0x5640b179f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.449+0000 7f73eff83740 1 -- 172.21.15.12:0/2511320921 >> 172.21.15.12:0/2511320921 conn(0x5640b17670a0 msgr2=0x5640b179af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.449+0000 7f73eff83740 1 -- 172.21.15.12:0/2511320921 shutdown_connections 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.449+0000 7f73eff83740 1 -- 172.21.15.12:0/2511320921 wait complete. 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.449+0000 7f73eff83740 1 -- start start 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.449+0000 7f73eff83740 1 Processor -- start 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73eff83740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 0x5640b17a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73eff83740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179feb0 0x5640b17a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73eff83740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b17a5020 0x5640b17b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73eff83740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5640b17a7f70 con 0x5640b179f4c0 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73eff83740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5640b16beb40 con 0x5640b179feb0 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73e759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 0x5640b17a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73eff83740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5640b17a80f0 con 0x5640b17a5020 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73e759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 0x5640b17a98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41252/0 (socket says 172.21.15.12:41252) 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73e759e640 1 -- 172.21.15.12:0/2295355239 learned_addr learned my addr 172.21.15.12:0/2295355239 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179feb0 0x5640b17a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73e759e640 1 --2- 172.21.15.12:0/2295355239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b17a5020 0x5640b17b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73e659c640 1 -- 172.21.15.12:0/2295355239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b17a5020 msgr2=0x5640b17b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73e659c640 1 --2- 172.21.15.12:0/2295355239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b17a5020 0x5640b17b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73e659c640 1 -- 172.21.15.12:0/2295355239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 msgr2=0x5640b17a98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73e659c640 1 --2- 172.21.15.12:0/2295355239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 0x5640b17a98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73e659c640 1 -- 172.21.15.12:0/2295355239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640b153ff70 con 0x5640b179feb0 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73e759e640 1 --2- 172.21.15.12:0/2295355239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 0x5640b17a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73eff83740 10 client.0 _ll_register_callbacks cb 0x5640b16b30f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.450+0000 7f73eff83740 1 -- 172.21.15.12:0/2295355239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d0031890 con 0x5640b179feb0 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.451+0000 7f73e659c640 1 --2- 172.21.15.12:0/2295355239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179feb0 0x5640b17a9e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5640b179bdf0 tx=0x7f73d00095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36ab00f34eeb9c43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.451+0000 7f73ce7fc640 10 client.26198 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.451+0000 7f73ce7fc640 1 -- 172.21.15.12:0/2295355239 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0056050 con 0x5640b179feb0 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.451+0000 7f73ce7fc640 1 -- 172.21.15.12:0/2295355239 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f73d00047a0 con 0x5640b179feb0 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.451+0000 7f73ce7fc640 1 -- 172.21.15.12:0/2295355239 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f73d004c070 con 0x5640b179feb0 2024-05-09T09:28:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.451+0000 7f73eff83740 10 client.26198 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.451+0000 7f73eff83740 1 -- 172.21.15.12:0/2295355239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=42}) -- 0x5640b13d7170 con 0x5640b179feb0 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.451+0000 7f73ce7fc640 1 -- 172.21.15.12:0/2295355239 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f73d0059020 con 0x5640b179feb0 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.451+0000 7f73eff83740 10 client.26198 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.451+0000 7f73eff83740 10 client.26198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.451+0000 7f73eff83740 20 client.26198 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.451+0000 7f73eff83740 1 --2- 172.21.15.12:0/2295355239 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5640b17ca080 0x5640b17cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.452+0000 7f73eff83740 4 client.26198 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.452+0000 7f73eff83740 1 -- 172.21.15.12:0/2295355239 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640b17110c0 con 0x5640b17ca080 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.452+0000 7f73e759e640 1 --2- 172.21.15.12:0/2295355239 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5640b17ca080 0x5640b17cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.452+0000 7f73e759e640 1 --2- 172.21.15.12:0/2295355239 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5640b17ca080 0x5640b17cc4f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44075b3d6dd1e302 server_cookie=bb6848648932e66 in_seq=0 out_seq=0 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.452+0000 7f73ce7fc640 10 client.26198 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.453+0000 7f73ce7fc640 1 -- 172.21.15.12:0/2295355239 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640b17110c0 con 0x5640b17ca080 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.453+0000 7f73ce7fc640 10 client.26198 handle_command_reply: tid=0 2024-05-09T09:28:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.453+0000 7f73eff83740 1 client.26198 shutdown 2024-05-09T09:28:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.453+0000 7f73eff83740 1 -- 172.21.15.12:0/2295355239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179feb0 msgr2=0x5640b17a9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.453+0000 7f73eff83740 1 --2- 172.21.15.12:0/2295355239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179feb0 0x5640b17a9e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5640b179bdf0 tx=0x7f73d00095a0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.453+0000 7f73eff83740 1 -- 172.21.15.12:0/2295355239 shutdown_connections 2024-05-09T09:28:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.453+0000 7f73eff83740 1 --2- 172.21.15.12:0/2295355239 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5640b17ca080 0x5640b17cc4f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.453+0000 7f73eff83740 1 --2- 172.21.15.12:0/2295355239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b17a5020 0x5640b17b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.453+0000 7f73eff83740 1 --2- 172.21.15.12:0/2295355239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179feb0 0x5640b17a9e30 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5640b179bdf0 tx=0x7f73d00095a0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.453+0000 7f73eff83740 1 --2- 172.21.15.12:0/2295355239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b179f4c0 0x5640b17a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.453+0000 7f73eff83740 1 -- 172.21.15.12:0/2295355239 >> 172.21.15.12:0/2295355239 conn(0x5640b17670a0 msgr2=0x5640b179a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.453+0000 7f73ce7fc640 0 client.26198 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.454+0000 7f73eff83740 1 -- 172.21.15.12:0/2295355239 shutdown_connections 2024-05-09T09:28:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.454+0000 7f73eff83740 1 -- 172.21.15.12:0/2295355239 wait complete. 2024-05-09T09:28:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.454+0000 7f73eff83740 20 client.26198 trim_cache size 0 max 16384 2024-05-09T09:28:26.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.476+0000 7f73eff83740 1 Processor -- start 2024-05-09T09:28:26.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.476+0000 7f73eff83740 1 -- start start 2024-05-09T09:28:26.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.476+0000 7f73eff83740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179f4c0 0x5640b17a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.476+0000 7f73eff83740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 0x5640b17b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.476+0000 7f73eff83740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179ae20 0x5640b1799f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.476+0000 7f73eff83740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73e80733a0 con 0x5640b17a5020 2024-05-09T09:28:26.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.476+0000 7f73eff83740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81a18e0 con 0x5640b179ae20 2024-05-09T09:28:26.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.476+0000 7f73eff83740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81a02f0 con 0x5640b179f4c0 2024-05-09T09:28:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.476+0000 7f73e659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 0x5640b17b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.476+0000 7f73e659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 0x5640b17b0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41266/0 (socket says 172.21.15.12:41266) 2024-05-09T09:28:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.477+0000 7f73e659c640 1 -- 172.21.15.12:0/3018163399 learned_addr learned my addr 172.21.15.12:0/3018163399 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.477+0000 7f73e759e640 1 --2- 172.21.15.12:0/3018163399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179f4c0 0x5640b17a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.477+0000 7f73e759e640 1 --2- 172.21.15.12:0/3018163399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179ae20 0x5640b1799f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.477+0000 7f73e659c640 1 -- 172.21.15.12:0/3018163399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179f4c0 msgr2=0x5640b17a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.477+0000 7f73e659c640 1 --2- 172.21.15.12:0/3018163399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179f4c0 0x5640b17a9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.477+0000 7f73e659c640 1 -- 172.21.15.12:0/3018163399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179ae20 msgr2=0x5640b1799f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.477+0000 7f73e659c640 1 --2- 172.21.15.12:0/3018163399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179ae20 0x5640b1799f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.477+0000 7f73e659c640 1 -- 172.21.15.12:0/3018163399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640b179bbf0 con 0x5640b17a5020 2024-05-09T09:28:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.477+0000 7f73e659c640 1 --2- 172.21.15.12:0/3018163399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 0x5640b17b0e30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f73d002a430 tx=0x7f73d0048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ae2359166f26c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:26.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.478+0000 7f73dcff9640 1 -- 172.21.15.12:0/3018163399 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0056040 con 0x5640b17a5020 2024-05-09T09:28:26.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.478+0000 7f73dcff9640 1 -- 172.21.15.12:0/3018163399 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f73d0060070 con 0x5640b17a5020 2024-05-09T09:28:26.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.478+0000 7f73dcff9640 1 -- 172.21.15.12:0/3018163399 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0035050 con 0x5640b17a5020 2024-05-09T09:28:26.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.479+0000 7f73eff83740 1 -- 172.21.15.12:0/3018163399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 msgr2=0x5640b17b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.479+0000 7f73eff83740 1 --2- 172.21.15.12:0/3018163399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 0x5640b17b0e30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f73d002a430 tx=0x7f73d0048000 comp rx=0 tx=0).stop 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.479+0000 7f73eff83740 1 -- 172.21.15.12:0/3018163399 shutdown_connections 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.479+0000 7f73eff83740 1 --2- 172.21.15.12:0/3018163399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179ae20 0x5640b1799f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.479+0000 7f73eff83740 1 --2- 172.21.15.12:0/3018163399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 0x5640b17b0e30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.479+0000 7f73eff83740 1 --2- 172.21.15.12:0/3018163399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179f4c0 0x5640b17a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.479+0000 7f73eff83740 1 -- 172.21.15.12:0/3018163399 >> 172.21.15.12:0/3018163399 conn(0x5640b17670a0 msgr2=0x5640b17997f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.479+0000 7f73eff83740 1 -- 172.21.15.12:0/3018163399 shutdown_connections 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.479+0000 7f73eff83740 1 -- 172.21.15.12:0/3018163399 wait complete. 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.479+0000 7f73eff83740 1 -- start start 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73eff83740 1 Processor -- start 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73eff83740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179ae20 0x5640b17aee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73eff83740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179f4c0 0x5640b17af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73eff83740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 0x5640b17b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73eff83740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81a1210 con 0x5640b17a5020 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73eff83740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81a0a80 con 0x5640b179ae20 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73eff83740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f73e8071380 con 0x5640b179f4c0 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73eff83740 10 client.0 _ll_register_callbacks cb 0x7f73e0009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73e759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 0x5640b17b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73e659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179f4c0 0x5640b17af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73e759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 0x5640b17b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41282/0 (socket says 172.21.15.12:41282) 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73e759e640 1 -- 172.21.15.12:0/58929061 learned_addr learned my addr 172.21.15.12:0/58929061 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73e659c640 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179f4c0 0x5640b17af390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:38010/0 (socket says 172.21.15.12:38010) 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73e759e640 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179ae20 0x5640b17aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73e659c640 1 -- 172.21.15.12:0/58929061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179ae20 msgr2=0x5640b17aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73e659c640 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179ae20 0x5640b17aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73e659c640 1 -- 172.21.15.12:0/58929061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 msgr2=0x5640b17b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73e659c640 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 0x5640b17b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73e659c640 1 -- 172.21.15.12:0/58929061 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640b17b0940 con 0x5640b179f4c0 2024-05-09T09:28:26.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.480+0000 7f73e759e640 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 0x5640b17b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:26.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.481+0000 7f73e659c640 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179f4c0 0x5640b17af390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f73d002a930 tx=0x7f73d002abc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e17346c72ab9b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:26.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.481+0000 7f73ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:28:26.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.481+0000 7f73ce7fc640 1 -- 172.21.15.12:0/58929061 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0035070 con 0x5640b179f4c0 2024-05-09T09:28:26.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.481+0000 7f73eff83740 1 -- 172.21.15.12:0/58929061 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d0060070 con 0x5640b179f4c0 2024-05-09T09:28:26.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.481+0000 7f73ce7fc640 1 -- 172.21.15.12:0/58929061 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f73d005c550 con 0x5640b179f4c0 2024-05-09T09:28:26.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.482+0000 7f73ce7fc640 1 -- 172.21.15.12:0/58929061 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f73d006c050 con 0x5640b179f4c0 2024-05-09T09:28:26.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.482+0000 7f73eff83740 10 client.26314 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:26.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.482+0000 7f73eff83740 1 -- 172.21.15.12:0/58929061 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5640b1830f90 con 0x5640b179f4c0 2024-05-09T09:28:26.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.482+0000 7f73ce7fc640 1 -- 172.21.15.12:0/58929061 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f73d0060070 con 0x5640b179f4c0 2024-05-09T09:28:26.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.482+0000 7f73ce7fc640 1 -- 172.21.15.12:0/58929061 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f73d0059070 con 0x5640b179f4c0 2024-05-09T09:28:26.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.482+0000 7f73eff83740 10 client.26314 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:26.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.482+0000 7f73eff83740 10 client.26314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:26.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.482+0000 7f73eff83740 20 client.26314 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:26.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.482+0000 7f73eff83740 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5640b18318e0 0x5640b1833d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.482+0000 7f73eff83740 4 client.26314 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:28:26.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.482+0000 7f73eff83740 1 -- 172.21.15.12:0/58929061 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5640b170b410 con 0x5640b18318e0 2024-05-09T09:28:26.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.483+0000 7f73e759e640 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5640b18318e0 0x5640b1833d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.483+0000 7f73ce7fc640 10 client.26314 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:26.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.483+0000 7f73e759e640 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5640b18318e0 0x5640b1833d60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89ba87e9c37a1ed0 server_cookie=43bfe627c8400a02 in_seq=0 out_seq=0 2024-05-09T09:28:26.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:26 smithi012 ceph-mon[30140]: pgmap v454: 129 pgs: 129 active+clean; 146 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 852 KiB/s wr, 360 op/s 2024-05-09T09:28:26.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:26 smithi012 ceph-mon[30140]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.484+0000 7f73ce7fc640 1 -- 172.21.15.12:0/58929061 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5640b170b410 con 0x5640b18318e0 2024-05-09T09:28:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.484+0000 7f73ce7fc640 10 client.26314 handle_command_reply: tid=0 2024-05-09T09:28:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.484+0000 7f73eff83740 1 client.26314 shutdown 2024-05-09T09:28:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.484+0000 7f73eff83740 1 -- 172.21.15.12:0/58929061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179f4c0 msgr2=0x5640b17af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.484+0000 7f73eff83740 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179f4c0 0x5640b17af390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f73d002a930 tx=0x7f73d002abc0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.484+0000 7f73eff83740 1 -- 172.21.15.12:0/58929061 shutdown_connections 2024-05-09T09:28:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.484+0000 7f73eff83740 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5640b18318e0 0x5640b1833d60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73eff83740 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640b17a5020 0x5640b17b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73eff83740 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640b179f4c0 0x5640b17af390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73ce7fc640 0 client.26314 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73eff83740 1 --2- 172.21.15.12:0/58929061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b179ae20 0x5640b17aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73eff83740 1 -- 172.21.15.12:0/58929061 >> 172.21.15.12:0/58929061 conn(0x5640b17670a0 msgr2=0x5640b179a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73eff83740 1 -- 172.21.15.12:0/58929061 shutdown_connections 2024-05-09T09:28:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73eff83740 1 -- 172.21.15.12:0/58929061 wait complete. 2024-05-09T09:28:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73eff83740 20 client.26314 trim_cache size 0 max 16384 2024-05-09T09:28:26.490 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:28:26.490 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T09:28:26.490 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "76ee4393-a523-4a83-84f2-2aab1b9ce837", 2024-05-09T09:28:26.490 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T09:28:26.490 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:28:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73ed19b640 1 -- 172.21.15.12:0/1112767424 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f73d40a2c10 msgr2=0x7f73d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73ed19b640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f73d40a2c10 0x7f73d40a50d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f73d8086cf0 tx=0x7f73d800cf90 comp rx=0 tx=0).stop 2024-05-09T09:28:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73ed19b640 1 -- 172.21.15.12:0/1112767424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73e8100770 msgr2=0x7f73e8197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73ed19b640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73e8100770 0x7f73e8197bd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f73d800ebc0 tx=0x7f73d800c6a0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73ed19b640 1 -- 172.21.15.12:0/1112767424 shutdown_connections 2024-05-09T09:28:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73ed19b640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f73d40a2c10 0x7f73d40a50d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73ed19b640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8102320 0x7f73e819f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73ed19b640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73e8101960 0x7f73e8198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73ed19b640 1 --2- 172.21.15.12:0/1112767424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73e8100770 0x7f73e8197bd0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.485+0000 7f73ed19b640 1 -- 172.21.15.12:0/1112767424 >> 172.21.15.12:0/1112767424 conn(0x7f73e80fbce0 msgr2=0x7f73e80689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.486+0000 7f73ed19b640 1 -- 172.21.15.12:0/1112767424 shutdown_connections 2024-05-09T09:28:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.486+0000 7f73ed19b640 1 -- 172.21.15.12:0/1112767424 wait complete. 2024-05-09T09:28:26.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:26 smithi170 ceph-mon[33632]: pgmap v454: 129 pgs: 129 active+clean; 146 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 852 KiB/s wr, 360 op/s 2024-05-09T09:28:26.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:26 smithi170 ceph-mon[33632]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:26.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:28:26.500 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:28:26.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:26 smithi016 ceph-mon[33335]: pgmap v454: 129 pgs: 129 active+clean; 146 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 852 KiB/s wr, 360 op/s 2024-05-09T09:28:26.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:26 smithi016 ceph-mon[33335]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T09:28:26.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.632+0000 7f19e37e7640 1 -- 172.21.15.12:0/949796763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19dc100770 msgr2=0x7f19dc100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.632+0000 7f19e37e7640 1 --2- 172.21.15.12:0/949796763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19dc100770 0x7f19dc100b90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009900 tx=0x7f19d802a410 comp rx=0 tx=0).stop 2024-05-09T09:28:26.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.632+0000 7f19e37e7640 1 -- 172.21.15.12:0/949796763 shutdown_connections 2024-05-09T09:28:26.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.632+0000 7f19e37e7640 1 --2- 172.21.15.12:0/949796763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19dc102320 0x7f19dc108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.632+0000 7f19e37e7640 1 --2- 172.21.15.12:0/949796763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19dc101960 0x7f19dc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.632+0000 7f19e37e7640 1 --2- 172.21.15.12:0/949796763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19dc100770 0x7f19dc100b90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.632+0000 7f19e37e7640 1 -- 172.21.15.12:0/949796763 >> 172.21.15.12:0/949796763 conn(0x7f19dc0fbce0 msgr2=0x7f19dc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:26.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.632+0000 7f19e37e7640 1 -- 172.21.15.12:0/949796763 shutdown_connections 2024-05-09T09:28:26.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.633+0000 7f19e37e7640 1 -- 172.21.15.12:0/949796763 wait complete. 2024-05-09T09:28:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.633+0000 7f19e37e7640 1 Processor -- start 2024-05-09T09:28:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.633+0000 7f19e37e7640 1 -- start start 2024-05-09T09:28:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.633+0000 7f19e37e7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19dc100770 0x7f19dc197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.633+0000 7f19e37e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19dc101960 0x7f19dc197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.633+0000 7f19e37e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19dc102320 0x7f19dc19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.633+0000 7f19e37e7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc10d500 con 0x7f19dc100770 2024-05-09T09:28:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.633+0000 7f19e37e7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc10d380 con 0x7f19dc102320 2024-05-09T09:28:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.633+0000 7f19e37e7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc10d680 con 0x7f19dc101960 2024-05-09T09:28:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19e1583640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19dc100770 0x7f19dc197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19e0d82640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19dc101960 0x7f19dc197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19e1583640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19dc100770 0x7f19dc197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41302/0 (socket says 172.21.15.12:41302) 2024-05-09T09:28:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19e1583640 1 -- 172.21.15.12:0/700008190 learned_addr learned my addr 172.21.15.12:0/700008190 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19e1d84640 1 --2- 172.21.15.12:0/700008190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19dc102320 0x7f19dc19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19e1583640 1 -- 172.21.15.12:0/700008190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19dc101960 msgr2=0x7f19dc197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19e1583640 1 --2- 172.21.15.12:0/700008190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19dc101960 0x7f19dc197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19e1583640 1 -- 172.21.15.12:0/700008190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19dc102320 msgr2=0x7f19dc19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19e1583640 1 --2- 172.21.15.12:0/700008190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19dc102320 0x7f19dc19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19e1583640 1 -- 172.21.15.12:0/700008190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc19f620 con 0x7f19dc100770 2024-05-09T09:28:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19e1583640 1 --2- 172.21.15.12:0/700008190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19dc100770 0x7f19dc197a00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009850 tx=0x7f19d80052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19d27fc640 1 -- 172.21.15.12:0/700008190 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d803f070 con 0x7f19dc100770 2024-05-09T09:28:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19d27fc640 1 -- 172.21.15.12:0/700008190 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f19d8003cc0 con 0x7f19dc100770 2024-05-09T09:28:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19e37e7640 1 -- 172.21.15.12:0/700008190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19dc19f8b0 con 0x7f19dc100770 2024-05-09T09:28:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19d27fc640 1 -- 172.21.15.12:0/700008190 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8035030 con 0x7f19dc100770 2024-05-09T09:28:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.634+0000 7f19e37e7640 1 -- 172.21.15.12:0/700008190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19dc19fda0 con 0x7f19dc100770 2024-05-09T09:28:26.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.638+0000 7f19e65d5740 1 Processor -- start 2024-05-09T09:28:26.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.638+0000 7f19e65d5740 1 -- start start 2024-05-09T09:28:26.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e65d5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed98d90 0x564d4ed991b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e65d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed99780 0x564d4ed9dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e65d5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4eda0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e65d5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564d4ecf4d00 con 0x564d4ed9e2d0 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e65d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564d4ec9c4c0 con 0x564d4ed99780 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19d27fc640 1 -- 172.21.15.12:0/700008190 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19d8042070 con 0x7f19dc100770 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e65d5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564d4ecf0cc0 con 0x564d4ed98d90 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e1583640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed99780 0x564d4ed9dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e0d82640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4eda0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e1d84640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed98d90 0x564d4ed991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e0d82640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4eda0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41306/0 (socket says 172.21.15.12:41306) 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e1d84640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed98d90 0x564d4ed991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:38028/0 (socket says 172.21.15.12:38028) 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e1583640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed99780 0x564d4ed9dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:43396/0 (socket says 172.21.15.12:43396) 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e1d84640 1 -- 172.21.15.12:0/3166793810 learned_addr learned my addr 172.21.15.12:0/3166793810 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e1d84640 1 -- 172.21.15.12:0/3166793810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed99780 msgr2=0x564d4ed9dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e1d84640 1 --2- 172.21.15.12:0/3166793810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed99780 0x564d4ed9dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e1d84640 1 -- 172.21.15.12:0/3166793810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 msgr2=0x564d4eda0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e1d84640 1 --2- 172.21.15.12:0/3166793810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4eda0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e1d84640 1 -- 172.21.15.12:0/3166793810 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4ead27d0 con 0x564d4ed98d90 2024-05-09T09:28:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.639+0000 7f19e0d82640 1 --2- 172.21.15.12:0/3166793810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4eda0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:28:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.640+0000 7f19e1d84640 1 --2- 172.21.15.12:0/3166793810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed98d90 0x564d4ed991b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f19d4011440 tx=0x7f19d4011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5ada0dd1d85028d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:26.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.640+0000 7f19c77fe640 1 -- 172.21.15.12:0/3166793810 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4003c30 con 0x564d4ed98d90 2024-05-09T09:28:26.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.640+0000 7f19c77fe640 1 -- 172.21.15.12:0/3166793810 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f19d4003dd0 con 0x564d4ed98d90 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.640+0000 7f19c77fe640 1 -- 172.21.15.12:0/3166793810 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d400fab0 con 0x564d4ed98d90 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.640+0000 7f19d27fc640 1 --2- 172.21.15.12:0/700008190 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f19c80a2b90 0x7f19c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.640+0000 7f19d27fc640 1 -- 172.21.15.12:0/700008190 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f19d80efe60 con 0x7f19dc100770 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.641+0000 7f19e65d5740 1 -- 172.21.15.12:0/3166793810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed98d90 msgr2=0x564d4ed991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.641+0000 7f19e65d5740 1 --2- 172.21.15.12:0/3166793810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed98d90 0x564d4ed991b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f19d4011440 tx=0x7f19d4011900 comp rx=0 tx=0).stop 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.641+0000 7f19e65d5740 1 -- 172.21.15.12:0/3166793810 shutdown_connections 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.641+0000 7f19e65d5740 1 --2- 172.21.15.12:0/3166793810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4eda0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.641+0000 7f19e65d5740 1 --2- 172.21.15.12:0/3166793810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed99780 0x564d4ed9dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.641+0000 7f19e1583640 1 --2- 172.21.15.12:0/700008190 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f19c80a2b90 0x7f19c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.641+0000 7f19e65d5740 1 --2- 172.21.15.12:0/3166793810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed98d90 0x564d4ed991b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.641+0000 7f19e65d5740 1 -- 172.21.15.12:0/3166793810 >> 172.21.15.12:0/3166793810 conn(0x564d4ed8b520 msgr2=0x564d4ed8b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.641+0000 7f19e65d5740 1 -- 172.21.15.12:0/3166793810 shutdown_connections 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.642+0000 7f19e1583640 1 --2- 172.21.15.12:0/700008190 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f19c80a2b90 0x7f19c80a5050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x564d4ed8f2d0 tx=0x7f19d8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.642+0000 7f19e65d5740 1 -- 172.21.15.12:0/3166793810 wait complete. 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.642+0000 7f19e65d5740 1 -- start start 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.642+0000 7f19e65d5740 1 Processor -- start 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.642+0000 7f19e65d5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed98d90 0x564d4eda29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.642+0000 7f19e65d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed99780 0x564d4eda2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e65d5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4eda9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e1d84640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed98d90 0x564d4eda29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e1d84640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed98d90 0x564d4eda29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:38040/0 (socket says 172.21.15.12:38040) 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e1d84640 1 -- 172.21.15.12:0/2167575931 learned_addr learned my addr 172.21.15.12:0/2167575931 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e0d82640 1 --2- 172.21.15.12:0/2167575931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed99780 0x564d4eda2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e65d5740 1 -- 172.21.15.12:0/2167575931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564d4eda1240 con 0x564d4ed9e2d0 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e65d5740 1 -- 172.21.15.12:0/2167575931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564d4ecafb90 con 0x564d4ed99780 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e65d5740 1 -- 172.21.15.12:0/2167575931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564d4eda13c0 con 0x564d4ed98d90 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e65d5740 10 client.0 _ll_register_callbacks cb 0x564d4ecebda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:26.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e1d84640 1 --2- 172.21.15.12:0/2167575931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4eda9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e0d82640 1 -- 172.21.15.12:0/2167575931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed98d90 msgr2=0x564d4eda29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e0d82640 1 --2- 172.21.15.12:0/2167575931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed98d90 0x564d4eda29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e0d82640 1 -- 172.21.15.12:0/2167575931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 msgr2=0x564d4eda9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e0d82640 1 --2- 172.21.15.12:0/2167575931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4eda9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e0d82640 1 -- 172.21.15.12:0/2167575931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4ea083f0 con 0x564d4ed99780 2024-05-09T09:28:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19e0d82640 1 --2- 172.21.15.12:0/2167575931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed99780 0x564d4eda2ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f19dc198f10 tx=0x7f19cc004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3316d206f21302ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:28:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.643+0000 7f19c4ff9640 1 -- 172.21.15.12:0/2167575931 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc01b440 con 0x564d4ed99780 2024-05-09T09:28:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.644+0000 7f19c4ff9640 1 -- 172.21.15.12:0/2167575931 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f19cc01b5e0 con 0x564d4ed99780 2024-05-09T09:28:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.644+0000 7f19e65d5740 1 -- 172.21.15.12:0/2167575931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19d4003dd0 con 0x564d4ed99780 2024-05-09T09:28:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.644+0000 7f19c4ff9640 1 -- 172.21.15.12:0/2167575931 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f19cc024050 con 0x564d4ed99780 2024-05-09T09:28:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.644+0000 7f19e65d5740 10 client.26216 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:26.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.644+0000 7f19e65d5740 1 -- 172.21.15.12:0/2167575931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564d4e9c80f0 con 0x564d4ed99780 2024-05-09T09:28:26.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.644+0000 7f19c4ff9640 1 -- 172.21.15.12:0/2167575931 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f19d4003dd0 con 0x564d4ed99780 2024-05-09T09:28:26.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.644+0000 7f19c4ff9640 1 -- 172.21.15.12:0/2167575931 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f19cc019020 con 0x564d4ed99780 2024-05-09T09:28:26.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.644+0000 7f19e65d5740 10 client.26216 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:26.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.644+0000 7f19e65d5740 10 client.26216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:26.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.644+0000 7f19e65d5740 20 client.26216 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:26.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.644+0000 7f19e65d5740 1 --2- 172.21.15.12:0/2167575931 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564d4edae830 0x564d4edb0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.644+0000 7f19e65d5740 4 client.26216 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:26.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.644+0000 7f19e65d5740 1 -- 172.21.15.12:0/2167575931 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d4ed02430 con 0x564d4edae830 2024-05-09T09:28:26.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.645+0000 7f19e1d84640 1 --2- 172.21.15.12:0/2167575931 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564d4edae830 0x564d4edb0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.645+0000 7f19c4ff9640 10 client.26216 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:26.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.645+0000 7f19e1d84640 1 --2- 172.21.15.12:0/2167575931 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564d4edae830 0x564d4edb0c50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=442693a270b420fa server_cookie=27299fa39eb97b3b in_seq=0 out_seq=0 2024-05-09T09:28:26.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.646+0000 7f19c4ff9640 1 -- 172.21.15.12:0/2167575931 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d4ed02430 con 0x564d4edae830 2024-05-09T09:28:26.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.646+0000 7f19c4ff9640 10 client.26216 handle_command_reply: tid=0 2024-05-09T09:28:26.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.646+0000 7f19e65d5740 1 client.26216 shutdown 2024-05-09T09:28:26.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.646+0000 7f19e65d5740 1 -- 172.21.15.12:0/2167575931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed99780 msgr2=0x564d4eda2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.646+0000 7f19e65d5740 1 --2- 172.21.15.12:0/2167575931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed99780 0x564d4eda2ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f19dc198f10 tx=0x7f19cc004610 comp rx=0 tx=0).stop 2024-05-09T09:28:26.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.646+0000 7f19e65d5740 1 -- 172.21.15.12:0/2167575931 shutdown_connections 2024-05-09T09:28:26.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.646+0000 7f19e65d5740 1 --2- 172.21.15.12:0/2167575931 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564d4edae830 0x564d4edb0c50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.646+0000 7f19e65d5740 1 --2- 172.21.15.12:0/2167575931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4eda9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.646+0000 7f19e65d5740 1 --2- 172.21.15.12:0/2167575931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed99780 0x564d4eda2ef0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.646+0000 7f19e65d5740 1 --2- 172.21.15.12:0/2167575931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed98d90 0x564d4eda29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.646+0000 7f19e65d5740 1 -- 172.21.15.12:0/2167575931 >> 172.21.15.12:0/2167575931 conn(0x564d4ed8b520 msgr2=0x564d4ed98420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:26.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.646+0000 7f19c4ff9640 0 client.26216 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:26.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.647+0000 7f19e65d5740 1 -- 172.21.15.12:0/2167575931 shutdown_connections 2024-05-09T09:28:26.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.647+0000 7f19e65d5740 1 -- 172.21.15.12:0/2167575931 wait complete. 2024-05-09T09:28:26.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.647+0000 7f19e65d5740 20 client.26216 trim_cache size 0 max 16384 2024-05-09T09:28:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e65d5740 1 Processor -- start 2024-05-09T09:28:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e65d5740 1 -- start start 2024-05-09T09:28:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e65d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed98d90 0x564d4ed9d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e65d5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed99780 0x564d4eda9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e65d5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4ed9a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e65d5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1a8290 con 0x564d4ed9e2d0 2024-05-09T09:28:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e65d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1aa010 con 0x564d4ed98d90 2024-05-09T09:28:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e65d5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1a8a20 con 0x564d4ed99780 2024-05-09T09:28:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e1d84640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4ed9a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e1d84640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4ed9a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41316/0 (socket says 172.21.15.12:41316) 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e1d84640 1 -- 172.21.15.12:0/363785653 learned_addr learned my addr 172.21.15.12:0/363785653 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e1d84640 1 --2- 172.21.15.12:0/363785653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed98d90 0x564d4ed9d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e0d82640 1 --2- 172.21.15.12:0/363785653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed99780 0x564d4eda9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e1d84640 1 -- 172.21.15.12:0/363785653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed99780 msgr2=0x564d4eda9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e1d84640 1 --2- 172.21.15.12:0/363785653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed99780 0x564d4eda9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e1d84640 1 -- 172.21.15.12:0/363785653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed98d90 msgr2=0x564d4ed9d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e1d84640 1 --2- 172.21.15.12:0/363785653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed98d90 0x564d4ed9d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e1d84640 1 -- 172.21.15.12:0/363785653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4ed99e90 con 0x564d4ed9e2d0 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.668+0000 7f19e1d84640 1 --2- 172.21.15.12:0/363785653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4ed9a3a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f19d4002230 tx=0x7f19d4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e557220a619eb245 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.669+0000 7f19c6ffd640 1 -- 172.21.15.12:0/363785653 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4030c10 con 0x564d4ed9e2d0 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.669+0000 7f19c6ffd640 1 -- 172.21.15.12:0/363785653 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f19d4030db0 con 0x564d4ed9e2d0 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.669+0000 7f19c6ffd640 1 -- 172.21.15.12:0/363785653 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d402f8e0 con 0x564d4ed9e2d0 2024-05-09T09:28:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.669+0000 7f19e65d5740 1 -- 172.21.15.12:0/363785653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 msgr2=0x564d4ed9a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.669+0000 7f19e65d5740 1 --2- 172.21.15.12:0/363785653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4ed9a3a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f19d4002230 tx=0x7f19d4005fd0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.669+0000 7f19e65d5740 1 -- 172.21.15.12:0/363785653 shutdown_connections 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.669+0000 7f19e65d5740 1 --2- 172.21.15.12:0/363785653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed9e2d0 0x564d4ed9a3a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.669+0000 7f19e65d5740 1 --2- 172.21.15.12:0/363785653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed99780 0x564d4eda9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.669+0000 7f19e65d5740 1 --2- 172.21.15.12:0/363785653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed98d90 0x564d4ed9d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.669+0000 7f19e65d5740 1 -- 172.21.15.12:0/363785653 >> 172.21.15.12:0/363785653 conn(0x564d4ed8b520 msgr2=0x564d4eda29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.670+0000 7f19e65d5740 1 -- 172.21.15.12:0/363785653 shutdown_connections 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.670+0000 7f19e65d5740 1 -- 172.21.15.12:0/363785653 wait complete. 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.670+0000 7f19e65d5740 1 -- start start 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.670+0000 7f19e65d5740 1 Processor -- start 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.670+0000 7f19e65d5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed98d90 0x564d4eda71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.670+0000 7f19e65d5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed99780 0x564d4eda7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.670+0000 7f19e65d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed9e2d0 0x564d4eda8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.670+0000 7f19e65d5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1a9940 con 0x564d4ed98d90 2024-05-09T09:28:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.670+0000 7f19e65d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1a91b0 con 0x564d4ed9e2d0 2024-05-09T09:28:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.670+0000 7f19e65d5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1a6270 con 0x564d4ed99780 2024-05-09T09:28:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.670+0000 7f19e65d5740 10 client.0 _ll_register_callbacks cb 0x7f19cc005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.670+0000 7f19e1d84640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed98d90 0x564d4eda71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19e0d82640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed99780 0x564d4eda7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19e1d84640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed98d90 0x564d4eda71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41322/0 (socket says 172.21.15.12:41322) 2024-05-09T09:28:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19e1d84640 1 -- 172.21.15.12:0/2075588149 learned_addr learned my addr 172.21.15.12:0/2075588149 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19e1d84640 1 --2- 172.21.15.12:0/2075588149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed9e2d0 0x564d4eda8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19e0d82640 1 -- 172.21.15.12:0/2075588149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed9e2d0 msgr2=0x564d4eda8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19e0d82640 1 --2- 172.21.15.12:0/2075588149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed9e2d0 0x564d4eda8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19e0d82640 1 -- 172.21.15.12:0/2075588149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed98d90 msgr2=0x564d4eda71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19e0d82640 1 --2- 172.21.15.12:0/2075588149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed98d90 0x564d4eda71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19e0d82640 1 -- 172.21.15.12:0/2075588149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4eda8d00 con 0x564d4ed99780 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19e1d84640 1 --2- 172.21.15.12:0/2075588149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed9e2d0 0x564d4eda8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19e0d82640 1 --2- 172.21.15.12:0/2075588149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed99780 0x564d4eda7700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f19cc011c70 tx=0x7f19cc0058c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffd4d55ddcb89fba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19c4ff9640 1 -- 172.21.15.12:0/2075588149 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc021040 con 0x564d4ed99780 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19e65d5740 1 -- 172.21.15.12:0/2075588149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19d4030db0 con 0x564d4ed99780 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.671+0000 7f19c4ff9640 1 -- 172.21.15.12:0/2075588149 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f19cc011850 con 0x564d4ed99780 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.672+0000 7f19c4ff9640 1 -- 172.21.15.12:0/2075588149 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f19cc039050 con 0x564d4ed99780 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.672+0000 7f19e65d5740 10 client.26332 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.672+0000 7f19e65d5740 1 -- 172.21.15.12:0/2075588149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564d4edaa080 con 0x564d4ed99780 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.672+0000 7f19c4ff9640 1 -- 172.21.15.12:0/2075588149 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f19d4030db0 con 0x564d4ed99780 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.673+0000 7f19c4ff9640 1 -- 172.21.15.12:0/2075588149 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f19cc019040 con 0x564d4ed99780 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.673+0000 7f19e65d5740 10 client.26332 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.673+0000 7f19e65d5740 10 client.26332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.673+0000 7f19e65d5740 20 client.26332 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.673+0000 7f19e65d5740 1 --2- 172.21.15.12:0/2075588149 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564d4ee15420 0x564d4ee17840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.673+0000 7f19e65d5740 4 client.26332 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.673+0000 7f19e65d5740 1 -- 172.21.15.12:0/2075588149 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564d4ecfc780 con 0x564d4ee15420 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.673+0000 7f19e1d84640 1 --2- 172.21.15.12:0/2075588149 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564d4ee15420 0x564d4ee17840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.673+0000 7f19c4ff9640 10 client.26332 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.673+0000 7f19e1d84640 1 --2- 172.21.15.12:0/2075588149 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564d4ee15420 0x564d4ee17840 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4885da6775b1298 server_cookie=55b9c28dd6424594 in_seq=0 out_seq=0 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.673+0000 7f19c4ff9640 1 -- 172.21.15.12:0/2075588149 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564d4ecfc780 con 0x564d4ee15420 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.673+0000 7f19c4ff9640 10 client.26332 handle_command_reply: tid=0 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.674+0000 7f19e65d5740 1 client.26332 shutdown 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.674+0000 7f19e65d5740 1 -- 172.21.15.12:0/2075588149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed99780 msgr2=0x564d4eda7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.674+0000 7f19e65d5740 1 --2- 172.21.15.12:0/2075588149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed99780 0x564d4eda7700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f19cc011c70 tx=0x7f19cc0058c0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.674+0000 7f19e65d5740 1 -- 172.21.15.12:0/2075588149 shutdown_connections 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.674+0000 7f19e65d5740 1 --2- 172.21.15.12:0/2075588149 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x564d4ee15420 0x564d4ee17840 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.674+0000 7f19e65d5740 1 --2- 172.21.15.12:0/2075588149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d4ed9e2d0 0x564d4eda8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.674+0000 7f19e65d5740 1 --2- 172.21.15.12:0/2075588149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d4ed99780 0x564d4eda7700 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.674+0000 7f19e65d5740 1 --2- 172.21.15.12:0/2075588149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d4ed98d90 0x564d4eda71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.674+0000 7f19c4ff9640 0 client.26332 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:26.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.674+0000 7f19e65d5740 1 -- 172.21.15.12:0/2075588149 >> 172.21.15.12:0/2075588149 conn(0x564d4ed8b520 msgr2=0x564d4ed9a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:26.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.674+0000 7f19e65d5740 1 -- 172.21.15.12:0/2075588149 shutdown_connections 2024-05-09T09:28:26.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.674+0000 7f19e65d5740 1 -- 172.21.15.12:0/2075588149 wait complete. 2024-05-09T09:28:26.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.674+0000 7f19e65d5740 20 client.26332 trim_cache size 0 max 16384 2024-05-09T09:28:26.682 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:28:26.682 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:28:26.682 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T09:28:26.683 INFO:teuthology.orchestra.run.smithi012.stdout: "76ee4393-a523-4a83-84f2-2aab1b9ce837": { 2024-05-09T09:28:26.683 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T09:28:26.683 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "76ee4393-a523-4a83-84f2-2aab1b9ce837", 2024-05-09T09:28:26.683 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T09:28:26.683 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:28:26.683 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:28:26.683 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:28:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.675+0000 7f19e37e7640 1 -- 172.21.15.12:0/700008190 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f19c80a2b90 msgr2=0x7f19c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.675+0000 7f19e37e7640 1 --2- 172.21.15.12:0/700008190 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f19c80a2b90 0x7f19c80a5050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x564d4ed8f2d0 tx=0x7f19d8005fd0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.675+0000 7f19e37e7640 1 -- 172.21.15.12:0/700008190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19dc100770 msgr2=0x7f19dc197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.675+0000 7f19e37e7640 1 --2- 172.21.15.12:0/700008190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19dc100770 0x7f19dc197a00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009850 tx=0x7f19d80052d0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.675+0000 7f19e37e7640 1 -- 172.21.15.12:0/700008190 shutdown_connections 2024-05-09T09:28:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.675+0000 7f19e37e7640 1 --2- 172.21.15.12:0/700008190 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f19c80a2b90 0x7f19c80a5050 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.675+0000 7f19e37e7640 1 --2- 172.21.15.12:0/700008190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19dc102320 0x7f19dc19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.675+0000 7f19e37e7640 1 --2- 172.21.15.12:0/700008190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19dc101960 0x7f19dc197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.675+0000 7f19e37e7640 1 --2- 172.21.15.12:0/700008190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19dc100770 0x7f19dc197a00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.675+0000 7f19e37e7640 1 -- 172.21.15.12:0/700008190 >> 172.21.15.12:0/700008190 conn(0x7f19dc0fbce0 msgr2=0x7f19dc104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.675+0000 7f19e37e7640 1 -- 172.21.15.12:0/700008190 shutdown_connections 2024-05-09T09:28:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:26.675+0000 7f19e37e7640 1 -- 172.21.15.12:0/700008190 wait complete. 2024-05-09T09:28:26.688 INFO:tasks.cephfs.filesystem:scrub status for tag:76ee4393-a523-4a83-84f2-2aab1b9ce837 - {'path': '/', 'tag': '76ee4393-a523-4a83-84f2-2aab1b9ce837', 'options': 'recursive,force'} 2024-05-09T09:28:28.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:28 smithi012 ceph-mon[30140]: scrub queued for path: / 2024-05-09T09:28:28.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:28 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:28:28.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:28 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:28:28.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:28 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:28:28.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:28 smithi012 ceph-mon[30140]: pgmap v455: 129 pgs: 129 active+clean; 144 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 987 KiB/s wr, 352 op/s 2024-05-09T09:28:28.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:28 smithi170 ceph-mon[33632]: scrub queued for path: / 2024-05-09T09:28:28.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:28 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:28:28.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:28 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:28:28.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:28 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:28:28.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:28 smithi170 ceph-mon[33632]: pgmap v455: 129 pgs: 129 active+clean; 144 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 987 KiB/s wr, 352 op/s 2024-05-09T09:28:28.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:28 smithi016 ceph-mon[33335]: scrub queued for path: / 2024-05-09T09:28:28.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:28 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:28:28.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:28 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:28:28.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:28 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:28:28.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:28 smithi016 ceph-mon[33335]: pgmap v455: 129 pgs: 129 active+clean; 144 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 987 KiB/s wr, 352 op/s 2024-05-09T09:28:30.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:30 smithi012 ceph-mon[30140]: pgmap v456: 129 pgs: 129 active+clean; 143 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1010 KiB/s wr, 329 op/s 2024-05-09T09:28:30.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:30 smithi170 ceph-mon[33632]: pgmap v456: 129 pgs: 129 active+clean; 143 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1010 KiB/s wr, 329 op/s 2024-05-09T09:28:30.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:30 smithi016 ceph-mon[33335]: pgmap v456: 129 pgs: 129 active+clean; 143 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1010 KiB/s wr, 329 op/s 2024-05-09T09:28:31.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:31 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:28:31.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:31 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:28:31.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:31 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:28:32.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:32 smithi012 ceph-mon[30140]: pgmap v457: 129 pgs: 129 active+clean; 143 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 986 KiB/s wr, 380 op/s 2024-05-09T09:28:32.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:32 smithi170 ceph-mon[33632]: pgmap v457: 129 pgs: 129 active+clean; 143 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 986 KiB/s wr, 380 op/s 2024-05-09T09:28:32.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:32 smithi016 ceph-mon[33335]: pgmap v457: 129 pgs: 129 active+clean; 143 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 986 KiB/s wr, 380 op/s 2024-05-09T09:28:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.486+0000 7f0517c9e640 1 -- 172.21.15.12:0/3766149728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0510101980 msgr2=0x7f0510101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.486+0000 7f0517c9e640 1 --2- 172.21.15.12:0/3766149728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0510101980 0x7f0510101e00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0500009900 tx=0x7f050002a410 comp rx=0 tx=0).stop 2024-05-09T09:28:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.486+0000 7f0517c9e640 1 -- 172.21.15.12:0/3766149728 shutdown_connections 2024-05-09T09:28:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.486+0000 7f0517c9e640 1 --2- 172.21.15.12:0/3766149728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102340 0x7f0510108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.486+0000 7f0517c9e640 1 --2- 172.21.15.12:0/3766149728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0510101980 0x7f0510101e00 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.486+0000 7f0517c9e640 1 --2- 172.21.15.12:0/3766149728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0510100790 0x7f0510100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.486+0000 7f0517c9e640 1 -- 172.21.15.12:0/3766149728 >> 172.21.15.12:0/3766149728 conn(0x7f05100fbd00 msgr2=0x7f05100fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.486+0000 7f0517c9e640 1 -- 172.21.15.12:0/3766149728 shutdown_connections 2024-05-09T09:28:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.486+0000 7f0517c9e640 1 -- 172.21.15.12:0/3766149728 wait complete. 2024-05-09T09:28:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.487+0000 7f0517c9e640 1 Processor -- start 2024-05-09T09:28:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.487+0000 7f0517c9e640 1 -- start start 2024-05-09T09:28:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.487+0000 7f0517c9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0510100790 0x7f0510199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.487+0000 7f0517c9e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0510101980 0x7f051019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.487+0000 7f0517c9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102340 0x7f05101a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.487+0000 7f0517c9e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f051010b330 con 0x7f0510101980 2024-05-09T09:28:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.487+0000 7f0517c9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f051010b1b0 con 0x7f0510102340 2024-05-09T09:28:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.487+0000 7f0517c9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f051010b4b0 con 0x7f0510100790 2024-05-09T09:28:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f0515239640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0510101980 0x7f051019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f0515239640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0510101980 0x7f051019a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35018/0 (socket says 172.21.15.12:35018) 2024-05-09T09:28:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f0515239640 1 -- 172.21.15.12:0/748437156 learned_addr learned my addr 172.21.15.12:0/748437156 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f051623b640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102340 0x7f05101a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f0515a3a640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0510100790 0x7f0510199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f051623b640 1 -- 172.21.15.12:0/748437156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0510100790 msgr2=0x7f0510199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f051623b640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0510100790 0x7f0510199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f051623b640 1 -- 172.21.15.12:0/748437156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0510101980 msgr2=0x7f051019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f051623b640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0510101980 0x7f051019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f051623b640 1 -- 172.21.15.12:0/748437156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05101a1960 con 0x7f0510102340 2024-05-09T09:28:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f051623b640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102340 0x7f05101a12f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0508009fd0 tx=0x7f050800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f0506ffd640 1 -- 172.21.15.12:0/748437156 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050800ce20 con 0x7f0510102340 2024-05-09T09:28:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f0506ffd640 1 -- 172.21.15.12:0/748437156 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0508005020 con 0x7f0510102340 2024-05-09T09:28:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f0517c9e640 1 -- 172.21.15.12:0/748437156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05101a1c50 con 0x7f0510102340 2024-05-09T09:28:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f0506ffd640 1 -- 172.21.15.12:0/748437156 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05080106b0 con 0x7f0510102340 2024-05-09T09:28:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.488+0000 7f0517c9e640 1 -- 172.21.15.12:0/748437156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05101a2150 con 0x7f0510102340 2024-05-09T09:28:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.491+0000 7f0506ffd640 1 -- 172.21.15.12:0/748437156 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f05080051d0 con 0x7f0510102340 2024-05-09T09:28:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.492+0000 7f051a9ae740 1 Processor -- start 2024-05-09T09:28:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.492+0000 7f051a9ae740 1 -- start start 2024-05-09T09:28:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.492+0000 7f051a9ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2a250 0x556aa8e2a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.492+0000 7f051a9ae740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 0x556aa8e2f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.492+0000 7f051a9ae740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2fdb0 0x556aa8e32240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.492+0000 7f051a9ae740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556aa8d8e340 con 0x556aa8e2ac40 2024-05-09T09:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.492+0000 7f051a9ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556aa8d36290 con 0x556aa8e2a250 2024-05-09T09:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f051a9ae740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556aa8d8a300 con 0x556aa8e2fdb0 2024-05-09T09:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f0515a3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 0x556aa8e2f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f0515239640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2a250 0x556aa8e2a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f0515a3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 0x556aa8e2f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35032/0 (socket says 172.21.15.12:35032) 2024-05-09T09:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f0515a3a640 1 -- 172.21.15.12:0/1296305954 learned_addr learned my addr 172.21.15.12:0/1296305954 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f0515239640 1 --2- 172.21.15.12:0/1296305954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2fdb0 0x556aa8e32240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f0506ffd640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f04fc0a2c30 0x7f04fc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f0515a3a640 1 -- 172.21.15.12:0/1296305954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2fdb0 msgr2=0x556aa8e32240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f0515a3a640 1 --2- 172.21.15.12:0/1296305954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2fdb0 0x556aa8e32240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f0515a3a640 1 -- 172.21.15.12:0/1296305954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2a250 msgr2=0x556aa8e2a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f0515a3a640 1 --2- 172.21.15.12:0/1296305954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2a250 0x556aa8e2a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f0515a3a640 1 -- 172.21.15.12:0/1296305954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa8d5a8e0 con 0x556aa8e2ac40 2024-05-09T09:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f051623b640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f04fc0a2c30 0x7f04fc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f0506ffd640 1 -- 172.21.15.12:0/748437156 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f05080cfbc0 con 0x7f0510102340 2024-05-09T09:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f0515a3a640 1 --2- 172.21.15.12:0/1296305954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 0x556aa8e2f740 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f050c009ca0 tx=0x7f050c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d86acb2d36c587 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f04fbfff640 1 -- 172.21.15.12:0/1296305954 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c009e50 con 0x556aa8e2ac40 2024-05-09T09:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f04fbfff640 1 -- 172.21.15.12:0/1296305954 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f050c0035f0 con 0x556aa8e2ac40 2024-05-09T09:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.493+0000 7f04fbfff640 1 -- 172.21.15.12:0/1296305954 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c019650 con 0x556aa8e2ac40 2024-05-09T09:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.494+0000 7f051a9ae740 1 -- 172.21.15.12:0/1296305954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 msgr2=0x556aa8e2f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.494+0000 7f051623b640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f04fc0a2c30 0x7f04fc0a50f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f050809bfe0 tx=0x7f050806f110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.494+0000 7f051a9ae740 1 --2- 172.21.15.12:0/1296305954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 0x556aa8e2f740 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f050c009ca0 tx=0x7f050c011810 comp rx=0 tx=0).stop 2024-05-09T09:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.494+0000 7f051a9ae740 1 -- 172.21.15.12:0/1296305954 shutdown_connections 2024-05-09T09:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.494+0000 7f051a9ae740 1 --2- 172.21.15.12:0/1296305954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2fdb0 0x556aa8e32240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.494+0000 7f051a9ae740 1 --2- 172.21.15.12:0/1296305954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 0x556aa8e2f740 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.494+0000 7f051a9ae740 1 --2- 172.21.15.12:0/1296305954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2a250 0x556aa8e2a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.494+0000 7f051a9ae740 1 -- 172.21.15.12:0/1296305954 >> 172.21.15.12:0/1296305954 conn(0x556aa8e258c0 msgr2=0x556aa8e25cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.494+0000 7f051a9ae740 1 -- 172.21.15.12:0/1296305954 shutdown_connections 2024-05-09T09:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.494+0000 7f051a9ae740 1 -- 172.21.15.12:0/1296305954 wait complete. 2024-05-09T09:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f051a9ae740 1 -- start start 2024-05-09T09:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f051a9ae740 1 Processor -- start 2024-05-09T09:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f051a9ae740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2a250 0x556aa8e34410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f051a9ae740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 0x556aa8e34950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f051a9ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f051a9ae740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556aa8e32d60 con 0x556aa8e2ac40 2024-05-09T09:28:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f051a9ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556aa8d49790 con 0x556aa8e2fdb0 2024-05-09T09:28:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f051a9ae740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556aa8e32ee0 con 0x556aa8e2a250 2024-05-09T09:28:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f051a9ae740 10 client.0 _ll_register_callbacks cb 0x556aa8d84a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f0515a3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 0x556aa8e34950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f0515239640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2a250 0x556aa8e34410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f0515a3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 0x556aa8e34950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35036/0 (socket says 172.21.15.12:35036) 2024-05-09T09:28:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f0515a3a640 1 -- 172.21.15.12:0/3750092006 learned_addr learned my addr 172.21.15.12:0/3750092006 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.495+0000 7f0515239640 1 --2- 172.21.15.12:0/3750092006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f0515a3a640 1 -- 172.21.15.12:0/3750092006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2a250 msgr2=0x556aa8e34410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f0515a3a640 1 --2- 172.21.15.12:0/3750092006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2a250 0x556aa8e34410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f0515a3a640 1 -- 172.21.15.12:0/3750092006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2fdb0 msgr2=0x556aa8e3b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f0515a3a640 1 --2- 172.21.15.12:0/3750092006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f0515a3a640 1 -- 172.21.15.12:0/3750092006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa8aa2210 con 0x556aa8e2ac40 2024-05-09T09:28:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f0515a3a640 1 --2- 172.21.15.12:0/3750092006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 0x556aa8e34950 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f050c01b040 tx=0x7f050c0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cadabe3c961f47 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f04f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f04f97fa640 1 -- 172.21.15.12:0/3750092006 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c028020 con 0x556aa8e2ac40 2024-05-09T09:28:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f04f97fa640 1 -- 172.21.15.12:0/3750092006 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f050c016040 con 0x556aa8e2ac40 2024-05-09T09:28:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f051a9ae740 1 -- 172.21.15.12:0/3750092006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f050c0035f0 con 0x556aa8e2ac40 2024-05-09T09:28:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f04f97fa640 1 -- 172.21.15.12:0/3750092006 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f050c033050 con 0x556aa8e2ac40 2024-05-09T09:28:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f051a9ae740 10 client.16797 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f051a9ae740 1 -- 172.21.15.12:0/3750092006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556aa8b6e620 con 0x556aa8e2ac40 2024-05-09T09:28:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f04f97fa640 1 -- 172.21.15.12:0/3750092006 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f050c0035f0 con 0x556aa8e2ac40 2024-05-09T09:28:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.496+0000 7f04f97fa640 1 -- 172.21.15.12:0/3750092006 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f050c0049c0 con 0x556aa8e2ac40 2024-05-09T09:28:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.497+0000 7f051a9ae740 10 client.16797 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.497+0000 7f051a9ae740 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.497+0000 7f051a9ae740 20 client.16797 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.497+0000 7f051a9ae740 1 --2- 172.21.15.12:0/3750092006 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556aa8e3fe70 0x556aa8e422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.497+0000 7f051a9ae740 4 client.16797 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.497+0000 7f051a9ae740 1 -- 172.21.15.12:0/3750092006 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aa8d9ba70 con 0x556aa8e3fe70 2024-05-09T09:28:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.497+0000 7f0515239640 1 --2- 172.21.15.12:0/3750092006 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556aa8e3fe70 0x556aa8e422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.497+0000 7f04f97fa640 10 client.16797 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.497+0000 7f0515239640 1 --2- 172.21.15.12:0/3750092006 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556aa8e3fe70 0x556aa8e422d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0f7a590617a97c server_cookie=bda5b87a1f50d2a2 in_seq=0 out_seq=0 2024-05-09T09:28:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.499+0000 7f04f97fa640 1 -- 172.21.15.12:0/3750092006 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556aa8d9ba70 con 0x556aa8e3fe70 2024-05-09T09:28:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.499+0000 7f04f97fa640 10 client.16797 handle_command_reply: tid=0 2024-05-09T09:28:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.499+0000 7f051a9ae740 1 client.16797 shutdown 2024-05-09T09:28:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.500+0000 7f051a9ae740 1 -- 172.21.15.12:0/3750092006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 msgr2=0x556aa8e34950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.500+0000 7f051a9ae740 1 --2- 172.21.15.12:0/3750092006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 0x556aa8e34950 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f050c01b040 tx=0x7f050c0042d0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.500+0000 7f051a9ae740 1 -- 172.21.15.12:0/3750092006 shutdown_connections 2024-05-09T09:28:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.500+0000 7f051a9ae740 1 --2- 172.21.15.12:0/3750092006 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556aa8e3fe70 0x556aa8e422d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.500+0000 7f051a9ae740 1 --2- 172.21.15.12:0/3750092006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.500+0000 7f051a9ae740 1 --2- 172.21.15.12:0/3750092006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2ac40 0x556aa8e34950 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.500+0000 7f051a9ae740 1 --2- 172.21.15.12:0/3750092006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2a250 0x556aa8e34410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.500+0000 7f04f97fa640 0 client.16797 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.500+0000 7f051a9ae740 1 -- 172.21.15.12:0/3750092006 >> 172.21.15.12:0/3750092006 conn(0x556aa8e258c0 msgr2=0x556aa8e24a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.500+0000 7f051a9ae740 1 -- 172.21.15.12:0/3750092006 shutdown_connections 2024-05-09T09:28:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.500+0000 7f051a9ae740 1 -- 172.21.15.12:0/3750092006 wait complete. 2024-05-09T09:28:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.500+0000 7f051a9ae740 20 client.16797 trim_cache size 0 max 16384 2024-05-09T09:28:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.515+0000 7f051a9ae740 1 Processor -- start 2024-05-09T09:28:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.515+0000 7f051a9ae740 1 -- start start 2024-05-09T09:28:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.515+0000 7f051a9ae740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2a250 0x556aa8e47be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.515+0000 7f051a9ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2ac40 0x556aa8e48120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.515+0000 7f051a9ae740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.515+0000 7f051a9ae740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f05101ac6e0 con 0x556aa8e2fdb0 2024-05-09T09:28:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.515+0000 7f051a9ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f05101ae460 con 0x556aa8e2ac40 2024-05-09T09:28:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.515+0000 7f051a9ae740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05101ace70 con 0x556aa8e2a250 2024-05-09T09:28:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.515+0000 7f0515239640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.515+0000 7f0515239640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35040/0 (socket says 172.21.15.12:35040) 2024-05-09T09:28:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.515+0000 7f0515239640 1 -- 172.21.15.12:0/831902128 learned_addr learned my addr 172.21.15.12:0/831902128 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.515+0000 7f0515a3a640 1 --2- 172.21.15.12:0/831902128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2ac40 0x556aa8e48120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.515+0000 7f0515239640 1 --2- 172.21.15.12:0/831902128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2a250 0x556aa8e47be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.516+0000 7f0515239640 1 -- 172.21.15.12:0/831902128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2a250 msgr2=0x556aa8e47be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.516+0000 7f0515239640 1 --2- 172.21.15.12:0/831902128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2a250 0x556aa8e47be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.516+0000 7f0515239640 1 -- 172.21.15.12:0/831902128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2ac40 msgr2=0x556aa8e48120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.516+0000 7f0515239640 1 --2- 172.21.15.12:0/831902128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2ac40 0x556aa8e48120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.516+0000 7f0515239640 1 -- 172.21.15.12:0/831902128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa8e26630 con 0x556aa8e2fdb0 2024-05-09T09:28:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.516+0000 7f0515a3a640 1 --2- 172.21.15.12:0/831902128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2ac40 0x556aa8e48120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.516+0000 7f0515239640 1 --2- 172.21.15.12:0/831902128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3b950 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0500004650 tx=0x7f0500005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f8b363ddb3d559 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.516+0000 7f04fb7fe640 1 -- 172.21.15.12:0/831902128 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500035040 con 0x556aa8e2fdb0 2024-05-09T09:28:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.516+0000 7f04fb7fe640 1 -- 172.21.15.12:0/831902128 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0500005020 con 0x556aa8e2fdb0 2024-05-09T09:28:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.516+0000 7f04fb7fe640 1 -- 172.21.15.12:0/831902128 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500004a00 con 0x556aa8e2fdb0 2024-05-09T09:28:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 -- 172.21.15.12:0/831902128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2fdb0 msgr2=0x556aa8e3b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 --2- 172.21.15.12:0/831902128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3b950 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0500004650 tx=0x7f0500005dd0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 -- 172.21.15.12:0/831902128 shutdown_connections 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 --2- 172.21.15.12:0/831902128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3b950 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 --2- 172.21.15.12:0/831902128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2ac40 0x556aa8e48120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 --2- 172.21.15.12:0/831902128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2a250 0x556aa8e47be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 -- 172.21.15.12:0/831902128 >> 172.21.15.12:0/831902128 conn(0x556aa8e258c0 msgr2=0x556aa8e25ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 -- 172.21.15.12:0/831902128 shutdown_connections 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 -- 172.21.15.12:0/831902128 wait complete. 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 -- start start 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 Processor -- start 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2a250 0x556aa8e39400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2ac40 0x556aa8e39940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f05101add90 con 0x556aa8e2a250 2024-05-09T09:28:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f05101ad600 con 0x556aa8e2ac40 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05101aa6c0 con 0x556aa8e2fdb0 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.517+0000 7f051a9ae740 10 client.0 _ll_register_callbacks cb 0x7f050c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f0515239640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2a250 0x556aa8e39400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f0515239640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2a250 0x556aa8e39400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35056/0 (socket says 172.21.15.12:35056) 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f0515239640 1 -- 172.21.15.12:0/1421045504 learned_addr learned my addr 172.21.15.12:0/1421045504 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f0515a3a640 1 --2- 172.21.15.12:0/1421045504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2ac40 0x556aa8e39940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f0515239640 1 --2- 172.21.15.12:0/1421045504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f0515239640 1 -- 172.21.15.12:0/1421045504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2fdb0 msgr2=0x556aa8e3a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f0515239640 1 --2- 172.21.15.12:0/1421045504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f0515239640 1 -- 172.21.15.12:0/1421045504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2ac40 msgr2=0x556aa8e39940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f0515239640 1 --2- 172.21.15.12:0/1421045504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2ac40 0x556aa8e39940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f0515239640 1 -- 172.21.15.12:0/1421045504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa8e3af40 con 0x556aa8e2a250 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f0515239640 1 --2- 172.21.15.12:0/1421045504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2a250 0x556aa8e39400 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f050004eab0 tx=0x7f0500009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676894bc6859264e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f04f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f04f97fa640 1 -- 172.21.15.12:0/1421045504 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500035040 con 0x556aa8e2a250 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f04f97fa640 1 -- 172.21.15.12:0/1421045504 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f0500032020 con 0x556aa8e2a250 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f051a9ae740 1 -- 172.21.15.12:0/1421045504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0500005020 con 0x556aa8e2a250 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f04f97fa640 1 -- 172.21.15.12:0/1421045504 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f050006a050 con 0x556aa8e2a250 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f051a9ae740 10 client.16803 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.518+0000 7f051a9ae740 1 -- 172.21.15.12:0/1421045504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556aa8ea6810 con 0x556aa8e2a250 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.519+0000 7f04f97fa640 1 -- 172.21.15.12:0/1421045504 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0500005020 con 0x556aa8e2a250 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.519+0000 7f04f97fa640 1 -- 172.21.15.12:0/1421045504 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f0500032810 con 0x556aa8e2a250 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.519+0000 7f051a9ae740 10 client.16803 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.519+0000 7f051a9ae740 10 client.16803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.519+0000 7f051a9ae740 20 client.16803 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.519+0000 7f051a9ae740 1 --2- 172.21.15.12:0/1421045504 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556aa8ea71b0 0x556aa8ea95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.519+0000 7f051a9ae740 4 client.16803 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:28:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.519+0000 7f051a9ae740 1 -- 172.21.15.12:0/1421045504 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556aa8e2b8d0 con 0x556aa8ea71b0 2024-05-09T09:28:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.519+0000 7f0515a3a640 1 --2- 172.21.15.12:0/1421045504 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556aa8ea71b0 0x556aa8ea95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.520+0000 7f04f97fa640 10 client.16803 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:33.519+0000 7f0515a3a640 1 --2- 172.21.15.12:0/1421045504 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556aa8ea71b0 0x556aa8ea95d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d820d631472292f server_cookie=7b284e9cb9e2d6f5 in_seq=0 out_seq=0 2024-05-09T09:28:34.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:34 smithi012 ceph-mon[30140]: pgmap v458: 129 pgs: 129 active+clean; 146 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-05-09T09:28:34.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:34 smithi170 ceph-mon[33632]: pgmap v458: 129 pgs: 129 active+clean; 146 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-05-09T09:28:34.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:34 smithi016 ceph-mon[33335]: pgmap v458: 129 pgs: 129 active+clean; 146 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-05-09T09:28:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.110+0000 7f04f97fa640 1 -- 172.21.15.12:0/1421045504 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556aa8e2b8d0 con 0x556aa8ea71b0 2024-05-09T09:28:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.110+0000 7f04f97fa640 10 client.16803 handle_command_reply: tid=0 2024-05-09T09:28:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.110+0000 7f051a9ae740 1 client.16803 shutdown 2024-05-09T09:28:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.111+0000 7f051a9ae740 1 -- 172.21.15.12:0/1421045504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2a250 msgr2=0x556aa8e39400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.111+0000 7f051a9ae740 1 --2- 172.21.15.12:0/1421045504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2a250 0x556aa8e39400 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f050004eab0 tx=0x7f0500009260 comp rx=0 tx=0).stop 2024-05-09T09:28:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.111+0000 7f051a9ae740 1 -- 172.21.15.12:0/1421045504 shutdown_connections 2024-05-09T09:28:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.111+0000 7f051a9ae740 1 --2- 172.21.15.12:0/1421045504 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x556aa8ea71b0 0x556aa8ea95d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.111+0000 7f051a9ae740 1 --2- 172.21.15.12:0/1421045504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aa8e2fdb0 0x556aa8e3a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.111+0000 7f051a9ae740 1 --2- 172.21.15.12:0/1421045504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aa8e2ac40 0x556aa8e39940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.111+0000 7f051a9ae740 1 --2- 172.21.15.12:0/1421045504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556aa8e2a250 0x556aa8e39400 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.111+0000 7f051a9ae740 1 -- 172.21.15.12:0/1421045504 >> 172.21.15.12:0/1421045504 conn(0x556aa8e258c0 msgr2=0x556aa8e2bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.111+0000 7f04f97fa640 0 client.16803 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.111+0000 7f051a9ae740 1 -- 172.21.15.12:0/1421045504 shutdown_connections 2024-05-09T09:28:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.111+0000 7f051a9ae740 1 -- 172.21.15.12:0/1421045504 wait complete. 2024-05-09T09:28:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.111+0000 7f051a9ae740 20 client.16803 trim_cache size 0 max 16384 2024-05-09T09:28:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:28:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:28:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:28:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:28:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.112+0000 7f0517c9e640 1 -- 172.21.15.12:0/748437156 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f04fc0a2c30 msgr2=0x7f04fc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.112+0000 7f0517c9e640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f04fc0a2c30 0x7f04fc0a50f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f050809bfe0 tx=0x7f050806f110 comp rx=0 tx=0).stop 2024-05-09T09:28:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.112+0000 7f0517c9e640 1 -- 172.21.15.12:0/748437156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102340 msgr2=0x7f05101a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.112+0000 7f0517c9e640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102340 0x7f05101a12f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0508009fd0 tx=0x7f050800ef10 comp rx=0 tx=0).stop 2024-05-09T09:28:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.112+0000 7f0517c9e640 1 -- 172.21.15.12:0/748437156 shutdown_connections 2024-05-09T09:28:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.112+0000 7f0517c9e640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f04fc0a2c30 0x7f04fc0a50f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.112+0000 7f0517c9e640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0510102340 0x7f05101a12f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.112+0000 7f0517c9e640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0510101980 0x7f051019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.112+0000 7f0517c9e640 1 --2- 172.21.15.12:0/748437156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0510100790 0x7f0510199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.112+0000 7f0517c9e640 1 -- 172.21.15.12:0/748437156 >> 172.21.15.12:0/748437156 conn(0x7f05100fbd00 msgr2=0x7f0510190a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.112+0000 7f0517c9e640 1 -- 172.21.15.12:0/748437156 shutdown_connections 2024-05-09T09:28:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:35.112+0000 7f0517c9e640 1 -- 172.21.15.12:0/748437156 wait complete. 2024-05-09T09:28:36.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:36 smithi012 ceph-mon[30140]: pgmap v459: 129 pgs: 129 active+clean; 152 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 1.5 MiB/s wr, 360 op/s 2024-05-09T09:28:36.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:36 smithi170 ceph-mon[33632]: pgmap v459: 129 pgs: 129 active+clean; 152 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 1.5 MiB/s wr, 360 op/s 2024-05-09T09:28:36.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:36 smithi016 ceph-mon[33335]: pgmap v459: 129 pgs: 129 active+clean; 152 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 1.5 MiB/s wr, 360 op/s 2024-05-09T09:28:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:38 smithi012 ceph-mon[30140]: pgmap v460: 129 pgs: 129 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 1.7 MiB/s wr, 508 op/s 2024-05-09T09:28:38.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:38 smithi170 ceph-mon[33632]: pgmap v460: 129 pgs: 129 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 1.7 MiB/s wr, 508 op/s 2024-05-09T09:28:38.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:38 smithi016 ceph-mon[33335]: pgmap v460: 129 pgs: 129 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 1.7 MiB/s wr, 508 op/s 2024-05-09T09:28:40.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:40 smithi012 ceph-mon[30140]: pgmap v461: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.8 MiB/s wr, 580 op/s 2024-05-09T09:28:40.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:40 smithi170 ceph-mon[33632]: pgmap v461: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.8 MiB/s wr, 580 op/s 2024-05-09T09:28:40.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:40 smithi016 ceph-mon[33335]: pgmap v461: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.8 MiB/s wr, 580 op/s 2024-05-09T09:28:41.308 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2024-05-09T09:28:42.311 INFO:tasks.workunit.client.0.smithi012.stderr:+ tar c linux-4.0.5 2024-05-09T09:28:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:42 smithi012 ceph-mon[30140]: pgmap v462: 129 pgs: 129 active+clean; 159 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 747 op/s 2024-05-09T09:28:42.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:42 smithi170 ceph-mon[33632]: pgmap v462: 129 pgs: 129 active+clean; 159 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 747 op/s 2024-05-09T09:28:42.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:42 smithi016 ceph-mon[33335]: pgmap v462: 129 pgs: 129 active+clean; 159 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 747 op/s 2024-05-09T09:28:44.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:44 smithi012 ceph-mon[30140]: pgmap v463: 129 pgs: 129 active+clean; 164 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 675 op/s 2024-05-09T09:28:44.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:44 smithi170 ceph-mon[33632]: pgmap v463: 129 pgs: 129 active+clean; 164 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 675 op/s 2024-05-09T09:28:44.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:44 smithi016 ceph-mon[33335]: pgmap v463: 129 pgs: 129 active+clean; 164 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 675 op/s 2024-05-09T09:28:44.638 INFO:tasks.workunit.client.0.smithi012.stdout:Ok 2024-05-09T09:28:44.639 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo Ok 2024-05-09T09:28:44.639 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T09:28:44.640 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T09:28:46.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:46 smithi012 ceph-mon[30140]: pgmap v464: 129 pgs: 129 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.6 MiB/s wr, 657 op/s 2024-05-09T09:28:46.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:46 smithi170 ceph-mon[33632]: pgmap v464: 129 pgs: 129 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.6 MiB/s wr, 657 op/s 2024-05-09T09:28:46.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:46 smithi016 ceph-mon[33335]: pgmap v464: 129 pgs: 129 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.6 MiB/s wr, 657 op/s 2024-05-09T09:28:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.210+0000 7f445b970640 1 -- 172.21.15.12:0/2540270867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4454100790 msgr2=0x7f4454100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.210+0000 7f445b970640 1 --2- 172.21.15.12:0/2540270867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4454100790 0x7f4454100bb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4450009900 tx=0x7f445002a410 comp rx=0 tx=0).stop 2024-05-09T09:28:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.210+0000 7f445b970640 1 -- 172.21.15.12:0/2540270867 shutdown_connections 2024-05-09T09:28:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.210+0000 7f445b970640 1 --2- 172.21.15.12:0/2540270867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4454102340 0x7f4454108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.210+0000 7f445b970640 1 --2- 172.21.15.12:0/2540270867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4454101980 0x7f4454101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.210+0000 7f445b970640 1 --2- 172.21.15.12:0/2540270867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4454100790 0x7f4454100bb0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.210+0000 7f445b970640 1 -- 172.21.15.12:0/2540270867 >> 172.21.15.12:0/2540270867 conn(0x7f44540fbd00 msgr2=0x7f44540fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.210+0000 7f445b970640 1 -- 172.21.15.12:0/2540270867 shutdown_connections 2024-05-09T09:28:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.210+0000 7f445b970640 1 -- 172.21.15.12:0/2540270867 wait complete. 2024-05-09T09:28:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.210+0000 7f445b970640 1 Processor -- start 2024-05-09T09:28:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.210+0000 7f445b970640 1 -- start start 2024-05-09T09:28:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445b970640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4454100790 0x7f4454197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445b970640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4454101980 0x7f44541980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445b970640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4454102340 0x7f445419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445b970640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f445410c3f0 con 0x7f4454100790 2024-05-09T09:28:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445b970640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f445410c270 con 0x7f4454101980 2024-05-09T09:28:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445b970640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f445410c570 con 0x7f4454102340 2024-05-09T09:28:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445970c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4454100790 0x7f4454197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445970c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4454100790 0x7f4454197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52342/0 (socket says 172.21.15.12:52342) 2024-05-09T09:28:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445970c640 1 -- 172.21.15.12:0/2933759406 learned_addr learned my addr 172.21.15.12:0/2933759406 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f4459f0d640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4454102340 0x7f445419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f4458f0b640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4454101980 0x7f44541980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445970c640 1 -- 172.21.15.12:0/2933759406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4454102340 msgr2=0x7f445419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445970c640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4454102340 0x7f445419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445970c640 1 -- 172.21.15.12:0/2933759406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4454101980 msgr2=0x7f44541980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445970c640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4454101980 0x7f44541980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445970c640 1 -- 172.21.15.12:0/2933759406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445419f760 con 0x7f4454100790 2024-05-09T09:28:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f4458f0b640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4454101980 0x7f44541980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:28:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445970c640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4454100790 0x7f4454197b90 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4450005ab0 tx=0x7f44500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f444a7fc640 1 -- 172.21.15.12:0/2933759406 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445003f030 con 0x7f4454100790 2024-05-09T09:28:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f444a7fc640 1 -- 172.21.15.12:0/2933759406 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4450003ac0 con 0x7f4454100790 2024-05-09T09:28:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445b970640 1 -- 172.21.15.12:0/2933759406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445419f9f0 con 0x7f4454100790 2024-05-09T09:28:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f444a7fc640 1 -- 172.21.15.12:0/2933759406 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450035070 con 0x7f4454100790 2024-05-09T09:28:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.211+0000 7f445b970640 1 -- 172.21.15.12:0/2933759406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445419fee0 con 0x7f4454100790 2024-05-09T09:28:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.214+0000 7f445e680740 1 Processor -- start 2024-05-09T09:28:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.214+0000 7f444a7fc640 1 -- 172.21.15.12:0/2933759406 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4450042030 con 0x7f4454100790 2024-05-09T09:28:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.214+0000 7f445e680740 1 -- start start 2024-05-09T09:28:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.214+0000 7f445e680740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e57c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.214+0000 7f445e680740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 0x55a73e581740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.214+0000 7f445e680740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 0x55a73e584240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.214+0000 7f445e680740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a73e4e0340 con 0x55a73e57c250 2024-05-09T09:28:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.214+0000 7f445e680740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a73e488290 con 0x55a73e57cc40 2024-05-09T09:28:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.214+0000 7f445e680740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a73e4dc300 con 0x55a73e581db0 2024-05-09T09:28:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f4459f0d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e57c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f445970c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 0x55a73e581740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f4459f0d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e57c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52358/0 (socket says 172.21.15.12:52358) 2024-05-09T09:28:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f4459f0d640 1 -- 172.21.15.12:0/2627590325 learned_addr learned my addr 172.21.15.12:0/2627590325 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f4458f0b640 1 --2- 172.21.15.12:0/2627590325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 0x55a73e584240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f4459f0d640 1 -- 172.21.15.12:0/2627590325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 msgr2=0x55a73e584240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f4459f0d640 1 --2- 172.21.15.12:0/2627590325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 0x55a73e584240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f4459f0d640 1 -- 172.21.15.12:0/2627590325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 msgr2=0x55a73e581740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f4459f0d640 1 --2- 172.21.15.12:0/2627590325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 0x55a73e581740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f4459f0d640 1 -- 172.21.15.12:0/2627590325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73e4ac8e0 con 0x55a73e57c250 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f444a7fc640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f44400a2c30 0x7f44400a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f4459f0d640 1 --2- 172.21.15.12:0/2627590325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e57c670 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f444c0039f0 tx=0x7f444c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257f4a3af9ca5d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f445970c640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f44400a2c30 0x7f44400a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f443f7fe640 1 -- 172.21.15.12:0/2627590325 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c01a470 con 0x55a73e57c250 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f443f7fe640 1 -- 172.21.15.12:0/2627590325 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f444c015070 con 0x55a73e57c250 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f443f7fe640 1 -- 172.21.15.12:0/2627590325 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c010880 con 0x55a73e57c250 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.215+0000 7f444a7fc640 1 -- 172.21.15.12:0/2933759406 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f44500efa20 con 0x7f4454100790 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.216+0000 7f445e680740 1 -- 172.21.15.12:0/2627590325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 msgr2=0x55a73e57c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.216+0000 7f445e680740 1 --2- 172.21.15.12:0/2627590325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e57c670 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f444c0039f0 tx=0x7f444c003eb0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.216+0000 7f445970c640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f44400a2c30 0x7f44400a50f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f44500ed4c0 tx=0x7f44500fa7c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.216+0000 7f445e680740 1 -- 172.21.15.12:0/2627590325 shutdown_connections 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.216+0000 7f445e680740 1 --2- 172.21.15.12:0/2627590325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 0x55a73e584240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.216+0000 7f445e680740 1 --2- 172.21.15.12:0/2627590325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 0x55a73e581740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.216+0000 7f445e680740 1 --2- 172.21.15.12:0/2627590325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e57c670 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.216+0000 7f445e680740 1 -- 172.21.15.12:0/2627590325 >> 172.21.15.12:0/2627590325 conn(0x55a73e5778c0 msgr2=0x55a73e577cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.216+0000 7f445e680740 1 -- 172.21.15.12:0/2627590325 shutdown_connections 2024-05-09T09:28:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.216+0000 7f445e680740 1 -- 172.21.15.12:0/2627590325 wait complete. 2024-05-09T09:28:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f445e680740 1 -- start start 2024-05-09T09:28:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f445e680740 1 Processor -- start 2024-05-09T09:28:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f445e680740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e586490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f445e680740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 0x55a73e5869d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f445e680740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 0x55a73e58da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f445e680740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a73e584d60 con 0x55a73e57c250 2024-05-09T09:28:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f445e680740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a73e49b790 con 0x55a73e57cc40 2024-05-09T09:28:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f445e680740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a73e584ee0 con 0x55a73e581db0 2024-05-09T09:28:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f445e680740 10 client.0 _ll_register_callbacks cb 0x55a73e4d6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f4459f0d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e586490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f4458f0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 0x55a73e5869d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f4459f0d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e586490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52364/0 (socket says 172.21.15.12:52364) 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f4459f0d640 1 -- 172.21.15.12:0/2097915609 learned_addr learned my addr 172.21.15.12:0/2097915609 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.217+0000 7f4459f0d640 1 --2- 172.21.15.12:0/2097915609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 0x55a73e58da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f4459f0d640 1 -- 172.21.15.12:0/2097915609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 msgr2=0x55a73e58da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f4459f0d640 1 --2- 172.21.15.12:0/2097915609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 0x55a73e58da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f4459f0d640 1 -- 172.21.15.12:0/2097915609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 msgr2=0x55a73e5869d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f4459f0d640 1 --2- 172.21.15.12:0/2097915609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 0x55a73e5869d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f4459f0d640 1 -- 172.21.15.12:0/2097915609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73e1f4210 con 0x55a73e57c250 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f4459f0d640 1 --2- 172.21.15.12:0/2097915609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e586490 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f444c006010 tx=0x7f444c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b29c89bbbbe3bcc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f443cff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f443cff9640 1 -- 172.21.15.12:0/2097915609 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c004c50 con 0x55a73e57c250 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f443cff9640 1 -- 172.21.15.12:0/2097915609 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f444c022040 con 0x55a73e57c250 2024-05-09T09:28:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f445e680740 1 -- 172.21.15.12:0/2097915609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f444c015070 con 0x55a73e57c250 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f443cff9640 1 -- 172.21.15.12:0/2097915609 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f444c033050 con 0x55a73e57c250 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f445e680740 10 client.16815 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f445e680740 1 -- 172.21.15.12:0/2097915609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a73e2c0620 con 0x55a73e57c250 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.218+0000 7f443cff9640 1 -- 172.21.15.12:0/2097915609 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f444c015070 con 0x55a73e57c250 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.219+0000 7f443cff9640 1 -- 172.21.15.12:0/2097915609 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f444c010c50 con 0x55a73e57c250 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.219+0000 7f445e680740 10 client.16815 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.219+0000 7f445e680740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.219+0000 7f445e680740 20 client.16815 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.219+0000 7f445e680740 1 --2- 172.21.15.12:0/2097915609 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a73e591d60 0x55a73e5941a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.219+0000 7f445e680740 4 client.16815 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.219+0000 7f445e680740 1 -- 172.21.15.12:0/2097915609 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a73e4eda70 con 0x55a73e591d60 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.219+0000 7f4458f0b640 1 --2- 172.21.15.12:0/2097915609 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a73e591d60 0x55a73e5941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.219+0000 7f4458f0b640 1 --2- 172.21.15.12:0/2097915609 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a73e591d60 0x55a73e5941a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb73e2fe1caaf89a server_cookie=5b11441828d79c98 in_seq=0 out_seq=0 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.219+0000 7f443cff9640 10 client.16815 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f443cff9640 1 -- 172.21.15.12:0/2097915609 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a73e4eda70 con 0x55a73e591d60 2024-05-09T09:28:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f443cff9640 10 client.16815 handle_command_reply: tid=0 2024-05-09T09:28:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f445e680740 1 client.16815 shutdown 2024-05-09T09:28:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f445e680740 1 -- 172.21.15.12:0/2097915609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 msgr2=0x55a73e586490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f445e680740 1 --2- 172.21.15.12:0/2097915609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e586490 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f444c006010 tx=0x7f444c009900 comp rx=0 tx=0).stop 2024-05-09T09:28:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f445e680740 1 -- 172.21.15.12:0/2097915609 shutdown_connections 2024-05-09T09:28:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f445e680740 1 --2- 172.21.15.12:0/2097915609 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a73e591d60 0x55a73e5941a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f445e680740 1 --2- 172.21.15.12:0/2097915609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 0x55a73e58da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f445e680740 1 --2- 172.21.15.12:0/2097915609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 0x55a73e5869d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f443cff9640 0 client.16815 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f445e680740 1 --2- 172.21.15.12:0/2097915609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e586490 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f445e680740 1 -- 172.21.15.12:0/2097915609 >> 172.21.15.12:0/2097915609 conn(0x55a73e5778c0 msgr2=0x55a73e576a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f445e680740 1 -- 172.21.15.12:0/2097915609 shutdown_connections 2024-05-09T09:28:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.221+0000 7f445e680740 1 -- 172.21.15.12:0/2097915609 wait complete. 2024-05-09T09:28:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.222+0000 7f445e680740 20 client.16815 trim_cache size 0 max 16384 2024-05-09T09:28:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f445e680740 1 Processor -- start 2024-05-09T09:28:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f445e680740 1 -- start start 2024-05-09T09:28:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f445e680740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e585990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f445e680740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 0x55a73e58d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f445e680740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 0x55a73e599a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f445e680740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f44541a8390 con 0x55a73e57c250 2024-05-09T09:28:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f445e680740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f44541aa110 con 0x55a73e57cc40 2024-05-09T09:28:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f445e680740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f44541a8b20 con 0x55a73e581db0 2024-05-09T09:28:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f4459f0d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e585990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f4459f0d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e585990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52372/0 (socket says 172.21.15.12:52372) 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f4459f0d640 1 -- 172.21.15.12:0/2502298858 learned_addr learned my addr 172.21.15.12:0/2502298858 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f4458f0b640 1 --2- 172.21.15.12:0/2502298858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 0x55a73e58d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f4459f0d640 1 -- 172.21.15.12:0/2502298858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 msgr2=0x55a73e599a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f4459f0d640 1 --2- 172.21.15.12:0/2502298858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 0x55a73e599a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f4459f0d640 1 -- 172.21.15.12:0/2502298858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 msgr2=0x55a73e58d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.238+0000 7f4459f0d640 1 --2- 172.21.15.12:0/2502298858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 0x55a73e58d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.239+0000 7f4459f0d640 1 -- 172.21.15.12:0/2502298858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73e57be90 con 0x55a73e57c250 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.239+0000 7f4459f0d640 1 --2- 172.21.15.12:0/2502298858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e585990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f444c002230 tx=0x7f444c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153302a93b1bd8e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.239+0000 7f443effd640 1 -- 172.21.15.12:0/2502298858 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c005bd0 con 0x55a73e57c250 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.239+0000 7f443effd640 1 -- 172.21.15.12:0/2502298858 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f444c015070 con 0x55a73e57c250 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.239+0000 7f443effd640 1 -- 172.21.15.12:0/2502298858 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c011850 con 0x55a73e57c250 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.239+0000 7f445e680740 1 -- 172.21.15.12:0/2502298858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 msgr2=0x55a73e585990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.239+0000 7f445e680740 1 --2- 172.21.15.12:0/2502298858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e585990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f444c002230 tx=0x7f444c003eb0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.240+0000 7f445e680740 1 -- 172.21.15.12:0/2502298858 shutdown_connections 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.240+0000 7f445e680740 1 --2- 172.21.15.12:0/2502298858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e581db0 0x55a73e599a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.240+0000 7f445e680740 1 --2- 172.21.15.12:0/2502298858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57cc40 0x55a73e58d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.240+0000 7f445e680740 1 --2- 172.21.15.12:0/2502298858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e57c250 0x55a73e585990 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.240+0000 7f445e680740 1 -- 172.21.15.12:0/2502298858 >> 172.21.15.12:0/2502298858 conn(0x55a73e5778c0 msgr2=0x55a73e585280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.240+0000 7f445e680740 1 -- 172.21.15.12:0/2502298858 shutdown_connections 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.240+0000 7f445e680740 1 -- 172.21.15.12:0/2502298858 wait complete. 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.240+0000 7f445e680740 1 -- start start 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.240+0000 7f445e680740 1 Processor -- start 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f445e680740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57c250 0x55a73e58adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f445e680740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e57cc40 0x55a73e58b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f445e680740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e581db0 0x55a73e58c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f445e680740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f44541a9a40 con 0x55a73e581db0 2024-05-09T09:28:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f445e680740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f44541a92b0 con 0x55a73e57c250 2024-05-09T09:28:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f445e680740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f44541a6370 con 0x55a73e57cc40 2024-05-09T09:28:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f445e680740 10 client.0 _ll_register_callbacks cb 0x7f44500fa690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f4459f0d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e581db0 0x55a73e58c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f4458f0b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e57cc40 0x55a73e58b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f4459f0d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e581db0 0x55a73e58c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52374/0 (socket says 172.21.15.12:52374) 2024-05-09T09:28:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f4459f0d640 1 -- 172.21.15.12:0/630762688 learned_addr learned my addr 172.21.15.12:0/630762688 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f4459f0d640 1 --2- 172.21.15.12:0/630762688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57c250 0x55a73e58adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f4458f0b640 1 -- 172.21.15.12:0/630762688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57c250 msgr2=0x55a73e58adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f4458f0b640 1 --2- 172.21.15.12:0/630762688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57c250 0x55a73e58adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f4458f0b640 1 -- 172.21.15.12:0/630762688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e581db0 msgr2=0x55a73e58c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f4458f0b640 1 --2- 172.21.15.12:0/630762688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e581db0 0x55a73e58c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f4458f0b640 1 -- 172.21.15.12:0/630762688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73e58c930 con 0x55a73e57cc40 2024-05-09T09:28:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f4459f0d640 1 --2- 172.21.15.12:0/630762688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e581db0 0x55a73e58c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f4458f0b640 1 --2- 172.21.15.12:0/630762688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e57cc40 0x55a73e58b330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55a73e5949b0 tx=0x7f4444014840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e0f5c6d6cbe16d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.241+0000 7f443cff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f443cff9640 1 -- 172.21.15.12:0/630762688 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444004250 con 0x55a73e57cc40 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f443cff9640 1 -- 172.21.15.12:0/630762688 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f44440043f0 con 0x55a73e57cc40 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f445e680740 1 -- 172.21.15.12:0/630762688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f444c015070 con 0x55a73e57cc40 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f443cff9640 1 -- 172.21.15.12:0/630762688 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f444402c050 con 0x55a73e57cc40 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f445e680740 10 client.26365 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f445e680740 1 -- 172.21.15.12:0/630762688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a73e58dd10 con 0x55a73e57cc40 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f443cff9640 1 -- 172.21.15.12:0/630762688 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f444c015070 con 0x55a73e57cc40 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f443cff9640 1 -- 172.21.15.12:0/630762688 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f444400f040 con 0x55a73e57cc40 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f445e680740 10 client.26365 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f445e680740 10 client.26365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f445e680740 20 client.26365 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f445e680740 1 --2- 172.21.15.12:0/630762688 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a73e5f8b10 0x55a73e5faf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f445e680740 4 client.26365 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:28:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f445e680740 1 -- 172.21.15.12:0/630762688 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a73e4e7dc0 con 0x55a73e5f8b10 2024-05-09T09:28:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.242+0000 7f4459f0d640 1 --2- 172.21.15.12:0/630762688 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a73e5f8b10 0x55a73e5faf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.243+0000 7f4459f0d640 1 --2- 172.21.15.12:0/630762688 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a73e5f8b10 0x55a73e5faf30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b9b74ee87c84399 server_cookie=2f25a932667873c2 in_seq=0 out_seq=0 2024-05-09T09:28:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.243+0000 7f443cff9640 10 client.26365 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.426+0000 7f443cff9640 1 -- 172.21.15.12:0/630762688 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a73e4e7dc0 con 0x55a73e5f8b10 2024-05-09T09:28:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.426+0000 7f443cff9640 10 client.26365 handle_command_reply: tid=0 2024-05-09T09:28:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.427+0000 7f445e680740 1 client.26365 shutdown 2024-05-09T09:28:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.427+0000 7f445e680740 1 -- 172.21.15.12:0/630762688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e57cc40 msgr2=0x55a73e58b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.427+0000 7f445e680740 1 --2- 172.21.15.12:0/630762688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e57cc40 0x55a73e58b330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55a73e5949b0 tx=0x7f4444014840 comp rx=0 tx=0).stop 2024-05-09T09:28:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.427+0000 7f445e680740 1 -- 172.21.15.12:0/630762688 shutdown_connections 2024-05-09T09:28:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.427+0000 7f445e680740 1 --2- 172.21.15.12:0/630762688 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a73e5f8b10 0x55a73e5faf30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.427+0000 7f445e680740 1 --2- 172.21.15.12:0/630762688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a73e581db0 0x55a73e58c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.427+0000 7f443cff9640 0 client.26365 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.427+0000 7f445e680740 1 --2- 172.21.15.12:0/630762688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a73e57cc40 0x55a73e58b330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.427+0000 7f445e680740 1 --2- 172.21.15.12:0/630762688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73e57c250 0x55a73e58adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.427+0000 7f445e680740 1 -- 172.21.15.12:0/630762688 >> 172.21.15.12:0/630762688 conn(0x55a73e5778c0 msgr2=0x55a73e5860a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.427+0000 7f445e680740 1 -- 172.21.15.12:0/630762688 shutdown_connections 2024-05-09T09:28:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.427+0000 7f445e680740 1 -- 172.21.15.12:0/630762688 wait complete. 2024-05-09T09:28:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.427+0000 7f445e680740 20 client.26365 trim_cache size 0 max 16384 2024-05-09T09:28:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:28:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:28:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:28:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:28:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.428+0000 7f445b970640 1 -- 172.21.15.12:0/2933759406 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f44400a2c30 msgr2=0x7f44400a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.428+0000 7f445b970640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f44400a2c30 0x7f44400a50f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f44500ed4c0 tx=0x7f44500fa7c0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.428+0000 7f445b970640 1 -- 172.21.15.12:0/2933759406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4454100790 msgr2=0x7f4454197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.428+0000 7f445b970640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4454100790 0x7f4454197b90 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4450005ab0 tx=0x7f44500042f0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.428+0000 7f445b970640 1 -- 172.21.15.12:0/2933759406 shutdown_connections 2024-05-09T09:28:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.428+0000 7f445b970640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f44400a2c30 0x7f44400a50f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.428+0000 7f445b970640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4454102340 0x7f445419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.428+0000 7f445b970640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4454101980 0x7f44541980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.428+0000 7f445b970640 1 --2- 172.21.15.12:0/2933759406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4454100790 0x7f4454197b90 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.428+0000 7f445b970640 1 -- 172.21.15.12:0/2933759406 >> 172.21.15.12:0/2933759406 conn(0x7f44540fbd00 msgr2=0x7f44540fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.428+0000 7f445b970640 1 -- 172.21.15.12:0/2933759406 shutdown_connections 2024-05-09T09:28:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:28:48.428+0000 7f445b970640 1 -- 172.21.15.12:0/2933759406 wait complete. 2024-05-09T09:28:48.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:48 smithi012 ceph-mon[30140]: pgmap v465: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.2 MiB/s wr, 610 op/s 2024-05-09T09:28:48.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:48 smithi170 ceph-mon[33632]: pgmap v465: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.2 MiB/s wr, 610 op/s 2024-05-09T09:28:48.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:48 smithi016 ceph-mon[33335]: pgmap v465: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.2 MiB/s wr, 610 op/s 2024-05-09T09:28:50.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:50 smithi012 ceph-mon[30140]: pgmap v466: 129 pgs: 129 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.9 MiB/s wr, 428 op/s 2024-05-09T09:28:50.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:50 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:28:50.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:50 smithi170 ceph-mon[33632]: pgmap v466: 129 pgs: 129 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.9 MiB/s wr, 428 op/s 2024-05-09T09:28:50.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:50 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:28:50.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:50 smithi016 ceph-mon[33335]: pgmap v466: 129 pgs: 129 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.9 MiB/s wr, 428 op/s 2024-05-09T09:28:50.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:50 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:28:51.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:51 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:28:51.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:51 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:28:51.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:51 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:28:51.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:51 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:28:51.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:51 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:28:51.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:51 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:28:51.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:51 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:28:51.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:51 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:28:51.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:51 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:28:52.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:52 smithi012 ceph-mon[30140]: pgmap v467: 129 pgs: 129 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 429 op/s 2024-05-09T09:28:52.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:52 smithi170 ceph-mon[33632]: pgmap v467: 129 pgs: 129 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 429 op/s 2024-05-09T09:28:52.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:52 smithi016 ceph-mon[33335]: pgmap v467: 129 pgs: 129 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 429 op/s 2024-05-09T09:28:54.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:54 smithi012 ceph-mon[30140]: pgmap v468: 129 pgs: 129 active+clean; 67 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.4 MiB/s wr, 315 op/s 2024-05-09T09:28:54.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:54 smithi170 ceph-mon[33632]: pgmap v468: 129 pgs: 129 active+clean; 67 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.4 MiB/s wr, 315 op/s 2024-05-09T09:28:54.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:54 smithi016 ceph-mon[33335]: pgmap v468: 129 pgs: 129 active+clean; 67 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.4 MiB/s wr, 315 op/s 2024-05-09T09:28:56.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:56 smithi012 ceph-mon[30140]: pgmap v469: 129 pgs: 129 active+clean; 67 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.0 MiB/s wr, 366 op/s 2024-05-09T09:28:56.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:56 smithi170 ceph-mon[33632]: pgmap v469: 129 pgs: 129 active+clean; 67 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.0 MiB/s wr, 366 op/s 2024-05-09T09:28:56.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:56 smithi016 ceph-mon[33335]: pgmap v469: 129 pgs: 129 active+clean; 67 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.0 MiB/s wr, 366 op/s 2024-05-09T09:28:56.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:28:56.689 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:28:56.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.806+0000 7fcca33a2640 1 -- 172.21.15.12:0/413171983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc9c100770 msgr2=0x7fcc9c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:56.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.806+0000 7fcca33a2640 1 --2- 172.21.15.12:0/413171983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc9c100770 0x7fcc9c100b90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009900 tx=0x7fcc9802a410 comp rx=0 tx=0).stop 2024-05-09T09:28:56.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.806+0000 7fcca33a2640 1 -- 172.21.15.12:0/413171983 shutdown_connections 2024-05-09T09:28:56.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.806+0000 7fcca33a2640 1 --2- 172.21.15.12:0/413171983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc9c102320 0x7fcc9c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.806+0000 7fcca33a2640 1 --2- 172.21.15.12:0/413171983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc9c101960 0x7fcc9c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.806+0000 7fcca33a2640 1 --2- 172.21.15.12:0/413171983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc9c100770 0x7fcc9c100b90 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.806+0000 7fcca33a2640 1 -- 172.21.15.12:0/413171983 >> 172.21.15.12:0/413171983 conn(0x7fcc9c0fbce0 msgr2=0x7fcc9c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:56.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.806+0000 7fcca33a2640 1 -- 172.21.15.12:0/413171983 shutdown_connections 2024-05-09T09:28:56.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.806+0000 7fcca33a2640 1 -- 172.21.15.12:0/413171983 wait complete. 2024-05-09T09:28:56.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.806+0000 7fcca33a2640 1 Processor -- start 2024-05-09T09:28:56.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.806+0000 7fcca33a2640 1 -- start start 2024-05-09T09:28:56.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca33a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc9c100770 0x7fcc9c104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca33a2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc9c101960 0x7fcc9c102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca33a2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc9c102320 0x7fcc9c103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca33a2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c10c3d0 con 0x7fcc9c102320 2024-05-09T09:28:56.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca33a2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c10c250 con 0x7fcc9c100770 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca33a2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c10c550 con 0x7fcc9c101960 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca193f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc9c102320 0x7fcc9c103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca093d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc9c101960 0x7fcc9c102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca093d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc9c101960 0x7fcc9c102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:57626/0 (socket says 172.21.15.12:57626) 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca093d640 1 -- 172.21.15.12:0/1724523567 learned_addr learned my addr 172.21.15.12:0/1724523567 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca113e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc9c100770 0x7fcc9c104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca193f640 1 -- 172.21.15.12:0/1724523567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc9c101960 msgr2=0x7fcc9c102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca193f640 1 --2- 172.21.15.12:0/1724523567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc9c101960 0x7fcc9c102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca193f640 1 -- 172.21.15.12:0/1724523567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc9c100770 msgr2=0x7fcc9c104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca193f640 1 --2- 172.21.15.12:0/1724523567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc9c100770 0x7fcc9c104700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca193f640 1 -- 172.21.15.12:0/1724523567 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9c103b10 con 0x7fcc9c102320 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca193f640 1 --2- 172.21.15.12:0/1724523567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc9c102320 0x7fcc9c103280 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009fd0 tx=0x7fcc9400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcc927fc640 1 -- 172.21.15.12:0/1724523567 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400ce20 con 0x7fcc9c102320 2024-05-09T09:28:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcc927fc640 1 -- 172.21.15.12:0/1724523567 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcc94005020 con 0x7fcc9c102320 2024-05-09T09:28:56.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca33a2640 1 -- 172.21.15.12:0/1724523567 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9c1a1d40 con 0x7fcc9c102320 2024-05-09T09:28:56.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcc927fc640 1 -- 172.21.15.12:0/1724523567 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc940106b0 con 0x7fcc9c102320 2024-05-09T09:28:56.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.807+0000 7fcca33a2640 1 -- 172.21.15.12:0/1724523567 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc9c1a21d0 con 0x7fcc9c102320 2024-05-09T09:28:56.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.810+0000 7fcca6193740 1 Processor -- start 2024-05-09T09:28:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.810+0000 7fcc927fc640 1 -- 172.21.15.12:0/1724523567 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcc940051d0 con 0x7fcc9c102320 2024-05-09T09:28:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.810+0000 7fcca6193740 1 -- start start 2024-05-09T09:28:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.810+0000 7fcca6193740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dbc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca6193740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 0x5651c4dc1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca6193740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 0x5651c4dc3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca6193740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5651c4d20d00 con 0x5651c4dbc920 2024-05-09T09:28:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca6193740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5651c4cc84c0 con 0x5651c4dc1a90 2024-05-09T09:28:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca6193740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5651c4d1ccc0 con 0x5651c4dbbf30 2024-05-09T09:28:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca113e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 0x5651c4dc1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca093d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dbc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca113e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 0x5651c4dc1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47264/0 (socket says 172.21.15.12:47264) 2024-05-09T09:28:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca093d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dbc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:57634/0 (socket says 172.21.15.12:57634) 2024-05-09T09:28:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca113e640 1 -- 172.21.15.12:0/2366696833 learned_addr learned my addr 172.21.15.12:0/2366696833 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca093d640 1 --2- 172.21.15.12:0/2366696833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 0x5651c4dc3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca113e640 1 -- 172.21.15.12:0/2366696833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 msgr2=0x5651c4dbc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcc927fc640 1 --2- 172.21.15.12:0/1724523567 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcc880a2b90 0x7fcc880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca113e640 1 --2- 172.21.15.12:0/2366696833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dbc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca113e640 1 -- 172.21.15.12:0/2366696833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 msgr2=0x5651c4dc3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca113e640 1 --2- 172.21.15.12:0/2366696833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 0x5651c4dc3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca113e640 1 -- 172.21.15.12:0/2366696833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651c4afe7d0 con 0x5651c4dbc920 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca093d640 1 --2- 172.21.15.12:0/2366696833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dbc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca193f640 1 --2- 172.21.15.12:0/1724523567 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcc880a2b90 0x7fcc880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcc927fc640 1 -- 172.21.15.12:0/1724523567 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fcc940cfbc0 con 0x7fcc9c102320 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.811+0000 7fcca113e640 1 --2- 172.21.15.12:0/2366696833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 0x5651c4dc1420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009900 tx=0x7fcc98005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc62a0582cb6378 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.812+0000 7fcc877fe640 1 -- 172.21.15.12:0/2366696833 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98040070 con 0x5651c4dbc920 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.812+0000 7fcc877fe640 1 -- 172.21.15.12:0/2366696833 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcc98004020 con 0x5651c4dbc920 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.812+0000 7fcc877fe640 1 -- 172.21.15.12:0/2366696833 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98035040 con 0x5651c4dbc920 2024-05-09T09:28:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.812+0000 7fcca6193740 1 -- 172.21.15.12:0/2366696833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 msgr2=0x5651c4dc1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.812+0000 7fcca193f640 1 --2- 172.21.15.12:0/1724523567 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcc880a2b90 0x7fcc880a5050 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcc940cd660 tx=0x7fcc94013700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.812+0000 7fcca6193740 1 --2- 172.21.15.12:0/2366696833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 0x5651c4dc1420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009900 tx=0x7fcc98005dd0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.812+0000 7fcca6193740 1 -- 172.21.15.12:0/2366696833 shutdown_connections 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.812+0000 7fcca6193740 1 --2- 172.21.15.12:0/2366696833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 0x5651c4dc3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.812+0000 7fcca6193740 1 --2- 172.21.15.12:0/2366696833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 0x5651c4dc1420 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.812+0000 7fcca6193740 1 --2- 172.21.15.12:0/2366696833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dbc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.812+0000 7fcca6193740 1 -- 172.21.15.12:0/2366696833 >> 172.21.15.12:0/2366696833 conn(0x5651c4db75a0 msgr2=0x5651c4db79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca6193740 1 -- 172.21.15.12:0/2366696833 shutdown_connections 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca6193740 1 -- 172.21.15.12:0/2366696833 wait complete. 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca6193740 1 -- start start 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca6193740 1 Processor -- start 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca6193740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dc62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca6193740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 0x5651c4dc6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca6193740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 0x5651c4dcd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca6193740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5651c4dc49e0 con 0x5651c4dbc920 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca6193740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5651c4cdbb90 con 0x5651c4dc1a90 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca6193740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5651c4dc4b60 con 0x5651c4dbbf30 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca6193740 10 client.0 _ll_register_callbacks cb 0x5651c4d17da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca113e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 0x5651c4dc6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca113e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 0x5651c4dc6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47274/0 (socket says 172.21.15.12:47274) 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca113e640 1 -- 172.21.15.12:0/3228307476 learned_addr learned my addr 172.21.15.12:0/3228307476 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.813+0000 7fcca093d640 1 --2- 172.21.15.12:0/3228307476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dc62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcca093d640 1 --2- 172.21.15.12:0/3228307476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 0x5651c4dcd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcca113e640 1 -- 172.21.15.12:0/3228307476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 msgr2=0x5651c4dc62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcca113e640 1 --2- 172.21.15.12:0/3228307476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dc62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcca113e640 1 -- 172.21.15.12:0/3228307476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 msgr2=0x5651c4dcd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcca113e640 1 --2- 172.21.15.12:0/3228307476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 0x5651c4dcd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcca113e640 1 -- 172.21.15.12:0/3228307476 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651c4a343f0 con 0x5651c4dbc920 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcca113e640 1 --2- 172.21.15.12:0/3228307476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 0x5651c4dc6830 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcc98005c40 tx=0x7fcc980056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7f2efbdbcf4b39 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcc84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcc84ff9640 1 -- 172.21.15.12:0/3228307476 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98040030 con 0x5651c4dbc920 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcc84ff9640 1 -- 172.21.15.12:0/3228307476 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fcc98031610 con 0x5651c4dbc920 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcca6193740 1 -- 172.21.15.12:0/3228307476 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc98004020 con 0x5651c4dbc920 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcc84ff9640 1 -- 172.21.15.12:0/3228307476 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fcc98056050 con 0x5651c4dbc920 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcca6193740 10 client.16833 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcca6193740 1 -- 172.21.15.12:0/3228307476 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5651c49f40f0 con 0x5651c4dbc920 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.814+0000 7fcc84ff9640 1 -- 172.21.15.12:0/3228307476 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcc98004020 con 0x5651c4dbc920 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.815+0000 7fcc84ff9640 1 -- 172.21.15.12:0/3228307476 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fcc98004780 con 0x5651c4dbc920 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.815+0000 7fcca6193740 10 client.16833 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.815+0000 7fcca6193740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.815+0000 7fcca6193740 20 client.16833 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.815+0000 7fcca6193740 1 --2- 172.21.15.12:0/3228307476 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5651c4dd1db0 0x5651c4dd41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.815+0000 7fcca6193740 4 client.16833 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.815+0000 7fcca6193740 1 -- 172.21.15.12:0/3228307476 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651c4d2e430 con 0x5651c4dd1db0 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.815+0000 7fcca093d640 1 --2- 172.21.15.12:0/3228307476 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5651c4dd1db0 0x5651c4dd41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.815+0000 7fcc84ff9640 10 client.16833 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.815+0000 7fcca093d640 1 --2- 172.21.15.12:0/3228307476 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5651c4dd1db0 0x5651c4dd41a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1f20d136cabd7eb server_cookie=ac0b6d0b941d257d in_seq=0 out_seq=0 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcc84ff9640 1 -- 172.21.15.12:0/3228307476 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651c4d2e430 con 0x5651c4dd1db0 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcc84ff9640 10 client.16833 handle_command_reply: tid=0 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcca6193740 1 client.16833 shutdown 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcca6193740 1 -- 172.21.15.12:0/3228307476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 msgr2=0x5651c4dc6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcca6193740 1 --2- 172.21.15.12:0/3228307476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 0x5651c4dc6830 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcc98005c40 tx=0x7fcc980056d0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcca6193740 1 -- 172.21.15.12:0/3228307476 shutdown_connections 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcca6193740 1 --2- 172.21.15.12:0/3228307476 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5651c4dd1db0 0x5651c4dd41a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcca6193740 1 --2- 172.21.15.12:0/3228307476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 0x5651c4dcd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcca6193740 1 --2- 172.21.15.12:0/3228307476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbc920 0x5651c4dc6830 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcc84ff9640 0 client.16833 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:56.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcca6193740 1 --2- 172.21.15.12:0/3228307476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dc62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcca6193740 1 -- 172.21.15.12:0/3228307476 >> 172.21.15.12:0/3228307476 conn(0x5651c4db75a0 msgr2=0x5651c4db67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:56.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcca6193740 1 -- 172.21.15.12:0/3228307476 shutdown_connections 2024-05-09T09:28:56.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.816+0000 7fcca6193740 1 -- 172.21.15.12:0/3228307476 wait complete. 2024-05-09T09:28:56.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.817+0000 7fcca6193740 20 client.16833 trim_cache size 0 max 16384 2024-05-09T09:28:56.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.835+0000 7fcca6193740 1 Processor -- start 2024-05-09T09:28:56.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.835+0000 7fcca6193740 1 -- start start 2024-05-09T09:28:56.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.835+0000 7fcca6193740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbbf30 0x5651c4dc62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.835+0000 7fcca6193740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dbc920 0x5651c4dc6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.835+0000 7fcca6193740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dc1a90 0x5651c4dcd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.835+0000 7fcca6193740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1ac7d0 con 0x5651c4dbbf30 2024-05-09T09:28:56.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.835+0000 7fcca6193740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1ae550 con 0x5651c4dbc920 2024-05-09T09:28:56.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.835+0000 7fcca6193740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1acf60 con 0x5651c4dc1a90 2024-05-09T09:28:56.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.835+0000 7fcca093d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbbf30 0x5651c4dc62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.835+0000 7fcca113e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dbc920 0x5651c4dc6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.835+0000 7fcca093d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbbf30 0x5651c4dc62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47288/0 (socket says 172.21.15.12:47288) 2024-05-09T09:28:56.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.835+0000 7fcca093d640 1 -- 172.21.15.12:0/1127843836 learned_addr learned my addr 172.21.15.12:0/1127843836 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:56.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.836+0000 7fcca093d640 1 --2- 172.21.15.12:0/1127843836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dc1a90 0x5651c4dcd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.836+0000 7fcca113e640 1 -- 172.21.15.12:0/1127843836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dc1a90 msgr2=0x5651c4dcd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:56.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.836+0000 7fcca113e640 1 --2- 172.21.15.12:0/1127843836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dc1a90 0x5651c4dcd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.836+0000 7fcca113e640 1 -- 172.21.15.12:0/1127843836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbbf30 msgr2=0x5651c4dc62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:56.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.836+0000 7fcca113e640 1 --2- 172.21.15.12:0/1127843836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbbf30 0x5651c4dc62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.836+0000 7fcca113e640 1 -- 172.21.15.12:0/1127843836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651c4db70a0 con 0x5651c4dbc920 2024-05-09T09:28:56.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.836+0000 7fcca113e640 1 --2- 172.21.15.12:0/1127843836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dbc920 0x5651c4dc6830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcc98004bf0 tx=0x7fcc98009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce801ee340e0a28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:56.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.836+0000 7fcc86ffd640 1 -- 172.21.15.12:0/1127843836 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98040040 con 0x5651c4dbc920 2024-05-09T09:28:56.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.836+0000 7fcc86ffd640 1 -- 172.21.15.12:0/1127843836 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcc9804d070 con 0x5651c4dbc920 2024-05-09T09:28:56.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.836+0000 7fcc86ffd640 1 -- 172.21.15.12:0/1127843836 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98035040 con 0x5651c4dbc920 2024-05-09T09:28:56.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.837+0000 7fcca6193740 1 -- 172.21.15.12:0/1127843836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dbc920 msgr2=0x5651c4dc6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:56.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.837+0000 7fcca6193740 1 --2- 172.21.15.12:0/1127843836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dbc920 0x5651c4dc6830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcc98004bf0 tx=0x7fcc98009780 comp rx=0 tx=0).stop 2024-05-09T09:28:56.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.838+0000 7fcca6193740 1 -- 172.21.15.12:0/1127843836 shutdown_connections 2024-05-09T09:28:56.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.838+0000 7fcca6193740 1 --2- 172.21.15.12:0/1127843836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dc1a90 0x5651c4dcd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.838+0000 7fcca6193740 1 --2- 172.21.15.12:0/1127843836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dbc920 0x5651c4dc6830 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcc98004bf0 tx=0x7fcc98009780 comp rx=0 tx=0).stop 2024-05-09T09:28:56.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.838+0000 7fcca6193740 1 --2- 172.21.15.12:0/1127843836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dbbf30 0x5651c4dc62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.838+0000 7fcca6193740 1 -- 172.21.15.12:0/1127843836 >> 172.21.15.12:0/1127843836 conn(0x5651c4db75a0 msgr2=0x5651c4db7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:56.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.838+0000 7fcca6193740 1 -- 172.21.15.12:0/1127843836 shutdown_connections 2024-05-09T09:28:56.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.838+0000 7fcca6193740 1 -- 172.21.15.12:0/1127843836 wait complete. 2024-05-09T09:28:56.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.839+0000 7fcca6193740 1 -- start start 2024-05-09T09:28:56.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.839+0000 7fcca6193740 1 Processor -- start 2024-05-09T09:28:56.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.839+0000 7fcca6193740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dcacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.839+0000 7fcca6193740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 0x5651c4dcb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.839+0000 7fcca6193740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dcc1d0 0x5651c4dcc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.840+0000 7fcca6193740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1ade80 con 0x5651c4dcc1d0 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.840+0000 7fcca6193740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1ad6f0 con 0x5651c4dc1a90 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.840+0000 7fcca6193740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1aa7b0 con 0x5651c4dbbf30 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.840+0000 7fcca6193740 10 client.0 _ll_register_callbacks cb 0x7fcc8c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.840+0000 7fcca093d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dcacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.840+0000 7fcca093d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dcc1d0 0x5651c4dcc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.840+0000 7fcca093d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dcacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:57666/0 (socket says 172.21.15.12:57666) 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.840+0000 7fcca093d640 1 -- 172.21.15.12:0/4172901135 learned_addr learned my addr 172.21.15.12:0/4172901135 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.841+0000 7fcca093d640 1 -- 172.21.15.12:0/4172901135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 msgr2=0x5651c4dcacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.841+0000 7fcca093d640 1 --2- 172.21.15.12:0/4172901135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dcacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.841+0000 7fcca093d640 1 -- 172.21.15.12:0/4172901135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 msgr2=0x5651c4dcb230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:28:56.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.841+0000 7fcca093d640 1 --2- 172.21.15.12:0/4172901135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 0x5651c4dcb230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.841+0000 7fcca093d640 1 -- 172.21.15.12:0/4172901135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651c4dccb90 con 0x5651c4dcc1d0 2024-05-09T09:28:56.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.841+0000 7fcca093d640 1 --2- 172.21.15.12:0/4172901135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dcc1d0 0x5651c4dcc650 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x5651c4dbd180 tx=0x7fcc980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bb15a34428acc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:56.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.842+0000 7fcc84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:56.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.842+0000 7fcc84ff9640 1 -- 172.21.15.12:0/4172901135 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c01c040 con 0x5651c4dcc1d0 2024-05-09T09:28:56.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.842+0000 7fcc84ff9640 1 -- 172.21.15.12:0/4172901135 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fcc8c009040 con 0x5651c4dcc1d0 2024-05-09T09:28:56.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.842+0000 7fcca6193740 1 -- 172.21.15.12:0/4172901135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9804d070 con 0x5651c4dcc1d0 2024-05-09T09:28:56.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.843+0000 7fcc84ff9640 1 -- 172.21.15.12:0/4172901135 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fcc8c03a050 con 0x5651c4dcc1d0 2024-05-09T09:28:56.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.843+0000 7fcca6193740 10 client.16839 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:56.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.843+0000 7fcca6193740 1 -- 172.21.15.12:0/4172901135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5651c4e38420 con 0x5651c4dcc1d0 2024-05-09T09:28:56.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.844+0000 7fcc84ff9640 1 -- 172.21.15.12:0/4172901135 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcc9804d070 con 0x5651c4dcc1d0 2024-05-09T09:28:56.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.844+0000 7fcc84ff9640 1 -- 172.21.15.12:0/4172901135 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fcc8c022040 con 0x5651c4dcc1d0 2024-05-09T09:28:56.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.844+0000 7fcca6193740 10 client.16839 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:56.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.844+0000 7fcca6193740 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:56.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.844+0000 7fcca6193740 20 client.16839 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:56.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.845+0000 7fcca6193740 1 --2- 172.21.15.12:0/4172901135 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5651c4e38e70 0x5651c4e3b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:56.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.845+0000 7fcca6193740 4 client.16839 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:28:56.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.845+0000 7fcca6193740 1 -- 172.21.15.12:0/4172901135 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651c4d28780 con 0x5651c4e38e70 2024-05-09T09:28:56.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.845+0000 7fcca113e640 1 --2- 172.21.15.12:0/4172901135 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5651c4e38e70 0x5651c4e3b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:56.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.846+0000 7fcca113e640 1 --2- 172.21.15.12:0/4172901135 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5651c4e38e70 0x5651c4e3b290 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5bda95a567faf65 server_cookie=b09ce98b58e0e8e in_seq=0 out_seq=0 2024-05-09T09:28:56.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.846+0000 7fcc84ff9640 10 client.16839 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:56.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.848+0000 7fcc84ff9640 1 -- 172.21.15.12:0/4172901135 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5651c4d28780 con 0x5651c4e38e70 2024-05-09T09:28:56.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.848+0000 7fcc84ff9640 10 client.16839 handle_command_reply: tid=0 2024-05-09T09:28:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.848+0000 7fcca6193740 1 client.16839 shutdown 2024-05-09T09:28:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.848+0000 7fcca6193740 1 -- 172.21.15.12:0/4172901135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dcc1d0 msgr2=0x5651c4dcc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.848+0000 7fcca6193740 1 --2- 172.21.15.12:0/4172901135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dcc1d0 0x5651c4dcc650 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x5651c4dbd180 tx=0x7fcc980046c0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.848+0000 7fcca6193740 1 -- 172.21.15.12:0/4172901135 shutdown_connections 2024-05-09T09:28:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.848+0000 7fcca6193740 1 --2- 172.21.15.12:0/4172901135 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5651c4e38e70 0x5651c4e3b290 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.848+0000 7fcca6193740 1 --2- 172.21.15.12:0/4172901135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5651c4dcc1d0 0x5651c4dcc650 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.848+0000 7fcca6193740 1 --2- 172.21.15.12:0/4172901135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651c4dc1a90 0x5651c4dcb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.848+0000 7fcca6193740 1 --2- 172.21.15.12:0/4172901135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5651c4dbbf30 0x5651c4dcacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.848+0000 7fcca6193740 1 -- 172.21.15.12:0/4172901135 >> 172.21.15.12:0/4172901135 conn(0x5651c4db75a0 msgr2=0x5651c4dbdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.848+0000 7fcc84ff9640 0 client.16839 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.849+0000 7fcca6193740 1 -- 172.21.15.12:0/4172901135 shutdown_connections 2024-05-09T09:28:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.849+0000 7fcca6193740 1 -- 172.21.15.12:0/4172901135 wait complete. 2024-05-09T09:28:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.849+0000 7fcca6193740 20 client.16839 trim_cache size 0 max 16384 2024-05-09T09:28:56.850 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:28:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T09:28:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T09:28:56.850 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:28:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.850+0000 7fcca33a2640 1 -- 172.21.15.12:0/1724523567 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcc880a2b90 msgr2=0x7fcc880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.850+0000 7fcca33a2640 1 --2- 172.21.15.12:0/1724523567 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcc880a2b90 0x7fcc880a5050 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcc940cd660 tx=0x7fcc94013700 comp rx=0 tx=0).stop 2024-05-09T09:28:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.850+0000 7fcca33a2640 1 -- 172.21.15.12:0/1724523567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc9c102320 msgr2=0x7fcc9c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.850+0000 7fcca33a2640 1 --2- 172.21.15.12:0/1724523567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc9c102320 0x7fcc9c103280 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009fd0 tx=0x7fcc9400ef10 comp rx=0 tx=0).stop 2024-05-09T09:28:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.850+0000 7fcca33a2640 1 -- 172.21.15.12:0/1724523567 shutdown_connections 2024-05-09T09:28:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.850+0000 7fcca33a2640 1 --2- 172.21.15.12:0/1724523567 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcc880a2b90 0x7fcc880a5050 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.850+0000 7fcca33a2640 1 --2- 172.21.15.12:0/1724523567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc9c102320 0x7fcc9c103280 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.850+0000 7fcca33a2640 1 --2- 172.21.15.12:0/1724523567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc9c101960 0x7fcc9c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.850+0000 7fcca33a2640 1 --2- 172.21.15.12:0/1724523567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc9c100770 0x7fcc9c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.850+0000 7fcca33a2640 1 -- 172.21.15.12:0/1724523567 >> 172.21.15.12:0/1724523567 conn(0x7fcc9c0fbce0 msgr2=0x7fcc9c0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.850+0000 7fcca33a2640 1 -- 172.21.15.12:0/1724523567 shutdown_connections 2024-05-09T09:28:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:56.850+0000 7fcca33a2640 1 -- 172.21.15.12:0/1724523567 wait complete. 2024-05-09T09:28:56.862 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:28:56.862 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:28:57.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.009+0000 7f8b41076640 1 -- 172.21.15.12:0/132977521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3c0ffab0 msgr2=0x7f8b3c0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.009+0000 7f8b41076640 1 --2- 172.21.15.12:0/132977521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3c0ffab0 0x7f8b3c0ffed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c008570 tx=0x7f8b2c02e390 comp rx=0 tx=0).stop 2024-05-09T09:28:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.009+0000 7f8b41076640 1 -- 172.21.15.12:0/132977521 shutdown_connections 2024-05-09T09:28:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.009+0000 7f8b41076640 1 --2- 172.21.15.12:0/132977521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b3c101660 0x7f8b3c107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.009+0000 7f8b41076640 1 --2- 172.21.15.12:0/132977521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b3c100ca0 0x7f8b3c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.009+0000 7f8b41076640 1 --2- 172.21.15.12:0/132977521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3c0ffab0 0x7f8b3c0ffed0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.009+0000 7f8b41076640 1 -- 172.21.15.12:0/132977521 >> 172.21.15.12:0/132977521 conn(0x7f8b3c0fb020 msgr2=0x7f8b3c0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.011+0000 7f8b41076640 1 -- 172.21.15.12:0/132977521 shutdown_connections 2024-05-09T09:28:57.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.012+0000 7f8b41076640 1 -- 172.21.15.12:0/132977521 wait complete. 2024-05-09T09:28:57.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.013+0000 7f8b41076640 1 Processor -- start 2024-05-09T09:28:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.013+0000 7f8b41076640 1 -- start start 2024-05-09T09:28:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.013+0000 7f8b41076640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b3c0ffab0 0x7f8b3c071480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.013+0000 7f8b41076640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3c100ca0 0x7f8b3c0719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.013+0000 7f8b41076640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b3c101660 0x7f8b3c072a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.013+0000 7f8b41076640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c06bc90 con 0x7f8b3c0ffab0 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.013+0000 7f8b41076640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c06bb10 con 0x7f8b3c101660 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.013+0000 7f8b41076640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c06be10 con 0x7f8b3c100ca0 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.014+0000 7f8b3b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b3c101660 0x7f8b3c072a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.014+0000 7f8b3b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b3c101660 0x7f8b3c072a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:59038/0 (socket says 172.21.15.12:59038) 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.014+0000 7f8b3b59e640 1 -- 172.21.15.12:0/3960596527 learned_addr learned my addr 172.21.15.12:0/3960596527 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.014+0000 7f8b3a59c640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3c100ca0 0x7f8b3c0719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.014+0000 7f8b3ad9d640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b3c0ffab0 0x7f8b3c071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.014+0000 7f8b3b59e640 1 -- 172.21.15.12:0/3960596527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3c100ca0 msgr2=0x7f8b3c0719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.014+0000 7f8b3b59e640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3c100ca0 0x7f8b3c0719c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.014+0000 7f8b3b59e640 1 -- 172.21.15.12:0/3960596527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b3c0ffab0 msgr2=0x7f8b3c071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.014+0000 7f8b3b59e640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b3c0ffab0 0x7f8b3c071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.014+0000 7f8b3b59e640 1 -- 172.21.15.12:0/3960596527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c073070 con 0x7f8b3c101660 2024-05-09T09:28:57.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.015+0000 7f8b3b59e640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b3c101660 0x7f8b3c072a00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000da60 tx=0x7f8b3000df20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.015+0000 7f8b23fff640 1 -- 172.21.15.12:0/3960596527 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30008ba0 con 0x7f8b3c101660 2024-05-09T09:28:57.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.016+0000 7f8b23fff640 1 -- 172.21.15.12:0/3960596527 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8b30008d40 con 0x7f8b3c101660 2024-05-09T09:28:57.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.016+0000 7f8b41076640 1 -- 172.21.15.12:0/3960596527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3c073360 con 0x7f8b3c101660 2024-05-09T09:28:57.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.016+0000 7f8b41076640 1 -- 172.21.15.12:0/3960596527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b3c1aeeb0 con 0x7f8b3c101660 2024-05-09T09:28:57.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.016+0000 7f8b23fff640 1 -- 172.21.15.12:0/3960596527 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30004dc0 con 0x7f8b3c101660 2024-05-09T09:28:57.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.017+0000 7f8b41076640 1 -- 172.21.15.12:0/3960596527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b28005290 con 0x7f8b3c101660 2024-05-09T09:28:57.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.020+0000 7f8b23fff640 1 -- 172.21.15.12:0/3960596527 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8b300046c0 con 0x7f8b3c101660 2024-05-09T09:28:57.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.021+0000 7f8b23fff640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8b240a2ce0 0x7f8b240a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.021+0000 7f8b23fff640 1 -- 172.21.15.12:0/3960596527 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8b300cefc0 con 0x7f8b3c101660 2024-05-09T09:28:57.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.022+0000 7f8b3ad9d640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8b240a2ce0 0x7f8b240a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.023+0000 7f8b3ad9d640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8b240a2ce0 0x7f8b240a51a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c008330 tx=0x7f8b2c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.027+0000 7f8b23fff640 1 -- 172.21.15.12:0/3960596527 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8b3009baf0 con 0x7f8b3c101660 2024-05-09T09:28:57.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.427+0000 7f8b41076640 1 -- 172.21.15.12:0/3960596527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b28005580 con 0x7f8b3c101660 2024-05-09T09:28:57.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.428+0000 7f8b23fff640 1 -- 172.21.15.12:0/3960596527 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11966 (secure 0 0 0) 0x7f8b3009b3f0 con 0x7f8b3c101660 2024-05-09T09:28:57.428 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:28:57.428 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:28:24.476138+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25883":{"gid":25883,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3067829157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3067829157},{"type":"v1","addr":"172.21.15.16:6837","nonce":3067829157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[24502,24523,24515,14628,24484]},"id":1}]} 2024-05-09T09:28:57.429 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 42 2024-05-09T09:28:57.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.431+0000 7f8b41076640 1 -- 172.21.15.12:0/3960596527 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8b240a2ce0 msgr2=0x7f8b240a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.431+0000 7f8b41076640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8b240a2ce0 0x7f8b240a51a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c008330 tx=0x7f8b2c000bc0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.431+0000 7f8b41076640 1 -- 172.21.15.12:0/3960596527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b3c101660 msgr2=0x7f8b3c072a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.431+0000 7f8b41076640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b3c101660 0x7f8b3c072a00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000da60 tx=0x7f8b3000df20 comp rx=0 tx=0).stop 2024-05-09T09:28:57.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.431+0000 7f8b41076640 1 -- 172.21.15.12:0/3960596527 shutdown_connections 2024-05-09T09:28:57.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.431+0000 7f8b41076640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8b240a2ce0 0x7f8b240a51a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.431+0000 7f8b41076640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b3c101660 0x7f8b3c072a00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.431+0000 7f8b41076640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3c100ca0 0x7f8b3c0719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.431+0000 7f8b41076640 1 --2- 172.21.15.12:0/3960596527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b3c0ffab0 0x7f8b3c071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.431+0000 7f8b41076640 1 -- 172.21.15.12:0/3960596527 >> 172.21.15.12:0/3960596527 conn(0x7f8b3c0fb020 msgr2=0x7f8b3c103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.431+0000 7f8b41076640 1 -- 172.21.15.12:0/3960596527 shutdown_connections 2024-05-09T09:28:57.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.431+0000 7f8b41076640 1 -- 172.21.15.12:0/3960596527 wait complete. 2024-05-09T09:28:57.443 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-09T09:28:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.560+0000 7f177f831640 1 -- 172.21.15.12:0/3932958981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1778100770 msgr2=0x7f1778100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.560+0000 7f177f831640 1 --2- 172.21.15.12:0/3932958981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1778100770 0x7f1778100b90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1774009900 tx=0x7f177402a410 comp rx=0 tx=0).stop 2024-05-09T09:28:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.560+0000 7f177f831640 1 -- 172.21.15.12:0/3932958981 shutdown_connections 2024-05-09T09:28:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.560+0000 7f177f831640 1 --2- 172.21.15.12:0/3932958981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1778102320 0x7f1778108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.560+0000 7f177f831640 1 --2- 172.21.15.12:0/3932958981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1778101960 0x7f1778101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.560+0000 7f177f831640 1 --2- 172.21.15.12:0/3932958981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1778100770 0x7f1778100b90 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.560+0000 7f177f831640 1 -- 172.21.15.12:0/3932958981 >> 172.21.15.12:0/3932958981 conn(0x7f17780fbce0 msgr2=0x7f17780fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.560+0000 7f177f831640 1 -- 172.21.15.12:0/3932958981 shutdown_connections 2024-05-09T09:28:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.560+0000 7f177f831640 1 -- 172.21.15.12:0/3932958981 wait complete. 2024-05-09T09:28:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.561+0000 7f177f831640 1 Processor -- start 2024-05-09T09:28:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.561+0000 7f177f831640 1 -- start start 2024-05-09T09:28:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.561+0000 7f177f831640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1778100770 0x7f1778199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.561+0000 7f177f831640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1778101960 0x7f177819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.561+0000 7f177f831640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1778102320 0x7f17781a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.561+0000 7f177f831640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f177810c3c0 con 0x7f1778102320 2024-05-09T09:28:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.561+0000 7f177f831640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f177810c240 con 0x7f1778100770 2024-05-09T09:28:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.561+0000 7f177f831640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f177810c540 con 0x7f1778101960 2024-05-09T09:28:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.561+0000 7f177ddce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1778102320 0x7f17781a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.561+0000 7f177ddce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1778102320 0x7f17781a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47332/0 (socket says 172.21.15.12:47332) 2024-05-09T09:28:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.561+0000 7f177ddce640 1 -- 172.21.15.12:0/2138750426 learned_addr learned my addr 172.21.15.12:0/2138750426 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.561+0000 7f177cdcc640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1778101960 0x7f177819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.561+0000 7f177d5cd640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1778100770 0x7f1778199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.562+0000 7f177ddce640 1 -- 172.21.15.12:0/2138750426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1778101960 msgr2=0x7f177819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.562+0000 7f177ddce640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1778101960 0x7f177819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.562+0000 7f177ddce640 1 -- 172.21.15.12:0/2138750426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1778100770 msgr2=0x7f1778199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.562+0000 7f177ddce640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1778100770 0x7f1778199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.562+0000 7f177ddce640 1 -- 172.21.15.12:0/2138750426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17781a1a30 con 0x7f1778102320 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.562+0000 7f177cdcc640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1778101960 0x7f177819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.562+0000 7f177ddce640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1778102320 0x7f17781a13c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f177000efd0 tx=0x7f177000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.562+0000 7f176e7fc640 1 -- 172.21.15.12:0/2138750426 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770003b70 con 0x7f1778102320 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.562+0000 7f177f831640 1 -- 172.21.15.12:0/2138750426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17781a1d20 con 0x7f1778102320 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.562+0000 7f176e7fc640 1 -- 172.21.15.12:0/2138750426 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1770003d10 con 0x7f1778102320 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.562+0000 7f176e7fc640 1 -- 172.21.15.12:0/2138750426 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17700109f0 con 0x7f1778102320 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.562+0000 7f177f831640 1 -- 172.21.15.12:0/2138750426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17781a2210 con 0x7f1778102320 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.565+0000 7f176e7fc640 1 -- 172.21.15.12:0/2138750426 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1770004000 con 0x7f1778102320 2024-05-09T09:28:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f1782618740 1 Processor -- start 2024-05-09T09:28:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f1782618740 1 -- start start 2024-05-09T09:28:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f1782618740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e5f40 0x560df29e6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f1782618740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e6930 0x560df29eb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f1782618740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29edf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f1782618740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560df294ac40 con 0x560df29ebaa0 2024-05-09T09:28:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f1782618740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560df28f2580 con 0x560df29e6930 2024-05-09T09:28:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f1782618740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560df2946c00 con 0x560df29e5f40 2024-05-09T09:28:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f177cdcc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e5f40 0x560df29e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f177d5cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e6930 0x560df29eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f177cdcc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e5f40 0x560df29e6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:57704/0 (socket says 172.21.15.12:57704) 2024-05-09T09:28:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f177cdcc640 1 -- 172.21.15.12:0/1542168813 learned_addr learned my addr 172.21.15.12:0/1542168813 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f177cdcc640 1 --2- 172.21.15.12:0/1542168813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29edf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f176e7fc640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17640a2b90 0x7f17640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f177cdcc640 1 -- 172.21.15.12:0/1542168813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e6930 msgr2=0x560df29eb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f177cdcc640 1 --2- 172.21.15.12:0/1542168813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e6930 0x560df29eb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f177cdcc640 1 -- 172.21.15.12:0/1542168813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 msgr2=0x560df29edf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f177cdcc640 1 --2- 172.21.15.12:0/1542168813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29edf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f177cdcc640 1 -- 172.21.15.12:0/1542168813 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df265e3f0 con 0x560df29e5f40 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.566+0000 7f177ddce640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17640a2b90 0x7f17640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.567+0000 7f176e7fc640 1 -- 172.21.15.12:0/2138750426 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f17700cf7f0 con 0x7f1778102320 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.567+0000 7f177cdcc640 1 --2- 172.21.15.12:0/1542168813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e5f40 0x560df29e6360 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1768011be0 tx=0x7f1768009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fda3d35da688097b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.567+0000 7f17637fe640 1 -- 172.21.15.12:0/1542168813 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176800fc50 con 0x560df29e5f40 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.567+0000 7f17637fe640 1 -- 172.21.15.12:0/1542168813 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f176800fdf0 con 0x560df29e5f40 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.567+0000 7f17637fe640 1 -- 172.21.15.12:0/1542168813 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176800e710 con 0x560df29e5f40 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.568+0000 7f1782618740 1 -- 172.21.15.12:0/1542168813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e5f40 msgr2=0x560df29e6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.568+0000 7f177ddce640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17640a2b90 0x7f17640a5050 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f177001b000 tx=0x7f1770013700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.568+0000 7f1782618740 1 --2- 172.21.15.12:0/1542168813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e5f40 0x560df29e6360 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1768011be0 tx=0x7f1768009ba0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.568+0000 7f1782618740 1 -- 172.21.15.12:0/1542168813 shutdown_connections 2024-05-09T09:28:57.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.568+0000 7f1782618740 1 --2- 172.21.15.12:0/1542168813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29edf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.568+0000 7f1782618740 1 --2- 172.21.15.12:0/1542168813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e6930 0x560df29eb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.568+0000 7f1782618740 1 --2- 172.21.15.12:0/1542168813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e5f40 0x560df29e6360 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.568+0000 7f1782618740 1 -- 172.21.15.12:0/1542168813 >> 172.21.15.12:0/1542168813 conn(0x560df29e15b0 msgr2=0x560df29e19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.568+0000 7f1782618740 1 -- 172.21.15.12:0/1542168813 shutdown_connections 2024-05-09T09:28:57.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.568+0000 7f1782618740 1 -- 172.21.15.12:0/1542168813 wait complete. 2024-05-09T09:28:57.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.568+0000 7f1782618740 1 -- start start 2024-05-09T09:28:57.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f1782618740 1 Processor -- start 2024-05-09T09:28:57.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f1782618740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e5f40 0x560df29f0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f1782618740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 0x560df29f08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f1782618740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29f7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f1782618740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560df29eea50 con 0x560df29ebaa0 2024-05-09T09:28:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f1782618740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560df2905c40 con 0x560df29e5f40 2024-05-09T09:28:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f1782618740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560df29eebd0 con 0x560df29e6930 2024-05-09T09:28:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f1782618740 10 client.0 _ll_register_callbacks cb 0x560df265dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f177cdcc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e5f40 0x560df29f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f177cdcc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e5f40 0x560df29f0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:59080/0 (socket says 172.21.15.12:59080) 2024-05-09T09:28:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f177cdcc640 1 -- 172.21.15.12:0/54866170 learned_addr learned my addr 172.21.15.12:0/54866170 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f177cdcc640 1 --2- 172.21.15.12:0/54866170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f177d5cd640 1 --2- 172.21.15.12:0/54866170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 0x560df29f08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f177cdcc640 1 -- 172.21.15.12:0/54866170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 msgr2=0x560df29f08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f177cdcc640 1 --2- 172.21.15.12:0/54866170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 0x560df29f08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f177cdcc640 1 -- 172.21.15.12:0/54866170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 msgr2=0x560df29f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.569+0000 7f177cdcc640 1 --2- 172.21.15.12:0/54866170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.570+0000 7f177cdcc640 1 -- 172.21.15.12:0/54866170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df27287d0 con 0x560df29e5f40 2024-05-09T09:28:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.570+0000 7f177cdcc640 1 --2- 172.21.15.12:0/54866170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e5f40 0x560df29f0390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1768009b70 tx=0x7f17680096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b019dcea25e9a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.570+0000 7f1760ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:28:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.570+0000 7f1760ff9640 1 -- 172.21.15.12:0/54866170 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176801fc10 con 0x560df29e5f40 2024-05-09T09:28:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.570+0000 7f1760ff9640 1 -- 172.21.15.12:0/54866170 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f1768027040 con 0x560df29e5f40 2024-05-09T09:28:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.570+0000 7f1782618740 1 -- 172.21.15.12:0/54866170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f176800fdf0 con 0x560df29e5f40 2024-05-09T09:28:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.570+0000 7f1760ff9640 1 -- 172.21.15.12:0/54866170 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f176802c410 con 0x560df29e5f40 2024-05-09T09:28:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.570+0000 7f1782618740 10 client.26288 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.570+0000 7f1782618740 1 -- 172.21.15.12:0/54866170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560df29fac80 con 0x560df29e5f40 2024-05-09T09:28:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.570+0000 7f1760ff9640 1 -- 172.21.15.12:0/54866170 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f176800fdf0 con 0x560df29e5f40 2024-05-09T09:28:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.571+0000 7f1760ff9640 1 -- 172.21.15.12:0/54866170 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f176801d020 con 0x560df29e5f40 2024-05-09T09:28:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.571+0000 7f1782618740 10 client.26288 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.571+0000 7f1782618740 10 client.26288 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:28:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.571+0000 7f1782618740 20 client.26288 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.571+0000 7f1782618740 1 --2- 172.21.15.12:0/54866170 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x560df29fc0c0 0x560df29fe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.571+0000 7f1782618740 4 client.26288 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.571+0000 7f1782618740 1 -- 172.21.15.12:0/54866170 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560df2958370 con 0x560df29fc0c0 2024-05-09T09:28:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.571+0000 7f177d5cd640 1 --2- 172.21.15.12:0/54866170 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x560df29fc0c0 0x560df29fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.571+0000 7f1760ff9640 10 client.26288 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:28:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.571+0000 7f177d5cd640 1 --2- 172.21.15.12:0/54866170 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x560df29fc0c0 0x560df29fe500 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4eaa6d54f33770dc server_cookie=9fdf4e23fb8b71f0 in_seq=0 out_seq=0 2024-05-09T09:28:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1760ff9640 1 -- 172.21.15.12:0/54866170 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560df2958370 con 0x560df29fc0c0 2024-05-09T09:28:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1760ff9640 10 client.26288 handle_command_reply: tid=0 2024-05-09T09:28:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1782618740 1 client.26288 shutdown 2024-05-09T09:28:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1782618740 1 -- 172.21.15.12:0/54866170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e5f40 msgr2=0x560df29f0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1782618740 1 --2- 172.21.15.12:0/54866170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e5f40 0x560df29f0390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1768009b70 tx=0x7f17680096c0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1782618740 1 -- 172.21.15.12:0/54866170 shutdown_connections 2024-05-09T09:28:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1782618740 1 --2- 172.21.15.12:0/54866170 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x560df29fc0c0 0x560df29fe500 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1782618740 1 --2- 172.21.15.12:0/54866170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1782618740 1 --2- 172.21.15.12:0/54866170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 0x560df29f08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1760ff9640 0 client.26288 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:28:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1782618740 1 --2- 172.21.15.12:0/54866170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e5f40 0x560df29f0390 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1782618740 1 -- 172.21.15.12:0/54866170 >> 172.21.15.12:0/54866170 conn(0x560df29e15b0 msgr2=0x560df29e07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1782618740 1 -- 172.21.15.12:0/54866170 shutdown_connections 2024-05-09T09:28:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1782618740 1 -- 172.21.15.12:0/54866170 wait complete. 2024-05-09T09:28:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.572+0000 7f1782618740 20 client.26288 trim_cache size 0 max 16384 2024-05-09T09:28:57.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.586+0000 7f1782618740 1 Processor -- start 2024-05-09T09:28:57.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.586+0000 7f1782618740 1 -- start start 2024-05-09T09:28:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.586+0000 7f1782618740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e5f40 0x560df29f06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.586+0000 7f1782618740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 0x560df29f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.586+0000 7f1782618740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29dff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f1782618740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f17781ac7d0 con 0x560df29ebaa0 2024-05-09T09:28:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f1782618740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17781ae550 con 0x560df29e5f40 2024-05-09T09:28:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f1782618740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f17781acf60 con 0x560df29e6930 2024-05-09T09:28:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f177cdcc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29dff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f177cdcc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29dff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47358/0 (socket says 172.21.15.12:47358) 2024-05-09T09:28:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f177cdcc640 1 -- 172.21.15.12:0/1083199876 learned_addr learned my addr 172.21.15.12:0/1083199876 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f177d5cd640 1 --2- 172.21.15.12:0/1083199876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 0x560df29f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f177cdcc640 1 --2- 172.21.15.12:0/1083199876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e5f40 0x560df29f06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f177cdcc640 1 -- 172.21.15.12:0/1083199876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 msgr2=0x560df29f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f177cdcc640 1 --2- 172.21.15.12:0/1083199876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 0x560df29f78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f177cdcc640 1 -- 172.21.15.12:0/1083199876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e5f40 msgr2=0x560df29f06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f177cdcc640 1 --2- 172.21.15.12:0/1083199876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e5f40 0x560df29f06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f177cdcc640 1 -- 172.21.15.12:0/1083199876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df29d2d20 con 0x560df29ebaa0 2024-05-09T09:28:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f177cdcc640 1 --2- 172.21.15.12:0/1083199876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29dff20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1768005cc0 tx=0x7f1768005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf06499f0f8109ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f1762ffd640 1 -- 172.21.15.12:0/1083199876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176801fc10 con 0x560df29ebaa0 2024-05-09T09:28:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f1762ffd640 1 -- 172.21.15.12:0/1083199876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1768009e70 con 0x560df29ebaa0 2024-05-09T09:28:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.587+0000 7f1762ffd640 1 -- 172.21.15.12:0/1083199876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176800edd0 con 0x560df29ebaa0 2024-05-09T09:28:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.588+0000 7f1782618740 1 -- 172.21.15.12:0/1083199876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 msgr2=0x560df29dff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.588+0000 7f1782618740 1 --2- 172.21.15.12:0/1083199876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29dff20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1768005cc0 tx=0x7f1768005c50 comp rx=0 tx=0).stop 2024-05-09T09:28:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.588+0000 7f1782618740 1 -- 172.21.15.12:0/1083199876 shutdown_connections 2024-05-09T09:28:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.588+0000 7f1782618740 1 --2- 172.21.15.12:0/1083199876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29ebaa0 0x560df29dff20 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.588+0000 7f1782618740 1 --2- 172.21.15.12:0/1083199876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 0x560df29f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.588+0000 7f1782618740 1 --2- 172.21.15.12:0/1083199876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29e5f40 0x560df29f06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.588+0000 7f1782618740 1 -- 172.21.15.12:0/1083199876 >> 172.21.15.12:0/1083199876 conn(0x560df29e15b0 msgr2=0x560df29effa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.588+0000 7f1782618740 1 -- 172.21.15.12:0/1083199876 shutdown_connections 2024-05-09T09:28:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.589+0000 7f1782618740 1 -- 172.21.15.12:0/1083199876 wait complete. 2024-05-09T09:28:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.589+0000 7f1782618740 1 -- start start 2024-05-09T09:28:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.589+0000 7f1782618740 1 Processor -- start 2024-05-09T09:28:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.589+0000 7f1782618740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29e5f40 0x560df29f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.589+0000 7f1782618740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 0x560df29f5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f1782618740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29ebaa0 0x560df29f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f1782618740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f17781ade80 con 0x560df29e5f40 2024-05-09T09:28:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f1782618740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17781ad6f0 con 0x560df29ebaa0 2024-05-09T09:28:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f1782618740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f17781aa7b0 con 0x560df29e6930 2024-05-09T09:28:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f1782618740 10 client.0 _ll_register_callbacks cb 0x7f17680116b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f177cdcc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29e5f40 0x560df29f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f177cdcc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29e5f40 0x560df29f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47362/0 (socket says 172.21.15.12:47362) 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f177cdcc640 1 -- 172.21.15.12:0/2521257233 learned_addr learned my addr 172.21.15.12:0/2521257233 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f177d5cd640 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 0x560df29f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f177cdcc640 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29ebaa0 0x560df29f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f177d5cd640 1 -- 172.21.15.12:0/2521257233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29ebaa0 msgr2=0x560df29f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f177d5cd640 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29ebaa0 0x560df29f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f177d5cd640 1 -- 172.21.15.12:0/2521257233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29e5f40 msgr2=0x560df29f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f177d5cd640 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29e5f40 0x560df29f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f177d5cd640 1 -- 172.21.15.12:0/2521257233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df29f6890 con 0x560df29e6930 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f177cdcc640 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29e5f40 0x560df29f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f177d5cd640 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 0x560df29f5290 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1774009230 tx=0x7f17740318d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50c1b49377670692 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f1760ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f1760ff9640 1 -- 172.21.15.12:0/2521257233 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774035040 con 0x560df29e6930 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f1760ff9640 1 -- 172.21.15.12:0/2521257233 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f1774004040 con 0x560df29e6930 2024-05-09T09:28:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.590+0000 7f1782618740 1 -- 172.21.15.12:0/2521257233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1768009e70 con 0x560df29e6930 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.591+0000 7f1760ff9640 1 -- 172.21.15.12:0/2521257233 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1774057050 con 0x560df29e6930 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.591+0000 7f1782618740 10 client.26407 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.591+0000 7f1782618740 1 -- 172.21.15.12:0/2521257233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560df29f7c00 con 0x560df29e6930 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.591+0000 7f1760ff9640 1 -- 172.21.15.12:0/2521257233 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1768009e70 con 0x560df29e6930 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.591+0000 7f1760ff9640 1 -- 172.21.15.12:0/2521257233 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f1774046290 con 0x560df29e6930 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.591+0000 7f1782618740 10 client.26407 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.591+0000 7f1782618740 10 client.26407 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.591+0000 7f1782618740 20 client.26407 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.591+0000 7f1782618740 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x560df2a62f40 0x560df2a65360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.591+0000 7f1782618740 4 client.26407 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.591+0000 7f1782618740 1 -- 172.21.15.12:0/2521257233 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560df29526c0 con 0x560df2a62f40 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.591+0000 7f177cdcc640 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x560df2a62f40 0x560df2a65360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.592+0000 7f1760ff9640 10 client.26407 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.592+0000 7f177cdcc640 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x560df2a62f40 0x560df2a65360 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=889bd5cabb74c3b4 server_cookie=d90026bd4de19a5b in_seq=0 out_seq=0 2024-05-09T09:28:57.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.594+0000 7f1760ff9640 1 -- 172.21.15.12:0/2521257233 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560df29526c0 con 0x560df2a62f40 2024-05-09T09:28:57.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.594+0000 7f1760ff9640 10 client.26407 handle_command_reply: tid=0 2024-05-09T09:28:57.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.594+0000 7f1782618740 1 client.26407 shutdown 2024-05-09T09:28:57.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.594+0000 7f1782618740 1 -- 172.21.15.12:0/2521257233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 msgr2=0x560df29f5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.594+0000 7f1782618740 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 0x560df29f5290 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1774009230 tx=0x7f17740318d0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.594+0000 7f1782618740 1 -- 172.21.15.12:0/2521257233 shutdown_connections 2024-05-09T09:28:57.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.594+0000 7f1782618740 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x560df2a62f40 0x560df2a65360 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.594+0000 7f1782618740 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560df29ebaa0 0x560df29f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.594+0000 7f1782618740 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560df29e6930 0x560df29f5290 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.594+0000 7f1760ff9640 0 client.26407 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:28:57.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.594+0000 7f1782618740 1 --2- 172.21.15.12:0/2521257233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560df29e5f40 0x560df29f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.594+0000 7f1782618740 1 -- 172.21.15.12:0/2521257233 >> 172.21.15.12:0/2521257233 conn(0x560df29e15b0 msgr2=0x560df29e4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.594+0000 7f1782618740 1 -- 172.21.15.12:0/2521257233 shutdown_connections 2024-05-09T09:28:57.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f1782618740 1 -- 172.21.15.12:0/2521257233 wait complete. 2024-05-09T09:28:57.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f1782618740 20 client.26407 trim_cache size 0 max 16384 2024-05-09T09:28:57.599 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:28:57.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f177f831640 1 -- 172.21.15.12:0/2138750426 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17640a2b90 msgr2=0x7f17640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f177f831640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17640a2b90 0x7f17640a5050 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f177001b000 tx=0x7f1770013700 comp rx=0 tx=0).stop 2024-05-09T09:28:57.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f177f831640 1 -- 172.21.15.12:0/2138750426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1778102320 msgr2=0x7f17781a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f177f831640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1778102320 0x7f17781a13c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f177000efd0 tx=0x7f177000c6a0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f177f831640 1 -- 172.21.15.12:0/2138750426 shutdown_connections 2024-05-09T09:28:57.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f177f831640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f17640a2b90 0x7f17640a5050 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f177f831640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1778102320 0x7f17781a13c0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f177f831640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1778101960 0x7f177819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f177f831640 1 --2- 172.21.15.12:0/2138750426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1778100770 0x7f1778199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f177f831640 1 -- 172.21.15.12:0/2138750426 >> 172.21.15.12:0/2138750426 conn(0x7f17780fbce0 msgr2=0x7f17780689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f177f831640 1 -- 172.21.15.12:0/2138750426 shutdown_connections 2024-05-09T09:28:57.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.595+0000 7f177f831640 1 -- 172.21.15.12:0/2138750426 wait complete. 2024-05-09T09:28:57.609 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-09T09:28:57.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.779+0000 7f99a39a5640 1 -- 172.21.15.12:0/3896027966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c068550 msgr2=0x7f999c1003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.779+0000 7f99a39a5640 1 --2- 172.21.15.12:0/3896027966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c068550 0x7f999c1003e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9994009be0 tx=0x7f999402a6c0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.779+0000 7f99a39a5640 1 -- 172.21.15.12:0/3896027966 shutdown_connections 2024-05-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.779+0000 7f99a39a5640 1 --2- 172.21.15.12:0/3896027966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c101080 0x7f999c101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.779+0000 7f99a39a5640 1 --2- 172.21.15.12:0/3896027966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c068550 0x7f999c1003e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.779+0000 7f99a39a5640 1 --2- 172.21.15.12:0/3896027966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f999c068130 0x7f999c0ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.779+0000 7f99a39a5640 1 -- 172.21.15.12:0/3896027966 >> 172.21.15.12:0/3896027966 conn(0x7f999c0fb460 msgr2=0x7f999c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.779+0000 7f99a39a5640 1 -- 172.21.15.12:0/3896027966 shutdown_connections 2024-05-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.779+0000 7f99a39a5640 1 -- 172.21.15.12:0/3896027966 wait complete. 2024-05-09T09:28:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.780+0000 7f99a39a5640 1 Processor -- start 2024-05-09T09:28:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.780+0000 7f99a39a5640 1 -- start start 2024-05-09T09:28:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.780+0000 7f99a39a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c068130 0x7f999c070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.780+0000 7f99a39a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c068550 0x7f999c0707b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.780+0000 7f99a39a5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f999c101080 0x7f999c1a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.780+0000 7f99a39a5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f999c10b0b0 con 0x7f999c101080 2024-05-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.780+0000 7f99a39a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f999c10af30 con 0x7f999c068130 2024-05-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.780+0000 7f99a39a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f999c10b230 con 0x7f999c068550 2024-05-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.781+0000 7f99a1f42640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f999c101080 0x7f999c1a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.781+0000 7f99a1f42640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f999c101080 0x7f999c1a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47382/0 (socket says 172.21.15.12:47382) 2024-05-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.781+0000 7f99a1f42640 1 -- 172.21.15.12:0/3429535876 learned_addr learned my addr 172.21.15.12:0/3429535876 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.781+0000 7f99a0f40640 1 --2- 172.21.15.12:0/3429535876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c068550 0x7f999c0707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.781+0000 7f99a1f42640 1 -- 172.21.15.12:0/3429535876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c068550 msgr2=0x7f999c0707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.781+0000 7f99a1f42640 1 --2- 172.21.15.12:0/3429535876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c068550 0x7f999c0707b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.781+0000 7f99a1f42640 1 -- 172.21.15.12:0/3429535876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c068130 msgr2=0x7f999c070270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.781+0000 7f99a1f42640 1 --2- 172.21.15.12:0/3429535876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c068130 0x7f999c070270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.781+0000 7f99a1f42640 1 -- 172.21.15.12:0/3429535876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c1aa330 con 0x7f999c101080 2024-05-09T09:28:57.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.782+0000 7f99a1f42640 1 --2- 172.21.15.12:0/3429535876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f999c101080 0x7f999c1a9df0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f999800e970 tx=0x7f999800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.782+0000 7f99927fc640 1 -- 172.21.15.12:0/3429535876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999800ccc0 con 0x7f999c101080 2024-05-09T09:28:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.783+0000 7f99a39a5640 1 -- 172.21.15.12:0/3429535876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c1aa500 con 0x7f999c101080 2024-05-09T09:28:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.783+0000 7f99a39a5640 1 -- 172.21.15.12:0/3429535876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c1aaa50 con 0x7f999c101080 2024-05-09T09:28:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.783+0000 7f99927fc640 1 -- 172.21.15.12:0/3429535876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f999800ce60 con 0x7f999c101080 2024-05-09T09:28:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.783+0000 7f99927fc640 1 -- 172.21.15.12:0/3429535876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998010430 con 0x7f999c101080 2024-05-09T09:28:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.787+0000 7f99a678c740 1 Processor -- start 2024-05-09T09:28:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.789+0000 7f99a678c740 1 -- start start 2024-05-09T09:28:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.789+0000 7f99a678c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b1750 0x55fe176b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.789+0000 7f99a678c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b2140 0x55fe176b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.789+0000 7f99a678c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b72b0 0x55fe176b9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.789+0000 7f99a678c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fe1760dc40 con 0x55fe176b2140 2024-05-09T09:28:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.789+0000 7f99a678c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe175b5580 con 0x55fe176b72b0 2024-05-09T09:28:57.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.789+0000 7f99a678c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fe17609c00 con 0x55fe176b1750 2024-05-09T09:28:57.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.789+0000 7f99927fc640 1 -- 172.21.15.12:0/3429535876 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f99980046c0 con 0x7f999c101080 2024-05-09T09:28:57.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.790+0000 7f99a1741640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b2140 0x55fe176b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.790+0000 7f99a1741640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b2140 0x55fe176b6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47388/0 (socket says 172.21.15.12:47388) 2024-05-09T09:28:57.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.790+0000 7f99a1741640 1 -- 172.21.15.12:0/2952355341 learned_addr learned my addr 172.21.15.12:0/2952355341 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:57.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.790+0000 7f99a1741640 1 -- 172.21.15.12:0/2952355341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b1750 msgr2=0x55fe176b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.790+0000 7f99a1741640 1 --2- 172.21.15.12:0/2952355341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b1750 0x55fe176b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.790+0000 7f99a1741640 1 -- 172.21.15.12:0/2952355341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b72b0 msgr2=0x55fe176b9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.790+0000 7f99a1741640 1 --2- 172.21.15.12:0/2952355341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b72b0 0x55fe176b9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.790+0000 7f99a1741640 1 -- 172.21.15.12:0/2952355341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe173213f0 con 0x55fe176b2140 2024-05-09T09:28:57.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.791+0000 7f99927fc640 1 --2- 172.21.15.12:0/3429535876 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99880a2d60 0x7f99880a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.791+0000 7f99a1741640 1 --2- 172.21.15.12:0/2952355341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b2140 0x55fe176b6c40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f998c009ab0 tx=0x7f998c009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8be6fb4da39ba9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.791+0000 7f99927fc640 1 -- 172.21.15.12:0/3429535876 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f99980cf790 con 0x7f999c101080 2024-05-09T09:28:57.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.791+0000 7f99877fe640 1 -- 172.21.15.12:0/2952355341 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c00ed10 con 0x55fe176b2140 2024-05-09T09:28:57.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.791+0000 7f99877fe640 1 -- 172.21.15.12:0/2952355341 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f998c0054f0 con 0x55fe176b2140 2024-05-09T09:28:57.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.792+0000 7f99877fe640 1 -- 172.21.15.12:0/2952355341 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c010690 con 0x55fe176b2140 2024-05-09T09:28:57.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.792+0000 7f99a0f40640 1 --2- 172.21.15.12:0/3429535876 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99880a2d60 0x7f99880a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.792+0000 7f99a678c740 1 -- 172.21.15.12:0/2952355341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b2140 msgr2=0x55fe176b6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.792+0000 7f99a678c740 1 --2- 172.21.15.12:0/2952355341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b2140 0x55fe176b6c40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f998c009ab0 tx=0x7f998c009f70 comp rx=0 tx=0).stop 2024-05-09T09:28:57.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.793+0000 7f99a678c740 1 -- 172.21.15.12:0/2952355341 shutdown_connections 2024-05-09T09:28:57.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.793+0000 7f99a678c740 1 --2- 172.21.15.12:0/2952355341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b72b0 0x55fe176b9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.793+0000 7f99a678c740 1 --2- 172.21.15.12:0/2952355341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b2140 0x55fe176b6c40 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.793+0000 7f99a678c740 1 --2- 172.21.15.12:0/2952355341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b1750 0x55fe176b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.793+0000 7f99a678c740 1 -- 172.21.15.12:0/2952355341 >> 172.21.15.12:0/2952355341 conn(0x55fe176acdc0 msgr2=0x55fe176ad1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.795+0000 7f99a678c740 1 -- 172.21.15.12:0/2952355341 shutdown_connections 2024-05-09T09:28:57.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.795+0000 7f99a0f40640 1 --2- 172.21.15.12:0/3429535876 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99880a2d60 0x7f99880a5220 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9994030a00 tx=0x7f999402a6c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.796+0000 7f99a678c740 1 -- 172.21.15.12:0/2952355341 wait complete. 2024-05-09T09:28:57.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.797+0000 7f99a678c740 1 -- start start 2024-05-09T09:28:57.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.797+0000 7f99a678c740 1 Processor -- start 2024-05-09T09:28:57.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.797+0000 7f99a678c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b1750 0x55fe176bb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.798+0000 7f99a678c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b2140 0x55fe176bbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.798+0000 7f99a678c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b72b0 0x55fe176c2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.798+0000 7f99a678c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fe176ba2b0 con 0x55fe176b72b0 2024-05-09T09:28:57.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.798+0000 7f99a678c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe175c8c40 con 0x55fe176b1750 2024-05-09T09:28:57.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.798+0000 7f99a678c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fe176ba430 con 0x55fe176b2140 2024-05-09T09:28:57.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.798+0000 7f99a678c740 10 client.0 _ll_register_callbacks cb 0x55fe17320be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:57.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.798+0000 7f99a1741640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b1750 0x55fe176bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.798+0000 7f99a1741640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b1750 0x55fe176bb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:59132/0 (socket says 172.21.15.12:59132) 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.798+0000 7f99a1741640 1 -- 172.21.15.12:0/722525946 learned_addr learned my addr 172.21.15.12:0/722525946 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.799+0000 7f99a1f42640 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b72b0 0x55fe176c2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.799+0000 7f99a0f40640 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b2140 0x55fe176bbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.799+0000 7f99a1f42640 1 -- 172.21.15.12:0/722525946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b2140 msgr2=0x55fe176bbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.799+0000 7f99a1f42640 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b2140 0x55fe176bbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.799+0000 7f99a1f42640 1 -- 172.21.15.12:0/722525946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b1750 msgr2=0x55fe176bb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.799+0000 7f99a1f42640 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b1750 0x55fe176bb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.799+0000 7f99a1f42640 1 -- 172.21.15.12:0/722525946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe173eb7d0 con 0x55fe176b72b0 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.799+0000 7f99a0f40640 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b2140 0x55fe176bbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.799+0000 7f99a1f42640 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b72b0 0x55fe176c2df0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f999800c6a0 tx=0x7f999809c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96091080023cfa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.799+0000 7f9984ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.800+0000 7f9984ff9640 1 -- 172.21.15.12:0/722525946 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99980a43a0 con 0x55fe176b72b0 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.800+0000 7f99a678c740 1 -- 172.21.15.12:0/722525946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f998c0054f0 con 0x55fe176b72b0 2024-05-09T09:28:57.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.800+0000 7f9984ff9640 1 -- 172.21.15.12:0/722525946 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f99980a4540 con 0x55fe176b72b0 2024-05-09T09:28:57.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.800+0000 7f9984ff9640 1 -- 172.21.15.12:0/722525946 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9998016be0 con 0x55fe176b72b0 2024-05-09T09:28:57.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.800+0000 7f99a678c740 10 client.16872 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:57.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.801+0000 7f99a678c740 1 -- 172.21.15.12:0/722525946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fe176c61d0 con 0x55fe176b72b0 2024-05-09T09:28:57.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.801+0000 7f9984ff9640 1 -- 172.21.15.12:0/722525946 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f998c0054f0 con 0x55fe176b72b0 2024-05-09T09:28:57.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.801+0000 7f9984ff9640 1 -- 172.21.15.12:0/722525946 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f9998097910 con 0x55fe176b72b0 2024-05-09T09:28:57.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.801+0000 7f99a678c740 10 client.16872 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:57.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.801+0000 7f99a678c740 10 client.16872 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:28:57.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.802+0000 7f99a678c740 20 client.16872 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:57.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.802+0000 7f99a678c740 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55fe176c7810 0x55fe176c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.802+0000 7f99a678c740 4 client.16872 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:57.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.802+0000 7f99a678c740 1 -- 172.21.15.12:0/722525946 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe1761b370 con 0x55fe176c7810 2024-05-09T09:28:57.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.802+0000 7f99a1741640 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55fe176c7810 0x55fe176c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.805+0000 7f99a1741640 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55fe176c7810 0x55fe176c9cd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8091f4b85a6d349c server_cookie=6076e838bb38459 in_seq=0 out_seq=0 2024-05-09T09:28:57.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.805+0000 7f9984ff9640 10 client.16872 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:28:57.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.806+0000 7f9984ff9640 1 -- 172.21.15.12:0/722525946 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe1761b370 con 0x55fe176c7810 2024-05-09T09:28:57.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.806+0000 7f9984ff9640 10 client.16872 handle_command_reply: tid=0 2024-05-09T09:28:57.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.806+0000 7f99a678c740 1 client.16872 shutdown 2024-05-09T09:28:57.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.806+0000 7f99a678c740 1 -- 172.21.15.12:0/722525946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b72b0 msgr2=0x55fe176c2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.806+0000 7f99a678c740 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b72b0 0x55fe176c2df0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f999800c6a0 tx=0x7f999809c400 comp rx=0 tx=0).stop 2024-05-09T09:28:57.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.806+0000 7f99a678c740 1 -- 172.21.15.12:0/722525946 shutdown_connections 2024-05-09T09:28:57.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.806+0000 7f99a678c740 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55fe176c7810 0x55fe176c9cd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.807+0000 7f99a678c740 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b72b0 0x55fe176c2df0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.807+0000 7f99a678c740 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b2140 0x55fe176bbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.807+0000 7f99a678c740 1 --2- 172.21.15.12:0/722525946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b1750 0x55fe176bb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.807+0000 7f99a678c740 1 -- 172.21.15.12:0/722525946 >> 172.21.15.12:0/722525946 conn(0x55fe176acdc0 msgr2=0x55fe176a37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.807+0000 7f9984ff9640 0 client.16872 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:28:57.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.807+0000 7f99a678c740 1 -- 172.21.15.12:0/722525946 shutdown_connections 2024-05-09T09:28:57.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.808+0000 7f99a678c740 1 -- 172.21.15.12:0/722525946 wait complete. 2024-05-09T09:28:57.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.808+0000 7f99a678c740 20 client.16872 trim_cache size 0 max 16384 2024-05-09T09:28:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.841+0000 7f99a678c740 1 Processor -- start 2024-05-09T09:28:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.841+0000 7f99a678c740 1 -- start start 2024-05-09T09:28:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.841+0000 7f99a678c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b1750 0x55fe176c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.841+0000 7f99a678c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b2140 0x55fe176c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.841+0000 7f99a678c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b72b0 0x55fe176c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.841+0000 7f99a678c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f998000fd80 con 0x55fe176b1750 2024-05-09T09:28:57.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.841+0000 7f99a678c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9980011b00 con 0x55fe176b2140 2024-05-09T09:28:57.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.841+0000 7f99a678c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9980010510 con 0x55fe176b72b0 2024-05-09T09:28:57.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.842+0000 7f99a0f40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b2140 0x55fe176c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.842+0000 7f99a0f40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b2140 0x55fe176c2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:59136/0 (socket says 172.21.15.12:59136) 2024-05-09T09:28:57.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.842+0000 7f99a0f40640 1 -- 172.21.15.12:0/2002236286 learned_addr learned my addr 172.21.15.12:0/2002236286 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:57.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.842+0000 7f99a1f42640 1 --2- 172.21.15.12:0/2002236286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b72b0 0x55fe176c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.842+0000 7f99a0f40640 1 -- 172.21.15.12:0/2002236286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b72b0 msgr2=0x55fe176c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.842+0000 7f99a0f40640 1 --2- 172.21.15.12:0/2002236286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b72b0 0x55fe176c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.842+0000 7f99a0f40640 1 -- 172.21.15.12:0/2002236286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b1750 msgr2=0x55fe176c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.842+0000 7f99a0f40640 1 --2- 172.21.15.12:0/2002236286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b1750 0x55fe176c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.842+0000 7f99a0f40640 1 -- 172.21.15.12:0/2002236286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe176ad830 con 0x55fe176b2140 2024-05-09T09:28:57.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.843+0000 7f99a0f40640 1 --2- 172.21.15.12:0/2002236286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b2140 0x55fe176c2db0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f99940308d0 tx=0x7f999403c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69e151d66804205b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.844+0000 7f9986ffd640 1 -- 172.21.15.12:0/2002236286 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994040030 con 0x55fe176b2140 2024-05-09T09:28:57.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.844+0000 7f9986ffd640 1 -- 172.21.15.12:0/2002236286 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9994004040 con 0x55fe176b2140 2024-05-09T09:28:57.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.844+0000 7f9986ffd640 1 -- 172.21.15.12:0/2002236286 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994036040 con 0x55fe176b2140 2024-05-09T09:28:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.845+0000 7f99a678c740 1 -- 172.21.15.12:0/2002236286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b2140 msgr2=0x55fe176c2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.845+0000 7f99a678c740 1 --2- 172.21.15.12:0/2002236286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b2140 0x55fe176c2db0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f99940308d0 tx=0x7f999403c000 comp rx=0 tx=0).stop 2024-05-09T09:28:57.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.847+0000 7f99a678c740 1 -- 172.21.15.12:0/2002236286 shutdown_connections 2024-05-09T09:28:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.847+0000 7f99a678c740 1 --2- 172.21.15.12:0/2002236286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b72b0 0x55fe176c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.847+0000 7f99a678c740 1 --2- 172.21.15.12:0/2002236286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b2140 0x55fe176c2db0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.847+0000 7f99a678c740 1 --2- 172.21.15.12:0/2002236286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b1750 0x55fe176c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.847+0000 7f99a678c740 1 -- 172.21.15.12:0/2002236286 >> 172.21.15.12:0/2002236286 conn(0x55fe176acdc0 msgr2=0x55fe176ba8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.849+0000 7f99a678c740 1 -- 172.21.15.12:0/2002236286 shutdown_connections 2024-05-09T09:28:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.849+0000 7f99a678c740 1 -- 172.21.15.12:0/2002236286 wait complete. 2024-05-09T09:28:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.849+0000 7f99a678c740 1 -- start start 2024-05-09T09:28:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.849+0000 7f99a678c740 1 Processor -- start 2024-05-09T09:28:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.850+0000 7f99a678c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b1750 0x55fe176c0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.850+0000 7f99a678c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b2140 0x55fe176c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.850+0000 7f99a678c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b72b0 0x55fe176c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.850+0000 7f99a678c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9980011430 con 0x55fe176b72b0 2024-05-09T09:28:57.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.850+0000 7f99a678c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9980010ca0 con 0x55fe176b2140 2024-05-09T09:28:57.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.850+0000 7f99a678c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f998000dd60 con 0x55fe176b1750 2024-05-09T09:28:57.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.851+0000 7f99a678c740 10 client.0 _ll_register_callbacks cb 0x7f999809b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:57.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.851+0000 7f99a1741640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b1750 0x55fe176c0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.851+0000 7f99a1f42640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b72b0 0x55fe176c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.851+0000 7f99a1f42640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b72b0 0x55fe176c18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47422/0 (socket says 172.21.15.12:47422) 2024-05-09T09:28:57.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.851+0000 7f99a1f42640 1 -- 172.21.15.12:0/1512643346 learned_addr learned my addr 172.21.15.12:0/1512643346 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:57.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.852+0000 7f99a1f42640 1 -- 172.21.15.12:0/1512643346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b1750 msgr2=0x55fe176c0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.852+0000 7f99a1f42640 1 --2- 172.21.15.12:0/1512643346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b1750 0x55fe176c0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.852+0000 7f99a1f42640 1 -- 172.21.15.12:0/1512643346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b2140 msgr2=0x55fe176c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:57.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.852+0000 7f99a1f42640 1 --2- 172.21.15.12:0/1512643346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b2140 0x55fe176c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.852+0000 7f99a1f42640 1 -- 172.21.15.12:0/1512643346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe176c1eb0 con 0x55fe176b72b0 2024-05-09T09:28:57.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.852+0000 7f99a1f42640 1 --2- 172.21.15.12:0/1512643346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b72b0 0x55fe176c18c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f999809c470 tx=0x7f99980a4980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f194f0378bfda07b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:57.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.853+0000 7f9984ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:57.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.853+0000 7f9984ff9640 1 -- 172.21.15.12:0/1512643346 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999809ab40 con 0x55fe176b72b0 2024-05-09T09:28:57.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.853+0000 7f9984ff9640 1 -- 172.21.15.12:0/1512643346 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f99980cf430 con 0x55fe176b72b0 2024-05-09T09:28:57.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.853+0000 7f99a678c740 1 -- 172.21.15.12:0/1512643346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9994004040 con 0x55fe176b72b0 2024-05-09T09:28:57.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.853+0000 7f9984ff9640 1 -- 172.21.15.12:0/1512643346 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9998080050 con 0x55fe176b72b0 2024-05-09T09:28:57.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.853+0000 7f99a678c740 10 client.16875 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:57.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.854+0000 7f99a678c740 1 -- 172.21.15.12:0/1512643346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fe1772de50 con 0x55fe176b72b0 2024-05-09T09:28:57.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.854+0000 7f9984ff9640 1 -- 172.21.15.12:0/1512643346 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9994004040 con 0x55fe176b72b0 2024-05-09T09:28:57.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.854+0000 7f9984ff9640 1 -- 172.21.15.12:0/1512643346 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f999807cbc0 con 0x55fe176b72b0 2024-05-09T09:28:57.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.855+0000 7f99a678c740 10 client.16875 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:57.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.855+0000 7f99a678c740 10 client.16875 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:28:57.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.855+0000 7f99a678c740 20 client.16875 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:57.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.855+0000 7f99a678c740 1 --2- 172.21.15.12:0/1512643346 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55fe1772e840 0x55fe17730c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:57.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.855+0000 7f99a678c740 4 client.16875 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:28:57.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.855+0000 7f99a678c740 1 -- 172.21.15.12:0/1512643346 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe176156c0 con 0x55fe1772e840 2024-05-09T09:28:57.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.857+0000 7f99a1741640 1 --2- 172.21.15.12:0/1512643346 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55fe1772e840 0x55fe17730c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:57.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.858+0000 7f99a1741640 1 --2- 172.21.15.12:0/1512643346 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55fe1772e840 0x55fe17730c60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca7349e784283392 server_cookie=9a40c5363ba64f06 in_seq=0 out_seq=0 2024-05-09T09:28:57.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.858+0000 7f9984ff9640 10 client.16875 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:28:57.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.859+0000 7f9984ff9640 1 -- 172.21.15.12:0/1512643346 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe176156c0 con 0x55fe1772e840 2024-05-09T09:28:57.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.859+0000 7f9984ff9640 10 client.16875 handle_command_reply: tid=0 2024-05-09T09:28:57.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.859+0000 7f99a678c740 1 client.16875 shutdown 2024-05-09T09:28:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.860+0000 7f99a678c740 1 -- 172.21.15.12:0/1512643346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b72b0 msgr2=0x55fe176c18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.860+0000 7f99a678c740 1 --2- 172.21.15.12:0/1512643346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b72b0 0x55fe176c18c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f999809c470 tx=0x7f99980a4980 comp rx=0 tx=0).stop 2024-05-09T09:28:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.860+0000 7f99a678c740 1 -- 172.21.15.12:0/1512643346 shutdown_connections 2024-05-09T09:28:57.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.860+0000 7f99a678c740 1 --2- 172.21.15.12:0/1512643346 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55fe1772e840 0x55fe17730c60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.860+0000 7f99a678c740 1 --2- 172.21.15.12:0/1512643346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe176b72b0 0x55fe176c18c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.860+0000 7f99a678c740 1 --2- 172.21.15.12:0/1512643346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe176b2140 0x55fe176c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.860+0000 7f99a678c740 1 --2- 172.21.15.12:0/1512643346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fe176b1750 0x55fe176c0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.860+0000 7f9984ff9640 0 client.16875 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:28:57.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.860+0000 7f99a678c740 1 -- 172.21.15.12:0/1512643346 >> 172.21.15.12:0/1512643346 conn(0x55fe176acdc0 msgr2=0x55fe176afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.861+0000 7f99a678c740 1 -- 172.21.15.12:0/1512643346 shutdown_connections 2024-05-09T09:28:57.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.861+0000 7f99a678c740 1 -- 172.21.15.12:0/1512643346 wait complete. 2024-05-09T09:28:57.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.862+0000 7f99a678c740 20 client.16875 trim_cache size 0 max 16384 2024-05-09T09:28:57.866 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:28:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.863+0000 7f99a39a5640 1 -- 172.21.15.12:0/3429535876 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99880a2d60 msgr2=0x7f99880a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.864+0000 7f99a39a5640 1 --2- 172.21.15.12:0/3429535876 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99880a2d60 0x7f99880a5220 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9994030a00 tx=0x7f999402a6c0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.864+0000 7f99a39a5640 1 -- 172.21.15.12:0/3429535876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f999c101080 msgr2=0x7f999c1a9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.864+0000 7f99a39a5640 1 --2- 172.21.15.12:0/3429535876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f999c101080 0x7f999c1a9df0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f999800e970 tx=0x7f999800ed70 comp rx=0 tx=0).stop 2024-05-09T09:28:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.864+0000 7f99a39a5640 1 -- 172.21.15.12:0/3429535876 shutdown_connections 2024-05-09T09:28:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.864+0000 7f99a39a5640 1 --2- 172.21.15.12:0/3429535876 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f99880a2d60 0x7f99880a5220 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.864+0000 7f99a39a5640 1 --2- 172.21.15.12:0/3429535876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f999c101080 0x7f999c1a9df0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.864+0000 7f99a39a5640 1 --2- 172.21.15.12:0/3429535876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c068550 0x7f999c0707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.864+0000 7f99a39a5640 1 --2- 172.21.15.12:0/3429535876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c068130 0x7f999c070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.864+0000 7f99a39a5640 1 -- 172.21.15.12:0/3429535876 >> 172.21.15.12:0/3429535876 conn(0x7f999c0fb460 msgr2=0x7f999c0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.865+0000 7f99a39a5640 1 -- 172.21.15.12:0/3429535876 shutdown_connections 2024-05-09T09:28:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:57.865+0000 7f99a39a5640 1 -- 172.21.15.12:0/3429535876 wait complete. 2024-05-09T09:28:57.885 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:28:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.051+0000 7fd5b4561640 1 -- 172.21.15.12:0/876438576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5ac100770 msgr2=0x7fd5ac100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.051+0000 7fd5b4561640 1 --2- 172.21.15.12:0/876438576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5ac100770 0x7fd5ac100b90 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009900 tx=0x7fd5a802a410 comp rx=0 tx=0).stop 2024-05-09T09:28:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.051+0000 7fd5b4561640 1 -- 172.21.15.12:0/876438576 shutdown_connections 2024-05-09T09:28:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.051+0000 7fd5b4561640 1 --2- 172.21.15.12:0/876438576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5ac102320 0x7fd5ac108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.051+0000 7fd5b4561640 1 --2- 172.21.15.12:0/876438576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5ac101960 0x7fd5ac101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.051+0000 7fd5b4561640 1 --2- 172.21.15.12:0/876438576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5ac100770 0x7fd5ac100b90 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.051+0000 7fd5b4561640 1 -- 172.21.15.12:0/876438576 >> 172.21.15.12:0/876438576 conn(0x7fd5ac0fbce0 msgr2=0x7fd5ac0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.051+0000 7fd5b4561640 1 -- 172.21.15.12:0/876438576 shutdown_connections 2024-05-09T09:28:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.051+0000 7fd5b4561640 1 -- 172.21.15.12:0/876438576 wait complete. 2024-05-09T09:28:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.052+0000 7fd5b4561640 1 Processor -- start 2024-05-09T09:28:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.052+0000 7fd5b4561640 1 -- start start 2024-05-09T09:28:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.052+0000 7fd5b4561640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5ac100770 0x7fd5ac197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.052+0000 7fd5b4561640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5ac101960 0x7fd5ac198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.052+0000 7fd5b4561640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5ac102320 0x7fd5ac19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.052+0000 7fd5b4561640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac10c3d0 con 0x7fd5ac100770 2024-05-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.052+0000 7fd5b4561640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac10c250 con 0x7fd5ac101960 2024-05-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.052+0000 7fd5b4561640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac10c550 con 0x7fd5ac102320 2024-05-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.052+0000 7fd5b22fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5ac100770 0x7fd5ac197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.052+0000 7fd5b22fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5ac100770 0x7fd5ac197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47442/0 (socket says 172.21.15.12:47442) 2024-05-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.052+0000 7fd5b22fd640 1 -- 172.21.15.12:0/3822020375 learned_addr learned my addr 172.21.15.12:0/3822020375 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.052+0000 7fd5b2afe640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5ac102320 0x7fd5ac19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.052+0000 7fd5b1afc640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5ac101960 0x7fd5ac198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.053+0000 7fd5b22fd640 1 -- 172.21.15.12:0/3822020375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5ac102320 msgr2=0x7fd5ac19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.053+0000 7fd5b22fd640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5ac102320 0x7fd5ac19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.053+0000 7fd5b22fd640 1 -- 172.21.15.12:0/3822020375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5ac101960 msgr2=0x7fd5ac198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.053+0000 7fd5b22fd640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5ac101960 0x7fd5ac198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.053+0000 7fd5b22fd640 1 -- 172.21.15.12:0/3822020375 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac19f7c0 con 0x7fd5ac100770 2024-05-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.053+0000 7fd5b22fd640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5ac100770 0x7fd5ac197bf0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8005ab0 tx=0x7fd5a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.053+0000 7fd5a37fe640 1 -- 172.21.15.12:0/3822020375 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a803f030 con 0x7fd5ac100770 2024-05-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.053+0000 7fd5a37fe640 1 -- 172.21.15.12:0/3822020375 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd5a8003ac0 con 0x7fd5ac100770 2024-05-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.053+0000 7fd5b4561640 1 -- 172.21.15.12:0/3822020375 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ac19fa50 con 0x7fd5ac100770 2024-05-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.053+0000 7fd5a37fe640 1 -- 172.21.15.12:0/3822020375 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8035070 con 0x7fd5ac100770 2024-05-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.053+0000 7fd5b4561640 1 -- 172.21.15.12:0/3822020375 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ac19ff40 con 0x7fd5ac100770 2024-05-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.055+0000 7fd5a37fe640 1 -- 172.21.15.12:0/3822020375 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd5a8042030 con 0x7fd5ac100770 2024-05-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.056+0000 7fd5b726f740 1 Processor -- start 2024-05-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.056+0000 7fd5b726f740 1 -- start start 2024-05-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.056+0000 7fd5b726f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c972360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.056+0000 7fd5b726f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c972930 0x556e1c977430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.056+0000 7fd5b726f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 0x556e1c979f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.056+0000 7fd5b726f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556e1c8d6c40 con 0x556e1c972930 2024-05-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.056+0000 7fd5b726f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556e1c87e580 con 0x556e1c971f40 2024-05-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.056+0000 7fd5b726f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556e1c8d2c00 con 0x556e1c977aa0 2024-05-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b22fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c972930 0x556e1c977430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b1afc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 0x556e1c979f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b22fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c972930 0x556e1c977430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47450/0 (socket says 172.21.15.12:47450) 2024-05-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b22fd640 1 -- 172.21.15.12:0/1729860736 learned_addr learned my addr 172.21.15.12:0/1729860736 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5a37fe640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd5980a2b90 0x7fd5980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b2afe640 1 --2- 172.21.15.12:0/1729860736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c972360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b1afc640 1 -- 172.21.15.12:0/1729860736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 msgr2=0x556e1c972360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b1afc640 1 --2- 172.21.15.12:0/1729860736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c972360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b1afc640 1 -- 172.21.15.12:0/1729860736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c972930 msgr2=0x556e1c977430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b1afc640 1 --2- 172.21.15.12:0/1729860736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c972930 0x556e1c977430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b1afc640 1 -- 172.21.15.12:0/1729860736 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e1c5ea3f0 con 0x556e1c977aa0 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b2afe640 1 --2- 172.21.15.12:0/1729860736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c972360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b2afe640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd5980a2b90 0x7fd5980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5a37fe640 1 -- 172.21.15.12:0/3822020375 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd5a80ef9a0 con 0x7fd5ac100770 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b1afc640 1 --2- 172.21.15.12:0/1729860736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 0x556e1c979f30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009bd0 tx=0x7fd59c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9f28944fc5d5c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5a0bfe640 1 -- 172.21.15.12:0/1729860736 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c004230 con 0x556e1c977aa0 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5a0bfe640 1 -- 172.21.15.12:0/1729860736 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd59c009e40 con 0x556e1c977aa0 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5b2afe640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd5980a2b90 0x7fd5980a5050 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4005c40 tx=0x7fd5a4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.057+0000 7fd5a0bfe640 1 -- 172.21.15.12:0/1729860736 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c004d30 con 0x556e1c977aa0 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.058+0000 7fd5b726f740 1 -- 172.21.15.12:0/1729860736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 msgr2=0x556e1c979f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.058+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1729860736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 0x556e1c979f30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009bd0 tx=0x7fd59c011810 comp rx=0 tx=0).stop 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.058+0000 7fd5b726f740 1 -- 172.21.15.12:0/1729860736 shutdown_connections 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.058+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1729860736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 0x556e1c979f30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.058+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1729860736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c972930 0x556e1c977430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.058+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1729860736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c972360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.058+0000 7fd5b726f740 1 -- 172.21.15.12:0/1729860736 >> 172.21.15.12:0/1729860736 conn(0x556e1c96d5b0 msgr2=0x556e1c96d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.058+0000 7fd5b726f740 1 -- 172.21.15.12:0/1729860736 shutdown_connections 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.058+0000 7fd5b726f740 1 -- 172.21.15.12:0/1729860736 wait complete. 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b726f740 1 -- start start 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b726f740 1 Processor -- start 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b726f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c97c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b726f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c972930 0x556e1c97c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b726f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c977aa0 0x556e1c9838c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b726f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556e1c97aa50 con 0x556e1c977aa0 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b726f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556e1c891c40 con 0x556e1c971f40 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b726f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556e1c97abd0 con 0x556e1c972930 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b726f740 10 client.0 _ll_register_callbacks cb 0x556e1c5e9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b22fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c97c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b1afc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c972930 0x556e1c97c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b2afe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c977aa0 0x556e1c9838c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b22fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c97c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:59182/0 (socket says 172.21.15.12:59182) 2024-05-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b22fd640 1 -- 172.21.15.12:0/1215609260 learned_addr learned my addr 172.21.15.12:0/1215609260 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b1afc640 1 -- 172.21.15.12:0/1215609260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 msgr2=0x556e1c97c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b1afc640 1 --2- 172.21.15.12:0/1215609260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c97c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b1afc640 1 -- 172.21.15.12:0/1215609260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c977aa0 msgr2=0x556e1c9838c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b1afc640 1 --2- 172.21.15.12:0/1215609260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c977aa0 0x556e1c9838c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b1afc640 1 -- 172.21.15.12:0/1215609260 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e1c6b47d0 con 0x556e1c972930 2024-05-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.059+0000 7fd5b22fd640 1 --2- 172.21.15.12:0/1215609260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c97c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.060+0000 7fd5b1afc640 1 --2- 172.21.15.12:0/1215609260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c972930 0x556e1c97c880 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009bd0 tx=0x7fd59c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a582463181979e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.060+0000 7fd595ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.060+0000 7fd595ffb640 1 -- 172.21.15.12:0/1215609260 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c027020 con 0x556e1c972930 2024-05-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.060+0000 7fd5b726f740 1 -- 172.21.15.12:0/1215609260 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556e1c8e4370 con 0x556e1c972930 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.060+0000 7fd595ffb640 1 -- 172.21.15.12:0/1215609260 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd59c005170 con 0x556e1c972930 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.060+0000 7fd595ffb640 1 -- 172.21.15.12:0/1215609260 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd59c032050 con 0x556e1c972930 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.060+0000 7fd5b726f740 10 client.26425 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.060+0000 7fd5b726f740 1 -- 172.21.15.12:0/1215609260 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556e1c986ba0 con 0x556e1c972930 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.060+0000 7fd595ffb640 1 -- 172.21.15.12:0/1215609260 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x556e1c8e4370 con 0x556e1c972930 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.060+0000 7fd595ffb640 1 -- 172.21.15.12:0/1215609260 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fd59c018070 con 0x556e1c972930 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.061+0000 7fd5b726f740 10 client.26425 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.061+0000 7fd5b726f740 10 client.26425 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.061+0000 7fd5b726f740 20 client.26425 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.061+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1215609260 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556e1c9880e0 0x556e1c98a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.061+0000 7fd5b726f740 4 client.26425 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.061+0000 7fd5b726f740 1 -- 172.21.15.12:0/1215609260 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e1c8de6c0 con 0x556e1c9880e0 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.061+0000 7fd5b22fd640 1 --2- 172.21.15.12:0/1215609260 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556e1c9880e0 0x556e1c98a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.061+0000 7fd595ffb640 10 client.26425 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.061+0000 7fd5b22fd640 1 --2- 172.21.15.12:0/1215609260 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556e1c9880e0 0x556e1c98a560 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1f5e46cc4fb98d6 server_cookie=91b2b01d0dc82060 in_seq=0 out_seq=0 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd595ffb640 1 -- 172.21.15.12:0/1215609260 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e1c8de6c0 con 0x556e1c9880e0 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd595ffb640 10 client.26425 handle_command_reply: tid=0 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd5b726f740 1 client.26425 shutdown 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd5b726f740 1 -- 172.21.15.12:0/1215609260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c972930 msgr2=0x556e1c97c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1215609260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c972930 0x556e1c97c880 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009bd0 tx=0x7fd59c0042f0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd5b726f740 1 -- 172.21.15.12:0/1215609260 shutdown_connections 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1215609260 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556e1c9880e0 0x556e1c98a560 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1215609260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c977aa0 0x556e1c9838c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1215609260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c972930 0x556e1c97c880 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1215609260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c97c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd595ffb640 0 client.26425 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd5b726f740 1 -- 172.21.15.12:0/1215609260 >> 172.21.15.12:0/1215609260 conn(0x556e1c96d5b0 msgr2=0x556e1c96c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd5b726f740 1 -- 172.21.15.12:0/1215609260 shutdown_connections 2024-05-09T09:28:58.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd5b726f740 1 -- 172.21.15.12:0/1215609260 wait complete. 2024-05-09T09:28:58.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.063+0000 7fd5b726f740 20 client.26425 trim_cache size 0 max 16384 2024-05-09T09:28:58.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.078+0000 7fd5b726f740 1 Processor -- start 2024-05-09T09:28:58.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.078+0000 7fd5b726f740 1 -- start start 2024-05-09T09:28:58.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.078+0000 7fd5b726f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c971f40 0x556e1c96c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.078+0000 7fd5b726f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c972930 0x556e1c96cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.078+0000 7fd5b726f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 0x556e1c983880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.078+0000 7fd5b726f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1a8340 con 0x556e1c971f40 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.078+0000 7fd5b726f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1aa0c0 con 0x556e1c972930 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.078+0000 7fd5b726f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1a8ad0 con 0x556e1c977aa0 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.078+0000 7fd5b22fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c971f40 0x556e1c96c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.078+0000 7fd5b2afe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 0x556e1c983880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.078+0000 7fd5b22fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c971f40 0x556e1c96c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47462/0 (socket says 172.21.15.12:47462) 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.078+0000 7fd5b22fd640 1 -- 172.21.15.12:0/2789044282 learned_addr learned my addr 172.21.15.12:0/2789044282 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.078+0000 7fd5b1afc640 1 --2- 172.21.15.12:0/2789044282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c972930 0x556e1c96cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.079+0000 7fd5b22fd640 1 -- 172.21.15.12:0/2789044282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 msgr2=0x556e1c983880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.079+0000 7fd5b22fd640 1 --2- 172.21.15.12:0/2789044282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 0x556e1c983880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.079+0000 7fd5b22fd640 1 -- 172.21.15.12:0/2789044282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c972930 msgr2=0x556e1c96cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.079+0000 7fd5b22fd640 1 --2- 172.21.15.12:0/2789044282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c972930 0x556e1c96cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.079+0000 7fd5b22fd640 1 -- 172.21.15.12:0/2789044282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e1c96e320 con 0x556e1c971f40 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.079+0000 7fd5b1afc640 1 --2- 172.21.15.12:0/2789044282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c972930 0x556e1c96cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:58.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.079+0000 7fd5b22fd640 1 --2- 172.21.15.12:0/2789044282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c971f40 0x556e1c96c630 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80c4510 tx=0x7fd5a80ecf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38a25fbd7625852 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.079+0000 7fd597fff640 1 -- 172.21.15.12:0/2789044282 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a80f5020 con 0x556e1c971f40 2024-05-09T09:28:58.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.079+0000 7fd597fff640 1 -- 172.21.15.12:0/2789044282 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd5a80047a0 con 0x556e1c971f40 2024-05-09T09:28:58.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.079+0000 7fd597fff640 1 -- 172.21.15.12:0/2789044282 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a80f1e60 con 0x556e1c971f40 2024-05-09T09:28:58.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 -- 172.21.15.12:0/2789044282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c971f40 msgr2=0x556e1c96c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 --2- 172.21.15.12:0/2789044282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c971f40 0x556e1c96c630 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80c4510 tx=0x7fd5a80ecf40 comp rx=0 tx=0).stop 2024-05-09T09:28:58.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 -- 172.21.15.12:0/2789044282 shutdown_connections 2024-05-09T09:28:58.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 --2- 172.21.15.12:0/2789044282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 0x556e1c983880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 --2- 172.21.15.12:0/2789044282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c972930 0x556e1c96cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 --2- 172.21.15.12:0/2789044282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c971f40 0x556e1c96c630 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 -- 172.21.15.12:0/2789044282 >> 172.21.15.12:0/2789044282 conn(0x556e1c96d5b0 msgr2=0x556e1c96bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 -- 172.21.15.12:0/2789044282 shutdown_connections 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 -- 172.21.15.12:0/2789044282 wait complete. 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 -- start start 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 Processor -- start 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c980da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c972930 0x556e1c9812e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 0x556e1c982280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1a99f0 con 0x556e1c972930 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1a9260 con 0x556e1c971f40 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1a6320 con 0x556e1c977aa0 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.080+0000 7fd5b726f740 10 client.0 _ll_register_callbacks cb 0x7fd5a80b7f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b1afc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c972930 0x556e1c9812e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b1afc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c972930 0x556e1c9812e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47468/0 (socket says 172.21.15.12:47468) 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b1afc640 1 -- 172.21.15.12:0/1730043899 learned_addr learned my addr 172.21.15.12:0/1730043899 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b2afe640 1 --2- 172.21.15.12:0/1730043899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 0x556e1c982280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b22fd640 1 --2- 172.21.15.12:0/1730043899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c980da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b1afc640 1 -- 172.21.15.12:0/1730043899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 msgr2=0x556e1c982280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b1afc640 1 --2- 172.21.15.12:0/1730043899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 0x556e1c982280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b1afc640 1 -- 172.21.15.12:0/1730043899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 msgr2=0x556e1c980da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b1afc640 1 --2- 172.21.15.12:0/1730043899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c980da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b1afc640 1 -- 172.21.15.12:0/1730043899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e1c982870 con 0x556e1c972930 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b1afc640 1 --2- 172.21.15.12:0/1730043899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c972930 0x556e1c9812e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0059f0 tx=0x7fd59c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2392a1f2516bd8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd595ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd595ffb640 1 -- 172.21.15.12:0/1730043899 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c027030 con 0x556e1c972930 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd595ffb640 1 -- 172.21.15.12:0/1730043899 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd59c0037f0 con 0x556e1c972930 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b726f740 1 -- 172.21.15.12:0/1730043899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5a80047a0 con 0x556e1c972930 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd595ffb640 1 -- 172.21.15.12:0/1730043899 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd59c03f050 con 0x556e1c972930 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b726f740 10 client.16893 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.081+0000 7fd5b726f740 1 -- 172.21.15.12:0/1730043899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556e1c983b70 con 0x556e1c972930 2024-05-09T09:28:58.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.082+0000 7fd595ffb640 1 -- 172.21.15.12:0/1730043899 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd5a80047a0 con 0x556e1c972930 2024-05-09T09:28:58.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.082+0000 7fd595ffb640 1 -- 172.21.15.12:0/1730043899 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fd59c018040 con 0x556e1c972930 2024-05-09T09:28:58.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.082+0000 7fd5b726f740 10 client.16893 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:58.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.082+0000 7fd5b726f740 10 client.16893 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:28:58.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.082+0000 7fd5b726f740 20 client.16893 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:58.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.082+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1730043899 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556e1c9eed60 0x556e1c9f1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.082+0000 7fd5b726f740 4 client.16893 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:28:58.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.082+0000 7fd5b726f740 1 -- 172.21.15.12:0/1730043899 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e1c85b820 con 0x556e1c9eed60 2024-05-09T09:28:58.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.082+0000 7fd5b22fd640 1 --2- 172.21.15.12:0/1730043899 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556e1c9eed60 0x556e1c9f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.082+0000 7fd5b22fd640 1 --2- 172.21.15.12:0/1730043899 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556e1c9eed60 0x556e1c9f1180 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c310c811f95fba2 server_cookie=de7cb0d49e8b9147 in_seq=0 out_seq=0 2024-05-09T09:28:58.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.082+0000 7fd595ffb640 10 client.16893 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:28:58.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.083+0000 7fd595ffb640 1 -- 172.21.15.12:0/1730043899 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e1c85b820 con 0x556e1c9eed60 2024-05-09T09:28:58.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.083+0000 7fd595ffb640 10 client.16893 handle_command_reply: tid=0 2024-05-09T09:28:58.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.083+0000 7fd5b726f740 1 client.16893 shutdown 2024-05-09T09:28:58.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.083+0000 7fd5b726f740 1 -- 172.21.15.12:0/1730043899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c972930 msgr2=0x556e1c9812e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.083+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1730043899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c972930 0x556e1c9812e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0059f0 tx=0x7fd59c005dc0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.083+0000 7fd5b726f740 1 -- 172.21.15.12:0/1730043899 shutdown_connections 2024-05-09T09:28:58.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.083+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1730043899 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x556e1c9eed60 0x556e1c9f1180 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.083+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1730043899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e1c977aa0 0x556e1c982280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd595ffb640 0 client.16893 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:28:58.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1730043899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556e1c972930 0x556e1c9812e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b726f740 1 --2- 172.21.15.12:0/1730043899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e1c971f40 0x556e1c980da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b726f740 1 -- 172.21.15.12:0/1730043899 >> 172.21.15.12:0/1730043899 conn(0x556e1c96d5b0 msgr2=0x556e1c96c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b726f740 1 -- 172.21.15.12:0/1730043899 shutdown_connections 2024-05-09T09:28:58.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b726f740 1 -- 172.21.15.12:0/1730043899 wait complete. 2024-05-09T09:28:58.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b726f740 20 client.16893 trim_cache size 0 max 16384 2024-05-09T09:28:58.090 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:28:58.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b4561640 1 -- 172.21.15.12:0/3822020375 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd5980a2b90 msgr2=0x7fd5980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b4561640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd5980a2b90 0x7fd5980a5050 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4005c40 tx=0x7fd5a4004a40 comp rx=0 tx=0).stop 2024-05-09T09:28:58.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b4561640 1 -- 172.21.15.12:0/3822020375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5ac100770 msgr2=0x7fd5ac197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b4561640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5ac100770 0x7fd5ac197bf0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8005ab0 tx=0x7fd5a80042f0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b4561640 1 -- 172.21.15.12:0/3822020375 shutdown_connections 2024-05-09T09:28:58.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b4561640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd5980a2b90 0x7fd5980a5050 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b4561640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5ac102320 0x7fd5ac19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b4561640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5ac101960 0x7fd5ac198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b4561640 1 --2- 172.21.15.12:0/3822020375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5ac100770 0x7fd5ac197bf0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b4561640 1 -- 172.21.15.12:0/3822020375 >> 172.21.15.12:0/3822020375 conn(0x7fd5ac0fbce0 msgr2=0x7fd5ac0fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b4561640 1 -- 172.21.15.12:0/3822020375 shutdown_connections 2024-05-09T09:28:58.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.084+0000 7fd5b4561640 1 -- 172.21.15.12:0/3822020375 wait complete. 2024-05-09T09:28:58.096 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:28:58.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.210+0000 7fc53f970640 1 -- 172.21.15.12:0/896295269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc538101960 msgr2=0x7fc538101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.210+0000 7fc53f970640 1 --2- 172.21.15.12:0/896295269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc538101960 0x7fc538101de0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc528009900 tx=0x7fc52802a410 comp rx=0 tx=0).stop 2024-05-09T09:28:58.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.210+0000 7fc53f970640 1 -- 172.21.15.12:0/896295269 shutdown_connections 2024-05-09T09:28:58.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.210+0000 7fc53f970640 1 --2- 172.21.15.12:0/896295269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538102320 0x7fc538108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.210+0000 7fc53f970640 1 --2- 172.21.15.12:0/896295269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc538101960 0x7fc538101de0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.210+0000 7fc53f970640 1 --2- 172.21.15.12:0/896295269 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc538100770 0x7fc538100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.210+0000 7fc53f970640 1 -- 172.21.15.12:0/896295269 >> 172.21.15.12:0/896295269 conn(0x7fc5380fbce0 msgr2=0x7fc5380fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.210+0000 7fc53f970640 1 -- 172.21.15.12:0/896295269 shutdown_connections 2024-05-09T09:28:58.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.210+0000 7fc53f970640 1 -- 172.21.15.12:0/896295269 wait complete. 2024-05-09T09:28:58.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53f970640 1 Processor -- start 2024-05-09T09:28:58.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53f970640 1 -- start start 2024-05-09T09:28:58.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53f970640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc538100770 0x7fc538197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53f970640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538101960 0x7fc538198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53f970640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc538102320 0x7fc53819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53f970640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc53810c420 con 0x7fc538102320 2024-05-09T09:28:58.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53f970640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc53810c2a0 con 0x7fc538100770 2024-05-09T09:28:58.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53f970640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc53810c5a0 con 0x7fc538101960 2024-05-09T09:28:58.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53df0d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc538102320 0x7fc53819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53cf0b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538101960 0x7fc538198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53cf0b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538101960 0x7fc538198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:57836/0 (socket says 172.21.15.12:57836) 2024-05-09T09:28:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53cf0b640 1 -- 172.21.15.12:0/2010376409 learned_addr learned my addr 172.21.15.12:0/2010376409 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53df0d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc538102320 0x7fc53819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47484/0 (socket says 172.21.15.12:47484) 2024-05-09T09:28:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53d70c640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc538100770 0x7fc538197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53cf0b640 1 -- 172.21.15.12:0/2010376409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc538100770 msgr2=0x7fc538197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53cf0b640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc538100770 0x7fc538197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53cf0b640 1 -- 172.21.15.12:0/2010376409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc538102320 msgr2=0x7fc53819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53cf0b640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc538102320 0x7fc53819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53cf0b640 1 -- 172.21.15.12:0/2010376409 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53819f7d0 con 0x7fc538101960 2024-05-09T09:28:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.211+0000 7fc53df0d640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc538102320 0x7fc53819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.212+0000 7fc53cf0b640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538101960 0x7fc538198140 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc52802a930 tx=0x7fc528004f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.212+0000 7fc52e7fc640 1 -- 172.21.15.12:0/2010376409 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52803f070 con 0x7fc538101960 2024-05-09T09:28:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.212+0000 7fc53f970640 1 -- 172.21.15.12:0/2010376409 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53819fa60 con 0x7fc538101960 2024-05-09T09:28:58.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.212+0000 7fc52e7fc640 1 -- 172.21.15.12:0/2010376409 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc528005430 con 0x7fc538101960 2024-05-09T09:28:58.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.212+0000 7fc53f970640 1 -- 172.21.15.12:0/2010376409 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53819ffa0 con 0x7fc538101960 2024-05-09T09:28:58.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.212+0000 7fc52e7fc640 1 -- 172.21.15.12:0/2010376409 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528049060 con 0x7fc538101960 2024-05-09T09:28:58.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.214+0000 7fc52e7fc640 1 -- 172.21.15.12:0/2010376409 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc528042070 con 0x7fc538101960 2024-05-09T09:28:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc542683740 1 Processor -- start 2024-05-09T09:28:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc542683740 1 -- start start 2024-05-09T09:28:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc542683740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c6306ef40 0x555c6306f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc542683740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c6306f930 0x555c63074430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc542683740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c63074aa0 0x555c63076f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc542683740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555c62fd3c40 con 0x555c63074aa0 2024-05-09T09:28:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc542683740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555c62f7b580 con 0x555c6306ef40 2024-05-09T09:28:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc542683740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555c62fcfc00 con 0x555c6306f930 2024-05-09T09:28:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc53cf0b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c63074aa0 0x555c63076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc53d70c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c6306f930 0x555c63074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc53d70c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c6306f930 0x555c63074430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:57852/0 (socket says 172.21.15.12:57852) 2024-05-09T09:28:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc53d70c640 1 -- 172.21.15.12:0/1416353697 learned_addr learned my addr 172.21.15.12:0/1416353697 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc53cf0b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c63074aa0 0x555c63076f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47500/0 (socket says 172.21.15.12:47500) 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc52e7fc640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc5240a2b90 0x7fc5240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc53df0d640 1 --2- 172.21.15.12:0/1416353697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c6306ef40 0x555c6306f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.215+0000 7fc53df0d640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc5240a2b90 0x7fc5240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.216+0000 7fc53d70c640 1 -- 172.21.15.12:0/1416353697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c6306ef40 msgr2=0x555c6306f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.216+0000 7fc53d70c640 1 --2- 172.21.15.12:0/1416353697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c6306ef40 0x555c6306f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.216+0000 7fc53d70c640 1 -- 172.21.15.12:0/1416353697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c63074aa0 msgr2=0x555c63076f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.216+0000 7fc53d70c640 1 --2- 172.21.15.12:0/1416353697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c63074aa0 0x555c63076f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.216+0000 7fc52e7fc640 1 -- 172.21.15.12:0/2010376409 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc5280effa0 con 0x7fc538101960 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.216+0000 7fc53d70c640 1 -- 172.21.15.12:0/1416353697 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c62ce73f0 con 0x555c6306f930 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.216+0000 7fc53df0d640 1 --2- 172.21.15.12:0/1416353697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c6306ef40 0x555c6306f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.216+0000 7fc53d70c640 1 --2- 172.21.15.12:0/1416353697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c6306f930 0x555c63074430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc534009ca0 tx=0x7fc534011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f10ef6fed6bbac1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.216+0000 7fc53df0d640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc5240a2b90 0x7fc5240a5050 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc530009b80 tx=0x7fc530009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.216+0000 7fc5237fe640 1 -- 172.21.15.12:0/1416353697 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534004220 con 0x555c6306f930 2024-05-09T09:28:58.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.216+0000 7fc5237fe640 1 -- 172.21.15.12:0/1416353697 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc5340043c0 con 0x555c6306f930 2024-05-09T09:28:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.216+0000 7fc5237fe640 1 -- 172.21.15.12:0/1416353697 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534019860 con 0x555c6306f930 2024-05-09T09:28:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.217+0000 7fc542683740 1 -- 172.21.15.12:0/1416353697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c6306f930 msgr2=0x555c63074430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.217+0000 7fc542683740 1 --2- 172.21.15.12:0/1416353697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c6306f930 0x555c63074430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc534009ca0 tx=0x7fc534011810 comp rx=0 tx=0).stop 2024-05-09T09:28:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.217+0000 7fc542683740 1 -- 172.21.15.12:0/1416353697 shutdown_connections 2024-05-09T09:28:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.217+0000 7fc542683740 1 --2- 172.21.15.12:0/1416353697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c63074aa0 0x555c63076f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.217+0000 7fc542683740 1 --2- 172.21.15.12:0/1416353697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c6306f930 0x555c63074430 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.217+0000 7fc542683740 1 --2- 172.21.15.12:0/1416353697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c6306ef40 0x555c6306f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.217+0000 7fc542683740 1 -- 172.21.15.12:0/1416353697 >> 172.21.15.12:0/1416353697 conn(0x555c6306a5b0 msgr2=0x555c6306a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.217+0000 7fc542683740 1 -- 172.21.15.12:0/1416353697 shutdown_connections 2024-05-09T09:28:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.217+0000 7fc542683740 1 -- 172.21.15.12:0/1416353697 wait complete. 2024-05-09T09:28:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.217+0000 7fc542683740 1 -- start start 2024-05-09T09:28:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.217+0000 7fc542683740 1 Processor -- start 2024-05-09T09:28:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc542683740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 0x555c63079310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc542683740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c6306f930 0x555c63079850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc542683740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c63074aa0 0x555c63080890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc542683740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555c63077a50 con 0x555c6306ef40 2024-05-09T09:28:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc542683740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555c62f8ec40 con 0x555c63074aa0 2024-05-09T09:28:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc542683740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555c63077bd0 con 0x555c6306f930 2024-05-09T09:28:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc542683740 10 client.0 _ll_register_callbacks cb 0x555c62ce6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc53d70c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 0x555c63079310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc53cf0b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c6306f930 0x555c63079850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc53d70c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 0x555c63079310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47514/0 (socket says 172.21.15.12:47514) 2024-05-09T09:28:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc53d70c640 1 -- 172.21.15.12:0/1331078927 learned_addr learned my addr 172.21.15.12:0/1331078927 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc53df0d640 1 --2- 172.21.15.12:0/1331078927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c63074aa0 0x555c63080890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc53d70c640 1 -- 172.21.15.12:0/1331078927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c6306f930 msgr2=0x555c63079850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc53d70c640 1 --2- 172.21.15.12:0/1331078927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c6306f930 0x555c63079850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc53d70c640 1 -- 172.21.15.12:0/1331078927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c63074aa0 msgr2=0x555c63080890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc53d70c640 1 --2- 172.21.15.12:0/1331078927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c63074aa0 0x555c63080890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc53d70c640 1 -- 172.21.15.12:0/1331078927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c62db17d0 con 0x555c6306ef40 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc53d70c640 1 --2- 172.21.15.12:0/1331078927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 0x555c63079310 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc5340058c0 tx=0x7fc534005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d134248027c37414 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc520ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc520ff9640 1 -- 172.21.15.12:0/1331078927 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534028070 con 0x555c6306ef40 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc520ff9640 1 -- 172.21.15.12:0/1331078927 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc534003880 con 0x555c6306ef40 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.218+0000 7fc542683740 1 -- 172.21.15.12:0/1331078927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5340043c0 con 0x555c6306ef40 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.219+0000 7fc520ff9640 1 -- 172.21.15.12:0/1331078927 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc534034050 con 0x555c6306ef40 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.219+0000 7fc542683740 10 client.16905 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.219+0000 7fc542683740 1 -- 172.21.15.12:0/1331078927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555c63083c00 con 0x555c6306ef40 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.219+0000 7fc520ff9640 1 -- 172.21.15.12:0/1331078927 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc5340043c0 con 0x555c6306ef40 2024-05-09T09:28:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.219+0000 7fc520ff9640 1 -- 172.21.15.12:0/1331078927 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fc534004910 con 0x555c6306ef40 2024-05-09T09:28:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.219+0000 7fc542683740 10 client.16905 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.219+0000 7fc542683740 10 client.16905 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:28:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.219+0000 7fc542683740 20 client.16905 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.219+0000 7fc542683740 1 --2- 172.21.15.12:0/1331078927 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555c630851f0 0x555c63087670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.219+0000 7fc542683740 4 client.16905 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.219+0000 7fc542683740 1 -- 172.21.15.12:0/1331078927 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c62fe1370 con 0x555c630851f0 2024-05-09T09:28:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.219+0000 7fc53cf0b640 1 --2- 172.21.15.12:0/1331078927 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555c630851f0 0x555c63087670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.220+0000 7fc520ff9640 10 client.16905 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:28:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.220+0000 7fc53cf0b640 1 --2- 172.21.15.12:0/1331078927 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555c630851f0 0x555c63087670 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a854ab997c4be0ba server_cookie=15ebc3747b2b7550 in_seq=0 out_seq=0 2024-05-09T09:28:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.220+0000 7fc520ff9640 1 -- 172.21.15.12:0/1331078927 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c62fe1370 con 0x555c630851f0 2024-05-09T09:28:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.220+0000 7fc520ff9640 10 client.16905 handle_command_reply: tid=0 2024-05-09T09:28:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.220+0000 7fc542683740 1 client.16905 shutdown 2024-05-09T09:28:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.221+0000 7fc542683740 1 -- 172.21.15.12:0/1331078927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 msgr2=0x555c63079310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.221+0000 7fc542683740 1 --2- 172.21.15.12:0/1331078927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 0x555c63079310 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc5340058c0 tx=0x7fc534005c50 comp rx=0 tx=0).stop 2024-05-09T09:28:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.221+0000 7fc542683740 1 -- 172.21.15.12:0/1331078927 shutdown_connections 2024-05-09T09:28:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.221+0000 7fc542683740 1 --2- 172.21.15.12:0/1331078927 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555c630851f0 0x555c63087670 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.221+0000 7fc542683740 1 --2- 172.21.15.12:0/1331078927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c63074aa0 0x555c63080890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.221+0000 7fc520ff9640 0 client.16905 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:28:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.221+0000 7fc542683740 1 --2- 172.21.15.12:0/1331078927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c6306f930 0x555c63079850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.221+0000 7fc542683740 1 --2- 172.21.15.12:0/1331078927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 0x555c63079310 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.221+0000 7fc542683740 1 -- 172.21.15.12:0/1331078927 >> 172.21.15.12:0/1331078927 conn(0x555c6306a5b0 msgr2=0x555c630697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.221+0000 7fc542683740 1 -- 172.21.15.12:0/1331078927 shutdown_connections 2024-05-09T09:28:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.221+0000 7fc542683740 1 -- 172.21.15.12:0/1331078927 wait complete. 2024-05-09T09:28:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.221+0000 7fc542683740 20 client.16905 trim_cache size 0 max 16384 2024-05-09T09:28:58.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc542683740 1 Processor -- start 2024-05-09T09:28:58.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc542683740 1 -- start start 2024-05-09T09:28:58.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc542683740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 0x555c630697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc542683740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c6306f930 0x555c630807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc542683740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c63074aa0 0x555c63078600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc542683740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc5381aa590 con 0x555c6306ef40 2024-05-09T09:28:58.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc542683740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5381ac310 con 0x555c6306f930 2024-05-09T09:28:58.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc542683740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc5381aad20 con 0x555c63074aa0 2024-05-09T09:28:58.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc53d70c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 0x555c630697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc53d70c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 0x555c630697b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47524/0 (socket says 172.21.15.12:47524) 2024-05-09T09:28:58.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc53d70c640 1 -- 172.21.15.12:0/2539565260 learned_addr learned my addr 172.21.15.12:0/2539565260 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc53df0d640 1 --2- 172.21.15.12:0/2539565260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c63074aa0 0x555c63078600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc53cf0b640 1 --2- 172.21.15.12:0/2539565260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c6306f930 0x555c630807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.236+0000 7fc53d70c640 1 -- 172.21.15.12:0/2539565260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c63074aa0 msgr2=0x555c63078600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.237+0000 7fc53d70c640 1 --2- 172.21.15.12:0/2539565260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c63074aa0 0x555c63078600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.237+0000 7fc53d70c640 1 -- 172.21.15.12:0/2539565260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c6306f930 msgr2=0x555c630807b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.237+0000 7fc53d70c640 1 --2- 172.21.15.12:0/2539565260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c6306f930 0x555c630807b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.237+0000 7fc53d70c640 1 -- 172.21.15.12:0/2539565260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c6306a0f0 con 0x555c6306ef40 2024-05-09T09:28:58.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.237+0000 7fc53cf0b640 1 --2- 172.21.15.12:0/2539565260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c6306f930 0x555c630807b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:28:58.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.237+0000 7fc53d70c640 1 --2- 172.21.15.12:0/2539565260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 0x555c630697b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc534004f20 tx=0x7fc534001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e03491c3b5dccb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.237+0000 7fc522ffd640 1 -- 172.21.15.12:0/2539565260 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534028070 con 0x555c6306ef40 2024-05-09T09:28:58.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.237+0000 7fc522ffd640 1 -- 172.21.15.12:0/2539565260 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc5340043a0 con 0x555c6306ef40 2024-05-09T09:28:58.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.237+0000 7fc522ffd640 1 -- 172.21.15.12:0/2539565260 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534019e00 con 0x555c6306ef40 2024-05-09T09:28:58.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.238+0000 7fc542683740 1 -- 172.21.15.12:0/2539565260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 msgr2=0x555c630697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.238+0000 7fc542683740 1 --2- 172.21.15.12:0/2539565260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 0x555c630697b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc534004f20 tx=0x7fc534001e50 comp rx=0 tx=0).stop 2024-05-09T09:28:58.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.238+0000 7fc542683740 1 -- 172.21.15.12:0/2539565260 shutdown_connections 2024-05-09T09:28:58.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.238+0000 7fc542683740 1 --2- 172.21.15.12:0/2539565260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c63074aa0 0x555c63078600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.238+0000 7fc542683740 1 --2- 172.21.15.12:0/2539565260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c6306f930 0x555c630807b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.238+0000 7fc542683740 1 --2- 172.21.15.12:0/2539565260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306ef40 0x555c630697b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.238+0000 7fc542683740 1 -- 172.21.15.12:0/2539565260 >> 172.21.15.12:0/2539565260 conn(0x555c6306a5b0 msgr2=0x555c6306a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.238+0000 7fc542683740 1 -- 172.21.15.12:0/2539565260 shutdown_connections 2024-05-09T09:28:58.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.238+0000 7fc542683740 1 -- 172.21.15.12:0/2539565260 wait complete. 2024-05-09T09:28:58.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.238+0000 7fc542683740 1 -- start start 2024-05-09T09:28:58.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.238+0000 7fc542683740 1 Processor -- start 2024-05-09T09:28:58.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc542683740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306f930 0x555c6307d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc542683740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c63074aa0 0x555c6307dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc542683740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c630e7790 0x555c630e9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc542683740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc5381abc40 con 0x555c6306f930 2024-05-09T09:28:58.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc542683740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5381ab4b0 con 0x555c630e7790 2024-05-09T09:28:58.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc542683740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc5381a8570 con 0x555c63074aa0 2024-05-09T09:28:58.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc542683740 10 client.0 _ll_register_callbacks cb 0x7fc5280b8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:58.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc53d70c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306f930 0x555c6307d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc53d70c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306f930 0x555c6307d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47538/0 (socket says 172.21.15.12:47538) 2024-05-09T09:28:58.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc53d70c640 1 -- 172.21.15.12:0/2637917532 learned_addr learned my addr 172.21.15.12:0/2637917532 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc53cf0b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c63074aa0 0x555c6307dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc53df0d640 1 --2- 172.21.15.12:0/2637917532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c630e7790 0x555c630e9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc53cf0b640 1 -- 172.21.15.12:0/2637917532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c630e7790 msgr2=0x555c630e9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc53cf0b640 1 --2- 172.21.15.12:0/2637917532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c630e7790 0x555c630e9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc53cf0b640 1 -- 172.21.15.12:0/2637917532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306f930 msgr2=0x555c6307d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc53cf0b640 1 --2- 172.21.15.12:0/2637917532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306f930 0x555c6307d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc53cf0b640 1 -- 172.21.15.12:0/2637917532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c6306b320 con 0x555c63074aa0 2024-05-09T09:28:58.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.239+0000 7fc53d70c640 1 --2- 172.21.15.12:0/2637917532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306f930 0x555c6307d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:58.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.240+0000 7fc53cf0b640 1 --2- 172.21.15.12:0/2637917532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c63074aa0 0x555c6307dd20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5280b88c0 tx=0x7fc528009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6bb459fe6f51a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.240+0000 7fc520ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:28:58.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.240+0000 7fc520ff9640 1 -- 172.21.15.12:0/2637917532 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5280bcb60 con 0x555c63074aa0 2024-05-09T09:28:58.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.240+0000 7fc520ff9640 1 -- 172.21.15.12:0/2637917532 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc5280ed780 con 0x555c63074aa0 2024-05-09T09:28:58.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.240+0000 7fc542683740 1 -- 172.21.15.12:0/2637917532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5340043a0 con 0x555c63074aa0 2024-05-09T09:28:58.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.240+0000 7fc520ff9640 1 -- 172.21.15.12:0/2637917532 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc528004770 con 0x555c63074aa0 2024-05-09T09:28:58.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.240+0000 7fc542683740 10 client.26449 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:58.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.240+0000 7fc542683740 1 -- 172.21.15.12:0/2637917532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555c630eb130 con 0x555c63074aa0 2024-05-09T09:28:58.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.240+0000 7fc520ff9640 1 -- 172.21.15.12:0/2637917532 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc5340043a0 con 0x555c63074aa0 2024-05-09T09:28:58.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.240+0000 7fc520ff9640 1 -- 172.21.15.12:0/2637917532 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fc5280d5150 con 0x555c63074aa0 2024-05-09T09:28:58.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.240+0000 7fc542683740 10 client.26449 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:58.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.240+0000 7fc542683740 10 client.26449 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:28:58.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.240+0000 7fc542683740 20 client.26449 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:58.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.241+0000 7fc542683740 1 --2- 172.21.15.12:0/2637917532 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555c630ebb50 0x555c630edf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.241+0000 7fc542683740 4 client.26449 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:28:58.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.241+0000 7fc542683740 1 -- 172.21.15.12:0/2637917532 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c63065d30 con 0x555c630ebb50 2024-05-09T09:28:58.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.241+0000 7fc53d70c640 1 --2- 172.21.15.12:0/2637917532 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555c630ebb50 0x555c630edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.241+0000 7fc520ff9640 10 client.26449 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:28:58.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.241+0000 7fc53d70c640 1 --2- 172.21.15.12:0/2637917532 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555c630ebb50 0x555c630edf70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35a066db1500869c server_cookie=255e916f31c280b7 in_seq=0 out_seq=0 2024-05-09T09:28:58.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.241+0000 7fc520ff9640 1 -- 172.21.15.12:0/2637917532 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c63065d30 con 0x555c630ebb50 2024-05-09T09:28:58.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.241+0000 7fc520ff9640 10 client.26449 handle_command_reply: tid=0 2024-05-09T09:28:58.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc542683740 1 client.26449 shutdown 2024-05-09T09:28:58.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc542683740 1 -- 172.21.15.12:0/2637917532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c63074aa0 msgr2=0x555c6307dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc542683740 1 --2- 172.21.15.12:0/2637917532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c63074aa0 0x555c6307dd20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5280b88c0 tx=0x7fc528009260 comp rx=0 tx=0).stop 2024-05-09T09:28:58.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc542683740 1 -- 172.21.15.12:0/2637917532 shutdown_connections 2024-05-09T09:28:58.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc542683740 1 --2- 172.21.15.12:0/2637917532 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555c630ebb50 0x555c630edf70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc542683740 1 --2- 172.21.15.12:0/2637917532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c630e7790 0x555c630e9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc542683740 1 --2- 172.21.15.12:0/2637917532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c63074aa0 0x555c6307dd20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc542683740 1 --2- 172.21.15.12:0/2637917532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c6306f930 0x555c6307d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc520ff9640 0 client.26449 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:28:58.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc542683740 1 -- 172.21.15.12:0/2637917532 >> 172.21.15.12:0/2637917532 conn(0x555c6306a5b0 msgr2=0x555c630697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc542683740 1 -- 172.21.15.12:0/2637917532 shutdown_connections 2024-05-09T09:28:58.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc542683740 1 -- 172.21.15.12:0/2637917532 wait complete. 2024-05-09T09:28:58.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc542683740 20 client.26449 trim_cache size 0 max 16384 2024-05-09T09:28:58.251 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:28:58.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc53f970640 1 -- 172.21.15.12:0/2010376409 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc5240a2b90 msgr2=0x7fc5240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc53f970640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc5240a2b90 0x7fc5240a5050 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc530009b80 tx=0x7fc530009500 comp rx=0 tx=0).stop 2024-05-09T09:28:58.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc53f970640 1 -- 172.21.15.12:0/2010376409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538101960 msgr2=0x7fc538198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.242+0000 7fc53f970640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538101960 0x7fc538198140 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc52802a930 tx=0x7fc528004f70 comp rx=0 tx=0).stop 2024-05-09T09:28:58.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.243+0000 7fc53f970640 1 -- 172.21.15.12:0/2010376409 shutdown_connections 2024-05-09T09:28:58.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.243+0000 7fc53f970640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fc5240a2b90 0x7fc5240a5050 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.243+0000 7fc53f970640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc538102320 0x7fc53819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.243+0000 7fc53f970640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538101960 0x7fc538198140 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.243+0000 7fc53f970640 1 --2- 172.21.15.12:0/2010376409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc538100770 0x7fc538197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.243+0000 7fc53f970640 1 -- 172.21.15.12:0/2010376409 >> 172.21.15.12:0/2010376409 conn(0x7fc5380fbce0 msgr2=0x7fc5380fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.243+0000 7fc53f970640 1 -- 172.21.15.12:0/2010376409 shutdown_connections 2024-05-09T09:28:58.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.243+0000 7fc53f970640 1 -- 172.21.15.12:0/2010376409 wait complete. 2024-05-09T09:28:58.254 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:28:58.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.368+0000 7fb1e9261640 1 -- 172.21.15.12:0/284197882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e4100770 msgr2=0x7fb1e4100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.368+0000 7fb1e9261640 1 --2- 172.21.15.12:0/284197882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e4100770 0x7fb1e4100b90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009900 tx=0x7fb1d402a410 comp rx=0 tx=0).stop 2024-05-09T09:28:58.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.368+0000 7fb1e9261640 1 -- 172.21.15.12:0/284197882 shutdown_connections 2024-05-09T09:28:58.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.368+0000 7fb1e9261640 1 --2- 172.21.15.12:0/284197882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e4102320 0x7fb1e4108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.368+0000 7fb1e9261640 1 --2- 172.21.15.12:0/284197882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4101960 0x7fb1e4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.368+0000 7fb1e9261640 1 --2- 172.21.15.12:0/284197882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e4100770 0x7fb1e4100b90 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.368+0000 7fb1e9261640 1 -- 172.21.15.12:0/284197882 >> 172.21.15.12:0/284197882 conn(0x7fb1e40fbce0 msgr2=0x7fb1e40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.368+0000 7fb1e9261640 1 -- 172.21.15.12:0/284197882 shutdown_connections 2024-05-09T09:28:58.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.368+0000 7fb1e9261640 1 -- 172.21.15.12:0/284197882 wait complete. 2024-05-09T09:28:58.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e9261640 1 Processor -- start 2024-05-09T09:28:58.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e9261640 1 -- start start 2024-05-09T09:28:58.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e9261640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e4100770 0x7fb1e4069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e9261640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e4101960 0x7fb1e419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e9261640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4102320 0x7fb1e419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e9261640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e410c3d0 con 0x7fb1e4100770 2024-05-09T09:28:58.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e9261640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e410c250 con 0x7fb1e4101960 2024-05-09T09:28:58.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e9261640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e410c550 con 0x7fb1e4102320 2024-05-09T09:28:58.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e4100770 0x7fb1e4069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e4100770 0x7fb1e4069550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47564/0 (socket says 172.21.15.12:47564) 2024-05-09T09:28:58.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e2ffd640 1 -- 172.21.15.12:0/848917769 learned_addr learned my addr 172.21.15.12:0/848917769 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e37fe640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4102320 0x7fb1e419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1dbfff640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e4101960 0x7fb1e419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e2ffd640 1 -- 172.21.15.12:0/848917769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4102320 msgr2=0x7fb1e419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e2ffd640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4102320 0x7fb1e419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e2ffd640 1 -- 172.21.15.12:0/848917769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e4101960 msgr2=0x7fb1e419c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e2ffd640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e4101960 0x7fb1e419c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.369+0000 7fb1e2ffd640 1 -- 172.21.15.12:0/848917769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e419f820 con 0x7fb1e4100770 2024-05-09T09:28:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.370+0000 7fb1e2ffd640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e4100770 0x7fb1e4069550 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4005ab0 tx=0x7fb1d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.370+0000 7fb1e0ff9640 1 -- 172.21.15.12:0/848917769 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d403f030 con 0x7fb1e4100770 2024-05-09T09:28:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.370+0000 7fb1e0ff9640 1 -- 172.21.15.12:0/848917769 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb1d4003ac0 con 0x7fb1e4100770 2024-05-09T09:28:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.370+0000 7fb1e9261640 1 -- 172.21.15.12:0/848917769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e419fab0 con 0x7fb1e4100770 2024-05-09T09:28:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.370+0000 7fb1e0ff9640 1 -- 172.21.15.12:0/848917769 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4035070 con 0x7fb1e4100770 2024-05-09T09:28:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.370+0000 7fb1e9261640 1 -- 172.21.15.12:0/848917769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e419ffa0 con 0x7fb1e4100770 2024-05-09T09:28:58.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.373+0000 7fb1e0ff9640 1 -- 172.21.15.12:0/848917769 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb1d4042030 con 0x7fb1e4100770 2024-05-09T09:28:58.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.373+0000 7fb1ebf68740 1 Processor -- start 2024-05-09T09:28:58.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.373+0000 7fb1ebf68740 1 -- start start 2024-05-09T09:28:58.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.374+0000 7fb1ebf68740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff4729770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.374+0000 7fb1ebf68740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff4729d40 0x55fff472e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.374+0000 7fb1ebf68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff472eeb0 0x55fff4731340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.374+0000 7fb1ebf68740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fff4689c40 con 0x55fff4729350 2024-05-09T09:28:58.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.374+0000 7fb1ebf68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fff4631580 con 0x55fff472eeb0 2024-05-09T09:28:58.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.374+0000 7fb1ebf68740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fff4685c00 con 0x55fff4729d40 2024-05-09T09:28:58.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.374+0000 7fb1e2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff4729d40 0x55fff472e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.374+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff4729770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.374+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff4729770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47566/0 (socket says 172.21.15.12:47566) 2024-05-09T09:28:58.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.374+0000 7fb1dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff472eeb0 0x55fff4731340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.374+0000 7fb1e37fe640 1 -- 172.21.15.12:0/4038308669 learned_addr learned my addr 172.21.15.12:0/4038308669 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.375+0000 7fb1e37fe640 1 -- 172.21.15.12:0/4038308669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff4729d40 msgr2=0x55fff472e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.375+0000 7fb1e37fe640 1 --2- 172.21.15.12:0/4038308669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff4729d40 0x55fff472e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.375+0000 7fb1e37fe640 1 -- 172.21.15.12:0/4038308669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff472eeb0 msgr2=0x55fff4731340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.375+0000 7fb1e37fe640 1 --2- 172.21.15.12:0/4038308669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff472eeb0 0x55fff4731340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.375+0000 7fb1e37fe640 1 -- 172.21.15.12:0/4038308669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fff439d3f0 con 0x55fff4729350 2024-05-09T09:28:58.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.375+0000 7fb1e0ff9640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb1d00a2b90 0x7fb1d00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.375+0000 7fb1e37fe640 1 --2- 172.21.15.12:0/4038308669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff4729770 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0039f0 tx=0x7fb1dc003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1479869c9b035564 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.375+0000 7fb1e2ffd640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb1d00a2b90 0x7fb1d00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.375+0000 7fb1d97fa640 1 -- 172.21.15.12:0/4038308669 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc01a470 con 0x55fff4729350 2024-05-09T09:28:58.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.375+0000 7fb1e0ff9640 1 -- 172.21.15.12:0/848917769 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb1d40ef9a0 con 0x7fb1e4100770 2024-05-09T09:28:58.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.375+0000 7fb1d97fa640 1 -- 172.21.15.12:0/4038308669 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb1dc015070 con 0x55fff4729350 2024-05-09T09:28:58.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.375+0000 7fb1d97fa640 1 -- 172.21.15.12:0/4038308669 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc010880 con 0x55fff4729350 2024-05-09T09:28:58.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.376+0000 7fb1ebf68740 1 -- 172.21.15.12:0/4038308669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 msgr2=0x55fff4729770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.376+0000 7fb1e2ffd640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb1d00a2b90 0x7fb1d00a5050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40bbb80 tx=0x7fb1d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.376+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/4038308669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff4729770 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0039f0 tx=0x7fb1dc003eb0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.376+0000 7fb1ebf68740 1 -- 172.21.15.12:0/4038308669 shutdown_connections 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.376+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/4038308669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff472eeb0 0x55fff4731340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.376+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/4038308669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff4729d40 0x55fff472e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.376+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/4038308669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff4729770 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.376+0000 7fb1ebf68740 1 -- 172.21.15.12:0/4038308669 >> 172.21.15.12:0/4038308669 conn(0x55fff47249c0 msgr2=0x55fff4724dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.376+0000 7fb1ebf68740 1 -- 172.21.15.12:0/4038308669 shutdown_connections 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.376+0000 7fb1ebf68740 1 -- 172.21.15.12:0/4038308669 wait complete. 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.377+0000 7fb1ebf68740 1 -- start start 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.377+0000 7fb1ebf68740 1 Processor -- start 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.377+0000 7fb1ebf68740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff47337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.377+0000 7fb1ebf68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff4729d40 0x55fff4733d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.377+0000 7fb1ebf68740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff472eeb0 0x55fff473ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.377+0000 7fb1ebf68740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fff4731e60 con 0x55fff4729350 2024-05-09T09:28:58.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.377+0000 7fb1ebf68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fff4644c40 con 0x55fff4729d40 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.377+0000 7fb1ebf68740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fff4731fe0 con 0x55fff472eeb0 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.377+0000 7fb1ebf68740 10 client.0 _ll_register_callbacks cb 0x55fff439cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.377+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff47337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.377+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff47337d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47578/0 (socket says 172.21.15.12:47578) 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.377+0000 7fb1dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff4729d40 0x55fff4733d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.377+0000 7fb1e37fe640 1 -- 172.21.15.12:0/4118655063 learned_addr learned my addr 172.21.15.12:0/4118655063 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.378+0000 7fb1dbfff640 1 -- 172.21.15.12:0/4118655063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff472eeb0 msgr2=0x55fff473ad50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.378+0000 7fb1dbfff640 1 --2- 172.21.15.12:0/4118655063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff472eeb0 0x55fff473ad50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.378+0000 7fb1dbfff640 1 -- 172.21.15.12:0/4118655063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 msgr2=0x55fff47337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.378+0000 7fb1dbfff640 1 --2- 172.21.15.12:0/4118655063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff47337d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.378+0000 7fb1dbfff640 1 -- 172.21.15.12:0/4118655063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fff44677d0 con 0x55fff4729d40 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.378+0000 7fb1e37fe640 1 --2- 172.21.15.12:0/4118655063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff47337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.378+0000 7fb1dbfff640 1 --2- 172.21.15.12:0/4118655063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff4729d40 0x55fff4733d10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40692c0 tx=0x7fb1cc012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea1b503b25f72453 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.378+0000 7fb1ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:28:58.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.379+0000 7fb1ca7fc640 1 -- 172.21.15.12:0/4118655063 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc00f920 con 0x55fff4729d40 2024-05-09T09:28:58.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.379+0000 7fb1ca7fc640 1 -- 172.21.15.12:0/4118655063 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb1cc00fac0 con 0x55fff4729d40 2024-05-09T09:28:58.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.379+0000 7fb1ebf68740 1 -- 172.21.15.12:0/4118655063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1dc015070 con 0x55fff4729d40 2024-05-09T09:28:58.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.379+0000 7fb1ca7fc640 1 -- 172.21.15.12:0/4118655063 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb1cc00fcc0 con 0x55fff4729d40 2024-05-09T09:28:58.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.379+0000 7fb1ebf68740 10 client.26342 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:58.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.379+0000 7fb1ebf68740 1 -- 172.21.15.12:0/4118655063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fff473e130 con 0x55fff4729d40 2024-05-09T09:28:58.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.379+0000 7fb1ca7fc640 1 -- 172.21.15.12:0/4118655063 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb1dc015070 con 0x55fff4729d40 2024-05-09T09:28:58.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.379+0000 7fb1ca7fc640 1 -- 172.21.15.12:0/4118655063 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fb1cc01c420 con 0x55fff4729d40 2024-05-09T09:28:58.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.379+0000 7fb1ebf68740 10 client.26342 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:58.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.379+0000 7fb1ebf68740 10 client.26342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:58.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.379+0000 7fb1ebf68740 20 client.26342 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:58.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.379+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/4118655063 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fff473f8d0 0x55fff4741d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.380+0000 7fb1ebf68740 4 client.26342 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:58.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.380+0000 7fb1ebf68740 1 -- 172.21.15.12:0/4118655063 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fff4697370 con 0x55fff473f8d0 2024-05-09T09:28:58.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.380+0000 7fb1e37fe640 1 --2- 172.21.15.12:0/4118655063 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fff473f8d0 0x55fff4741d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.380+0000 7fb1e37fe640 1 --2- 172.21.15.12:0/4118655063 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fff473f8d0 0x55fff4741d50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13bc8acb2f865593 server_cookie=89d51d8c65bbc90a in_seq=0 out_seq=0 2024-05-09T09:28:58.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.380+0000 7fb1ca7fc640 10 client.26342 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:58.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ca7fc640 1 -- 172.21.15.12:0/4118655063 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fff4697370 con 0x55fff473f8d0 2024-05-09T09:28:58.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ca7fc640 10 client.26342 handle_command_reply: tid=0 2024-05-09T09:28:58.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ebf68740 1 client.26342 shutdown 2024-05-09T09:28:58.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ebf68740 1 -- 172.21.15.12:0/4118655063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff4729d40 msgr2=0x55fff4733d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/4118655063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff4729d40 0x55fff4733d10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40692c0 tx=0x7fb1cc012b60 comp rx=0 tx=0).stop 2024-05-09T09:28:58.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ebf68740 1 -- 172.21.15.12:0/4118655063 shutdown_connections 2024-05-09T09:28:58.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/4118655063 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fff473f8d0 0x55fff4741d50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/4118655063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff472eeb0 0x55fff473ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/4118655063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff4729d40 0x55fff4733d10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ca7fc640 0 client.26342 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:58.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/4118655063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff47337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ebf68740 1 -- 172.21.15.12:0/4118655063 >> 172.21.15.12:0/4118655063 conn(0x55fff47249c0 msgr2=0x55fff471f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ebf68740 1 -- 172.21.15.12:0/4118655063 shutdown_connections 2024-05-09T09:28:58.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ebf68740 1 -- 172.21.15.12:0/4118655063 wait complete. 2024-05-09T09:28:58.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.381+0000 7fb1ebf68740 20 client.26342 trim_cache size 0 max 16384 2024-05-09T09:28:58.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1ebf68740 1 Processor -- start 2024-05-09T09:28:58.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1ebf68740 1 -- start start 2024-05-09T09:28:58.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1ebf68740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff4732410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1ebf68740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff4729d40 0x55fff473ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1ebf68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff472eeb0 0x55fff4732950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1ebf68740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41a8470 con 0x55fff4729350 2024-05-09T09:28:58.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1ebf68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41aa1f0 con 0x55fff472eeb0 2024-05-09T09:28:58.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1ebf68740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41a8c00 con 0x55fff4729d40 2024-05-09T09:28:58.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff4732410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff4732410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47580/0 (socket says 172.21.15.12:47580) 2024-05-09T09:28:58.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1e37fe640 1 -- 172.21.15.12:0/1190221440 learned_addr learned my addr 172.21.15.12:0/1190221440 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1dbfff640 1 --2- 172.21.15.12:0/1190221440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff4729d40 0x55fff473ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1e37fe640 1 --2- 172.21.15.12:0/1190221440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff472eeb0 0x55fff4732950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1dbfff640 1 -- 172.21.15.12:0/1190221440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff472eeb0 msgr2=0x55fff4732950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1dbfff640 1 --2- 172.21.15.12:0/1190221440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff472eeb0 0x55fff4732950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1dbfff640 1 -- 172.21.15.12:0/1190221440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 msgr2=0x55fff4732410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1dbfff640 1 --2- 172.21.15.12:0/1190221440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff4732410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1dbfff640 1 -- 172.21.15.12:0/1190221440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fff47423d0 con 0x55fff4729d40 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.396+0000 7fb1e37fe640 1 --2- 172.21.15.12:0/1190221440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff4732410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.397+0000 7fb1dbfff640 1 --2- 172.21.15.12:0/1190221440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff4729d40 0x55fff473ad10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc000c00 tx=0x7fb1cc01dfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a732259efde5593 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.397+0000 7fb1d8ff9640 1 -- 172.21.15.12:0/1190221440 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc00f920 con 0x55fff4729d40 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.397+0000 7fb1d8ff9640 1 -- 172.21.15.12:0/1190221440 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb1cc0055e0 con 0x55fff4729d40 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.397+0000 7fb1d8ff9640 1 -- 172.21.15.12:0/1190221440 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc003850 con 0x55fff4729d40 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.397+0000 7fb1ebf68740 1 -- 172.21.15.12:0/1190221440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff4729d40 msgr2=0x55fff473ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.397+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/1190221440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff4729d40 0x55fff473ad10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc000c00 tx=0x7fb1cc01dfb0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 -- 172.21.15.12:0/1190221440 shutdown_connections 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/1190221440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff472eeb0 0x55fff4732950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/1190221440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff4729d40 0x55fff473ad10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/1190221440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729350 0x55fff4732410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 -- 172.21.15.12:0/1190221440 >> 172.21.15.12:0/1190221440 conn(0x55fff47249c0 msgr2=0x55fff47337d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 -- 172.21.15.12:0/1190221440 shutdown_connections 2024-05-09T09:28:58.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 -- 172.21.15.12:0/1190221440 wait complete. 2024-05-09T09:28:58.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 -- start start 2024-05-09T09:28:58.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 Processor -- start 2024-05-09T09:28:58.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff4729350 0x55fff47377c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729d40 0x55fff4737d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff472eeb0 0x55fff47a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41a9b20 con 0x55fff4729d40 2024-05-09T09:28:58.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41a9390 con 0x55fff4729350 2024-05-09T09:28:58.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41a6450 con 0x55fff472eeb0 2024-05-09T09:28:58.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1ebf68740 10 client.0 _ll_register_callbacks cb 0x7fb1cc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:58.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1dbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729d40 0x55fff4737d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff4729350 0x55fff47377c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff4729350 0x55fff47377c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.12:59322/0 (socket says 172.21.15.12:59322) 2024-05-09T09:28:58.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1dbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729d40 0x55fff4737d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47592/0 (socket says 172.21.15.12:47592) 2024-05-09T09:28:58.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1e37fe640 1 -- 172.21.15.12:0/163556443 learned_addr learned my addr 172.21.15.12:0/163556443 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:58.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.398+0000 7fb1e37fe640 1 --2- 172.21.15.12:0/163556443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff472eeb0 0x55fff47a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.399+0000 7fb1e37fe640 1 -- 172.21.15.12:0/163556443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff472eeb0 msgr2=0x55fff47a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.399+0000 7fb1e37fe640 1 --2- 172.21.15.12:0/163556443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff472eeb0 0x55fff47a3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.399+0000 7fb1e37fe640 1 -- 172.21.15.12:0/163556443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729d40 msgr2=0x55fff4737d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.399+0000 7fb1e37fe640 1 --2- 172.21.15.12:0/163556443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729d40 0x55fff4737d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.399+0000 7fb1e37fe640 1 -- 172.21.15.12:0/163556443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fff473add0 con 0x55fff4729350 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.399+0000 7fb1e37fe640 1 --2- 172.21.15.12:0/163556443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff4729350 0x55fff47377c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc005d70 tx=0x7fb1dc01e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97a44b666815a638 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.399+0000 7fb1ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.399+0000 7fb1ca7fc640 1 -- 172.21.15.12:0/163556443 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc02a900 con 0x55fff4729350 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.399+0000 7fb1ebf68740 1 -- 172.21.15.12:0/163556443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1cc0055e0 con 0x55fff4729350 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.399+0000 7fb1ca7fc640 1 -- 172.21.15.12:0/163556443 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb1dc004860 con 0x55fff4729350 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.399+0000 7fb1ca7fc640 1 -- 172.21.15.12:0/163556443 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb1dc004a60 con 0x55fff4729350 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.399+0000 7fb1ebf68740 10 client.26348 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.399+0000 7fb1ebf68740 1 -- 172.21.15.12:0/163556443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fff47a5280 con 0x55fff4729350 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.400+0000 7fb1ca7fc640 1 -- 172.21.15.12:0/163556443 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb1cc0055e0 con 0x55fff4729350 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.400+0000 7fb1ca7fc640 1 -- 172.21.15.12:0/163556443 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fb1dc01a7d0 con 0x55fff4729350 2024-05-09T09:28:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.400+0000 7fb1ebf68740 10 client.26348 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.400+0000 7fb1ebf68740 10 client.26348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.400+0000 7fb1ebf68740 20 client.26348 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.400+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/163556443 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fff47a5ac0 0x55fff47a7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.400+0000 7fb1ebf68740 4 client.26348 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.400+0000 7fb1ebf68740 1 -- 172.21.15.12:0/163556443 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fff46916c0 con 0x55fff47a5ac0 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.400+0000 7fb1dbfff640 1 --2- 172.21.15.12:0/163556443 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fff47a5ac0 0x55fff47a7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.400+0000 7fb1dbfff640 1 --2- 172.21.15.12:0/163556443 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fff47a5ac0 0x55fff47a7ee0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb81197b437429e server_cookie=4f986e0a826086c7 in_seq=0 out_seq=0 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.400+0000 7fb1ca7fc640 10 client.26348 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ca7fc640 1 -- 172.21.15.12:0/163556443 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fff46916c0 con 0x55fff47a5ac0 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ca7fc640 10 client.26348 handle_command_reply: tid=0 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ebf68740 1 client.26348 shutdown 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ebf68740 1 -- 172.21.15.12:0/163556443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff4729350 msgr2=0x55fff47377c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/163556443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff4729350 0x55fff47377c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc005d70 tx=0x7fb1dc01e040 comp rx=0 tx=0).stop 2024-05-09T09:28:58.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ebf68740 1 -- 172.21.15.12:0/163556443 shutdown_connections 2024-05-09T09:28:58.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/163556443 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55fff47a5ac0 0x55fff47a7ee0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/163556443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fff472eeb0 0x55fff47a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/163556443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fff4729d40 0x55fff4737d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ca7fc640 0 client.26348 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:58.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ebf68740 1 --2- 172.21.15.12:0/163556443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fff4729350 0x55fff47377c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ebf68740 1 -- 172.21.15.12:0/163556443 >> 172.21.15.12:0/163556443 conn(0x55fff47249c0 msgr2=0x55fff4732410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ebf68740 1 -- 172.21.15.12:0/163556443 shutdown_connections 2024-05-09T09:28:58.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ebf68740 1 -- 172.21.15.12:0/163556443 wait complete. 2024-05-09T09:28:58.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1ebf68740 20 client.26348 trim_cache size 0 max 16384 2024-05-09T09:28:58.407 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:28:58.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1e9261640 1 -- 172.21.15.12:0/848917769 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb1d00a2b90 msgr2=0x7fb1d00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1e9261640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb1d00a2b90 0x7fb1d00a5050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40bbb80 tx=0x7fb1d4009260 comp rx=0 tx=0).stop 2024-05-09T09:28:58.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1e9261640 1 -- 172.21.15.12:0/848917769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e4100770 msgr2=0x7fb1e4069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:58.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.401+0000 7fb1e9261640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e4100770 0x7fb1e4069550 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4005ab0 tx=0x7fb1d40042f0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.402+0000 7fb1e9261640 1 -- 172.21.15.12:0/848917769 shutdown_connections 2024-05-09T09:28:58.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.402+0000 7fb1e9261640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fb1d00a2b90 0x7fb1d00a5050 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.402+0000 7fb1e9261640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4102320 0x7fb1e419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.402+0000 7fb1e9261640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e4101960 0x7fb1e419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.402+0000 7fb1e9261640 1 --2- 172.21.15.12:0/848917769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e4100770 0x7fb1e4069550 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:58.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.402+0000 7fb1e9261640 1 -- 172.21.15.12:0/848917769 >> 172.21.15.12:0/848917769 conn(0x7fb1e40fbce0 msgr2=0x7fb1e4104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:58.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.402+0000 7fb1e9261640 1 -- 172.21.15.12:0/848917769 shutdown_connections 2024-05-09T09:28:58.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:58.402+0000 7fb1e9261640 1 -- 172.21.15.12:0/848917769 wait complete. 2024-05-09T09:28:58.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:58 smithi012 ceph-mon[30140]: pgmap v470: 129 pgs: 129 active+clean; 69 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.1 MiB/s wr, 424 op/s 2024-05-09T09:28:58.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:28:58 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3960596527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:28:58.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:58 smithi170 ceph-mon[33632]: pgmap v470: 129 pgs: 129 active+clean; 69 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.1 MiB/s wr, 424 op/s 2024-05-09T09:28:58.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:28:58 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3960596527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:28:58.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:58 smithi016 ceph-mon[33335]: pgmap v470: 129 pgs: 129 active+clean; 69 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.1 MiB/s wr, 424 op/s 2024-05-09T09:28:58.504 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:28:58 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3960596527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:28:59.413 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:28:59.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:28:59.414 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:28:59.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.530+0000 7f6908aa5640 1 -- 172.21.15.12:0/87711115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6904104cc0 msgr2=0x7f69041070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.530+0000 7f6908aa5640 1 --2- 172.21.15.12:0/87711115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6904104cc0 0x7f69041070b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009900 tx=0x7f68f802a410 comp rx=0 tx=0).stop 2024-05-09T09:28:59.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.531+0000 7f6908aa5640 1 -- 172.21.15.12:0/87711115 shutdown_connections 2024-05-09T09:28:59.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.531+0000 7f6908aa5640 1 --2- 172.21.15.12:0/87711115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69041075f0 0x7f69041099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.531+0000 7f6908aa5640 1 --2- 172.21.15.12:0/87711115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6904104cc0 0x7f69041070b0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.531+0000 7f6908aa5640 1 --2- 172.21.15.12:0/87711115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6904102390 0x7f6904104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.531+0000 7f6908aa5640 1 -- 172.21.15.12:0/87711115 >> 172.21.15.12:0/87711115 conn(0x7f69040fbce0 msgr2=0x7f69040fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:59.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.531+0000 7f6908aa5640 1 -- 172.21.15.12:0/87711115 shutdown_connections 2024-05-09T09:28:59.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.531+0000 7f6908aa5640 1 -- 172.21.15.12:0/87711115 wait complete. 2024-05-09T09:28:59.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.531+0000 7f6908aa5640 1 Processor -- start 2024-05-09T09:28:59.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f6908aa5640 1 -- start start 2024-05-09T09:28:59.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f6908aa5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6904102390 0x7f6904197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f6908aa5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6904104cc0 0x7f6904198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f6908aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69041075f0 0x7f690419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f6908aa5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f690410d4f0 con 0x7f6904102390 2024-05-09T09:28:59.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f6908aa5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f690410d370 con 0x7f69041075f0 2024-05-09T09:28:59.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f6908aa5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f690410d670 con 0x7f6904104cc0 2024-05-09T09:28:59.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f690259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6904102390 0x7f6904197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f690259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6904102390 0x7f6904197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47602/0 (socket says 172.21.15.12:47602) 2024-05-09T09:28:59.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f690259c640 1 -- 172.21.15.12:0/3862141902 learned_addr learned my addr 172.21.15.12:0/3862141902 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:59.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f6901d9b640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6904104cc0 0x7f6904198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f6902d9d640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69041075f0 0x7f690419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f690259c640 1 -- 172.21.15.12:0/3862141902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6904104cc0 msgr2=0x7f6904198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f690259c640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6904104cc0 0x7f6904198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f690259c640 1 -- 172.21.15.12:0/3862141902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69041075f0 msgr2=0x7f690419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f690259c640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69041075f0 0x7f690419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f690259c640 1 -- 172.21.15.12:0/3862141902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690419f830 con 0x7f6904102390 2024-05-09T09:28:59.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f6901d9b640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6904104cc0 0x7f6904198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:59.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.532+0000 7f690259c640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6904102390 0x7f6904197c20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f68f400d950 tx=0x7f68f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:59.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.533+0000 7f68f37fe640 1 -- 172.21.15.12:0/3862141902 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4017070 con 0x7f6904102390 2024-05-09T09:28:59.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.533+0000 7f6908aa5640 1 -- 172.21.15.12:0/3862141902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690419fb20 con 0x7f6904102390 2024-05-09T09:28:59.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.533+0000 7f68f37fe640 1 -- 172.21.15.12:0/3862141902 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f68f4009c30 con 0x7f6904102390 2024-05-09T09:28:59.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.533+0000 7f68f37fe640 1 -- 172.21.15.12:0/3862141902 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f400f6b0 con 0x7f6904102390 2024-05-09T09:28:59.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.533+0000 7f6908aa5640 1 -- 172.21.15.12:0/3862141902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69041a0070 con 0x7f6904102390 2024-05-09T09:28:59.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.535+0000 7f68f37fe640 1 -- 172.21.15.12:0/3862141902 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68f4010a50 con 0x7f6904102390 2024-05-09T09:28:59.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.535+0000 7f690b7b5740 1 Processor -- start 2024-05-09T09:28:59.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.535+0000 7f690b7b5740 1 -- start start 2024-05-09T09:28:59.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f690b7b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a2027008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f690b7b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a2027059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f690b7b5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202706020 0x55a2027084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f690b7b5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a202664990 con 0x55a202706020 2024-05-09T09:28:59.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f690b7b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a20260c5b0 con 0x55a202700eb0 2024-05-09T09:28:59.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f690b7b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a202660950 con 0x55a2027004c0 2024-05-09T09:28:59.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f690259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a2027059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f6902d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a2027008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f6901d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202706020 0x55a2027084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f6902d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a2027008e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:57948/0 (socket says 172.21.15.12:57948) 2024-05-09T09:28:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f68f37fe640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68ec0a2b90 0x7f68ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f6902d9d640 1 -- 172.21.15.12:0/3035275892 learned_addr learned my addr 172.21.15.12:0/3035275892 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f6901d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202706020 0x55a2027084b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47614/0 (socket says 172.21.15.12:47614) 2024-05-09T09:28:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f6902d9d640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68ec0a2b90 0x7f68ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f690259c640 1 -- 172.21.15.12:0/3035275892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 msgr2=0x55a2027008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f690259c640 1 --2- 172.21.15.12:0/3035275892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a2027008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f690259c640 1 -- 172.21.15.12:0/3035275892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202706020 msgr2=0x55a2027084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f690259c640 1 --2- 172.21.15.12:0/3035275892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202706020 0x55a2027084b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f690259c640 1 -- 172.21.15.12:0/3035275892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2024407e0 con 0x55a202700eb0 2024-05-09T09:28:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f6902d9d640 1 --2- 172.21.15.12:0/3035275892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a2027008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.536+0000 7f68f37fe640 1 -- 172.21.15.12:0/3862141902 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f68f40cf130 con 0x7f6904102390 2024-05-09T09:28:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.537+0000 7f690259c640 1 --2- 172.21.15.12:0/3035275892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a2027059b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f68f409b260 tx=0x7f68f409aff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=454f406316dc934f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.537+0000 7f6902d9d640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68ec0a2b90 0x7f68ec0a5050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55a202701710 tx=0x7f68fc004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.537+0000 7f68f0ff9640 1 -- 172.21.15.12:0/3035275892 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4020070 con 0x55a202700eb0 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.537+0000 7f68f0ff9640 1 -- 172.21.15.12:0/3035275892 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f68f4013050 con 0x55a202700eb0 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.537+0000 7f68f0ff9640 1 -- 172.21.15.12:0/3035275892 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f40a41d0 con 0x55a202700eb0 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 -- 172.21.15.12:0/3035275892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 msgr2=0x55a2027059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 --2- 172.21.15.12:0/3035275892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a2027059b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f68f409b260 tx=0x7f68f409aff0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 -- 172.21.15.12:0/3035275892 shutdown_connections 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 --2- 172.21.15.12:0/3035275892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202706020 0x55a2027084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 --2- 172.21.15.12:0/3035275892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a2027059b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 --2- 172.21.15.12:0/3035275892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a2027008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 -- 172.21.15.12:0/3035275892 >> 172.21.15.12:0/3035275892 conn(0x55a2026c80a0 msgr2=0x55a2026fbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 -- 172.21.15.12:0/3035275892 shutdown_connections 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 -- 172.21.15.12:0/3035275892 wait complete. 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 -- start start 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 Processor -- start 2024-05-09T09:28:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a20270a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a20270ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202706020 0x55a202711d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a202708f10 con 0x55a202706020 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a20261fb40 con 0x55a202700eb0 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.538+0000 7f690b7b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a202709090 con 0x55a2027004c0 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f690b7b5740 10 client.0 _ll_register_callbacks cb 0x55a2026140f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f6902d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202706020 0x55a202711d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f690259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a20270a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f6902d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202706020 0x55a202711d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47624/0 (socket says 172.21.15.12:47624) 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f6902d9d640 1 -- 172.21.15.12:0/4227688650 learned_addr learned my addr 172.21.15.12:0/4227688650 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f6901d9b640 1 --2- 172.21.15.12:0/4227688650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a20270ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f690259c640 1 -- 172.21.15.12:0/4227688650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 msgr2=0x55a20270ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f690259c640 1 --2- 172.21.15.12:0/4227688650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a20270ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f690259c640 1 -- 172.21.15.12:0/4227688650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202706020 msgr2=0x55a202711d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f690259c640 1 --2- 172.21.15.12:0/4227688650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202706020 0x55a202711d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f690259c640 1 -- 172.21.15.12:0/4227688650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2024a0f70 con 0x55a2027004c0 2024-05-09T09:28:59.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f6901d9b640 1 --2- 172.21.15.12:0/4227688650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a20270ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:59.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f690259c640 1 --2- 172.21.15.12:0/4227688650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a20270a7f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f68f40cf420 tx=0x7f68f40bf400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=804b5d282676b4b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:59.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f68e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:28:59.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f68e9ffb640 1 -- 172.21.15.12:0/4227688650 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4020040 con 0x55a2027004c0 2024-05-09T09:28:59.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f690b7b5740 1 -- 172.21.15.12:0/4227688650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a2026720c0 con 0x55a2027004c0 2024-05-09T09:28:59.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.539+0000 7f68e9ffb640 1 -- 172.21.15.12:0/4227688650 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f68f404a110 con 0x55a2027004c0 2024-05-09T09:28:59.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.540+0000 7f68e9ffb640 1 -- 172.21.15.12:0/4227688650 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f68f409bc60 con 0x55a2027004c0 2024-05-09T09:28:59.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.540+0000 7f690b7b5740 10 client.26476 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:59.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.540+0000 7f690b7b5740 1 -- 172.21.15.12:0/4227688650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a202338170 con 0x55a2027004c0 2024-05-09T09:28:59.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.540+0000 7f68e9ffb640 1 -- 172.21.15.12:0/4227688650 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55a2026720c0 con 0x55a2027004c0 2024-05-09T09:28:59.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.540+0000 7f68e9ffb640 1 -- 172.21.15.12:0/4227688650 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f68f40a3d40 con 0x55a2027004c0 2024-05-09T09:28:59.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.540+0000 7f690b7b5740 10 client.26476 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:59.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.540+0000 7f690b7b5740 10 client.26476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:59.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.540+0000 7f690b7b5740 20 client.26476 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:59.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.540+0000 7f690b7b5740 1 --2- 172.21.15.12:0/4227688650 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a202715ed0 0x55a2027182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.540+0000 7f690b7b5740 4 client.26476 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:59.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.540+0000 7f690b7b5740 1 -- 172.21.15.12:0/4227688650 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a20266c410 con 0x55a202715ed0 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.540+0000 7f6901d9b640 1 --2- 172.21.15.12:0/4227688650 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a202715ed0 0x55a2027182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.541+0000 7f6901d9b640 1 --2- 172.21.15.12:0/4227688650 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a202715ed0 0x55a2027182c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471528e7600d5748 server_cookie=3939b88a141eb3e3 in_seq=0 out_seq=0 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.541+0000 7f68e9ffb640 10 client.26476 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.541+0000 7f68e9ffb640 1 -- 172.21.15.12:0/4227688650 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a20266c410 con 0x55a202715ed0 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.541+0000 7f68e9ffb640 10 client.26476 handle_command_reply: tid=0 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.542+0000 7f690b7b5740 1 client.26476 shutdown 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.542+0000 7f690b7b5740 1 -- 172.21.15.12:0/4227688650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 msgr2=0x55a20270a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.542+0000 7f690b7b5740 1 --2- 172.21.15.12:0/4227688650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a20270a7f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f68f40cf420 tx=0x7f68f40bf400 comp rx=0 tx=0).stop 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.542+0000 7f690b7b5740 1 -- 172.21.15.12:0/4227688650 shutdown_connections 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.542+0000 7f690b7b5740 1 --2- 172.21.15.12:0/4227688650 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a202715ed0 0x55a2027182c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.542+0000 7f690b7b5740 1 --2- 172.21.15.12:0/4227688650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202706020 0x55a202711d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.542+0000 7f690b7b5740 1 --2- 172.21.15.12:0/4227688650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a20270ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.542+0000 7f690b7b5740 1 --2- 172.21.15.12:0/4227688650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a20270a7f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.542+0000 7f68e9ffb640 0 client.26476 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:59.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.542+0000 7f690b7b5740 1 -- 172.21.15.12:0/4227688650 >> 172.21.15.12:0/4227688650 conn(0x55a2026c80a0 msgr2=0x55a2026fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:59.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.542+0000 7f690b7b5740 1 -- 172.21.15.12:0/4227688650 shutdown_connections 2024-05-09T09:28:59.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.542+0000 7f690b7b5740 1 -- 172.21.15.12:0/4227688650 wait complete. 2024-05-09T09:28:59.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.542+0000 7f690b7b5740 20 client.26476 trim_cache size 0 max 16384 2024-05-09T09:28:59.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.561+0000 7f690b7b5740 1 Processor -- start 2024-05-09T09:28:59.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690b7b5740 1 -- start start 2024-05-09T09:28:59.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690b7b5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2027004c0 0x55a20270a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690b7b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a202711d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690b7b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a202706020 0x55a20270a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690b7b5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f69041aa670 con 0x55a2027004c0 2024-05-09T09:28:59.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690b7b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69041ac3f0 con 0x55a202700eb0 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690b7b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f69041aae00 con 0x55a202706020 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2027004c0 0x55a20270a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2027004c0 0x55a20270a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47630/0 (socket says 172.21.15.12:47630) 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690259c640 1 -- 172.21.15.12:0/2364339716 learned_addr learned my addr 172.21.15.12:0/2364339716 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f6902d9d640 1 --2- 172.21.15.12:0/2364339716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a202706020 0x55a20270a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f6901d9b640 1 --2- 172.21.15.12:0/2364339716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a202711d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690259c640 1 -- 172.21.15.12:0/2364339716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a202706020 msgr2=0x55a20270a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690259c640 1 --2- 172.21.15.12:0/2364339716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a202706020 0x55a20270a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690259c640 1 -- 172.21.15.12:0/2364339716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 msgr2=0x55a202711d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690259c640 1 --2- 172.21.15.12:0/2364339716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a202711d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690259c640 1 -- 172.21.15.12:0/2364339716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a202705d30 con 0x55a2027004c0 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f690259c640 1 --2- 172.21.15.12:0/2364339716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2027004c0 0x55a20270a400 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f68f4097930 tx=0x7f68f4086050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55430277efdefb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.562+0000 7f68ebfff640 1 -- 172.21.15.12:0/2364339716 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4020070 con 0x55a2027004c0 2024-05-09T09:28:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.563+0000 7f68ebfff640 1 -- 172.21.15.12:0/2364339716 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f68f4097590 con 0x55a2027004c0 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.563+0000 7f68ebfff640 1 -- 172.21.15.12:0/2364339716 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f409a420 con 0x55a2027004c0 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.563+0000 7f690b7b5740 1 -- 172.21.15.12:0/2364339716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2027004c0 msgr2=0x55a20270a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.563+0000 7f690b7b5740 1 --2- 172.21.15.12:0/2364339716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2027004c0 0x55a20270a400 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f68f4097930 tx=0x7f68f4086050 comp rx=0 tx=0).stop 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.563+0000 7f690b7b5740 1 -- 172.21.15.12:0/2364339716 shutdown_connections 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.563+0000 7f690b7b5740 1 --2- 172.21.15.12:0/2364339716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a202706020 0x55a20270a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.563+0000 7f690b7b5740 1 --2- 172.21.15.12:0/2364339716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202700eb0 0x55a202711d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.563+0000 7f690b7b5740 1 --2- 172.21.15.12:0/2364339716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2027004c0 0x55a20270a400 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.563+0000 7f690b7b5740 1 -- 172.21.15.12:0/2364339716 >> 172.21.15.12:0/2364339716 conn(0x55a2026c80a0 msgr2=0x55a2027022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.563+0000 7f690b7b5740 1 -- 172.21.15.12:0/2364339716 shutdown_connections 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.563+0000 7f690b7b5740 1 -- 172.21.15.12:0/2364339716 wait complete. 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690b7b5740 1 -- start start 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690b7b5740 1 Processor -- start 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690b7b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a20270eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690b7b5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202700eb0 0x55a20270f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690b7b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202706020 0x55a20277a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690b7b5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f69041abd20 con 0x55a202700eb0 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690b7b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69041ab590 con 0x55a202706020 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690b7b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f69041a8650 con 0x55a2027004c0 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690b7b5740 10 client.0 _ll_register_callbacks cb 0x7f68f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f6901d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202700eb0 0x55a20270f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a20270eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a20270eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:57980/0 (socket says 172.21.15.12:57980) 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690259c640 1 -- 172.21.15.12:0/758193644 learned_addr learned my addr 172.21.15.12:0/758193644 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f6902d9d640 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202706020 0x55a20277a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690259c640 1 -- 172.21.15.12:0/758193644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202706020 msgr2=0x55a20277a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.564+0000 7f690259c640 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202706020 0x55a20277a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f690259c640 1 -- 172.21.15.12:0/758193644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202700eb0 msgr2=0x55a20270f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f690259c640 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202700eb0 0x55a20270f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f690259c640 1 -- 172.21.15.12:0/758193644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a20277a8d0 con 0x55a2027004c0 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f6901d9b640 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202700eb0 0x55a20270f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f6902d9d640 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202706020 0x55a20277a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f690259c640 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a20270eca0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f68f4000c00 tx=0x7f68f4097130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c835d269ea3e6e7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f68e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f68e9ffb640 1 -- 172.21.15.12:0/758193644 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4020070 con 0x55a2027004c0 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f68e9ffb640 1 -- 172.21.15.12:0/758193644 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f68f409abb0 con 0x55a2027004c0 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f690b7b5740 1 -- 172.21.15.12:0/758193644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68f4097590 con 0x55a2027004c0 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f68e9ffb640 1 -- 172.21.15.12:0/758193644 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f68f4009de0 con 0x55a2027004c0 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f690b7b5740 10 client.26482 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f690b7b5740 1 -- 172.21.15.12:0/758193644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a20277a8d0 con 0x55a2027004c0 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f68e9ffb640 1 -- 172.21.15.12:0/758193644 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f68f4097590 con 0x55a2027004c0 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f68e9ffb640 1 -- 172.21.15.12:0/758193644 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f68f40afa80 con 0x55a2027004c0 2024-05-09T09:28:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f690b7b5740 10 client.26482 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.565+0000 7f690b7b5740 10 client.26482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.566+0000 7f690b7b5740 20 client.26482 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.566+0000 7f690b7b5740 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a20277c170 0x55a20277e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.566+0000 7f690b7b5740 4 client.26482 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.566+0000 7f690b7b5740 1 -- 172.21.15.12:0/758193644 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a2025e9da0 con 0x55a20277c170 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.566+0000 7f6901d9b640 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a20277c170 0x55a20277e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.566+0000 7f6901d9b640 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a20277c170 0x55a20277e590 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4880bfe0576867b7 server_cookie=c0853a01d334dee6 in_seq=0 out_seq=0 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.566+0000 7f68e9ffb640 10 client.26482 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f68e9ffb640 1 -- 172.21.15.12:0/758193644 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a2025e9da0 con 0x55a20277c170 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f68e9ffb640 10 client.26482 handle_command_reply: tid=0 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f690b7b5740 1 client.26482 shutdown 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f690b7b5740 1 -- 172.21.15.12:0/758193644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 msgr2=0x55a20270eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f690b7b5740 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a20270eca0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f68f4000c00 tx=0x7f68f4097130 comp rx=0 tx=0).stop 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f690b7b5740 1 -- 172.21.15.12:0/758193644 shutdown_connections 2024-05-09T09:28:59.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f690b7b5740 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55a20277c170 0x55a20277e590 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f690b7b5740 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a202706020 0x55a20277a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f690b7b5740 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a202700eb0 0x55a20270f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f690b7b5740 1 --2- 172.21.15.12:0/758193644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2027004c0 0x55a20270eca0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f68e9ffb640 0 client.26482 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:59.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f690b7b5740 1 -- 172.21.15.12:0/758193644 >> 172.21.15.12:0/758193644 conn(0x55a2026c80a0 msgr2=0x55a2026feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:59.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f690b7b5740 1 -- 172.21.15.12:0/758193644 shutdown_connections 2024-05-09T09:28:59.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f690b7b5740 1 -- 172.21.15.12:0/758193644 wait complete. 2024-05-09T09:28:59.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f690b7b5740 20 client.26482 trim_cache size 0 max 16384 2024-05-09T09:28:59.574 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:28:59.575 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T09:28:59.575 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "64775efd-96e9-40fe-8289-19d3875e5019", 2024-05-09T09:28:59.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T09:28:59.575 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:28:59.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f6908aa5640 1 -- 172.21.15.12:0/3862141902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68ec0a2b90 msgr2=0x7f68ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f6908aa5640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68ec0a2b90 0x7f68ec0a5050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55a202701710 tx=0x7f68fc004ab0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f6908aa5640 1 -- 172.21.15.12:0/3862141902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6904102390 msgr2=0x7f6904197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.567+0000 7f6908aa5640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6904102390 0x7f6904197c20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f68f400d950 tx=0x7f68f400de10 comp rx=0 tx=0).stop 2024-05-09T09:28:59.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.568+0000 7f6908aa5640 1 -- 172.21.15.12:0/3862141902 shutdown_connections 2024-05-09T09:28:59.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.568+0000 7f6908aa5640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f68ec0a2b90 0x7f68ec0a5050 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.568+0000 7f6908aa5640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69041075f0 0x7f690419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.568+0000 7f6908aa5640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6904104cc0 0x7f6904198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.568+0000 7f6908aa5640 1 --2- 172.21.15.12:0/3862141902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6904102390 0x7f6904197c20 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.568+0000 7f6908aa5640 1 -- 172.21.15.12:0/3862141902 >> 172.21.15.12:0/3862141902 conn(0x7f69040fbce0 msgr2=0x7f6904102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:59.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.568+0000 7f6908aa5640 1 -- 172.21.15.12:0/3862141902 shutdown_connections 2024-05-09T09:28:59.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.568+0000 7f6908aa5640 1 -- 172.21.15.12:0/3862141902 wait complete. 2024-05-09T09:28:59.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:28:59.579 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:28:59.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.698+0000 7f8d964dd640 1 -- 172.21.15.12:0/3051577562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d90100790 msgr2=0x7f8d90100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.698+0000 7f8d964dd640 1 --2- 172.21.15.12:0/3051577562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d90100790 0x7f8d90100bb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009900 tx=0x7f8d8002a410 comp rx=0 tx=0).stop 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.698+0000 7f8d964dd640 1 -- 172.21.15.12:0/3051577562 shutdown_connections 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.698+0000 7f8d964dd640 1 --2- 172.21.15.12:0/3051577562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d90102340 0x7f8d90108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.698+0000 7f8d964dd640 1 --2- 172.21.15.12:0/3051577562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d90101980 0x7f8d90101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.698+0000 7f8d964dd640 1 --2- 172.21.15.12:0/3051577562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d90100790 0x7f8d90100bb0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.698+0000 7f8d964dd640 1 -- 172.21.15.12:0/3051577562 >> 172.21.15.12:0/3051577562 conn(0x7f8d900fbd20 msgr2=0x7f8d900fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.698+0000 7f8d964dd640 1 -- 172.21.15.12:0/3051577562 shutdown_connections 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.698+0000 7f8d964dd640 1 -- 172.21.15.12:0/3051577562 wait complete. 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.698+0000 7f8d964dd640 1 Processor -- start 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.698+0000 7f8d964dd640 1 -- start start 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d964dd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d90100790 0x7f8d90199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d964dd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d90101980 0x7f8d9019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d964dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d90102340 0x7f8d901a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d964dd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9010c3f0 con 0x7f8d90101980 2024-05-09T09:28:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d964dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9010c270 con 0x7f8d90102340 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d964dd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9010c570 con 0x7f8d90100790 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d90101980 0x7f8d9019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d90101980 0x7f8d9019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47664/0 (socket says 172.21.15.12:47664) 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d90100790 0x7f8d90199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d87fff640 1 -- 172.21.15.12:0/820533677 learned_addr learned my addr 172.21.15.12:0/820533677 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d94a7a640 1 --2- 172.21.15.12:0/820533677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d90102340 0x7f8d901a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d87fff640 1 -- 172.21.15.12:0/820533677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d90100790 msgr2=0x7f8d90199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d87fff640 1 --2- 172.21.15.12:0/820533677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d90100790 0x7f8d90199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d87fff640 1 -- 172.21.15.12:0/820533677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d90102340 msgr2=0x7f8d901a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d87fff640 1 --2- 172.21.15.12:0/820533677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d90102340 0x7f8d901a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d87fff640 1 -- 172.21.15.12:0/820533677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d901a1a60 con 0x7f8d90101980 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d87fff640 1 --2- 172.21.15.12:0/820533677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d90101980 0x7f8d9019a380 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800e9f0 tx=0x7f8d7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d8dffb640 1 -- 172.21.15.12:0/820533677 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7800cd80 con 0x7f8d90101980 2024-05-09T09:28:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d964dd640 1 -- 172.21.15.12:0/820533677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d901a1d50 con 0x7f8d90101980 2024-05-09T09:28:59.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d8dffb640 1 -- 172.21.15.12:0/820533677 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8d780054f0 con 0x7f8d90101980 2024-05-09T09:28:59.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d8dffb640 1 -- 172.21.15.12:0/820533677 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78010690 con 0x7f8d90101980 2024-05-09T09:28:59.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.699+0000 7f8d964dd640 1 -- 172.21.15.12:0/820533677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d901a21f0 con 0x7f8d90101980 2024-05-09T09:28:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.702+0000 7f8d8dffb640 1 -- 172.21.15.12:0/820533677 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8d78004f20 con 0x7f8d90101980 2024-05-09T09:28:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.702+0000 7f8d991ea740 1 Processor -- start 2024-05-09T09:28:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.702+0000 7f8d991ea740 1 -- start start 2024-05-09T09:28:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.702+0000 7f8d991ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 0x55e9426fc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.702+0000 7f8d991ea740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e942701420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.702+0000 7f8d991ea740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e942703f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.702+0000 7f8d991ea740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e942660d00 con 0x55e9426fc920 2024-05-09T09:28:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.702+0000 7f8d991ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9426084c0 con 0x55e9426fbf30 2024-05-09T09:28:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.702+0000 7f8d991ea740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e94265ccc0 con 0x55e942701a90 2024-05-09T09:28:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e942701420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d87fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e942703f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e942701420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47674/0 (socket says 172.21.15.12:47674) 2024-05-09T09:28:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d8ffff640 1 -- 172.21.15.12:0/1517911544 learned_addr learned my addr 172.21.15.12:0/1517911544 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d8dffb640 1 --2- 172.21.15.12:0/820533677 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8d7c0a2b90 0x7f8d7c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d94a7a640 1 --2- 172.21.15.12:0/1517911544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 0x55e9426fc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d94a7a640 1 --2- 172.21.15.12:0/820533677 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8d7c0a2b90 0x7f8d7c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d8ffff640 1 -- 172.21.15.12:0/1517911544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 msgr2=0x55e942703f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d8ffff640 1 --2- 172.21.15.12:0/1517911544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e942703f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d8ffff640 1 -- 172.21.15.12:0/1517911544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 msgr2=0x55e9426fc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d8ffff640 1 --2- 172.21.15.12:0/1517911544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 0x55e9426fc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d8ffff640 1 -- 172.21.15.12:0/1517911544 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e94243e7d0 con 0x55e9426fc920 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d8dffb640 1 -- 172.21.15.12:0/820533677 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8d780cf7f0 con 0x7f8d90101980 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d94a7a640 1 --2- 172.21.15.12:0/1517911544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 0x55e9426fc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d8ffff640 1 --2- 172.21.15.12:0/1517911544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e942701420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8d901017e0 tx=0x7f8d80031c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b5d77007cf4268 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d94a7a640 1 --2- 172.21.15.12:0/820533677 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8d7c0a2b90 0x7f8d7c0a5050 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55e9426fd180 tx=0x7f8d8800a4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d867fc640 1 -- 172.21.15.12:0/1517911544 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80040070 con 0x55e9426fc920 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d867fc640 1 -- 172.21.15.12:0/1517911544 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8d80031810 con 0x55e9426fc920 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.703+0000 7f8d867fc640 1 -- 172.21.15.12:0/1517911544 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80035040 con 0x55e9426fc920 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.704+0000 7f8d991ea740 1 -- 172.21.15.12:0/1517911544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 msgr2=0x55e942701420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.704+0000 7f8d991ea740 1 --2- 172.21.15.12:0/1517911544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e942701420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8d901017e0 tx=0x7f8d80031c00 comp rx=0 tx=0).stop 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.704+0000 7f8d991ea740 1 -- 172.21.15.12:0/1517911544 shutdown_connections 2024-05-09T09:28:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.704+0000 7f8d991ea740 1 --2- 172.21.15.12:0/1517911544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e942703f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.704+0000 7f8d991ea740 1 --2- 172.21.15.12:0/1517911544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e942701420 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.704+0000 7f8d991ea740 1 --2- 172.21.15.12:0/1517911544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 0x55e9426fc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.704+0000 7f8d991ea740 1 -- 172.21.15.12:0/1517911544 >> 172.21.15.12:0/1517911544 conn(0x55e9426f75a0 msgr2=0x55e9426f79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.704+0000 7f8d991ea740 1 -- 172.21.15.12:0/1517911544 shutdown_connections 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.704+0000 7f8d991ea740 1 -- 172.21.15.12:0/1517911544 wait complete. 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.704+0000 7f8d991ea740 1 -- start start 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d991ea740 1 Processor -- start 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d991ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 0x55e9427062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d991ea740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e942706810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d991ea740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e94270d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d991ea740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e9427049e0 con 0x55e9426fc920 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d991ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e94261bb90 con 0x55e9426fbf30 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d991ea740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e942704b60 con 0x55e942701a90 2024-05-09T09:28:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d991ea740 10 client.0 _ll_register_callbacks cb 0x55e942657da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e942706810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e942706810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47688/0 (socket says 172.21.15.12:47688) 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d87fff640 1 -- 172.21.15.12:0/1856273 learned_addr learned my addr 172.21.15.12:0/1856273 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d94a7a640 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e94270d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d8ffff640 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 0x55e9427062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d87fff640 1 -- 172.21.15.12:0/1856273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 msgr2=0x55e94270d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d87fff640 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e94270d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d87fff640 1 -- 172.21.15.12:0/1856273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 msgr2=0x55e9427062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d87fff640 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 0x55e9427062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d87fff640 1 -- 172.21.15.12:0/1856273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9423743f0 con 0x55e9426fc920 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.705+0000 7f8d94a7a640 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e94270d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d87fff640 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e942706810 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8d780ccf20 tx=0x7f8d78009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4421073c9869190 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d6f7fe640 1 -- 172.21.15.12:0/1856273 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d780a48f0 con 0x55e9426fc920 2024-05-09T09:28:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d6f7fe640 1 -- 172.21.15.12:0/1856273 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8d78005210 con 0x55e9426fc920 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d991ea740 1 -- 172.21.15.12:0/1856273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d80031810 con 0x55e9426fc920 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d6f7fe640 1 -- 172.21.15.12:0/1856273 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8d7809c380 con 0x55e9426fc920 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d991ea740 10 client.16959 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d991ea740 1 -- 172.21.15.12:0/1856273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e9423340f0 con 0x55e9426fc920 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d6f7fe640 1 -- 172.21.15.12:0/1856273 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8d80031810 con 0x55e9426fc920 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d6f7fe640 1 -- 172.21.15.12:0/1856273 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f8d7809bee0 con 0x55e9426fc920 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d991ea740 10 client.16959 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d991ea740 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d991ea740 20 client.16959 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d991ea740 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e942712170 0x55e942714580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d991ea740 4 client.16959 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d991ea740 1 -- 172.21.15.12:0/1856273 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e94266e430 con 0x55e942712170 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.706+0000 7f8d8ffff640 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e942712170 0x55e942714580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.707+0000 7f8d8ffff640 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e942712170 0x55e942714580 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15abe7f6bb64ca48 server_cookie=f0083b7398d1123c in_seq=0 out_seq=0 2024-05-09T09:28:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.707+0000 7f8d6f7fe640 10 client.16959 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.707+0000 7f8d6f7fe640 1 -- 172.21.15.12:0/1856273 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e94266e430 con 0x55e942712170 2024-05-09T09:28:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.707+0000 7f8d6f7fe640 10 client.16959 handle_command_reply: tid=0 2024-05-09T09:28:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.708+0000 7f8d991ea740 1 client.16959 shutdown 2024-05-09T09:28:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.708+0000 7f8d991ea740 1 -- 172.21.15.12:0/1856273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 msgr2=0x55e942706810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.708+0000 7f8d991ea740 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e942706810 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8d780ccf20 tx=0x7f8d78009310 comp rx=0 tx=0).stop 2024-05-09T09:28:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.708+0000 7f8d991ea740 1 -- 172.21.15.12:0/1856273 shutdown_connections 2024-05-09T09:28:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.708+0000 7f8d991ea740 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e942712170 0x55e942714580 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.708+0000 7f8d991ea740 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e94270d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.708+0000 7f8d6f7fe640 0 client.16959 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.708+0000 7f8d991ea740 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e942706810 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.708+0000 7f8d991ea740 1 --2- 172.21.15.12:0/1856273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 0x55e9427062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.708+0000 7f8d991ea740 1 -- 172.21.15.12:0/1856273 >> 172.21.15.12:0/1856273 conn(0x55e9426f75a0 msgr2=0x55e9426f67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.708+0000 7f8d991ea740 1 -- 172.21.15.12:0/1856273 shutdown_connections 2024-05-09T09:28:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.708+0000 7f8d991ea740 1 -- 172.21.15.12:0/1856273 wait complete. 2024-05-09T09:28:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.708+0000 7f8d991ea740 20 client.16959 trim_cache size 0 max 16384 2024-05-09T09:28:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d991ea740 1 Processor -- start 2024-05-09T09:28:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d991ea740 1 -- start start 2024-05-09T09:28:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d991ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 0x55e9426fdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d991ea740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e94270d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d991ea740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e9426fe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d991ea740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901ac840 con 0x55e9426fc920 2024-05-09T09:28:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d991ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901ae5c0 con 0x55e9426fbf30 2024-05-09T09:28:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d991ea740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901acfd0 con 0x55e942701a90 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e94270d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e94270d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47704/0 (socket says 172.21.15.12:47704) 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d87fff640 1 -- 172.21.15.12:0/2008708442 learned_addr learned my addr 172.21.15.12:0/2008708442 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d94a7a640 1 --2- 172.21.15.12:0/2008708442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e9426fe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d87fff640 1 -- 172.21.15.12:0/2008708442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 msgr2=0x55e9426fe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d87fff640 1 --2- 172.21.15.12:0/2008708442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e9426fe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d87fff640 1 -- 172.21.15.12:0/2008708442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 msgr2=0x55e9426fdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d87fff640 1 --2- 172.21.15.12:0/2008708442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 0x55e9426fdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d87fff640 1 -- 172.21.15.12:0/2008708442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9427017b0 con 0x55e9426fc920 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.726+0000 7f8d87fff640 1 --2- 172.21.15.12:0/2008708442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e94270d810 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8d78098650 tx=0x7f8d78005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae66dfcb0e4451c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.727+0000 7f8d85ffb640 1 -- 172.21.15.12:0/2008708442 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7809b5e0 con 0x55e9426fc920 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.727+0000 7f8d85ffb640 1 -- 172.21.15.12:0/2008708442 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8d7809bc80 con 0x55e9426fc920 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.727+0000 7f8d85ffb640 1 -- 172.21.15.12:0/2008708442 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78097a20 con 0x55e9426fc920 2024-05-09T09:28:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.727+0000 7f8d991ea740 1 -- 172.21.15.12:0/2008708442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 msgr2=0x55e94270d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.727+0000 7f8d991ea740 1 --2- 172.21.15.12:0/2008708442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e94270d810 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8d78098650 tx=0x7f8d78005ab0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.727+0000 7f8d991ea740 1 -- 172.21.15.12:0/2008708442 shutdown_connections 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.727+0000 7f8d991ea740 1 --2- 172.21.15.12:0/2008708442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e9426fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.727+0000 7f8d991ea740 1 --2- 172.21.15.12:0/2008708442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fc920 0x55e94270d810 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.727+0000 7f8d991ea740 1 --2- 172.21.15.12:0/2008708442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fbf30 0x55e9426fdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.727+0000 7f8d991ea740 1 -- 172.21.15.12:0/2008708442 >> 172.21.15.12:0/2008708442 conn(0x55e9426f75a0 msgr2=0x55e9427062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.727+0000 7f8d991ea740 1 -- 172.21.15.12:0/2008708442 shutdown_connections 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.727+0000 7f8d991ea740 1 -- 172.21.15.12:0/2008708442 wait complete. 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d991ea740 1 -- start start 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d991ea740 1 Processor -- start 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d991ea740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fbf30 0x55e94270b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d991ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fc920 0x55e94270b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d991ea740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e94270c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d991ea740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901adef0 con 0x55e9426fbf30 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d991ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901ad760 con 0x55e9426fc920 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d991ea740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901aa820 con 0x55e942701a90 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d991ea740 10 client.0 _ll_register_callbacks cb 0x7f8d7800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fbf30 0x55e94270b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fbf30 0x55e94270b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47710/0 (socket says 172.21.15.12:47710) 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d8ffff640 1 -- 172.21.15.12:0/384857810 learned_addr learned my addr 172.21.15.12:0/384857810 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d87fff640 1 --2- 172.21.15.12:0/384857810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fc920 0x55e94270b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d94a7a640 1 --2- 172.21.15.12:0/384857810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e94270c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d8ffff640 1 -- 172.21.15.12:0/384857810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 msgr2=0x55e94270c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.728+0000 7f8d8ffff640 1 --2- 172.21.15.12:0/384857810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e94270c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d8ffff640 1 -- 172.21.15.12:0/384857810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fc920 msgr2=0x55e94270b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d8ffff640 1 --2- 172.21.15.12:0/384857810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fc920 0x55e94270b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d8ffff640 1 -- 172.21.15.12:0/384857810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e94270cbe0 con 0x55e9426fbf30 2024-05-09T09:28:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d8ffff640 1 --2- 172.21.15.12:0/384857810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fbf30 0x55e94270b0a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8d80000c00 tx=0x7f8d80031ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa43a0fcf31b9fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d6f7fe640 1 -- 172.21.15.12:0/384857810 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80052060 con 0x55e9426fbf30 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d6f7fe640 1 -- 172.21.15.12:0/384857810 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8d8004ddc0 con 0x55e9426fbf30 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d991ea740 1 -- 172.21.15.12:0/384857810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e942668780 con 0x55e9426fbf30 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d6f7fe640 1 -- 172.21.15.12:0/384857810 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8d80059050 con 0x55e9426fbf30 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d991ea740 10 client.16965 fetch_fsmap learned FSMap version 42 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d991ea740 1 -- 172.21.15.12:0/384857810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e942778830 con 0x55e9426fbf30 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d6f7fe640 1 -- 172.21.15.12:0/384857810 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55e942668780 con 0x55e9426fbf30 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d6f7fe640 1 -- 172.21.15.12:0/384857810 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f8d80004780 con 0x55e9426fbf30 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d991ea740 10 client.16965 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d991ea740 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d991ea740 20 client.16965 populate_metadata read hostname 'smithi012' 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d991ea740 1 --2- 172.21.15.12:0/384857810 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e942778e00 0x55e94277b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:28:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d991ea740 4 client.16965 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:28:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.729+0000 7f8d991ea740 1 -- 172.21.15.12:0/384857810 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9425e56f0 con 0x55e942778e00 2024-05-09T09:28:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d87fff640 1 --2- 172.21.15.12:0/384857810 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e942778e00 0x55e94277b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:28:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d87fff640 1 --2- 172.21.15.12:0/384857810 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e942778e00 0x55e94277b1f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23fef796e0663d39 server_cookie=ccbc4d4b09ff1b2d in_seq=0 out_seq=0 2024-05-09T09:28:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d6f7fe640 10 client.16965 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d6f7fe640 1 -- 172.21.15.12:0/384857810 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55e9425e56f0 con 0x55e942778e00 2024-05-09T09:28:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d6f7fe640 10 client.16965 handle_command_reply: tid=0 2024-05-09T09:28:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d991ea740 1 client.16965 shutdown 2024-05-09T09:28:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d991ea740 1 -- 172.21.15.12:0/384857810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fbf30 msgr2=0x55e94270b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d991ea740 1 --2- 172.21.15.12:0/384857810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fbf30 0x55e94270b0a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8d80000c00 tx=0x7f8d80031ae0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d991ea740 1 -- 172.21.15.12:0/384857810 shutdown_connections 2024-05-09T09:28:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d991ea740 1 --2- 172.21.15.12:0/384857810 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55e942778e00 0x55e94277b1f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d991ea740 1 --2- 172.21.15.12:0/384857810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e942701a90 0x55e94270c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d991ea740 1 --2- 172.21.15.12:0/384857810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9426fc920 0x55e94270b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d6f7fe640 0 client.16965 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:28:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d991ea740 1 --2- 172.21.15.12:0/384857810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9426fbf30 0x55e94270b0a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d991ea740 1 -- 172.21.15.12:0/384857810 >> 172.21.15.12:0/384857810 conn(0x55e9426f75a0 msgr2=0x55e9427062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d991ea740 1 -- 172.21.15.12:0/384857810 shutdown_connections 2024-05-09T09:28:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.730+0000 7f8d991ea740 1 -- 172.21.15.12:0/384857810 wait complete. 2024-05-09T09:28:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.731+0000 7f8d991ea740 20 client.16965 trim_cache size 0 max 16384 2024-05-09T09:28:59.737 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:28:59.738 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (0 inodes in the stack)", 2024-05-09T09:28:59.738 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T09:28:59.738 INFO:teuthology.orchestra.run.smithi012.stdout: "64775efd-96e9-40fe-8289-19d3875e5019": { 2024-05-09T09:28:59.738 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T09:28:59.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "64775efd-96e9-40fe-8289-19d3875e5019", 2024-05-09T09:28:59.738 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T09:28:59.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:28:59.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:28:59.738 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:28:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.731+0000 7f8d964dd640 1 -- 172.21.15.12:0/820533677 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8d7c0a2b90 msgr2=0x7f8d7c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.731+0000 7f8d964dd640 1 --2- 172.21.15.12:0/820533677 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8d7c0a2b90 0x7f8d7c0a5050 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55e9426fd180 tx=0x7f8d8800a4d0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.731+0000 7f8d964dd640 1 -- 172.21.15.12:0/820533677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d90101980 msgr2=0x7f8d9019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:28:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.731+0000 7f8d964dd640 1 --2- 172.21.15.12:0/820533677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d90101980 0x7f8d9019a380 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800e9f0 tx=0x7f8d7800edf0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.731+0000 7f8d964dd640 1 -- 172.21.15.12:0/820533677 shutdown_connections 2024-05-09T09:28:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.731+0000 7f8d964dd640 1 --2- 172.21.15.12:0/820533677 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8d7c0a2b90 0x7f8d7c0a5050 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.731+0000 7f8d964dd640 1 --2- 172.21.15.12:0/820533677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d90102340 0x7f8d901a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.731+0000 7f8d964dd640 1 --2- 172.21.15.12:0/820533677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d90101980 0x7f8d9019a380 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.731+0000 7f8d964dd640 1 --2- 172.21.15.12:0/820533677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d90100790 0x7f8d90199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:28:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.731+0000 7f8d964dd640 1 -- 172.21.15.12:0/820533677 >> 172.21.15.12:0/820533677 conn(0x7f8d900fbd20 msgr2=0x7f8d90069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:28:59.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.731+0000 7f8d964dd640 1 -- 172.21.15.12:0/820533677 shutdown_connections 2024-05-09T09:28:59.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:28:59.731+0000 7f8d964dd640 1 -- 172.21.15.12:0/820533677 wait complete. 2024-05-09T09:28:59.741 INFO:tasks.cephfs.filesystem:scrub status for tag:64775efd-96e9-40fe-8289-19d3875e5019 - {'path': '/', 'tag': '64775efd-96e9-40fe-8289-19d3875e5019', 'options': 'recursive,force'} 2024-05-09T09:29:00.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:00 smithi012 ceph-mon[30140]: pgmap v471: 129 pgs: 129 active+clean; 67 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 965 KiB/s wr, 461 op/s 2024-05-09T09:29:00.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:00 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:29:00.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:00 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:29:00.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:00 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:00.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:00 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:00.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:00 smithi170 ceph-mon[33632]: pgmap v471: 129 pgs: 129 active+clean; 67 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 965 KiB/s wr, 461 op/s 2024-05-09T09:29:00.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:00 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:29:00.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:00 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:29:00.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:00 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:00.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:00 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:00 smithi016 ceph-mon[33335]: pgmap v471: 129 pgs: 129 active+clean; 67 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 965 KiB/s wr, 461 op/s 2024-05-09T09:29:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:00 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:29:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:00 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:29:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:00 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:00.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:00 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:01.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:01 smithi012 ceph-mon[30140]: scrub queued for path: / 2024-05-09T09:29:01.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:01 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:01.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:01 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:29:01.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:01 smithi012 ceph-mon[30140]: scrub summary: active paths [/] 2024-05-09T09:29:01.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:01 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:01.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:01 smithi012 ceph-mon[30140]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:01.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:01 smithi170 ceph-mon[33632]: scrub queued for path: / 2024-05-09T09:29:01.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:01 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:01.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:01 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:29:01.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:01 smithi170 ceph-mon[33632]: scrub summary: active paths [/] 2024-05-09T09:29:01.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:01 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:01.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:01 smithi170 ceph-mon[33632]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:01.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:01 smithi016 ceph-mon[33335]: scrub queued for path: / 2024-05-09T09:29:01.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:01 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:01.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:01 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:29:01.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:01 smithi016 ceph-mon[33335]: scrub summary: active paths [/] 2024-05-09T09:29:01.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:01 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:01.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:01 smithi016 ceph-mon[33335]: scrub summary: idle+waiting paths [/] 2024-05-09T09:29:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.522+0000 7f2a2b168640 1 -- 172.21.15.12:0/2752562422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24104c40 msgr2=0x7f2a24107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.522+0000 7f2a2b168640 1 --2- 172.21.15.12:0/2752562422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24104c40 0x7f2a24107030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009900 tx=0x7f2a1002a410 comp rx=0 tx=0).stop 2024-05-09T09:29:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.522+0000 7f2a2b168640 1 -- 172.21.15.12:0/2752562422 shutdown_connections 2024-05-09T09:29:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.522+0000 7f2a2b168640 1 --2- 172.21.15.12:0/2752562422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a24107570 0x7f2a241099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.522+0000 7f2a2b168640 1 --2- 172.21.15.12:0/2752562422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24104c40 0x7f2a24107030 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.522+0000 7f2a2b168640 1 --2- 172.21.15.12:0/2752562422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24102310 0x7f2a24104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.522+0000 7f2a2b168640 1 -- 172.21.15.12:0/2752562422 >> 172.21.15.12:0/2752562422 conn(0x7f2a240fbd00 msgr2=0x7f2a240fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.522+0000 7f2a2b168640 1 -- 172.21.15.12:0/2752562422 shutdown_connections 2024-05-09T09:29:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.522+0000 7f2a2b168640 1 -- 172.21.15.12:0/2752562422 wait complete. 2024-05-09T09:29:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.523+0000 7f2a2b168640 1 Processor -- start 2024-05-09T09:29:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.523+0000 7f2a2b168640 1 -- start start 2024-05-09T09:29:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.523+0000 7f2a2b168640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24102310 0x7f2a24101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.523+0000 7f2a2b168640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a24104c40 0x7f2a241002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.523+0000 7f2a2b168640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24107570 0x7f2a24100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.523+0000 7f2a2b168640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2410d4d0 con 0x7f2a24102310 2024-05-09T09:29:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.523+0000 7f2a2b168640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2410d350 con 0x7f2a24104c40 2024-05-09T09:29:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.523+0000 7f2a2b168640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2410d650 con 0x7f2a24107570 2024-05-09T09:29:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.523+0000 7f2a28f04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24102310 0x7f2a24101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.523+0000 7f2a28f04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24102310 0x7f2a24101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47720/0 (socket says 172.21.15.12:47720) 2024-05-09T09:29:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.523+0000 7f2a28f04640 1 -- 172.21.15.12:0/2632901660 learned_addr learned my addr 172.21.15.12:0/2632901660 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.523+0000 7f2a23fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a24104c40 0x7f2a241002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.523+0000 7f2a29705640 1 --2- 172.21.15.12:0/2632901660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24107570 0x7f2a24100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.524+0000 7f2a28f04640 1 -- 172.21.15.12:0/2632901660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24107570 msgr2=0x7f2a24100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.524+0000 7f2a28f04640 1 --2- 172.21.15.12:0/2632901660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24107570 0x7f2a24100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.524+0000 7f2a28f04640 1 -- 172.21.15.12:0/2632901660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a24104c40 msgr2=0x7f2a241002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.524+0000 7f2a28f04640 1 --2- 172.21.15.12:0/2632901660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a24104c40 0x7f2a241002c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.524+0000 7f2a28f04640 1 -- 172.21.15.12:0/2632901660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a24101090 con 0x7f2a24102310 2024-05-09T09:29:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.524+0000 7f2a28f04640 1 --2- 172.21.15.12:0/2632901660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24102310 0x7f2a24101c80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800ea60 tx=0x7f2a1800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.524+0000 7f2a21ffb640 1 -- 172.21.15.12:0/2632901660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1800cde0 con 0x7f2a24102310 2024-05-09T09:29:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.524+0000 7f2a2b168640 1 -- 172.21.15.12:0/2632901660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a241a1c60 con 0x7f2a24102310 2024-05-09T09:29:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.524+0000 7f2a21ffb640 1 -- 172.21.15.12:0/2632901660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2a180035f0 con 0x7f2a24102310 2024-05-09T09:29:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.524+0000 7f2a21ffb640 1 -- 172.21.15.12:0/2632901660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18010690 con 0x7f2a24102310 2024-05-09T09:29:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.524+0000 7f2a2b168640 1 -- 172.21.15.12:0/2632901660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a241a21d0 con 0x7f2a24102310 2024-05-09T09:29:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.527+0000 7f2a21ffb640 1 -- 172.21.15.12:0/2632901660 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2a180037a0 con 0x7f2a24102310 2024-05-09T09:29:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.527+0000 7f2a2de72740 1 Processor -- start 2024-05-09T09:29:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.527+0000 7f2a2de72740 1 -- start start 2024-05-09T09:29:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a2de72740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a1250 0x5579a44a1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a2de72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1c40 0x5579a44a6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a2de72740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a6db0 0x5579a44a9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a2de72740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5579a4405340 con 0x5579a44a1250 2024-05-09T09:29:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a2de72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5579a43ad290 con 0x5579a44a1c40 2024-05-09T09:29:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a2de72740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5579a4401300 con 0x5579a44a6db0 2024-05-09T09:29:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a29705640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a1250 0x5579a44a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a29705640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a1250 0x5579a44a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47724/0 (socket says 172.21.15.12:47724) 2024-05-09T09:29:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a29705640 1 -- 172.21.15.12:0/508905973 learned_addr learned my addr 172.21.15.12:0/508905973 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a23fff640 1 --2- 172.21.15.12:0/508905973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a6db0 0x5579a44a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a28f04640 1 --2- 172.21.15.12:0/508905973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1c40 0x5579a44a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a21ffb640 1 --2- 172.21.15.12:0/2632901660 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2a140a2c30 0x7f2a140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a23fff640 1 -- 172.21.15.12:0/508905973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1c40 msgr2=0x5579a44a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a23fff640 1 --2- 172.21.15.12:0/508905973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1c40 0x5579a44a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a23fff640 1 -- 172.21.15.12:0/508905973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a1250 msgr2=0x5579a44a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a23fff640 1 --2- 172.21.15.12:0/508905973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a1250 0x5579a44a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a23fff640 1 -- 172.21.15.12:0/508905973 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a43d18e0 con 0x5579a44a6db0 2024-05-09T09:29:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a28f04640 1 --2- 172.21.15.12:0/508905973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1c40 0x5579a44a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:29:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a29705640 1 --2- 172.21.15.12:0/2632901660 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2a140a2c30 0x7f2a140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.528+0000 7f2a21ffb640 1 -- 172.21.15.12:0/2632901660 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2a180cf7c0 con 0x7f2a24102310 2024-05-09T09:29:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.529+0000 7f2a23fff640 1 --2- 172.21.15.12:0/508905973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a6db0 0x5579a44a9240 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009230 tx=0x7f2a10005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cf0c0b99037431b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.529+0000 7f2a29705640 1 --2- 172.21.15.12:0/2632901660 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2a140a2c30 0x7f2a140a50f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5579a44a24a0 tx=0x7f2a1c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.529+0000 7f2a0effd640 1 -- 172.21.15.12:0/508905973 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10040070 con 0x5579a44a6db0 2024-05-09T09:29:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.529+0000 7f2a0effd640 1 -- 172.21.15.12:0/508905973 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2a10004020 con 0x5579a44a6db0 2024-05-09T09:29:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.529+0000 7f2a0effd640 1 -- 172.21.15.12:0/508905973 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10035040 con 0x5579a44a6db0 2024-05-09T09:29:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.529+0000 7f2a2de72740 1 -- 172.21.15.12:0/508905973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a6db0 msgr2=0x5579a44a9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.529+0000 7f2a2de72740 1 --2- 172.21.15.12:0/508905973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a6db0 0x5579a44a9240 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009230 tx=0x7f2a10005e80 comp rx=0 tx=0).stop 2024-05-09T09:29:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 -- 172.21.15.12:0/508905973 shutdown_connections 2024-05-09T09:29:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 --2- 172.21.15.12:0/508905973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a6db0 0x5579a44a9240 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 --2- 172.21.15.12:0/508905973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1c40 0x5579a44a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 --2- 172.21.15.12:0/508905973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a1250 0x5579a44a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 -- 172.21.15.12:0/508905973 >> 172.21.15.12:0/508905973 conn(0x5579a449c8c0 msgr2=0x5579a449ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 -- 172.21.15.12:0/508905973 shutdown_connections 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 -- 172.21.15.12:0/508905973 wait complete. 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 -- start start 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 Processor -- start 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1250 0x5579a44ab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1c40 0x5579a44ab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 0x5579a44b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5579a44a9d00 con 0x5579a44a6db0 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5579a43c0790 con 0x5579a44a1c40 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5579a44a9e80 con 0x5579a44a1250 2024-05-09T09:29:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a2de72740 10 client.0 _ll_register_callbacks cb 0x5579a43fba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a29705640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 0x5579a44b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.530+0000 7f2a28f04640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1250 0x5579a44ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a28f04640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1250 0x5579a44ab430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.12:58086/0 (socket says 172.21.15.12:58086) 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a29705640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 0x5579a44b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47736/0 (socket says 172.21.15.12:47736) 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a28f04640 1 -- 172.21.15.12:0/2543546721 learned_addr learned my addr 172.21.15.12:0/2543546721 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a23fff640 1 --2- 172.21.15.12:0/2543546721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1c40 0x5579a44ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a28f04640 1 -- 172.21.15.12:0/2543546721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1c40 msgr2=0x5579a44ab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a28f04640 1 --2- 172.21.15.12:0/2543546721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1c40 0x5579a44ab970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a28f04640 1 -- 172.21.15.12:0/2543546721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 msgr2=0x5579a44b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a28f04640 1 --2- 172.21.15.12:0/2543546721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 0x5579a44b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a28f04640 1 -- 172.21.15.12:0/2543546721 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a4119210 con 0x5579a44a1250 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a29705640 1 --2- 172.21.15.12:0/2543546721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 0x5579a44b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a28f04640 1 --2- 172.21.15.12:0/2543546721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1250 0x5579a44ab430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2a180a4330 tx=0x7f2a1809b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe1392baf5c1f1cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a03fff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-05-09T09:29:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a03fff640 1 -- 172.21.15.12:0/2543546721 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a180b2070 con 0x5579a44a1250 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a03fff640 1 -- 172.21.15.12:0/2543546721 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2a180cfbb0 con 0x5579a44a1250 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a2de72740 1 -- 172.21.15.12:0/2543546721 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a10004020 con 0x5579a44a1250 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a03fff640 1 -- 172.21.15.12:0/2543546721 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2a18003ba0 con 0x5579a44a1250 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a2de72740 10 client.26509 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.531+0000 7f2a2de72740 1 -- 172.21.15.12:0/2543546721 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5579a41e5620 con 0x5579a44a1250 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.532+0000 7f2a03fff640 1 -- 172.21.15.12:0/2543546721 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2a10004020 con 0x5579a44a1250 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.532+0000 7f2a03fff640 1 -- 172.21.15.12:0/2543546721 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f2a180ba350 con 0x5579a44a1250 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.532+0000 7f2a2de72740 10 client.26509 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.532+0000 7f2a2de72740 10 client.26509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.532+0000 7f2a2de72740 20 client.26509 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.532+0000 7f2a2de72740 1 --2- 172.21.15.12:0/2543546721 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5579a44b6f70 0x5579a44b9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.532+0000 7f2a2de72740 4 client.26509 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.532+0000 7f2a2de72740 1 -- 172.21.15.12:0/2543546721 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579a4412a70 con 0x5579a44b6f70 2024-05-09T09:29:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.532+0000 7f2a23fff640 1 --2- 172.21.15.12:0/2543546721 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5579a44b6f70 0x5579a44b9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.532+0000 7f2a23fff640 1 --2- 172.21.15.12:0/2543546721 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5579a44b6f70 0x5579a44b9430 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10ea999c706bcfbe server_cookie=ae724c766a2d7de2 in_seq=0 out_seq=0 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.532+0000 7f2a03fff640 10 client.26509 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a03fff640 1 -- 172.21.15.12:0/2543546721 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579a4412a70 con 0x5579a44b6f70 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a03fff640 10 client.26509 handle_command_reply: tid=0 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a2de72740 1 client.26509 shutdown 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a2de72740 1 -- 172.21.15.12:0/2543546721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1250 msgr2=0x5579a44ab430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a2de72740 1 --2- 172.21.15.12:0/2543546721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1250 0x5579a44ab430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2a180a4330 tx=0x7f2a1809b9c0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a2de72740 1 -- 172.21.15.12:0/2543546721 shutdown_connections 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a2de72740 1 --2- 172.21.15.12:0/2543546721 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5579a44b6f70 0x5579a44b9430 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a2de72740 1 --2- 172.21.15.12:0/2543546721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 0x5579a44b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a03fff640 0 client.26509 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a2de72740 1 --2- 172.21.15.12:0/2543546721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1c40 0x5579a44ab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a2de72740 1 --2- 172.21.15.12:0/2543546721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1250 0x5579a44ab430 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a2de72740 1 -- 172.21.15.12:0/2543546721 >> 172.21.15.12:0/2543546721 conn(0x5579a449c8c0 msgr2=0x5579a449ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a2de72740 1 -- 172.21.15.12:0/2543546721 shutdown_connections 2024-05-09T09:29:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.533+0000 7f2a2de72740 1 -- 172.21.15.12:0/2543546721 wait complete. 2024-05-09T09:29:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.534+0000 7f2a2de72740 20 client.26509 trim_cache size 0 max 16384 2024-05-09T09:29:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.548+0000 7f2a2de72740 1 Processor -- start 2024-05-09T09:29:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.548+0000 7f2a2de72740 1 -- start start 2024-05-09T09:29:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.548+0000 7f2a2de72740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1250 0x5579a449cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.548+0000 7f2a2de72740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a1c40 0x5579a44bed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.548+0000 7f2a2de72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a6db0 0x5579a44bf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a2de72740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241ac700 con 0x5579a44a1c40 2024-05-09T09:29:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a2de72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241ae480 con 0x5579a44a6db0 2024-05-09T09:29:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a2de72740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241ace90 con 0x5579a44a1250 2024-05-09T09:29:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a23fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a1c40 0x5579a44bed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a23fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a1c40 0x5579a44bed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47740/0 (socket says 172.21.15.12:47740) 2024-05-09T09:29:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a23fff640 1 -- 172.21.15.12:0/703658982 learned_addr learned my addr 172.21.15.12:0/703658982 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a28f04640 1 --2- 172.21.15.12:0/703658982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1250 0x5579a449cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a29705640 1 --2- 172.21.15.12:0/703658982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a6db0 0x5579a44bf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a23fff640 1 -- 172.21.15.12:0/703658982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1250 msgr2=0x5579a449cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a23fff640 1 --2- 172.21.15.12:0/703658982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1250 0x5579a449cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a23fff640 1 -- 172.21.15.12:0/703658982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a6db0 msgr2=0x5579a44bf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a23fff640 1 --2- 172.21.15.12:0/703658982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a6db0 0x5579a44bf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a23fff640 1 -- 172.21.15.12:0/703658982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a44a2020 con 0x5579a44a1c40 2024-05-09T09:29:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a23fff640 1 --2- 172.21.15.12:0/703658982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a1c40 0x5579a44bed40 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2a10004730 tx=0x7f2a10009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c779ee250e1f22cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a0e7fc640 1 -- 172.21.15.12:0/703658982 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10040040 con 0x5579a44a1c40 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a0e7fc640 1 -- 172.21.15.12:0/703658982 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2a10031810 con 0x5579a44a1c40 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.549+0000 7f2a0e7fc640 1 -- 172.21.15.12:0/703658982 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10035040 con 0x5579a44a1c40 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.550+0000 7f2a2de72740 1 -- 172.21.15.12:0/703658982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a1c40 msgr2=0x5579a44bed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.550+0000 7f2a2de72740 1 --2- 172.21.15.12:0/703658982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a1c40 0x5579a44bed40 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2a10004730 tx=0x7f2a10009260 comp rx=0 tx=0).stop 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.550+0000 7f2a2de72740 1 -- 172.21.15.12:0/703658982 shutdown_connections 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.550+0000 7f2a2de72740 1 --2- 172.21.15.12:0/703658982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a6db0 0x5579a44bf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.550+0000 7f2a2de72740 1 --2- 172.21.15.12:0/703658982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a1c40 0x5579a44bed40 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.550+0000 7f2a2de72740 1 --2- 172.21.15.12:0/703658982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1250 0x5579a449cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.550+0000 7f2a2de72740 1 -- 172.21.15.12:0/703658982 >> 172.21.15.12:0/703658982 conn(0x5579a449c8c0 msgr2=0x5579a44a64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.550+0000 7f2a2de72740 1 -- 172.21.15.12:0/703658982 shutdown_connections 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.550+0000 7f2a2de72740 1 -- 172.21.15.12:0/703658982 wait complete. 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.550+0000 7f2a2de72740 1 -- start start 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.550+0000 7f2a2de72740 1 Processor -- start 2024-05-09T09:29:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a2de72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1250 0x5579a44af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a2de72740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1c40 0x5579a44af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a2de72740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 0x5579a44b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a2de72740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241addb0 con 0x5579a44a6db0 2024-05-09T09:29:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a2de72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241ad620 con 0x5579a44a1250 2024-05-09T09:29:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a2de72740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241aa6e0 con 0x5579a44a1c40 2024-05-09T09:29:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a2de72740 10 client.0 _ll_register_callbacks cb 0x7f2a18003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a29705640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 0x5579a44b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a23fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1c40 0x5579a44af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a29705640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 0x5579a44b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47742/0 (socket says 172.21.15.12:47742) 2024-05-09T09:29:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a29705640 1 -- 172.21.15.12:0/2416980339 learned_addr learned my addr 172.21.15.12:0/2416980339 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a28f04640 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1250 0x5579a44af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a29705640 1 -- 172.21.15.12:0/2416980339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1c40 msgr2=0x5579a44af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a29705640 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1c40 0x5579a44af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a29705640 1 -- 172.21.15.12:0/2416980339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1250 msgr2=0x5579a44af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a29705640 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1250 0x5579a44af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a29705640 1 -- 172.21.15.12:0/2416980339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a44b0db0 con 0x5579a44a6db0 2024-05-09T09:29:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a23fff640 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1c40 0x5579a44af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:29:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a28f04640 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1250 0x5579a44af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:29:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a29705640 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 0x5579a44b07c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c005fd0 tx=0x7f2a1c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91663dd1593273bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.551+0000 7f2a03fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:29:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a03fff640 1 -- 172.21.15.12:0/2416980339 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c011ca0 con 0x5579a44a6db0 2024-05-09T09:29:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a03fff640 1 -- 172.21.15.12:0/2416980339 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2a1c02c420 con 0x5579a44a6db0 2024-05-09T09:29:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a2de72740 1 -- 172.21.15.12:0/2416980339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a10031810 con 0x5579a44a6db0 2024-05-09T09:29:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a03fff640 1 -- 172.21.15.12:0/2416980339 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2a1c011e40 con 0x5579a44a6db0 2024-05-09T09:29:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a2de72740 10 client.16983 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a2de72740 1 -- 172.21.15.12:0/2416980339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5579a44b2130 con 0x5579a44a6db0 2024-05-09T09:29:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a03fff640 1 -- 172.21.15.12:0/2416980339 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2a10031810 con 0x5579a44a6db0 2024-05-09T09:29:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a03fff640 1 -- 172.21.15.12:0/2416980339 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f2a1c02b590 con 0x5579a44a6db0 2024-05-09T09:29:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a2de72740 10 client.16983 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a2de72740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T09:29:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a2de72740 20 client.16983 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a2de72740 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5579a451de00 0x5579a4520220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a2de72740 4 client.16983 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:29:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a2de72740 1 -- 172.21.15.12:0/2416980339 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579a440cdc0 con 0x5579a451de00 2024-05-09T09:29:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.552+0000 7f2a28f04640 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5579a451de00 0x5579a4520220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.553+0000 7f2a03fff640 10 client.16983 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:29:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.553+0000 7f2a28f04640 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5579a451de00 0x5579a4520220 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e542a8ece1b99729 server_cookie=9a6ae2d954fed34 in_seq=0 out_seq=0 2024-05-09T09:29:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a03fff640 1 -- 172.21.15.12:0/2416980339 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579a440cdc0 con 0x5579a451de00 2024-05-09T09:29:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a03fff640 10 client.16983 handle_command_reply: tid=0 2024-05-09T09:29:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a2de72740 1 client.16983 shutdown 2024-05-09T09:29:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a2de72740 1 -- 172.21.15.12:0/2416980339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 msgr2=0x5579a44b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a2de72740 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 0x5579a44b07c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c005fd0 tx=0x7f2a1c004ab0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a2de72740 1 -- 172.21.15.12:0/2416980339 shutdown_connections 2024-05-09T09:29:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a2de72740 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5579a451de00 0x5579a4520220 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a2de72740 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579a44a6db0 0x5579a44b07c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a2de72740 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a44a1c40 0x5579a44af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a03fff640 0 client.16983 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:29:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a2de72740 1 --2- 172.21.15.12:0/2416980339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579a44a1250 0x5579a44af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a2de72740 1 -- 172.21.15.12:0/2416980339 >> 172.21.15.12:0/2416980339 conn(0x5579a449c8c0 msgr2=0x5579a449cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a2de72740 1 -- 172.21.15.12:0/2416980339 shutdown_connections 2024-05-09T09:29:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a2de72740 1 -- 172.21.15.12:0/2416980339 wait complete. 2024-05-09T09:29:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.576+0000 7f2a2de72740 20 client.16983 trim_cache size 0 max 16384 2024-05-09T09:29:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T09:29:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T09:29:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T09:29:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T09:29:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.577+0000 7f2a2b168640 1 -- 172.21.15.12:0/2632901660 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2a140a2c30 msgr2=0x7f2a140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.577+0000 7f2a2b168640 1 --2- 172.21.15.12:0/2632901660 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2a140a2c30 0x7f2a140a50f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5579a44a24a0 tx=0x7f2a1c004a40 comp rx=0 tx=0).stop 2024-05-09T09:29:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.577+0000 7f2a2b168640 1 -- 172.21.15.12:0/2632901660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24102310 msgr2=0x7f2a24101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.577+0000 7f2a2b168640 1 --2- 172.21.15.12:0/2632901660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24102310 0x7f2a24101c80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800ea60 tx=0x7f2a1800ef20 comp rx=0 tx=0).stop 2024-05-09T09:29:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.577+0000 7f2a2b168640 1 -- 172.21.15.12:0/2632901660 shutdown_connections 2024-05-09T09:29:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.577+0000 7f2a2b168640 1 --2- 172.21.15.12:0/2632901660 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f2a140a2c30 0x7f2a140a50f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.577+0000 7f2a2b168640 1 --2- 172.21.15.12:0/2632901660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24107570 0x7f2a24100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.577+0000 7f2a2b168640 1 --2- 172.21.15.12:0/2632901660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a24104c40 0x7f2a241002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.577+0000 7f2a2b168640 1 --2- 172.21.15.12:0/2632901660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24102310 0x7f2a24101c80 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.577+0000 7f2a2b168640 1 -- 172.21.15.12:0/2632901660 >> 172.21.15.12:0/2632901660 conn(0x7f2a240fbd00 msgr2=0x7f2a24102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.577+0000 7f2a2b168640 1 -- 172.21.15.12:0/2632901660 shutdown_connections 2024-05-09T09:29:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T09:29:01.577+0000 7f2a2b168640 1 -- 172.21.15.12:0/2632901660 wait complete. 2024-05-09T09:29:02.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:02 smithi012 ceph-mon[30140]: pgmap v472: 129 pgs: 129 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 860 KiB/s wr, 520 op/s 2024-05-09T09:29:02.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:02 smithi170 ceph-mon[33632]: pgmap v472: 129 pgs: 129 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 860 KiB/s wr, 520 op/s 2024-05-09T09:29:02.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:02 smithi016 ceph-mon[33335]: pgmap v472: 129 pgs: 129 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 860 KiB/s wr, 520 op/s 2024-05-09T09:29:03.038 INFO:tasks.workunit:Stopping ['fs/norstats'] on client.0... 2024-05-09T09:29:03.038 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T09:29:03.380 DEBUG:teuthology.parallel:result is None 2024-05-09T09:29:03.380 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T09:29:03.416 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T09:29:03.416 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-09T09:29:03.427 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi012.front.sepia.ceph.com 2024-05-09T09:29:03.465 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-09T09:29:03.474 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-09T09:29:04.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:04 smithi012 ceph-mon[30140]: pgmap v473: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 923 KiB/s wr, 461 op/s 2024-05-09T09:29:04.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:04 smithi170 ceph-mon[33632]: pgmap v473: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 923 KiB/s wr, 461 op/s 2024-05-09T09:29:04.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:04 smithi016 ceph-mon[33335]: pgmap v473: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 923 KiB/s wr, 461 op/s 2024-05-09T09:29:06.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:06 smithi012 ceph-mon[30140]: pgmap v474: 129 pgs: 129 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 937 KiB/s wr, 428 op/s 2024-05-09T09:29:06.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:06 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:29:06.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:06 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:29:06.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:06 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:29:06.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:06 smithi012 ceph-mon[30140]: scrub summary: idle 2024-05-09T09:29:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:06 smithi170 ceph-mon[33632]: pgmap v474: 129 pgs: 129 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 937 KiB/s wr, 428 op/s 2024-05-09T09:29:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:06 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:29:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:06 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:29:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:06 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:29:06.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:06 smithi170 ceph-mon[33632]: scrub summary: idle 2024-05-09T09:29:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:06 smithi016 ceph-mon[33335]: pgmap v474: 129 pgs: 129 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 937 KiB/s wr, 428 op/s 2024-05-09T09:29:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:06 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:29:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:06 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:29:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:06 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:29:06.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:06 smithi016 ceph-mon[33335]: scrub summary: idle 2024-05-09T09:29:08.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:08 smithi012 ceph-mon[30140]: pgmap v475: 129 pgs: 129 active+clean; 56 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 816 KiB/s wr, 394 op/s 2024-05-09T09:29:08.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:08 smithi170 ceph-mon[33632]: pgmap v475: 129 pgs: 129 active+clean; 56 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 816 KiB/s wr, 394 op/s 2024-05-09T09:29:08.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:08 smithi016 ceph-mon[33335]: pgmap v475: 129 pgs: 129 active+clean; 56 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 816 KiB/s wr, 394 op/s 2024-05-09T09:29:10.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:10 smithi012 ceph-mon[30140]: pgmap v476: 129 pgs: 129 active+clean; 56 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 599 KiB/s wr, 318 op/s 2024-05-09T09:29:10.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:10 smithi170 ceph-mon[33632]: pgmap v476: 129 pgs: 129 active+clean; 56 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 599 KiB/s wr, 318 op/s 2024-05-09T09:29:10.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:10 smithi016 ceph-mon[33335]: pgmap v476: 129 pgs: 129 active+clean; 56 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 599 KiB/s wr, 318 op/s 2024-05-09T09:29:12.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:12 smithi012 ceph-mon[30140]: pgmap v477: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 604 KiB/s wr, 266 op/s 2024-05-09T09:29:12.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:12 smithi170 ceph-mon[33632]: pgmap v477: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 604 KiB/s wr, 266 op/s 2024-05-09T09:29:12.507 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:12 smithi016 ceph-mon[33335]: pgmap v477: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 604 KiB/s wr, 266 op/s 2024-05-09T09:29:14.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:14 smithi012 ceph-mon[30140]: pgmap v478: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 473 KiB/s wr, 178 op/s 2024-05-09T09:29:14.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:14 smithi170 ceph-mon[33632]: pgmap v478: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 473 KiB/s wr, 178 op/s 2024-05-09T09:29:14.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:14 smithi016 ceph-mon[33335]: pgmap v478: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 473 KiB/s wr, 178 op/s 2024-05-09T09:29:16.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:16 smithi012 ceph-mon[30140]: pgmap v479: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 243 KiB/s wr, 119 op/s 2024-05-09T09:29:16.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:16 smithi170 ceph-mon[33632]: pgmap v479: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 243 KiB/s wr, 119 op/s 2024-05-09T09:29:16.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:16 smithi016 ceph-mon[33335]: pgmap v479: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 243 KiB/s wr, 119 op/s 2024-05-09T09:29:18.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:18 smithi012 ceph-mon[30140]: pgmap v480: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 61 op/s 2024-05-09T09:29:18.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:18 smithi170 ceph-mon[33632]: pgmap v480: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 61 op/s 2024-05-09T09:29:18.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:18 smithi016 ceph-mon[33335]: pgmap v480: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 61 op/s 2024-05-09T09:29:20.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:20 smithi012 ceph-mon[30140]: pgmap v481: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-05-09T09:29:20.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:20 smithi170 ceph-mon[33632]: pgmap v481: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-05-09T09:29:20.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:20 smithi016 ceph-mon[33335]: pgmap v481: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-05-09T09:29:22.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:22 smithi012 ceph-mon[30140]: pgmap v482: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-05-09T09:29:22.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:22 smithi170 ceph-mon[33632]: pgmap v482: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-05-09T09:29:22.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:22 smithi016 ceph-mon[33335]: pgmap v482: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-05-09T09:29:24.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:24 smithi012 ceph-mon[30140]: pgmap v483: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-05-09T09:29:24.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:24 smithi170 ceph-mon[33632]: pgmap v483: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-05-09T09:29:24.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:24 smithi016 ceph-mon[33335]: pgmap v483: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-05-09T09:29:26.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:26 smithi012 ceph-mon[30140]: pgmap v484: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-05-09T09:29:26.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:26 smithi170 ceph-mon[33632]: pgmap v484: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-05-09T09:29:26.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:26 smithi016 ceph-mon[33335]: pgmap v484: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-05-09T09:29:28.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:28 smithi170 ceph-mon[33632]: pgmap v485: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T09:29:28.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:28 smithi016 ceph-mon[33335]: pgmap v485: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T09:29:28.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:28 smithi012 ceph-mon[30140]: pgmap v485: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T09:29:29.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:29:29.742 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:29:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.864+0000 7f72a65ce640 1 -- 172.21.15.12:0/673891809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a0101960 msgr2=0x7f72a0101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.864+0000 7f72a65ce640 1 --2- 172.21.15.12:0/673891809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a0101960 0x7f72a0101de0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7294009900 tx=0x7f729402a410 comp rx=0 tx=0).stop 2024-05-09T09:29:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.864+0000 7f72a65ce640 1 -- 172.21.15.12:0/673891809 shutdown_connections 2024-05-09T09:29:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.864+0000 7f72a65ce640 1 --2- 172.21.15.12:0/673891809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a0102320 0x7f72a0108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.864+0000 7f72a65ce640 1 --2- 172.21.15.12:0/673891809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a0101960 0x7f72a0101de0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.864+0000 7f72a65ce640 1 --2- 172.21.15.12:0/673891809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72a0100770 0x7f72a0100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.864+0000 7f72a65ce640 1 -- 172.21.15.12:0/673891809 >> 172.21.15.12:0/673891809 conn(0x7f72a00fbce0 msgr2=0x7f72a00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.864+0000 7f72a65ce640 1 -- 172.21.15.12:0/673891809 shutdown_connections 2024-05-09T09:29:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.864+0000 7f72a65ce640 1 -- 172.21.15.12:0/673891809 wait complete. 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.864+0000 7f72a65ce640 1 Processor -- start 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.864+0000 7f72a65ce640 1 -- start start 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f72a65ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a0100770 0x7f72a0197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f72a65ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72a0101960 0x7f72a01980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f72a65ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a0102320 0x7f72a019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f72a65ce640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f72a010d4e0 con 0x7f72a0100770 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f72a65ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72a010d360 con 0x7f72a0102320 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f72a65ce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72a010d660 con 0x7f72a0101960 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f729ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a0100770 0x7f72a0197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f729ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a0100770 0x7f72a0197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49446/0 (socket says 172.21.15.12:49446) 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f729ffff640 1 -- 172.21.15.12:0/4193666517 learned_addr learned my addr 172.21.15.12:0/4193666517 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f729f7fe640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72a0101960 0x7f72a01980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f72a4b6b640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a0102320 0x7f72a019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f729ffff640 1 -- 172.21.15.12:0/4193666517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72a0101960 msgr2=0x7f72a01980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f729ffff640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72a0101960 0x7f72a01980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f729ffff640 1 -- 172.21.15.12:0/4193666517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a0102320 msgr2=0x7f72a019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f729ffff640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a0102320 0x7f72a019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f729ffff640 1 -- 172.21.15.12:0/4193666517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a019f780 con 0x7f72a0100770 2024-05-09T09:29:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.865+0000 7f729f7fe640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72a0101960 0x7f72a01980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:29:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.866+0000 7f729ffff640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a0100770 0x7f72a0197b60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f729000d770 tx=0x7f729000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.866+0000 7f729d7fa640 1 -- 172.21.15.12:0/4193666517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290017070 con 0x7f72a0100770 2024-05-09T09:29:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.866+0000 7f729d7fa640 1 -- 172.21.15.12:0/4193666517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7290009c30 con 0x7f72a0100770 2024-05-09T09:29:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.866+0000 7f72a65ce640 1 -- 172.21.15.12:0/4193666517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a019fa70 con 0x7f72a0100770 2024-05-09T09:29:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.866+0000 7f729d7fa640 1 -- 172.21.15.12:0/4193666517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290003e10 con 0x7f72a0100770 2024-05-09T09:29:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.866+0000 7f72a65ce640 1 -- 172.21.15.12:0/4193666517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a019ffc0 con 0x7f72a0100770 2024-05-09T09:29:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.868+0000 7f729d7fa640 1 -- 172.21.15.12:0/4193666517 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7290004020 con 0x7f72a0100770 2024-05-09T09:29:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.868+0000 7f72a93b8740 1 Processor -- start 2024-05-09T09:29:29.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.868+0000 7f72a93b8740 1 -- start start 2024-05-09T09:29:29.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f72a93b8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6340 0x55ca6e3a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f72a93b8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3ab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f72a93b8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3abea0 0x55ca6e3ae330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f72a93b8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ca6e306d00 con 0x55ca6e3abea0 2024-05-09T09:29:29.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f72a93b8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ca6e2ae4c0 con 0x55ca6e3a6340 2024-05-09T09:29:29.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f72a93b8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ca6e302cc0 con 0x55ca6e3a6d30 2024-05-09T09:29:29.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f729f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3abea0 0x55ca6e3ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f729f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3abea0 0x55ca6e3ae330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49460/0 (socket says 172.21.15.12:49460) 2024-05-09T09:29:29.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f729f7fe640 1 -- 172.21.15.12:0/3595778848 learned_addr learned my addr 172.21.15.12:0/3595778848 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:29.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f72a4b6b640 1 --2- 172.21.15.12:0/3595778848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6340 0x55ca6e3a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f729ffff640 1 --2- 172.21.15.12:0/3595778848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f729d7fa640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f72880a2b90 0x7f72880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f729f7fe640 1 -- 172.21.15.12:0/3595778848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 msgr2=0x55ca6e3ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f729f7fe640 1 --2- 172.21.15.12:0/3595778848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f729f7fe640 1 -- 172.21.15.12:0/3595778848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6340 msgr2=0x55ca6e3a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f729f7fe640 1 --2- 172.21.15.12:0/3595778848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6340 0x55ca6e3a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f729f7fe640 1 -- 172.21.15.12:0/3595778848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca6e0e47d0 con 0x55ca6e3abea0 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f729d7fa640 1 -- 172.21.15.12:0/4193666517 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f72900cf280 con 0x7f72a0100770 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f72a4b6b640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f72880a2b90 0x7f72880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.869+0000 7f729f7fe640 1 --2- 172.21.15.12:0/3595778848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3abea0 0x55ca6e3ae330 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7294031570 tx=0x7f7294031ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64dca2db8f1f15a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.870+0000 7f72867fc640 1 -- 172.21.15.12:0/3595778848 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294035030 con 0x55ca6e3abea0 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.870+0000 7f72867fc640 1 -- 172.21.15.12:0/3595778848 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f72940318d0 con 0x55ca6e3abea0 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.870+0000 7f72867fc640 1 -- 172.21.15.12:0/3595778848 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294004900 con 0x55ca6e3abea0 2024-05-09T09:29:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.870+0000 7f72a93b8740 1 -- 172.21.15.12:0/3595778848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3abea0 msgr2=0x55ca6e3ae330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.870+0000 7f72a4b6b640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f72880a2b90 0x7f72880a5050 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55ca6e3a7590 tx=0x7f7298004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.870+0000 7f72a93b8740 1 --2- 172.21.15.12:0/3595778848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3abea0 0x55ca6e3ae330 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7294031570 tx=0x7f7294031ef0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.870+0000 7f72a93b8740 1 -- 172.21.15.12:0/3595778848 shutdown_connections 2024-05-09T09:29:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.870+0000 7f72a93b8740 1 --2- 172.21.15.12:0/3595778848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3abea0 0x55ca6e3ae330 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.870+0000 7f72a93b8740 1 --2- 172.21.15.12:0/3595778848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.870+0000 7f72a93b8740 1 --2- 172.21.15.12:0/3595778848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6340 0x55ca6e3a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.870+0000 7f72a93b8740 1 -- 172.21.15.12:0/3595778848 >> 172.21.15.12:0/3595778848 conn(0x55ca6e3a19b0 msgr2=0x55ca6e3a1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.871+0000 7f72a93b8740 1 -- 172.21.15.12:0/3595778848 shutdown_connections 2024-05-09T09:29:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.871+0000 7f72a93b8740 1 -- 172.21.15.12:0/3595778848 wait complete. 2024-05-09T09:29:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.871+0000 7f72a93b8740 1 -- start start 2024-05-09T09:29:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.871+0000 7f72a93b8740 1 Processor -- start 2024-05-09T09:29:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.871+0000 7f72a93b8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 0x55ca6e3b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.871+0000 7f72a93b8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.871+0000 7f72a93b8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3abea0 0x55ca6e3b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.871+0000 7f72a93b8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ca6e3aedf0 con 0x55ca6e3a6340 2024-05-09T09:29:29.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.871+0000 7f72a93b8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ca6e2c1b90 con 0x55ca6e3a6d30 2024-05-09T09:29:29.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.871+0000 7f72a93b8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ca6e3aef70 con 0x55ca6e3abea0 2024-05-09T09:29:29.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.871+0000 7f729ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 0x55ca6e3b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.871+0000 7f72a93b8740 10 client.0 _ll_register_callbacks cb 0x55ca6e2fdda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f729ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 0x55ca6e3b06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49476/0 (socket says 172.21.15.12:49476) 2024-05-09T09:29:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f729ffff640 1 -- 172.21.15.12:0/969574866 learned_addr learned my addr 172.21.15.12:0/969574866 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f729f7fe640 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f72a4b6b640 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3abea0 0x55ca6e3b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f729ffff640 1 -- 172.21.15.12:0/969574866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3abea0 msgr2=0x55ca6e3b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f729ffff640 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3abea0 0x55ca6e3b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f729ffff640 1 -- 172.21.15.12:0/969574866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6d30 msgr2=0x55ca6e3b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f729ffff640 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f729ffff640 1 -- 172.21.15.12:0/969574866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca6e01a3f0 con 0x55ca6e3a6340 2024-05-09T09:29:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f72a4b6b640 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3abea0 0x55ca6e3b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:29:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f729ffff640 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 0x55ca6e3b06b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f72900cc9b0 tx=0x7f72900973b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18b1109dbee82bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f727f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f727f7fe640 1 -- 172.21.15.12:0/969574866 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290020050 con 0x55ca6e3a6340 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f727f7fe640 1 -- 172.21.15.12:0/969574866 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f7290099a10 con 0x55ca6e3a6340 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f72a93b8740 1 -- 172.21.15.12:0/969574866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72940318d0 con 0x55ca6e3a6340 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f727f7fe640 1 -- 172.21.15.12:0/969574866 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f729009b070 con 0x55ca6e3a6340 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f72a93b8740 10 client.16995 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.872+0000 7f72a93b8740 1 -- 172.21.15.12:0/969574866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ca6dfda0f0 con 0x55ca6e3a6340 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.873+0000 7f727f7fe640 1 -- 172.21.15.12:0/969574866 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f72940318d0 con 0x55ca6e3a6340 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.873+0000 7f727f7fe640 1 -- 172.21.15.12:0/969574866 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f7290097d80 con 0x55ca6e3a6340 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.873+0000 7f72a93b8740 10 client.16995 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.873+0000 7f72a93b8740 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.873+0000 7f72a93b8740 20 client.16995 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.873+0000 7f72a93b8740 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ca6e3bc490 0x55ca6e3be950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.873+0000 7f72a93b8740 4 client.16995 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.873+0000 7f72a93b8740 1 -- 172.21.15.12:0/969574866 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca6e314430 con 0x55ca6e3bc490 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.873+0000 7f729f7fe640 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ca6e3bc490 0x55ca6e3be950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.874+0000 7f727f7fe640 10 client.16995 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.874+0000 7f729f7fe640 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ca6e3bc490 0x55ca6e3be950 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e59f3cccdc2c10b server_cookie=757d68f2a2f62e5b in_seq=0 out_seq=0 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.874+0000 7f727f7fe640 1 -- 172.21.15.12:0/969574866 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca6e314430 con 0x55ca6e3bc490 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.874+0000 7f727f7fe640 10 client.16995 handle_command_reply: tid=0 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.874+0000 7f72a93b8740 1 client.16995 shutdown 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.875+0000 7f72a93b8740 1 -- 172.21.15.12:0/969574866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 msgr2=0x55ca6e3b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.875+0000 7f72a93b8740 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 0x55ca6e3b06b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f72900cc9b0 tx=0x7f72900973b0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.875+0000 7f72a93b8740 1 -- 172.21.15.12:0/969574866 shutdown_connections 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.875+0000 7f72a93b8740 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ca6e3bc490 0x55ca6e3be950 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.875+0000 7f72a93b8740 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3abea0 0x55ca6e3b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.875+0000 7f72a93b8740 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.875+0000 7f727f7fe640 0 client.16995 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:29:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.875+0000 7f72a93b8740 1 --2- 172.21.15.12:0/969574866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 0x55ca6e3b06b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.875+0000 7f72a93b8740 1 -- 172.21.15.12:0/969574866 >> 172.21.15.12:0/969574866 conn(0x55ca6e3a19b0 msgr2=0x55ca6e39c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.875+0000 7f72a93b8740 1 -- 172.21.15.12:0/969574866 shutdown_connections 2024-05-09T09:29:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.875+0000 7f72a93b8740 1 -- 172.21.15.12:0/969574866 wait complete. 2024-05-09T09:29:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.875+0000 7f72a93b8740 20 client.16995 trim_cache size 0 max 16384 2024-05-09T09:29:29.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f72a93b8740 1 Processor -- start 2024-05-09T09:29:29.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f72a93b8740 1 -- start start 2024-05-09T09:29:29.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f72a93b8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a19b0 0x55ca6e39c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f72a93b8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6340 0x55ca6e3afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f72a93b8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f72a93b8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01a8370 con 0x55ca6e3a19b0 2024-05-09T09:29:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f72a93b8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01aa0f0 con 0x55ca6e3a6340 2024-05-09T09:29:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f72a93b8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01a8b00 con 0x55ca6e3a6d30 2024-05-09T09:29:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f729ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a19b0 0x55ca6e39c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f729ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a19b0 0x55ca6e39c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49488/0 (socket says 172.21.15.12:49488) 2024-05-09T09:29:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f729ffff640 1 -- 172.21.15.12:0/988020614 learned_addr learned my addr 172.21.15.12:0/988020614 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f729ffff640 1 -- 172.21.15.12:0/988020614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 msgr2=0x55ca6e3b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f729ffff640 1 --2- 172.21.15.12:0/988020614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f729ffff640 1 -- 172.21.15.12:0/988020614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6340 msgr2=0x55ca6e3afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f729ffff640 1 --2- 172.21.15.12:0/988020614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6340 0x55ca6e3afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f729ffff640 1 -- 172.21.15.12:0/988020614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca6e3a7110 con 0x55ca6e3a19b0 2024-05-09T09:29:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f729ffff640 1 --2- 172.21.15.12:0/988020614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a19b0 0x55ca6e39c660 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f72900037b0 tx=0x7f7290003810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce89972e1c5a754 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.893+0000 7f7285ffb640 1 -- 172.21.15.12:0/988020614 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290020070 con 0x55ca6e3a19b0 2024-05-09T09:29:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.894+0000 7f7285ffb640 1 -- 172.21.15.12:0/988020614 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f72900b4020 con 0x55ca6e3a19b0 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.894+0000 7f7285ffb640 1 -- 172.21.15.12:0/988020614 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729009be10 con 0x55ca6e3a19b0 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.894+0000 7f72a93b8740 1 -- 172.21.15.12:0/988020614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a19b0 msgr2=0x55ca6e39c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.894+0000 7f72a93b8740 1 --2- 172.21.15.12:0/988020614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a19b0 0x55ca6e39c660 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f72900037b0 tx=0x7f7290003810 comp rx=0 tx=0).stop 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.894+0000 7f72a93b8740 1 -- 172.21.15.12:0/988020614 shutdown_connections 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.894+0000 7f72a93b8740 1 --2- 172.21.15.12:0/988020614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.894+0000 7f72a93b8740 1 --2- 172.21.15.12:0/988020614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a6340 0x55ca6e3afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.894+0000 7f72a93b8740 1 --2- 172.21.15.12:0/988020614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a19b0 0x55ca6e39c660 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.894+0000 7f72a93b8740 1 -- 172.21.15.12:0/988020614 >> 172.21.15.12:0/988020614 conn(0x55ca6e3989e0 msgr2=0x55ca6e39bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.894+0000 7f72a93b8740 1 -- 172.21.15.12:0/988020614 shutdown_connections 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.894+0000 7f72a93b8740 1 -- 172.21.15.12:0/988020614 wait complete. 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.894+0000 7f72a93b8740 1 -- start start 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f72a93b8740 1 Processor -- start 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f72a93b8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a19b0 0x55ca6e3b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f72a93b8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 0x55ca6e3b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f72a93b8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f72a93b8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01a9a20 con 0x55ca6e3a6340 2024-05-09T09:29:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f72a93b8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01a9290 con 0x55ca6e3a19b0 2024-05-09T09:29:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f72a93b8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01a6350 con 0x55ca6e3a6d30 2024-05-09T09:29:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f72a93b8740 10 client.0 _ll_register_callbacks cb 0x7f7290003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f729f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 0x55ca6e3b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f729f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 0x55ca6e3b7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49504/0 (socket says 172.21.15.12:49504) 2024-05-09T09:29:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f729f7fe640 1 -- 172.21.15.12:0/1189567155 learned_addr learned my addr 172.21.15.12:0/1189567155 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f72a4b6b640 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f729f7fe640 1 -- 172.21.15.12:0/1189567155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 msgr2=0x55ca6e3b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f729ffff640 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a19b0 0x55ca6e3b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f729f7fe640 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f729f7fe640 1 -- 172.21.15.12:0/1189567155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a19b0 msgr2=0x55ca6e3b6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f729f7fe640 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a19b0 0x55ca6e3b6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f729f7fe640 1 -- 172.21.15.12:0/1189567155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca6e3b5930 con 0x55ca6e3a6340 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f72a4b6b640 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.895+0000 7f729ffff640 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a19b0 0x55ca6e3b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f729f7fe640 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 0x55ca6e3b7050 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7294004650 tx=0x7f72940058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759c4f88fbe6286 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f727f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f727f7fe640 1 -- 172.21.15.12:0/1189567155 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294035040 con 0x55ca6e3a6340 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f72a93b8740 1 -- 172.21.15.12:0/1189567155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72900b4020 con 0x55ca6e3a6340 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f727f7fe640 1 -- 172.21.15.12:0/1189567155 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f7294051070 con 0x55ca6e3a6340 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f727f7fe640 1 -- 172.21.15.12:0/1189567155 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f729405f050 con 0x55ca6e3a6340 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f72a93b8740 10 client.17001 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f72a93b8740 1 -- 172.21.15.12:0/1189567155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ca6e4235e0 con 0x55ca6e3a6340 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f727f7fe640 1 -- 172.21.15.12:0/1189567155 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f72900b4020 con 0x55ca6e3a6340 2024-05-09T09:29:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f727f7fe640 1 -- 172.21.15.12:0/1189567155 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f7294050770 con 0x55ca6e3a6340 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f72a93b8740 10 client.17001 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f72a93b8740 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f72a93b8740 20 client.17001 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f72a93b8740 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ca6e3abea0 0x55ca6e425d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f72a93b8740 4 client.17001 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.896+0000 7f72a93b8740 1 -- 172.21.15.12:0/1189567155 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca6e30e780 con 0x55ca6e3abea0 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f729ffff640 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ca6e3abea0 0x55ca6e425d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f729ffff640 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ca6e3abea0 0x55ca6e425d00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63fa28aae920af94 server_cookie=99bc3a544244832a in_seq=0 out_seq=0 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f727f7fe640 10 client.17001 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f727f7fe640 1 -- 172.21.15.12:0/1189567155 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ca6e30e780 con 0x55ca6e3abea0 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f727f7fe640 10 client.17001 handle_command_reply: tid=0 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f72a93b8740 1 client.17001 shutdown 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f72a93b8740 1 -- 172.21.15.12:0/1189567155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 msgr2=0x55ca6e3b7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f72a93b8740 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 0x55ca6e3b7050 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7294004650 tx=0x7f72940058b0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f72a93b8740 1 -- 172.21.15.12:0/1189567155 shutdown_connections 2024-05-09T09:29:29.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f72a93b8740 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x55ca6e3abea0 0x55ca6e425d00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f72a93b8740 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca6e3a6d30 0x55ca6e3b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f72a93b8740 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca6e3a6340 0x55ca6e3b7050 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f727f7fe640 0 client.17001 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:29:29.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f72a93b8740 1 --2- 172.21.15.12:0/1189567155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6e3a19b0 0x55ca6e3b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.897+0000 7f72a93b8740 1 -- 172.21.15.12:0/1189567155 >> 172.21.15.12:0/1189567155 conn(0x55ca6e3989e0 msgr2=0x55ca6e39c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:29.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a93b8740 1 -- 172.21.15.12:0/1189567155 shutdown_connections 2024-05-09T09:29:29.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a93b8740 1 -- 172.21.15.12:0/1189567155 wait complete. 2024-05-09T09:29:29.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a93b8740 20 client.17001 trim_cache size 0 max 16384 2024-05-09T09:29:29.905 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:29:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T09:29:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T09:29:29.905 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:29:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a65ce640 1 -- 172.21.15.12:0/4193666517 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f72880a2b90 msgr2=0x7f72880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a65ce640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f72880a2b90 0x7f72880a5050 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55ca6e3a7590 tx=0x7f7298004a40 comp rx=0 tx=0).stop 2024-05-09T09:29:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a65ce640 1 -- 172.21.15.12:0/4193666517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a0100770 msgr2=0x7f72a0197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a65ce640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a0100770 0x7f72a0197b60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f729000d770 tx=0x7f729000dc30 comp rx=0 tx=0).stop 2024-05-09T09:29:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a65ce640 1 -- 172.21.15.12:0/4193666517 shutdown_connections 2024-05-09T09:29:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a65ce640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f72880a2b90 0x7f72880a5050 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a65ce640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a0102320 0x7f72a019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a65ce640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72a0101960 0x7f72a01980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a65ce640 1 --2- 172.21.15.12:0/4193666517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a0100770 0x7f72a0197b60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a65ce640 1 -- 172.21.15.12:0/4193666517 >> 172.21.15.12:0/4193666517 conn(0x7f72a00fbce0 msgr2=0x7f72a0104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a65ce640 1 -- 172.21.15.12:0/4193666517 shutdown_connections 2024-05-09T09:29:29.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:29.898+0000 7f72a65ce640 1 -- 172.21.15.12:0/4193666517 wait complete. 2024-05-09T09:29:29.909 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:29:29.910 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:29:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.015+0000 7f5686bc4640 1 -- 172.21.15.12:0/2897427120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5680102330 msgr2=0x7f5680108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.015+0000 7f5686bc4640 1 --2- 172.21.15.12:0/2897427120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5680102330 0x7f5680108bc0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5678009ca0 tx=0x7f567802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.015+0000 7f5686bc4640 1 -- 172.21.15.12:0/2897427120 shutdown_connections 2024-05-09T09:29:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.015+0000 7f5686bc4640 1 --2- 172.21.15.12:0/2897427120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5680102330 0x7f5680108bc0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.015+0000 7f5686bc4640 1 --2- 172.21.15.12:0/2897427120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5680101970 0x7f5680101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.015+0000 7f5686bc4640 1 --2- 172.21.15.12:0/2897427120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5680100780 0x7f5680100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.015+0000 7f5686bc4640 1 -- 172.21.15.12:0/2897427120 >> 172.21.15.12:0/2897427120 conn(0x7f56800fbd10 msgr2=0x7f56800fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.015+0000 7f5686bc4640 1 -- 172.21.15.12:0/2897427120 shutdown_connections 2024-05-09T09:29:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.015+0000 7f5686bc4640 1 -- 172.21.15.12:0/2897427120 wait complete. 2024-05-09T09:29:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.016+0000 7f5686bc4640 1 Processor -- start 2024-05-09T09:29:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.016+0000 7f5686bc4640 1 -- start start 2024-05-09T09:29:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.016+0000 7f5686bc4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5680100780 0x7f5680197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.016+0000 7f5686bc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5680101970 0x7f5680198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.016+0000 7f5686bc4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5680102330 0x7f568019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.016+0000 7f5686bc4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f568010d4f0 con 0x7f5680100780 2024-05-09T09:29:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.016+0000 7f5686bc4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f568010d370 con 0x7f5680101970 2024-05-09T09:29:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.016+0000 7f5686bc4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f568010d670 con 0x7f5680102330 2024-05-09T09:29:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.016+0000 7f5684960640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5680100780 0x7f5680197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.016+0000 7f5684960640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5680100780 0x7f5680197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49530/0 (socket says 172.21.15.12:49530) 2024-05-09T09:29:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.016+0000 7f5684960640 1 -- 172.21.15.12:0/2743219745 learned_addr learned my addr 172.21.15.12:0/2743219745 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.016+0000 7f567ffff640 1 --2- 172.21.15.12:0/2743219745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5680101970 0x7f5680198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.017+0000 7f5684960640 1 -- 172.21.15.12:0/2743219745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5680102330 msgr2=0x7f568019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.017+0000 7f5684960640 1 --2- 172.21.15.12:0/2743219745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5680102330 0x7f568019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.017+0000 7f5684960640 1 -- 172.21.15.12:0/2743219745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5680101970 msgr2=0x7f5680198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.017+0000 7f5684960640 1 --2- 172.21.15.12:0/2743219745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5680101970 0x7f5680198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.017+0000 7f5684960640 1 -- 172.21.15.12:0/2743219745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568019f750 con 0x7f5680100780 2024-05-09T09:29:30.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.017+0000 7f5684960640 1 --2- 172.21.15.12:0/2743219745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5680100780 0x7f5680197b30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f567400d950 tx=0x7f567400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.017+0000 7f567dffb640 1 -- 172.21.15.12:0/2743219745 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674017070 con 0x7f5680100780 2024-05-09T09:29:30.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.017+0000 7f567dffb640 1 -- 172.21.15.12:0/2743219745 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5674009c30 con 0x7f5680100780 2024-05-09T09:29:30.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.017+0000 7f567dffb640 1 -- 172.21.15.12:0/2743219745 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567400f620 con 0x7f5680100780 2024-05-09T09:29:30.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.017+0000 7f5686bc4640 1 -- 172.21.15.12:0/2743219745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568019fa40 con 0x7f5680100780 2024-05-09T09:29:30.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.017+0000 7f5686bc4640 1 -- 172.21.15.12:0/2743219745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568019ff40 con 0x7f5680100780 2024-05-09T09:29:30.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.018+0000 7f5686bc4640 1 -- 172.21.15.12:0/2743219745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5680107aa0 con 0x7f5680100780 2024-05-09T09:29:30.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.019+0000 7f567dffb640 1 -- 172.21.15.12:0/2743219745 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f567401a070 con 0x7f5680100780 2024-05-09T09:29:30.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.020+0000 7f567dffb640 1 --2- 172.21.15.12:0/2743219745 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f56700a2b90 0x7f56700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.020+0000 7f567dffb640 1 -- 172.21.15.12:0/2743219745 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f56740ceed0 con 0x7f5680100780 2024-05-09T09:29:30.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.020+0000 7f567ffff640 1 --2- 172.21.15.12:0/2743219745 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f56700a2b90 0x7f56700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.020+0000 7f567ffff640 1 --2- 172.21.15.12:0/2743219745 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f56700a2b90 0x7f56700a5050 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5680199040 tx=0x7f566c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.025+0000 7f567dffb640 1 -- 172.21.15.12:0/2743219745 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5674009de0 con 0x7f5680100780 2024-05-09T09:29:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.249+0000 7f5686bc4640 1 -- 172.21.15.12:0/2743219745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f568002d1c0 con 0x7f5680100780 2024-05-09T09:29:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.250+0000 7f567dffb640 1 -- 172.21.15.12:0/2743219745 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11966 (secure 0 0 0) 0x7f568002d1c0 con 0x7f5680100780 2024-05-09T09:29:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:29:30.250 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:28:24.476138+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25883":{"gid":25883,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3067829157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3067829157},{"type":"v1","addr":"172.21.15.16:6837","nonce":3067829157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[24502,24523,24515,14628,24484]},"id":1}]} 2024-05-09T09:29:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 42 2024-05-09T09:29:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.252+0000 7f5686bc4640 1 -- 172.21.15.12:0/2743219745 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f56700a2b90 msgr2=0x7f56700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.252+0000 7f5686bc4640 1 --2- 172.21.15.12:0/2743219745 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f56700a2b90 0x7f56700a5050 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5680199040 tx=0x7f566c00b390 comp rx=0 tx=0).stop 2024-05-09T09:29:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.252+0000 7f5686bc4640 1 -- 172.21.15.12:0/2743219745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5680100780 msgr2=0x7f5680197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.252+0000 7f5686bc4640 1 --2- 172.21.15.12:0/2743219745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5680100780 0x7f5680197b30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f567400d950 tx=0x7f567400de10 comp rx=0 tx=0).stop 2024-05-09T09:29:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.252+0000 7f5686bc4640 1 -- 172.21.15.12:0/2743219745 shutdown_connections 2024-05-09T09:29:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.252+0000 7f5686bc4640 1 --2- 172.21.15.12:0/2743219745 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f56700a2b90 0x7f56700a5050 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.252+0000 7f5686bc4640 1 --2- 172.21.15.12:0/2743219745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5680102330 0x7f568019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.252+0000 7f5686bc4640 1 --2- 172.21.15.12:0/2743219745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5680101970 0x7f5680198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.252+0000 7f5686bc4640 1 --2- 172.21.15.12:0/2743219745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5680100780 0x7f5680197b30 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.252+0000 7f5686bc4640 1 -- 172.21.15.12:0/2743219745 >> 172.21.15.12:0/2743219745 conn(0x7f56800fbd10 msgr2=0x7f5680104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.252+0000 7f5686bc4640 1 -- 172.21.15.12:0/2743219745 shutdown_connections 2024-05-09T09:29:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.252+0000 7f5686bc4640 1 -- 172.21.15.12:0/2743219745 wait complete. 2024-05-09T09:29:30.264 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-09T09:29:30.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.372+0000 7f4e6a590640 1 -- 172.21.15.12:0/3724821265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e641023d0 msgr2=0x7f4e641047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.372+0000 7f4e6a590640 1 --2- 172.21.15.12:0/3724821265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e641023d0 0x7f4e641047c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009900 tx=0x7f4e5402a410 comp rx=0 tx=0).stop 2024-05-09T09:29:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.372+0000 7f4e6a590640 1 -- 172.21.15.12:0/3724821265 shutdown_connections 2024-05-09T09:29:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.372+0000 7f4e6a590640 1 --2- 172.21.15.12:0/3724821265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e64107630 0x7f4e64109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.372+0000 7f4e6a590640 1 --2- 172.21.15.12:0/3724821265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e64104d00 0x7f4e641070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.372+0000 7f4e6a590640 1 --2- 172.21.15.12:0/3724821265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e641023d0 0x7f4e641047c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.372+0000 7f4e6a590640 1 -- 172.21.15.12:0/3724821265 >> 172.21.15.12:0/3724821265 conn(0x7f4e640fbd20 msgr2=0x7f4e640fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.372+0000 7f4e6a590640 1 -- 172.21.15.12:0/3724821265 shutdown_connections 2024-05-09T09:29:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.372+0000 7f4e6a590640 1 -- 172.21.15.12:0/3724821265 wait complete. 2024-05-09T09:29:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.373+0000 7f4e6a590640 1 Processor -- start 2024-05-09T09:29:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.373+0000 7f4e6a590640 1 -- start start 2024-05-09T09:29:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.373+0000 7f4e6a590640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e641023d0 0x7f4e64197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.373+0000 7f4e6a590640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e64104d00 0x7f4e641980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.373+0000 7f4e6a590640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e64107630 0x7f4e6419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.373+0000 7f4e6a590640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6410d570 con 0x7f4e64107630 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.373+0000 7f4e6a590640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6410d3f0 con 0x7f4e641023d0 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.373+0000 7f4e6a590640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6410d6f0 con 0x7f4e64104d00 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.373+0000 7f4e68b2d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e64107630 0x7f4e6419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.374+0000 7f4e68b2d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e64107630 0x7f4e6419f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49546/0 (socket says 172.21.15.12:49546) 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.374+0000 7f4e63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e641023d0 0x7f4e64197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.374+0000 7f4e68b2d640 1 -- 172.21.15.12:0/3752894535 learned_addr learned my addr 172.21.15.12:0/3752894535 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.374+0000 7f4e637fe640 1 --2- 172.21.15.12:0/3752894535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e64104d00 0x7f4e641980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.374+0000 7f4e68b2d640 1 -- 172.21.15.12:0/3752894535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e64104d00 msgr2=0x7f4e641980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.374+0000 7f4e68b2d640 1 --2- 172.21.15.12:0/3752894535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e64104d00 0x7f4e641980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.374+0000 7f4e68b2d640 1 -- 172.21.15.12:0/3752894535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e641023d0 msgr2=0x7f4e64197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.374+0000 7f4e68b2d640 1 --2- 172.21.15.12:0/3752894535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e641023d0 0x7f4e64197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.374+0000 7f4e68b2d640 1 -- 172.21.15.12:0/3752894535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6419f7d0 con 0x7f4e64107630 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.374+0000 7f4e68b2d640 1 --2- 172.21.15.12:0/3752894535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e64107630 0x7f4e6419f160 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00cce0 tx=0x7f4e5c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.374+0000 7f4e617fa640 1 -- 172.21.15.12:0/3752894535 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00f450 con 0x7f4e64107630 2024-05-09T09:29:30.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.374+0000 7f4e6a590640 1 -- 172.21.15.12:0/3752894535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6419fac0 con 0x7f4e64107630 2024-05-09T09:29:30.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.374+0000 7f4e617fa640 1 -- 172.21.15.12:0/3752894535 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4e5c00fac0 con 0x7f4e64107630 2024-05-09T09:29:30.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.375+0000 7f4e617fa640 1 -- 172.21.15.12:0/3752894535 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c018be0 con 0x7f4e64107630 2024-05-09T09:29:30.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.375+0000 7f4e6a590640 1 -- 172.21.15.12:0/3752894535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e641a0010 con 0x7f4e64107630 2024-05-09T09:29:30.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.377+0000 7f4e617fa640 1 -- 172.21.15.12:0/3752894535 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4e5c00fc70 con 0x7f4e64107630 2024-05-09T09:29:30.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.378+0000 7f4e617fa640 1 --2- 172.21.15.12:0/3752894535 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4e4c0a2c10 0x7f4e4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.378+0000 7f4e617fa640 1 -- 172.21.15.12:0/3752894535 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4e5c0cf9c0 con 0x7f4e64107630 2024-05-09T09:29:30.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.378+0000 7f4e6d37e740 1 Processor -- start 2024-05-09T09:29:30.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.378+0000 7f4e63fff640 1 --2- 172.21.15.12:0/3752894535 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4e4c0a2c10 0x7f4e4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.379+0000 7f4e6d37e740 1 -- start start 2024-05-09T09:29:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.379+0000 7f4e6d37e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5fac360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.379+0000 7f4e6d37e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 0x5630b5fb1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.379+0000 7f4e6d37e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fb1aa0 0x5630b5fb3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.379+0000 7f4e6d37e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5630b5f10c40 con 0x5630b5fac930 2024-05-09T09:29:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.379+0000 7f4e6d37e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5630b5eb8580 con 0x5630b5fb1aa0 2024-05-09T09:29:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.379+0000 7f4e6d37e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5630b5f0cc00 con 0x5630b5fabf40 2024-05-09T09:29:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.379+0000 7f4e637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 0x5630b5fb1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.380+0000 7f4e637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 0x5630b5fb1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49552/0 (socket says 172.21.15.12:49552) 2024-05-09T09:29:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.380+0000 7f4e637fe640 1 -- 172.21.15.12:0/1036105015 learned_addr learned my addr 172.21.15.12:0/1036105015 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.380+0000 7f4e68b2d640 1 --2- 172.21.15.12:0/1036105015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fb1aa0 0x5630b5fb3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.380+0000 7f4e63fff640 1 --2- 172.21.15.12:0/3752894535 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4e4c0a2c10 0x7f4e4c0a50d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4e54006010 tx=0x7f4e54031530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.380+0000 7f4e63fff640 1 --2- 172.21.15.12:0/1036105015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5fac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.380+0000 7f4e637fe640 1 -- 172.21.15.12:0/1036105015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 msgr2=0x5630b5fac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.380+0000 7f4e637fe640 1 --2- 172.21.15.12:0/1036105015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5fac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.380+0000 7f4e637fe640 1 -- 172.21.15.12:0/1036105015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fb1aa0 msgr2=0x5630b5fb3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.380+0000 7f4e637fe640 1 --2- 172.21.15.12:0/1036105015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fb1aa0 0x5630b5fb3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.380+0000 7f4e637fe640 1 -- 172.21.15.12:0/1036105015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b5c243f0 con 0x5630b5fac930 2024-05-09T09:29:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.380+0000 7f4e63fff640 1 --2- 172.21.15.12:0/1036105015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5fac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:29:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.380+0000 7f4e68b2d640 1 --2- 172.21.15.12:0/1036105015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fb1aa0 0x5630b5fb3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:29:30.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.380+0000 7f4e637fe640 1 --2- 172.21.15.12:0/1036105015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 0x5630b5fb1430 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800b6c0 tx=0x7f4e5800bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc51b90e4189cd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.381+0000 7f4e4a7fc640 1 -- 172.21.15.12:0/1036105015 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58003bd0 con 0x5630b5fac930 2024-05-09T09:29:30.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.381+0000 7f4e4a7fc640 1 -- 172.21.15.12:0/1036105015 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4e58015070 con 0x5630b5fac930 2024-05-09T09:29:30.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.381+0000 7f4e4a7fc640 1 -- 172.21.15.12:0/1036105015 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58004df0 con 0x5630b5fac930 2024-05-09T09:29:30.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.381+0000 7f4e6d37e740 1 -- 172.21.15.12:0/1036105015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 msgr2=0x5630b5fb1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.381+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/1036105015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 0x5630b5fb1430 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800b6c0 tx=0x7f4e5800bb80 comp rx=0 tx=0).stop 2024-05-09T09:29:30.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.381+0000 7f4e6d37e740 1 -- 172.21.15.12:0/1036105015 shutdown_connections 2024-05-09T09:29:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.381+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/1036105015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fb1aa0 0x5630b5fb3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.381+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/1036105015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 0x5630b5fb1430 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.381+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/1036105015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5fac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.381+0000 7f4e6d37e740 1 -- 172.21.15.12:0/1036105015 >> 172.21.15.12:0/1036105015 conn(0x5630b5fa75b0 msgr2=0x5630b5fa79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.381+0000 7f4e6d37e740 1 -- 172.21.15.12:0/1036105015 shutdown_connections 2024-05-09T09:29:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.381+0000 7f4e6d37e740 1 -- 172.21.15.12:0/1036105015 wait complete. 2024-05-09T09:29:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e6d37e740 1 -- start start 2024-05-09T09:29:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e6d37e740 1 Processor -- start 2024-05-09T09:29:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e6d37e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5fba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e6d37e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 0x5630b5fbace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e6d37e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fb1aa0 0x5630b5fc1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e6d37e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5630b5fb8e60 con 0x5630b5fac930 2024-05-09T09:29:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e6d37e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5630b5ecbc40 con 0x5630b5fb1aa0 2024-05-09T09:29:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e6d37e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5630b5fb8fe0 con 0x5630b5fabf40 2024-05-09T09:29:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e6d37e740 10 client.0 _ll_register_callbacks cb 0x5630b5c23be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 0x5630b5fbace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 0x5630b5fbace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49560/0 (socket says 172.21.15.12:49560) 2024-05-09T09:29:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e637fe640 1 -- 172.21.15.12:0/1114405708 learned_addr learned my addr 172.21.15.12:0/1114405708 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e68b2d640 1 --2- 172.21.15.12:0/1114405708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fb1aa0 0x5630b5fc1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.382+0000 7f4e63fff640 1 --2- 172.21.15.12:0/1114405708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5fba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e637fe640 1 -- 172.21.15.12:0/1114405708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 msgr2=0x5630b5fba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e637fe640 1 --2- 172.21.15.12:0/1114405708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5fba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e637fe640 1 -- 172.21.15.12:0/1114405708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fb1aa0 msgr2=0x5630b5fc1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e637fe640 1 --2- 172.21.15.12:0/1114405708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fb1aa0 0x5630b5fc1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e637fe640 1 -- 172.21.15.12:0/1114405708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b5cee7d0 con 0x5630b5fac930 2024-05-09T09:29:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e637fe640 1 --2- 172.21.15.12:0/1114405708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 0x5630b5fbace0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4e58018000 tx=0x7f4e5800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491e2c171349d53f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e437fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:29:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e437fe640 1 -- 172.21.15.12:0/1114405708 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58028060 con 0x5630b5fac930 2024-05-09T09:29:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e6d37e740 1 -- 172.21.15.12:0/1114405708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e58015070 con 0x5630b5fac930 2024-05-09T09:29:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e437fe640 1 -- 172.21.15.12:0/1114405708 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4e58003dd0 con 0x5630b5fac930 2024-05-09T09:29:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e437fe640 1 -- 172.21.15.12:0/1114405708 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4e5802f050 con 0x5630b5fac930 2024-05-09T09:29:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e6d37e740 10 client.17019 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e6d37e740 1 -- 172.21.15.12:0/1114405708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5630b5fc5090 con 0x5630b5fac930 2024-05-09T09:29:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e437fe640 1 -- 172.21.15.12:0/1114405708 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4e58015070 con 0x5630b5fac930 2024-05-09T09:29:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.383+0000 7f4e437fe640 1 -- 172.21.15.12:0/1114405708 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f4e58022070 con 0x5630b5fac930 2024-05-09T09:29:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.384+0000 7f4e6d37e740 10 client.17019 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.384+0000 7f4e6d37e740 10 client.17019 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:29:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.384+0000 7f4e6d37e740 20 client.17019 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.384+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/1114405708 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5630b5fc63e0 0x5630b5fc87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.384+0000 7f4e6d37e740 4 client.17019 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:29:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.384+0000 7f4e6d37e740 1 -- 172.21.15.12:0/1114405708 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630b5f1e370 con 0x5630b5fc63e0 2024-05-09T09:29:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.384+0000 7f4e63fff640 1 --2- 172.21.15.12:0/1114405708 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5630b5fc63e0 0x5630b5fc87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.384+0000 7f4e437fe640 10 client.17019 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:29:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.384+0000 7f4e63fff640 1 --2- 172.21.15.12:0/1114405708 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5630b5fc63e0 0x5630b5fc87f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=275e90d8c578a94 server_cookie=8a28518a9164d17 in_seq=0 out_seq=0 2024-05-09T09:29:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e437fe640 1 -- 172.21.15.12:0/1114405708 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630b5f1e370 con 0x5630b5fc63e0 2024-05-09T09:29:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e437fe640 10 client.17019 handle_command_reply: tid=0 2024-05-09T09:29:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e6d37e740 1 client.17019 shutdown 2024-05-09T09:29:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e6d37e740 1 -- 172.21.15.12:0/1114405708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 msgr2=0x5630b5fbace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/1114405708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 0x5630b5fbace0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4e58018000 tx=0x7f4e5800bf10 comp rx=0 tx=0).stop 2024-05-09T09:29:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e6d37e740 1 -- 172.21.15.12:0/1114405708 shutdown_connections 2024-05-09T09:29:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/1114405708 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5630b5fc63e0 0x5630b5fc87f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/1114405708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fb1aa0 0x5630b5fc1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/1114405708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fac930 0x5630b5fbace0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e437fe640 0 client.17019 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:29:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/1114405708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5fba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e6d37e740 1 -- 172.21.15.12:0/1114405708 >> 172.21.15.12:0/1114405708 conn(0x5630b5fa75b0 msgr2=0x5630b5fa67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e6d37e740 1 -- 172.21.15.12:0/1114405708 shutdown_connections 2024-05-09T09:29:30.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e6d37e740 1 -- 172.21.15.12:0/1114405708 wait complete. 2024-05-09T09:29:30.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.385+0000 7f4e6d37e740 20 client.17019 trim_cache size 0 max 16384 2024-05-09T09:29:30.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e6d37e740 1 Processor -- start 2024-05-09T09:29:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e6d37e740 1 -- start start 2024-05-09T09:29:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e6d37e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5faa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e6d37e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fac930 0x5630b5fc1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e6d37e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 0x5630b5faaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e6d37e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4e641a8340 con 0x5630b5fb1aa0 2024-05-09T09:29:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e6d37e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e641aa0c0 con 0x5630b5fac930 2024-05-09T09:29:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e6d37e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4e641a8ad0 con 0x5630b5fabf40 2024-05-09T09:29:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e68b2d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 0x5630b5faaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e68b2d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 0x5630b5faaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49566/0 (socket says 172.21.15.12:49566) 2024-05-09T09:29:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e68b2d640 1 -- 172.21.15.12:0/3158625486 learned_addr learned my addr 172.21.15.12:0/3158625486 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e63fff640 1 --2- 172.21.15.12:0/3158625486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5faa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e637fe640 1 --2- 172.21.15.12:0/3158625486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fac930 0x5630b5fc1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e68b2d640 1 -- 172.21.15.12:0/3158625486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 msgr2=0x5630b5faa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e68b2d640 1 --2- 172.21.15.12:0/3158625486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5faa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e68b2d640 1 -- 172.21.15.12:0/3158625486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fac930 msgr2=0x5630b5fc1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e68b2d640 1 --2- 172.21.15.12:0/3158625486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fac930 0x5630b5fc1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e68b2d640 1 -- 172.21.15.12:0/3158625486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b5fa7150 con 0x5630b5fb1aa0 2024-05-09T09:29:30.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e63fff640 1 --2- 172.21.15.12:0/3158625486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5faa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:29:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e637fe640 1 --2- 172.21.15.12:0/3158625486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fac930 0x5630b5fc1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:29:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.399+0000 7f4e68b2d640 1 --2- 172.21.15.12:0/3158625486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 0x5630b5faaa70 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0cd0f0 tx=0x7f4e5c0077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa876871b5a5e44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.400+0000 7f4e49ffb640 1 -- 172.21.15.12:0/3158625486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c09a150 con 0x5630b5fb1aa0 2024-05-09T09:29:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.400+0000 7f4e49ffb640 1 -- 172.21.15.12:0/3158625486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4e5c09e070 con 0x5630b5fb1aa0 2024-05-09T09:29:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.400+0000 7f4e49ffb640 1 -- 172.21.15.12:0/3158625486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c004d80 con 0x5630b5fb1aa0 2024-05-09T09:29:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.400+0000 7f4e6d37e740 1 -- 172.21.15.12:0/3158625486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 msgr2=0x5630b5faaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.400+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/3158625486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 0x5630b5faaa70 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0cd0f0 tx=0x7f4e5c0077a0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.400+0000 7f4e6d37e740 1 -- 172.21.15.12:0/3158625486 shutdown_connections 2024-05-09T09:29:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.400+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/3158625486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 0x5630b5faaa70 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.400+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/3158625486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fac930 0x5630b5fc1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.400+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/3158625486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5faa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.400+0000 7f4e6d37e740 1 -- 172.21.15.12:0/3158625486 >> 172.21.15.12:0/3158625486 conn(0x5630b5fa75b0 msgr2=0x5630b5fadd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.400+0000 7f4e6d37e740 1 -- 172.21.15.12:0/3158625486 shutdown_connections 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.400+0000 7f4e6d37e740 1 -- 172.21.15.12:0/3158625486 wait complete. 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.400+0000 7f4e6d37e740 1 -- start start 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e6d37e740 1 Processor -- start 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e6d37e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5fbf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e6d37e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fac930 0x5630b5fbf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e6d37e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 0x5630b5fc0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e6d37e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4e641a99f0 con 0x5630b5fb1aa0 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e6d37e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e641a9260 con 0x5630b5fac930 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e6d37e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4e641a6320 con 0x5630b5fabf40 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e6d37e740 10 client.0 _ll_register_callbacks cb 0x7f4e5c018400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e68b2d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 0x5630b5fc0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e68b2d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 0x5630b5fc0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49576/0 (socket says 172.21.15.12:49576) 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e68b2d640 1 -- 172.21.15.12:0/4289879557 learned_addr learned my addr 172.21.15.12:0/4289879557 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e63fff640 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5fbf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e637fe640 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fac930 0x5630b5fbf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e68b2d640 1 -- 172.21.15.12:0/4289879557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 msgr2=0x5630b5fbf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e68b2d640 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5fbf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e68b2d640 1 -- 172.21.15.12:0/4289879557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fac930 msgr2=0x5630b5fbf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e68b2d640 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fac930 0x5630b5fbf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e68b2d640 1 -- 172.21.15.12:0/4289879557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b5fc0d80 con 0x5630b5fb1aa0 2024-05-09T09:29:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.401+0000 7f4e637fe640 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fac930 0x5630b5fbf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:29:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e68b2d640 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 0x5630b5fc0790 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0cd460 tx=0x7f4e5c00f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a1ffe832cb6cb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e437fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:29:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e437fe640 1 -- 172.21.15.12:0/4289879557 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c09b4a0 con 0x5630b5fb1aa0 2024-05-09T09:29:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e437fe640 1 -- 172.21.15.12:0/4289879557 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4e5c004b20 con 0x5630b5fb1aa0 2024-05-09T09:29:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e6d37e740 1 -- 172.21.15.12:0/4289879557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e5c09e070 con 0x5630b5fb1aa0 2024-05-09T09:29:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e437fe640 1 -- 172.21.15.12:0/4289879557 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4e5c004d40 con 0x5630b5fb1aa0 2024-05-09T09:29:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e6d37e740 10 client.17025 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e6d37e740 1 -- 172.21.15.12:0/4289879557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5630b5fc2060 con 0x5630b5fb1aa0 2024-05-09T09:29:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e437fe640 1 -- 172.21.15.12:0/4289879557 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4e5c09e070 con 0x5630b5fb1aa0 2024-05-09T09:29:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e437fe640 1 -- 172.21.15.12:0/4289879557 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f4e5c0baa90 con 0x5630b5fb1aa0 2024-05-09T09:29:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e6d37e740 10 client.17025 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e6d37e740 10 client.17025 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-05-09T09:29:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e6d37e740 20 client.17025 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5630b602d1a0 0x5630b602f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e6d37e740 4 client.17025 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:29:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e6d37e740 1 -- 172.21.15.12:0/4289879557 --> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630b5f186c0 con 0x5630b602d1a0 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.402+0000 7f4e63fff640 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5630b602d1a0 0x5630b602f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e437fe640 10 client.17025 ms_handle_connect on v2:172.21.15.12:6840/1434721532 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e63fff640 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5630b602d1a0 0x5630b602f5c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9e76f390ad080901 server_cookie=e94e60668c021fd in_seq=0 out_seq=0 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e437fe640 1 -- 172.21.15.12:0/4289879557 <== mds.4 v2:172.21.15.12:6840/1434721532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630b5f186c0 con 0x5630b602d1a0 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e437fe640 10 client.17025 handle_command_reply: tid=0 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e6d37e740 1 client.17025 shutdown 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e6d37e740 1 -- 172.21.15.12:0/4289879557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 msgr2=0x5630b5fc0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 0x5630b5fc0790 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0cd460 tx=0x7f4e5c00f9b0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e6d37e740 1 -- 172.21.15.12:0/4289879557 shutdown_connections 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.12:6840/1434721532,v1:172.21.15.12:6841/1434721532] conn(0x5630b602d1a0 0x5630b602f5c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630b5fb1aa0 0x5630b5fc0790 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630b5fac930 0x5630b5fbf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e437fe640 0 client.17025 ms_handle_reset on v2:172.21.15.12:6840/1434721532 2024-05-09T09:29:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e6d37e740 1 --2- 172.21.15.12:0/4289879557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b5fabf40 0x5630b5fbf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e6d37e740 1 -- 172.21.15.12:0/4289879557 >> 172.21.15.12:0/4289879557 conn(0x5630b5fa75b0 msgr2=0x5630b5fa5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e6d37e740 1 -- 172.21.15.12:0/4289879557 shutdown_connections 2024-05-09T09:29:30.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e6d37e740 1 -- 172.21.15.12:0/4289879557 wait complete. 2024-05-09T09:29:30.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.403+0000 7f4e6d37e740 20 client.17025 trim_cache size 0 max 16384 2024-05-09T09:29:30.412 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:29:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.404+0000 7f4e6a590640 1 -- 172.21.15.12:0/3752894535 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4e4c0a2c10 msgr2=0x7f4e4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.404+0000 7f4e6a590640 1 --2- 172.21.15.12:0/3752894535 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4e4c0a2c10 0x7f4e4c0a50d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4e54006010 tx=0x7f4e54031530 comp rx=0 tx=0).stop 2024-05-09T09:29:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.404+0000 7f4e6a590640 1 -- 172.21.15.12:0/3752894535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e64107630 msgr2=0x7f4e6419f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.404+0000 7f4e6a590640 1 --2- 172.21.15.12:0/3752894535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e64107630 0x7f4e6419f160 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00cce0 tx=0x7f4e5c00a620 comp rx=0 tx=0).stop 2024-05-09T09:29:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.404+0000 7f4e6a590640 1 -- 172.21.15.12:0/3752894535 shutdown_connections 2024-05-09T09:29:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.404+0000 7f4e6a590640 1 --2- 172.21.15.12:0/3752894535 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f4e4c0a2c10 0x7f4e4c0a50d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.404+0000 7f4e6a590640 1 --2- 172.21.15.12:0/3752894535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e64107630 0x7f4e6419f160 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.404+0000 7f4e6a590640 1 --2- 172.21.15.12:0/3752894535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e64104d00 0x7f4e641980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.404+0000 7f4e6a590640 1 --2- 172.21.15.12:0/3752894535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e641023d0 0x7f4e64197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.404+0000 7f4e6a590640 1 -- 172.21.15.12:0/3752894535 >> 172.21.15.12:0/3752894535 conn(0x7f4e640fbd20 msgr2=0x7f4e64102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.404+0000 7f4e6a590640 1 -- 172.21.15.12:0/3752894535 shutdown_connections 2024-05-09T09:29:30.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.404+0000 7f4e6a590640 1 -- 172.21.15.12:0/3752894535 wait complete. 2024-05-09T09:29:30.415 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-09T09:29:30.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:30 smithi170 ceph-mon[33632]: pgmap v486: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-09T09:29:30.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:30 smithi016 ceph-mon[33335]: pgmap v486: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-09T09:29:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.523+0000 7f8d188d4640 1 -- 172.21.15.12:0/853505425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d14102310 msgr2=0x7f8d14104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.523+0000 7f8d188d4640 1 --2- 172.21.15.12:0/853505425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d14102310 0x7f8d14104700 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009900 tx=0x7f8d0402a410 comp rx=0 tx=0).stop 2024-05-09T09:29:30.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.523+0000 7f8d188d4640 1 -- 172.21.15.12:0/853505425 shutdown_connections 2024-05-09T09:29:30.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.523+0000 7f8d188d4640 1 --2- 172.21.15.12:0/853505425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d14107570 0x7f8d141099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.523+0000 7f8d188d4640 1 --2- 172.21.15.12:0/853505425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d14104c40 0x7f8d14107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.523+0000 7f8d188d4640 1 --2- 172.21.15.12:0/853505425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d14102310 0x7f8d14104700 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.523+0000 7f8d188d4640 1 -- 172.21.15.12:0/853505425 >> 172.21.15.12:0/853505425 conn(0x7f8d140fbd20 msgr2=0x7f8d140fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.523+0000 7f8d188d4640 1 -- 172.21.15.12:0/853505425 shutdown_connections 2024-05-09T09:29:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.523+0000 7f8d188d4640 1 -- 172.21.15.12:0/853505425 wait complete. 2024-05-09T09:29:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.524+0000 7f8d188d4640 1 Processor -- start 2024-05-09T09:29:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.524+0000 7f8d188d4640 1 -- start start 2024-05-09T09:29:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.524+0000 7f8d188d4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d14102310 0x7f8d14199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.524+0000 7f8d188d4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d14104c40 0x7f8d1419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.524+0000 7f8d188d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d14107570 0x7f8d141a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.524+0000 7f8d188d4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1410d510 con 0x7f8d14104c40 2024-05-09T09:29:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.524+0000 7f8d188d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1410d390 con 0x7f8d14107570 2024-05-09T09:29:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.524+0000 7f8d188d4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1410d690 con 0x7f8d14102310 2024-05-09T09:29:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.524+0000 7f8d11d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d14104c40 0x7f8d1419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.524+0000 7f8d11d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d14104c40 0x7f8d1419a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49590/0 (socket says 172.21.15.12:49590) 2024-05-09T09:29:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.524+0000 7f8d11d9b640 1 -- 172.21.15.12:0/2354244547 learned_addr learned my addr 172.21.15.12:0/2354244547 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.524+0000 7f8d1259c640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d14102310 0x7f8d14199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.524+0000 7f8d12d9d640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d14107570 0x7f8d141a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.525+0000 7f8d11d9b640 1 -- 172.21.15.12:0/2354244547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d14102310 msgr2=0x7f8d14199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.525+0000 7f8d11d9b640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d14102310 0x7f8d14199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.525+0000 7f8d11d9b640 1 -- 172.21.15.12:0/2354244547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d14107570 msgr2=0x7f8d141a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.525+0000 7f8d11d9b640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d14107570 0x7f8d141a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.525+0000 7f8d11d9b640 1 -- 172.21.15.12:0/2354244547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d141a1a20 con 0x7f8d14104c40 2024-05-09T09:29:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.525+0000 7f8d12d9d640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d14107570 0x7f8d141a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:29:30.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.525+0000 7f8d11d9b640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d14104c40 0x7f8d1419a340 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800c970 tx=0x7f8d0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.525+0000 7f8d037fe640 1 -- 172.21.15.12:0/2354244547 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08007bb0 con 0x7f8d14104c40 2024-05-09T09:29:30.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.525+0000 7f8d037fe640 1 -- 172.21.15.12:0/2354244547 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8d08007d50 con 0x7f8d14104c40 2024-05-09T09:29:30.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.525+0000 7f8d037fe640 1 -- 172.21.15.12:0/2354244547 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0800f450 con 0x7f8d14104c40 2024-05-09T09:29:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.525+0000 7f8d188d4640 1 -- 172.21.15.12:0/2354244547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d141a1d10 con 0x7f8d14104c40 2024-05-09T09:29:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.525+0000 7f8d188d4640 1 -- 172.21.15.12:0/2354244547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d141a2200 con 0x7f8d14104c40 2024-05-09T09:29:30.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.527+0000 7f8d037fe640 1 -- 172.21.15.12:0/2354244547 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8d08004f50 con 0x7f8d14104c40 2024-05-09T09:29:30.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.527+0000 7f8d1b5e6740 1 Processor -- start 2024-05-09T09:29:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1b5e6740 1 -- start start 2024-05-09T09:29:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1b5e6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953548f40 0x559953549360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1b5e6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x55995354e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1b5e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 0x559953550f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1b5e6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5599534adc40 con 0x559953549930 2024-05-09T09:29:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1b5e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559953455580 con 0x55995354eaa0 2024-05-09T09:29:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1b5e6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5599534a9c00 con 0x559953548f40 2024-05-09T09:29:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x55995354e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x55995354e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49600/0 (socket says 172.21.15.12:49600) 2024-05-09T09:29:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1259c640 1 -- 172.21.15.12:0/558166002 learned_addr learned my addr 172.21.15.12:0/558166002 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d11d9b640 1 --2- 172.21.15.12:0/558166002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 0x559953550f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d12d9d640 1 --2- 172.21.15.12:0/558166002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953548f40 0x559953549360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d037fe640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8cfc0a2b90 0x7f8cfc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1259c640 1 -- 172.21.15.12:0/558166002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953548f40 msgr2=0x559953549360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1259c640 1 --2- 172.21.15.12:0/558166002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953548f40 0x559953549360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1259c640 1 -- 172.21.15.12:0/558166002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 msgr2=0x559953550f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1259c640 1 --2- 172.21.15.12:0/558166002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 0x559953550f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d1259c640 1 -- 172.21.15.12:0/558166002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599531c13f0 con 0x559953549930 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d12d9d640 1 --2- 172.21.15.12:0/558166002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953548f40 0x559953549360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.528+0000 7f8d12d9d640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8cfc0a2b90 0x7f8cfc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.529+0000 7f8d037fe640 1 -- 172.21.15.12:0/2354244547 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8d080cf400 con 0x7f8d14104c40 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.529+0000 7f8d1259c640 1 --2- 172.21.15.12:0/558166002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x55995354e430 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009900 tx=0x7f8d04031cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d52ce1ed97f1a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.529+0000 7f8d00ff9640 1 -- 172.21.15.12:0/558166002 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04040070 con 0x559953549930 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.529+0000 7f8d00ff9640 1 -- 172.21.15.12:0/558166002 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8d04031810 con 0x559953549930 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.529+0000 7f8d00ff9640 1 -- 172.21.15.12:0/558166002 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04035040 con 0x559953549930 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.529+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/558166002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 msgr2=0x55995354e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.529+0000 7f8d12d9d640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8cfc0a2b90 0x7f8cfc0a5050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55995354a190 tx=0x7f8d0c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.529+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/558166002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x55995354e430 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009900 tx=0x7f8d04031cb0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/558166002 shutdown_connections 2024-05-09T09:29:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/558166002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 0x559953550f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/558166002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x55995354e430 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/558166002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953548f40 0x559953549360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/558166002 >> 172.21.15.12:0/558166002 conn(0x5599535445b0 msgr2=0x5599535449b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/558166002 shutdown_connections 2024-05-09T09:29:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/558166002 wait complete. 2024-05-09T09:29:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 -- start start 2024-05-09T09:29:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 Processor -- start 2024-05-09T09:29:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953548f40 0x5599535532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953549930 0x559953553820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 0x55995355a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5599535519f0 con 0x559953548f40 2024-05-09T09:29:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559953468c40 con 0x55995354eaa0 2024-05-09T09:29:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559953551b70 con 0x559953549930 2024-05-09T09:29:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1b5e6740 10 client.0 _ll_register_callbacks cb 0x5599531c0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953548f40 0x5599535532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953548f40 0x5599535532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49612/0 (socket says 172.21.15.12:49612) 2024-05-09T09:29:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.530+0000 7f8d1259c640 1 -- 172.21.15.12:0/2054399635 learned_addr learned my addr 172.21.15.12:0/2054399635 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8d12d9d640 1 --2- 172.21.15.12:0/2054399635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 0x55995355a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8d11d9b640 1 --2- 172.21.15.12:0/2054399635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953549930 0x559953553820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8d1259c640 1 -- 172.21.15.12:0/2054399635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953549930 msgr2=0x559953553820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8d1259c640 1 --2- 172.21.15.12:0/2054399635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953549930 0x559953553820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8d1259c640 1 -- 172.21.15.12:0/2054399635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 msgr2=0x55995355a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8d1259c640 1 --2- 172.21.15.12:0/2054399635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 0x55995355a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8d1259c640 1 -- 172.21.15.12:0/2054399635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55995328b7d0 con 0x559953548f40 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8d1259c640 1 --2- 172.21.15.12:0/2054399635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953548f40 0x5599535532e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8d040096c0 tx=0x7f8d0402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c447d01bfc43d5eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8cf9ffb640 1 -- 172.21.15.12:0/2054399635 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0404c030 con 0x559953548f40 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8cf9ffb640 1 -- 172.21.15.12:0/2054399635 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8d04031610 con 0x559953548f40 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2054399635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d04031810 con 0x559953548f40 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8cf9ffb640 1 -- 172.21.15.12:0/2054399635 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8d04059050 con 0x559953548f40 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8d1b5e6740 10 client.17037 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2054399635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55995355db40 con 0x559953548f40 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.531+0000 7f8cf9ffb640 1 -- 172.21.15.12:0/2054399635 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8d04031810 con 0x559953548f40 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.532+0000 7f8cf9ffb640 1 -- 172.21.15.12:0/2054399635 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f8d04044a00 con 0x559953548f40 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.532+0000 7f8d1b5e6740 10 client.17037 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.532+0000 7f8d1b5e6740 10 client.17037 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:29:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.532+0000 7f8d1b5e6740 20 client.17037 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:30.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.532+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/2054399635 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55995355ee90 0x559953561280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.532+0000 7f8d1b5e6740 4 client.17037 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:29:30.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.532+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2054399635 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599534bb370 con 0x55995355ee90 2024-05-09T09:29:30.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.532+0000 7f8d11d9b640 1 --2- 172.21.15.12:0/2054399635 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55995355ee90 0x559953561280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.532+0000 7f8d11d9b640 1 --2- 172.21.15.12:0/2054399635 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55995355ee90 0x559953561280 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46267f29fa348eb2 server_cookie=117050da03d7408e in_seq=0 out_seq=0 2024-05-09T09:29:30.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.532+0000 7f8cf9ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:29:30.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.533+0000 7f8cf9ffb640 1 -- 172.21.15.12:0/2054399635 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599534bb370 con 0x55995355ee90 2024-05-09T09:29:30.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.533+0000 7f8cf9ffb640 10 client.17037 handle_command_reply: tid=0 2024-05-09T09:29:30.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.533+0000 7f8d1b5e6740 1 client.17037 shutdown 2024-05-09T09:29:30.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.533+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2054399635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953548f40 msgr2=0x5599535532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.533+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/2054399635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953548f40 0x5599535532e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8d040096c0 tx=0x7f8d0402a410 comp rx=0 tx=0).stop 2024-05-09T09:29:30.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.533+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2054399635 shutdown_connections 2024-05-09T09:29:30.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.533+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/2054399635 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x55995355ee90 0x559953561280 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.533+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/2054399635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 0x55995355a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.533+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/2054399635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953549930 0x559953553820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.533+0000 7f8cf9ffb640 0 client.17037 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:29:30.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.533+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/2054399635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953548f40 0x5599535532e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.533+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2054399635 >> 172.21.15.12:0/2054399635 conn(0x5599535445b0 msgr2=0x5599535437b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.534+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2054399635 shutdown_connections 2024-05-09T09:29:30.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.534+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2054399635 wait complete. 2024-05-09T09:29:30.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.534+0000 7f8d1b5e6740 20 client.17037 trim_cache size 0 max 16384 2024-05-09T09:29:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d1b5e6740 1 Processor -- start 2024-05-09T09:29:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d1b5e6740 1 -- start start 2024-05-09T09:29:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d1b5e6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953548f40 0x559953543630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d1b5e6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x559953543b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d1b5e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 0x55995355a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d1b5e6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ac7d0 con 0x559953549930 2024-05-09T09:29:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d1b5e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ae550 con 0x55995354eaa0 2024-05-09T09:29:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d1b5e6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141acf60 con 0x559953548f40 2024-05-09T09:29:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d11d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x559953543b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d11d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x559953543b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49620/0 (socket says 172.21.15.12:49620) 2024-05-09T09:29:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d11d9b640 1 -- 172.21.15.12:0/3838527873 learned_addr learned my addr 172.21.15.12:0/3838527873 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d11d9b640 1 -- 172.21.15.12:0/3838527873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953548f40 msgr2=0x559953543630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d11d9b640 1 --2- 172.21.15.12:0/3838527873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953548f40 0x559953543630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d11d9b640 1 -- 172.21.15.12:0/3838527873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 msgr2=0x55995355a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d12d9d640 1 --2- 172.21.15.12:0/3838527873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 0x55995355a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.547+0000 7f8d11d9b640 1 --2- 172.21.15.12:0/3838527873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 0x55995355a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8d11d9b640 1 -- 172.21.15.12:0/3838527873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559953545320 con 0x559953549930 2024-05-09T09:29:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8d12d9d640 1 --2- 172.21.15.12:0/3838527873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 0x55995355a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:29:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8d11d9b640 1 --2- 172.21.15.12:0/3838527873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x559953543b70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8d08097cf0 tx=0x7f8d08086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c2ffe361511e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8cfbfff640 1 -- 172.21.15.12:0/3838527873 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0801d040 con 0x559953549930 2024-05-09T09:29:30.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8cfbfff640 1 -- 172.21.15.12:0/3838527873 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8d0809bf90 con 0x559953549930 2024-05-09T09:29:30.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8cfbfff640 1 -- 172.21.15.12:0/3838527873 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d080046c0 con 0x559953549930 2024-05-09T09:29:30.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/3838527873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 msgr2=0x559953543b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/3838527873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x559953543b70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8d08097cf0 tx=0x7f8d08086040 comp rx=0 tx=0).stop 2024-05-09T09:29:30.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/3838527873 shutdown_connections 2024-05-09T09:29:30.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/3838527873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55995354eaa0 0x55995355a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/3838527873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x559953543b70 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/3838527873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559953548f40 0x559953543630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/3838527873 >> 172.21.15.12:0/3838527873 conn(0x5599535445b0 msgr2=0x559953542f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/3838527873 shutdown_connections 2024-05-09T09:29:30.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.548+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/3838527873 wait complete. 2024-05-09T09:29:30.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d1b5e6740 1 -- start start 2024-05-09T09:29:30.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d1b5e6740 1 Processor -- start 2024-05-09T09:29:30.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d1b5e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559953548f40 0x559953557dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d1b5e6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x559953558300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d1b5e6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55995354eaa0 0x559953559310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d1b5e6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ade80 con 0x559953549930 2024-05-09T09:29:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d1b5e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ad6f0 con 0x559953548f40 2024-05-09T09:29:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d1b5e6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141aa7b0 con 0x55995354eaa0 2024-05-09T09:29:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d1b5e6740 10 client.0 _ll_register_callbacks cb 0x7f8d04031b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d11d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x559953558300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d11d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x559953558300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49626/0 (socket says 172.21.15.12:49626) 2024-05-09T09:29:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d11d9b640 1 -- 172.21.15.12:0/2551077446 learned_addr learned my addr 172.21.15.12:0/2551077446 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d1259c640 1 --2- 172.21.15.12:0/2551077446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559953548f40 0x559953557dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d11d9b640 1 -- 172.21.15.12:0/2551077446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55995354eaa0 msgr2=0x559953559310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d11d9b640 1 --2- 172.21.15.12:0/2551077446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55995354eaa0 0x559953559310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d11d9b640 1 -- 172.21.15.12:0/2551077446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559953548f40 msgr2=0x559953557dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d11d9b640 1 --2- 172.21.15.12:0/2551077446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559953548f40 0x559953557dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d11d9b640 1 -- 172.21.15.12:0/2551077446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559953559900 con 0x559953549930 2024-05-09T09:29:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8d11d9b640 1 --2- 172.21.15.12:0/2551077446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x559953558300 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8d08086010 tx=0x7f8d0809b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396457b6efe4be11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.549+0000 7f8cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:29:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8cf9ffb640 1 -- 172.21.15.12:0/2551077446 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d080cf0d0 con 0x559953549930 2024-05-09T09:29:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8cf9ffb640 1 -- 172.21.15.12:0/2551077446 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8d08097f00 con 0x559953549930 2024-05-09T09:29:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2551077446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d0809bf90 con 0x559953549930 2024-05-09T09:29:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8cf9ffb640 1 -- 172.21.15.12:0/2551077446 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8d0809baf0 con 0x559953549930 2024-05-09T09:29:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8d1b5e6740 10 client.17043 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2551077446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55995355ab70 con 0x559953549930 2024-05-09T09:29:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8cf9ffb640 1 -- 172.21.15.12:0/2551077446 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8d0809bf90 con 0x559953549930 2024-05-09T09:29:30.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8cf9ffb640 1 -- 172.21.15.12:0/2551077446 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f8d0809cbc0 con 0x559953549930 2024-05-09T09:29:30.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8d1b5e6740 10 client.17043 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:30.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8d1b5e6740 10 client.17043 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T09:29:30.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8d1b5e6740 20 client.17043 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:30.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/2551077446 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x5599535c59c0 0x5599535c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8d1b5e6740 4 client.17043 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:29:30.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2551077446 --> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599534b56c0 con 0x5599535c59c0 2024-05-09T09:29:30.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.550+0000 7f8d1259c640 1 --2- 172.21.15.12:0/2551077446 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x5599535c59c0 0x5599535c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8d1259c640 1 --2- 172.21.15.12:0/2551077446 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x5599535c59c0 0x5599535c7de0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc9c1bc4a75948ba server_cookie=87e619f607ceba0d in_seq=0 out_seq=0 2024-05-09T09:29:30.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8cf9ffb640 10 client.17043 ms_handle_connect on v2:172.21.15.170:6834/277363225 2024-05-09T09:29:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8cf9ffb640 1 -- 172.21.15.12:0/2551077446 <== mds.3 v2:172.21.15.170:6834/277363225 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599534b56c0 con 0x5599535c59c0 2024-05-09T09:29:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8cf9ffb640 10 client.17043 handle_command_reply: tid=0 2024-05-09T09:29:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8d1b5e6740 1 client.17043 shutdown 2024-05-09T09:29:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2551077446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 msgr2=0x559953558300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/2551077446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x559953558300 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8d08086010 tx=0x7f8d0809b9d0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2551077446 shutdown_connections 2024-05-09T09:29:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/2551077446 >> [v2:172.21.15.170:6834/277363225,v1:172.21.15.170:6835/277363225] conn(0x5599535c59c0 0x5599535c7de0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/2551077446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55995354eaa0 0x559953559310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/2551077446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559953549930 0x559953558300 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8cf9ffb640 0 client.17043 ms_handle_reset on v2:172.21.15.170:6834/277363225 2024-05-09T09:29:30.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8d1b5e6740 1 --2- 172.21.15.12:0/2551077446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559953548f40 0x559953557dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2551077446 >> 172.21.15.12:0/2551077446 conn(0x5599535445b0 msgr2=0x5599535437b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2551077446 shutdown_connections 2024-05-09T09:29:30.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8d1b5e6740 1 -- 172.21.15.12:0/2551077446 wait complete. 2024-05-09T09:29:30.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.551+0000 7f8d1b5e6740 20 client.17043 trim_cache size 0 max 16384 2024-05-09T09:29:30.563 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:29:30.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.552+0000 7f8d188d4640 1 -- 172.21.15.12:0/2354244547 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8cfc0a2b90 msgr2=0x7f8cfc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.552+0000 7f8d188d4640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8cfc0a2b90 0x7f8cfc0a5050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55995354a190 tx=0x7f8d0c004a40 comp rx=0 tx=0).stop 2024-05-09T09:29:30.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.552+0000 7f8d188d4640 1 -- 172.21.15.12:0/2354244547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d14104c40 msgr2=0x7f8d1419a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.552+0000 7f8d188d4640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d14104c40 0x7f8d1419a340 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800c970 tx=0x7f8d0800ce30 comp rx=0 tx=0).stop 2024-05-09T09:29:30.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.552+0000 7f8d188d4640 1 -- 172.21.15.12:0/2354244547 shutdown_connections 2024-05-09T09:29:30.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.552+0000 7f8d188d4640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8cfc0a2b90 0x7f8cfc0a5050 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.552+0000 7f8d188d4640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d14107570 0x7f8d141a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.552+0000 7f8d188d4640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d14104c40 0x7f8d1419a340 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.552+0000 7f8d188d4640 1 --2- 172.21.15.12:0/2354244547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d14102310 0x7f8d14199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.552+0000 7f8d188d4640 1 -- 172.21.15.12:0/2354244547 >> 172.21.15.12:0/2354244547 conn(0x7f8d140fbd20 msgr2=0x7f8d14101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.552+0000 7f8d188d4640 1 -- 172.21.15.12:0/2354244547 shutdown_connections 2024-05-09T09:29:30.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.552+0000 7f8d188d4640 1 -- 172.21.15.12:0/2354244547 wait complete. 2024-05-09T09:29:30.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:30 smithi012 ceph-mon[30140]: pgmap v486: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-09T09:29:30.567 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:29:30.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.675+0000 7f383d815640 1 -- 172.21.15.12:0/694699794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838103440 msgr2=0x7f3838109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.675+0000 7f383d815640 1 --2- 172.21.15.12:0/694699794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838103440 0x7f3838109cd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f3830009900 tx=0x7f383002a410 comp rx=0 tx=0).stop 2024-05-09T09:29:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.675+0000 7f383d815640 1 -- 172.21.15.12:0/694699794 shutdown_connections 2024-05-09T09:29:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.675+0000 7f383d815640 1 --2- 172.21.15.12:0/694699794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838103440 0x7f3838109cd0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.675+0000 7f383d815640 1 --2- 172.21.15.12:0/694699794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3838102a80 0x7f3838102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.675+0000 7f383d815640 1 --2- 172.21.15.12:0/694699794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3838101890 0x7f3838101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.675+0000 7f383d815640 1 -- 172.21.15.12:0/694699794 >> 172.21.15.12:0/694699794 conn(0x7f38380fce00 msgr2=0x7f38380ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.675+0000 7f383d815640 1 -- 172.21.15.12:0/694699794 shutdown_connections 2024-05-09T09:29:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.675+0000 7f383d815640 1 -- 172.21.15.12:0/694699794 wait complete. 2024-05-09T09:29:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f383d815640 1 Processor -- start 2024-05-09T09:29:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f383d815640 1 -- start start 2024-05-09T09:29:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f383d815640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838101890 0x7f3838197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f383d815640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3838102a80 0x7f38381980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f383d815640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3838103440 0x7f383819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f383d815640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f383810d4f0 con 0x7f3838101890 2024-05-09T09:29:30.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f383d815640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f383810d370 con 0x7f3838102a80 2024-05-09T09:29:30.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f383d815640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f383810d670 con 0x7f3838103440 2024-05-09T09:29:30.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f3836ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838101890 0x7f3838197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f3836ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838101890 0x7f3838197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49652/0 (socket says 172.21.15.12:49652) 2024-05-09T09:29:30.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f3836ffd640 1 -- 172.21.15.12:0/3482019605 learned_addr learned my addr 172.21.15.12:0/3482019605 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f38367fc640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3838102a80 0x7f38381980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f38377fe640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3838103440 0x7f383819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f3836ffd640 1 -- 172.21.15.12:0/3482019605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3838103440 msgr2=0x7f383819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f3836ffd640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3838103440 0x7f383819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f3836ffd640 1 -- 172.21.15.12:0/3482019605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3838102a80 msgr2=0x7f38381980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f3836ffd640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3838102a80 0x7f38381980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.676+0000 7f3836ffd640 1 -- 172.21.15.12:0/3482019605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383819f780 con 0x7f3838101890 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.677+0000 7f3836ffd640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838101890 0x7f3838197b60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f382800ea80 tx=0x7f382800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.677+0000 7f383c813640 1 -- 172.21.15.12:0/3482019605 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382800ce40 con 0x7f3838101890 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.677+0000 7f383c813640 1 -- 172.21.15.12:0/3482019605 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3828011940 con 0x7f3838101890 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.677+0000 7f383c813640 1 -- 172.21.15.12:0/3482019605 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382801ab30 con 0x7f3838101890 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.677+0000 7f383d815640 1 -- 172.21.15.12:0/3482019605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f383819fa70 con 0x7f3838101890 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.677+0000 7f383d815640 1 -- 172.21.15.12:0/3482019605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f383819ffc0 con 0x7f3838101890 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.679+0000 7f383c813640 1 -- 172.21.15.12:0/3482019605 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f382801acd0 con 0x7f3838101890 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38405ff740 1 Processor -- start 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38405ff740 1 -- start start 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38405ff740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 0x55789336a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38405ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 0x55789336f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38405ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336feb0 0x557893372340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38405ff740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5578932cac40 con 0x55789336a350 2024-05-09T09:29:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38405ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557893272580 con 0x55789336ad40 2024-05-09T09:29:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38405ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5578932c6c00 con 0x55789336feb0 2024-05-09T09:29:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38377fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 0x55789336a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38377fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 0x55789336a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49666/0 (socket says 172.21.15.12:49666) 2024-05-09T09:29:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38377fe640 1 -- 172.21.15.12:0/56982747 learned_addr learned my addr 172.21.15.12:0/56982747 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38367fc640 1 --2- 172.21.15.12:0/56982747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336feb0 0x557893372340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f383c813640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f38200a2b90 0x7f38200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f3836ffd640 1 --2- 172.21.15.12:0/56982747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 0x55789336f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38377fe640 1 -- 172.21.15.12:0/56982747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336feb0 msgr2=0x557893372340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38377fe640 1 --2- 172.21.15.12:0/56982747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336feb0 0x557893372340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38377fe640 1 -- 172.21.15.12:0/56982747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 msgr2=0x55789336f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38377fe640 1 --2- 172.21.15.12:0/56982747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 0x55789336f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38377fe640 1 -- 172.21.15.12:0/56982747 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557892fde3f0 con 0x55789336a350 2024-05-09T09:29:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f38377fe640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f38200a2b90 0x7f38200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.680+0000 7f383c813640 1 -- 172.21.15.12:0/3482019605 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f38280cf9d0 con 0x7f3838101890 2024-05-09T09:29:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.681+0000 7f38377fe640 1 --2- 172.21.15.12:0/56982747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 0x55789336a770 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3830033000 tx=0x7f3830032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c8692259bc6a10 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.681+0000 7f381dffb640 1 -- 172.21.15.12:0/56982747 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383004c070 con 0x55789336a350 2024-05-09T09:29:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.681+0000 7f381dffb640 1 -- 172.21.15.12:0/56982747 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3830005400 con 0x55789336a350 2024-05-09T09:29:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.681+0000 7f381dffb640 1 -- 172.21.15.12:0/56982747 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830035070 con 0x55789336a350 2024-05-09T09:29:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.681+0000 7f38405ff740 1 -- 172.21.15.12:0/56982747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 msgr2=0x55789336a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.681+0000 7f38377fe640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f38200a2b90 0x7f38200a5050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3830005ab0 tx=0x7f383004a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.681+0000 7f38405ff740 1 --2- 172.21.15.12:0/56982747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 0x55789336a770 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3830033000 tx=0x7f3830032d70 comp rx=0 tx=0).stop 2024-05-09T09:29:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 -- 172.21.15.12:0/56982747 shutdown_connections 2024-05-09T09:29:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 --2- 172.21.15.12:0/56982747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336feb0 0x557893372340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 --2- 172.21.15.12:0/56982747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 0x55789336f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 --2- 172.21.15.12:0/56982747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 0x55789336a770 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 -- 172.21.15.12:0/56982747 >> 172.21.15.12:0/56982747 conn(0x5578933615b0 msgr2=0x5578933619b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 -- 172.21.15.12:0/56982747 shutdown_connections 2024-05-09T09:29:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 -- 172.21.15.12:0/56982747 wait complete. 2024-05-09T09:29:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 -- start start 2024-05-09T09:29:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 Processor -- start 2024-05-09T09:29:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336a350 0x5578933746f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336ad40 0x557893374c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336feb0 0x55789337bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557893372e60 con 0x55789336ad40 2024-05-09T09:29:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557893285c40 con 0x55789336feb0 2024-05-09T09:29:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557893372fe0 con 0x55789336a350 2024-05-09T09:29:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.682+0000 7f38405ff740 10 client.0 _ll_register_callbacks cb 0x557892fddbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38367fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336ad40 0x557893374c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38367fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336ad40 0x557893374c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49672/0 (socket says 172.21.15.12:49672) 2024-05-09T09:29:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38367fc640 1 -- 172.21.15.12:0/324028962 learned_addr learned my addr 172.21.15.12:0/324028962 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f3836ffd640 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336a350 0x5578933746f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38377fe640 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336feb0 0x55789337bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38367fc640 1 -- 172.21.15.12:0/324028962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336a350 msgr2=0x5578933746f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38367fc640 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336a350 0x5578933746f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38367fc640 1 -- 172.21.15.12:0/324028962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336feb0 msgr2=0x55789337bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38367fc640 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336feb0 0x55789337bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38367fc640 1 -- 172.21.15.12:0/324028962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578930a87d0 con 0x55789336ad40 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38377fe640 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336feb0 0x55789337bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38367fc640 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336ad40 0x557893374c30 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f382c00b8f0 tx=0x7f382c00bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa63c1786cda694e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f3816ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f3816ffd640 1 -- 172.21.15.12:0/324028962 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c003d00 con 0x55789336ad40 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f3816ffd640 1 -- 172.21.15.12:0/324028962 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f382c005020 con 0x55789336ad40 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38405ff740 1 -- 172.21.15.12:0/324028962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3830005400 con 0x55789336ad40 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f3816ffd640 1 -- 172.21.15.12:0/324028962 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f382c003ea0 con 0x55789336ad40 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38405ff740 10 client.17055 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.683+0000 7f38405ff740 1 -- 172.21.15.12:0/324028962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55789337f100 con 0x55789336ad40 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.684+0000 7f3816ffd640 1 -- 172.21.15.12:0/324028962 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3830005400 con 0x55789336ad40 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.684+0000 7f3816ffd640 1 -- 172.21.15.12:0/324028962 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f382c01b500 con 0x55789336ad40 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.684+0000 7f38405ff740 10 client.17055 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.684+0000 7f38405ff740 10 client.17055 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.684+0000 7f38405ff740 20 client.17055 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.684+0000 7f38405ff740 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5578933808d0 0x557893382d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.684+0000 7f38405ff740 4 client.17055 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.684+0000 7f38405ff740 1 -- 172.21.15.12:0/324028962 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578932d8370 con 0x5578933808d0 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.684+0000 7f3836ffd640 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5578933808d0 0x557893382d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.684+0000 7f3836ffd640 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5578933808d0 0x557893382d50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5387ae8f938d4042 server_cookie=327728230ffc3a2e in_seq=0 out_seq=0 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.684+0000 7f3816ffd640 10 client.17055 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.685+0000 7f3816ffd640 1 -- 172.21.15.12:0/324028962 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578932d8370 con 0x5578933808d0 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.685+0000 7f3816ffd640 10 client.17055 handle_command_reply: tid=0 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.685+0000 7f38405ff740 1 client.17055 shutdown 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.685+0000 7f38405ff740 1 -- 172.21.15.12:0/324028962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336ad40 msgr2=0x557893374c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.685+0000 7f38405ff740 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336ad40 0x557893374c30 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f382c00b8f0 tx=0x7f382c00bcf0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.685+0000 7f38405ff740 1 -- 172.21.15.12:0/324028962 shutdown_connections 2024-05-09T09:29:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.685+0000 7f38405ff740 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5578933808d0 0x557893382d50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.685+0000 7f38405ff740 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336feb0 0x55789337bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.685+0000 7f38405ff740 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336ad40 0x557893374c30 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.685+0000 7f38405ff740 1 --2- 172.21.15.12:0/324028962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336a350 0x5578933746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.685+0000 7f3816ffd640 0 client.17055 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:29:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.685+0000 7f38405ff740 1 -- 172.21.15.12:0/324028962 >> 172.21.15.12:0/324028962 conn(0x5578933615b0 msgr2=0x5578933607b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.686+0000 7f38405ff740 1 -- 172.21.15.12:0/324028962 shutdown_connections 2024-05-09T09:29:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.686+0000 7f38405ff740 1 -- 172.21.15.12:0/324028962 wait complete. 2024-05-09T09:29:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.686+0000 7f38405ff740 20 client.17055 trim_cache size 0 max 16384 2024-05-09T09:29:30.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.699+0000 7f38405ff740 1 Processor -- start 2024-05-09T09:29:30.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.699+0000 7f38405ff740 1 -- start start 2024-05-09T09:29:30.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.699+0000 7f38405ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336a350 0x55789336c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.699+0000 7f38405ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 0x55789336c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.699+0000 7f38405ff740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336feb0 0x55789337bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.699+0000 7f38405ff740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f38381a8350 con 0x55789336feb0 2024-05-09T09:29:30.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.699+0000 7f38405ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38381aa0d0 con 0x55789336ad40 2024-05-09T09:29:30.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.699+0000 7f38405ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f38381a8ae0 con 0x55789336a350 2024-05-09T09:29:30.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.699+0000 7f38377fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336feb0 0x55789337bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.699+0000 7f38377fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336feb0 0x55789337bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49684/0 (socket says 172.21.15.12:49684) 2024-05-09T09:29:30.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.699+0000 7f38377fe640 1 -- 172.21.15.12:0/3714436616 learned_addr learned my addr 172.21.15.12:0/3714436616 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.700+0000 7f38377fe640 1 -- 172.21.15.12:0/3714436616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336a350 msgr2=0x55789336c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.700+0000 7f38367fc640 1 --2- 172.21.15.12:0/3714436616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 0x55789336c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.700+0000 7f38377fe640 1 --2- 172.21.15.12:0/3714436616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336a350 0x55789336c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.700+0000 7f38377fe640 1 -- 172.21.15.12:0/3714436616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 msgr2=0x55789336c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.700+0000 7f38377fe640 1 --2- 172.21.15.12:0/3714436616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 0x55789336c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.700+0000 7f38377fe640 1 -- 172.21.15.12:0/3714436616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557893352d20 con 0x55789336feb0 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.700+0000 7f38367fc640 1 --2- 172.21.15.12:0/3714436616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 0x55789336c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.700+0000 7f38377fe640 1 --2- 172.21.15.12:0/3714436616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336feb0 0x55789337bc30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3830033000 tx=0x7f383003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bea885195a8677 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.700+0000 7f381d7fa640 1 -- 172.21.15.12:0/3714436616 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383004c070 con 0x55789336feb0 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.700+0000 7f381d7fa640 1 -- 172.21.15.12:0/3714436616 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f38300465e0 con 0x55789336feb0 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.700+0000 7f381d7fa640 1 -- 172.21.15.12:0/3714436616 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830035040 con 0x55789336feb0 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.701+0000 7f38405ff740 1 -- 172.21.15.12:0/3714436616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336feb0 msgr2=0x55789337bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.701+0000 7f38405ff740 1 --2- 172.21.15.12:0/3714436616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336feb0 0x55789337bc30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3830033000 tx=0x7f383003c000 comp rx=0 tx=0).stop 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.701+0000 7f38405ff740 1 -- 172.21.15.12:0/3714436616 shutdown_connections 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.701+0000 7f38405ff740 1 --2- 172.21.15.12:0/3714436616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336feb0 0x55789337bc30 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.701+0000 7f38405ff740 1 --2- 172.21.15.12:0/3714436616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 0x55789336c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.701+0000 7f38405ff740 1 --2- 172.21.15.12:0/3714436616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336a350 0x55789336c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.701+0000 7f38405ff740 1 -- 172.21.15.12:0/3714436616 >> 172.21.15.12:0/3714436616 conn(0x5578933615b0 msgr2=0x5578933746f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.701+0000 7f38405ff740 1 -- 172.21.15.12:0/3714436616 shutdown_connections 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.701+0000 7f38405ff740 1 -- 172.21.15.12:0/3714436616 wait complete. 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.701+0000 7f38405ff740 1 -- start start 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.701+0000 7f38405ff740 1 Processor -- start 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f38405ff740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 0x557893379310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f38405ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 0x557893379850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f38405ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336feb0 0x55789337a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f38405ff740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f38381a9a00 con 0x55789336a350 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f38405ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38381a9270 con 0x55789336ad40 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f38405ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f38381a6330 con 0x55789336feb0 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f38405ff740 10 client.0 _ll_register_callbacks cb 0x7f382c0041d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f3836ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 0x557893379310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f3836ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 0x557893379310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49692/0 (socket says 172.21.15.12:49692) 2024-05-09T09:29:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f3836ffd640 1 -- 172.21.15.12:0/1013777599 learned_addr learned my addr 172.21.15.12:0/1013777599 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f38367fc640 1 --2- 172.21.15.12:0/1013777599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 0x557893379850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f38377fe640 1 --2- 172.21.15.12:0/1013777599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336feb0 0x55789337a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f3836ffd640 1 -- 172.21.15.12:0/1013777599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336feb0 msgr2=0x55789337a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f3836ffd640 1 --2- 172.21.15.12:0/1013777599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336feb0 0x55789337a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f3836ffd640 1 -- 172.21.15.12:0/1013777599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 msgr2=0x557893379850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f3836ffd640 1 --2- 172.21.15.12:0/1013777599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 0x557893379850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f3836ffd640 1 -- 172.21.15.12:0/1013777599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55789337ae50 con 0x55789336a350 2024-05-09T09:29:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f3836ffd640 1 --2- 172.21.15.12:0/1013777599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 0x557893379310 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f382809bb60 tx=0x7f38280cfbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05b6ff8a4bd3902 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f3816ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:29:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f3816ffd640 1 -- 172.21.15.12:0/1013777599 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38280a4ad0 con 0x55789336a350 2024-05-09T09:29:30.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f3816ffd640 1 -- 172.21.15.12:0/1013777599 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f382809a160 con 0x55789336a350 2024-05-09T09:29:30.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f38405ff740 1 -- 172.21.15.12:0/1013777599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38300465e0 con 0x55789336a350 2024-05-09T09:29:30.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f3816ffd640 1 -- 172.21.15.12:0/1013777599 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f38280a0210 con 0x55789336a350 2024-05-09T09:29:30.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f38405ff740 10 client.17061 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:30.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.702+0000 7f38405ff740 1 -- 172.21.15.12:0/1013777599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5578933e6e90 con 0x55789336a350 2024-05-09T09:29:30.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.703+0000 7f3816ffd640 1 -- 172.21.15.12:0/1013777599 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f38300465e0 con 0x55789336a350 2024-05-09T09:29:30.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.703+0000 7f3816ffd640 1 -- 172.21.15.12:0/1013777599 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f38280a8040 con 0x55789336a350 2024-05-09T09:29:30.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.703+0000 7f38405ff740 10 client.17061 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:30.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.703+0000 7f38405ff740 10 client.17061 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:29:30.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.703+0000 7f38405ff740 20 client.17061 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:30.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.703+0000 7f38405ff740 1 --2- 172.21.15.12:0/1013777599 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5578933e7770 0x5578933e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.703+0000 7f38405ff740 4 client.17061 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:29:30.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.703+0000 7f38405ff740 1 -- 172.21.15.12:0/1013777599 --> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578932d26c0 con 0x5578933e7770 2024-05-09T09:29:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.703+0000 7f38367fc640 1 --2- 172.21.15.12:0/1013777599 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5578933e7770 0x5578933e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.703+0000 7f38367fc640 1 --2- 172.21.15.12:0/1013777599 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5578933e7770 0x5578933e9b90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8478790570c31f84 server_cookie=6dd6ff1d44b34e4b in_seq=0 out_seq=0 2024-05-09T09:29:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.703+0000 7f3816ffd640 10 client.17061 ms_handle_connect on v2:172.21.15.170:6836/2973369023 2024-05-09T09:29:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f3816ffd640 1 -- 172.21.15.12:0/1013777599 <== mds.2 v2:172.21.15.170:6836/2973369023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578932d26c0 con 0x5578933e7770 2024-05-09T09:29:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f3816ffd640 10 client.17061 handle_command_reply: tid=0 2024-05-09T09:29:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f38405ff740 1 client.17061 shutdown 2024-05-09T09:29:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f38405ff740 1 -- 172.21.15.12:0/1013777599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 msgr2=0x557893379310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f38405ff740 1 --2- 172.21.15.12:0/1013777599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 0x557893379310 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f382809bb60 tx=0x7f38280cfbd0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f38405ff740 1 -- 172.21.15.12:0/1013777599 shutdown_connections 2024-05-09T09:29:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f38405ff740 1 --2- 172.21.15.12:0/1013777599 >> [v2:172.21.15.170:6836/2973369023,v1:172.21.15.170:6837/2973369023] conn(0x5578933e7770 0x5578933e9b90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f38405ff740 1 --2- 172.21.15.12:0/1013777599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55789336feb0 0x55789337a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f38405ff740 1 --2- 172.21.15.12:0/1013777599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55789336ad40 0x557893379850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f3816ffd640 0 client.17061 ms_handle_reset on v2:172.21.15.170:6836/2973369023 2024-05-09T09:29:30.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f38405ff740 1 --2- 172.21.15.12:0/1013777599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55789336a350 0x557893379310 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f38405ff740 1 -- 172.21.15.12:0/1013777599 >> 172.21.15.12:0/1013777599 conn(0x5578933615b0 msgr2=0x557893374620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f38405ff740 1 -- 172.21.15.12:0/1013777599 shutdown_connections 2024-05-09T09:29:30.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f38405ff740 1 -- 172.21.15.12:0/1013777599 wait complete. 2024-05-09T09:29:30.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.704+0000 7f38405ff740 20 client.17061 trim_cache size 0 max 16384 2024-05-09T09:29:30.719 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:29:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.705+0000 7f383d815640 1 -- 172.21.15.12:0/3482019605 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f38200a2b90 msgr2=0x7f38200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.705+0000 7f383d815640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f38200a2b90 0x7f38200a5050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3830005ab0 tx=0x7f383004a000 comp rx=0 tx=0).stop 2024-05-09T09:29:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.705+0000 7f383d815640 1 -- 172.21.15.12:0/3482019605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838101890 msgr2=0x7f3838197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.705+0000 7f383d815640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838101890 0x7f3838197b60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f382800ea80 tx=0x7f382800ef40 comp rx=0 tx=0).stop 2024-05-09T09:29:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.705+0000 7f383d815640 1 -- 172.21.15.12:0/3482019605 shutdown_connections 2024-05-09T09:29:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.705+0000 7f383d815640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f38200a2b90 0x7f38200a5050 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.705+0000 7f383d815640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3838103440 0x7f383819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.705+0000 7f383d815640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3838102a80 0x7f38381980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.705+0000 7f383d815640 1 --2- 172.21.15.12:0/3482019605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838101890 0x7f3838197b60 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.705+0000 7f383d815640 1 -- 172.21.15.12:0/3482019605 >> 172.21.15.12:0/3482019605 conn(0x7f38380fce00 msgr2=0x7f3838105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.705+0000 7f383d815640 1 -- 172.21.15.12:0/3482019605 shutdown_connections 2024-05-09T09:29:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.705+0000 7f383d815640 1 -- 172.21.15.12:0/3482019605 wait complete. 2024-05-09T09:29:30.722 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:29:30.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.830+0000 7ff32f264640 1 -- 172.21.15.12:0/252489302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff328102370 msgr2=0x7ff328104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.830+0000 7ff32f264640 1 --2- 172.21.15.12:0/252489302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff328102370 0x7ff328104760 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009900 tx=0x7ff31c02a410 comp rx=0 tx=0).stop 2024-05-09T09:29:30.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.830+0000 7ff32f264640 1 -- 172.21.15.12:0/252489302 shutdown_connections 2024-05-09T09:29:30.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.830+0000 7ff32f264640 1 --2- 172.21.15.12:0/252489302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3281075d0 0x7ff3281099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.830+0000 7ff32f264640 1 --2- 172.21.15.12:0/252489302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff328104ca0 0x7ff328107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.830+0000 7ff32f264640 1 --2- 172.21.15.12:0/252489302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff328102370 0x7ff328104760 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.830+0000 7ff32f264640 1 -- 172.21.15.12:0/252489302 >> 172.21.15.12:0/252489302 conn(0x7ff3280fbd20 msgr2=0x7ff3280fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.830+0000 7ff32f264640 1 -- 172.21.15.12:0/252489302 shutdown_connections 2024-05-09T09:29:30.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.830+0000 7ff32f264640 1 -- 172.21.15.12:0/252489302 wait complete. 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.830+0000 7ff32f264640 1 Processor -- start 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32f264640 1 -- start start 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32f264640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff328102370 0x7ff328197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32f264640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff328104ca0 0x7ff3281980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32f264640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3281075d0 0x7ff32819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32f264640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff32810d510 con 0x7ff3281075d0 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32f264640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff32810d390 con 0x7ff328104ca0 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32f264640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff32810d690 con 0x7ff328102370 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32d801640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3281075d0 0x7ff32819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32d801640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3281075d0 0x7ff32819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49714/0 (socket says 172.21.15.12:49714) 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32d801640 1 -- 172.21.15.12:0/1214702350 learned_addr learned my addr 172.21.15.12:0/1214702350 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff327fff640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff328104ca0 0x7ff3281980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32d000640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff328102370 0x7ff328197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32d801640 1 -- 172.21.15.12:0/1214702350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff328102370 msgr2=0x7ff328197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32d801640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff328102370 0x7ff328197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32d801640 1 -- 172.21.15.12:0/1214702350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff328104ca0 msgr2=0x7ff3281980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32d801640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff328104ca0 0x7ff3281980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32d801640 1 -- 172.21.15.12:0/1214702350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32819f780 con 0x7ff3281075d0 2024-05-09T09:29:30.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.831+0000 7ff32d801640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3281075d0 0x7ff32819f110 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff32000eaa0 tx=0x7ff32000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.832+0000 7ff325ffb640 1 -- 172.21.15.12:0/1214702350 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320004d20 con 0x7ff3281075d0 2024-05-09T09:29:30.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.832+0000 7ff325ffb640 1 -- 172.21.15.12:0/1214702350 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff320005020 con 0x7ff3281075d0 2024-05-09T09:29:30.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.832+0000 7ff325ffb640 1 -- 172.21.15.12:0/1214702350 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3200116d0 con 0x7ff3281075d0 2024-05-09T09:29:30.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.832+0000 7ff32f264640 1 -- 172.21.15.12:0/1214702350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32819fa70 con 0x7ff3281075d0 2024-05-09T09:29:30.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.832+0000 7ff32f264640 1 -- 172.21.15.12:0/1214702350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff32819ffc0 con 0x7ff3281075d0 2024-05-09T09:29:30.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.835+0000 7ff325ffb640 1 -- 172.21.15.12:0/1214702350 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff3200051d0 con 0x7ff3281075d0 2024-05-09T09:29:30.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.835+0000 7ff331f75740 1 Processor -- start 2024-05-09T09:29:30.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.835+0000 7ff331f75740 1 -- start start 2024-05-09T09:29:30.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.835+0000 7ff331f75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 0x555bca6a4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.835+0000 7ff331f75740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a4930 0x555bca6a9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.835+0000 7ff331f75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 0x555bca6abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.835+0000 7ff331f75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555bca608c40 con 0x555bca6a3f40 2024-05-09T09:29:30.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.835+0000 7ff331f75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555bca5b0580 con 0x555bca6a9aa0 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.835+0000 7ff331f75740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555bca604c00 con 0x555bca6a4930 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff327fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 0x555bca6a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff327fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 0x555bca6a4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49724/0 (socket says 172.21.15.12:49724) 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff327fff640 1 -- 172.21.15.12:0/2789939552 learned_addr learned my addr 172.21.15.12:0/2789939552 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff32d000640 1 --2- 172.21.15.12:0/2789939552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a4930 0x555bca6a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff325ffb640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff3180a2b90 0x7ff3180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff327fff640 1 -- 172.21.15.12:0/2789939552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a4930 msgr2=0x555bca6a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff327fff640 1 --2- 172.21.15.12:0/2789939552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a4930 0x555bca6a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff327fff640 1 -- 172.21.15.12:0/2789939552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 msgr2=0x555bca6abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff327fff640 1 --2- 172.21.15.12:0/2789939552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 0x555bca6abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff327fff640 1 -- 172.21.15.12:0/2789939552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bca31c3f0 con 0x555bca6a3f40 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff32d801640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff3180a2b90 0x7ff3180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff325ffb640 1 -- 172.21.15.12:0/1214702350 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff3200cf5d0 con 0x7ff3281075d0 2024-05-09T09:29:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff327fff640 1 --2- 172.21.15.12:0/2789939552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 0x555bca6a4360 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff31400ad00 tx=0x7ff3140117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916597e104186ab9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff312ffd640 1 -- 172.21.15.12:0/2789939552 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314012450 con 0x555bca6a3f40 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff312ffd640 1 -- 172.21.15.12:0/2789939552 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff314012ac0 con 0x555bca6a3f40 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.836+0000 7ff312ffd640 1 -- 172.21.15.12:0/2789939552 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31400fa50 con 0x555bca6a3f40 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.837+0000 7ff331f75740 1 -- 172.21.15.12:0/2789939552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 msgr2=0x555bca6a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.837+0000 7ff32d801640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff3180a2b90 0x7ff3180a5050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff320097e60 tx=0x7ff320084f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.837+0000 7ff331f75740 1 --2- 172.21.15.12:0/2789939552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 0x555bca6a4360 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff31400ad00 tx=0x7ff3140117b0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.837+0000 7ff331f75740 1 -- 172.21.15.12:0/2789939552 shutdown_connections 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.837+0000 7ff331f75740 1 --2- 172.21.15.12:0/2789939552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 0x555bca6abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.837+0000 7ff331f75740 1 --2- 172.21.15.12:0/2789939552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a4930 0x555bca6a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.837+0000 7ff331f75740 1 --2- 172.21.15.12:0/2789939552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 0x555bca6a4360 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.837+0000 7ff331f75740 1 -- 172.21.15.12:0/2789939552 >> 172.21.15.12:0/2789939552 conn(0x555bca69f5b0 msgr2=0x555bca69f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.837+0000 7ff331f75740 1 -- 172.21.15.12:0/2789939552 shutdown_connections 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.837+0000 7ff331f75740 1 -- 172.21.15.12:0/2789939552 wait complete. 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.837+0000 7ff331f75740 1 -- start start 2024-05-09T09:29:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.837+0000 7ff331f75740 1 Processor -- start 2024-05-09T09:29:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff331f75740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a3f40 0x555bca6ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff331f75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 0x555bca6ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff331f75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 0x555bca6b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff331f75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555bca6aca50 con 0x555bca6a4930 2024-05-09T09:29:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff331f75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555bca5c3c40 con 0x555bca6a9aa0 2024-05-09T09:29:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff331f75740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555bca6acbd0 con 0x555bca6a3f40 2024-05-09T09:29:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff331f75740 10 client.0 _ll_register_callbacks cb 0x555bca31bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff32d000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 0x555bca6ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff32d000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 0x555bca6ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49736/0 (socket says 172.21.15.12:49736) 2024-05-09T09:29:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff32d000640 1 -- 172.21.15.12:0/3350063169 learned_addr learned my addr 172.21.15.12:0/3350063169 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff327fff640 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a3f40 0x555bca6ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff327fff640 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 0x555bca6b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff32d000640 1 -- 172.21.15.12:0/3350063169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a3f40 msgr2=0x555bca6ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff32d000640 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a3f40 0x555bca6ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff32d000640 1 -- 172.21.15.12:0/3350063169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 msgr2=0x555bca6b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff32d000640 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 0x555bca6b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff32d000640 1 -- 172.21.15.12:0/3350063169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bca3e67d0 con 0x555bca6a4930 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.838+0000 7ff327fff640 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 0x555bca6b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff32d000640 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 0x555bca6ae880 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009900 tx=0x7ff31c031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538b2720db408ed5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff307fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff307fff640 1 -- 172.21.15.12:0/3350063169 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c035040 con 0x555bca6a4930 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff307fff640 1 -- 172.21.15.12:0/3350063169 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff31c004040 con 0x555bca6a4930 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff331f75740 1 -- 172.21.15.12:0/3350063169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff314012ac0 con 0x555bca6a4930 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff307fff640 1 -- 172.21.15.12:0/3350063169 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff31c004240 con 0x555bca6a4930 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff331f75740 10 client.17073 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff331f75740 1 -- 172.21.15.12:0/3350063169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555bca6b8ca0 con 0x555bca6a4930 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff307fff640 1 -- 172.21.15.12:0/3350063169 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff314012ac0 con 0x555bca6a4930 2024-05-09T09:29:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff307fff640 1 -- 172.21.15.12:0/3350063169 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7ff31c046500 con 0x555bca6a4930 2024-05-09T09:29:30.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff331f75740 10 client.17073 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:30.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff331f75740 10 client.17073 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:29:30.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff331f75740 20 client.17073 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:30.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff331f75740 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555bca6ba470 0x555bca6bc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff331f75740 4 client.17073 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:29:30.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.839+0000 7ff331f75740 1 -- 172.21.15.12:0/3350063169 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bca616370 con 0x555bca6ba470 2024-05-09T09:29:30.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.840+0000 7ff327fff640 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555bca6ba470 0x555bca6bc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.840+0000 7ff307fff640 10 client.17073 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:29:30.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.840+0000 7ff327fff640 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555bca6ba470 0x555bca6bc8b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a3eaa5ba74cbc5b server_cookie=7e55abe6dbbcaf9b in_seq=0 out_seq=0 2024-05-09T09:29:30.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff307fff640 1 -- 172.21.15.12:0/3350063169 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555bca616370 con 0x555bca6ba470 2024-05-09T09:29:30.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff307fff640 10 client.17073 handle_command_reply: tid=0 2024-05-09T09:29:30.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff331f75740 1 client.17073 shutdown 2024-05-09T09:29:30.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff331f75740 1 -- 172.21.15.12:0/3350063169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 msgr2=0x555bca6ae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff331f75740 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 0x555bca6ae880 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009900 tx=0x7ff31c031840 comp rx=0 tx=0).stop 2024-05-09T09:29:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff331f75740 1 -- 172.21.15.12:0/3350063169 shutdown_connections 2024-05-09T09:29:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff331f75740 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555bca6ba470 0x555bca6bc8b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff331f75740 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 0x555bca6b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff331f75740 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 0x555bca6ae880 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff331f75740 1 --2- 172.21.15.12:0/3350063169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a3f40 0x555bca6ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff307fff640 0 client.17073 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:29:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff331f75740 1 -- 172.21.15.12:0/3350063169 >> 172.21.15.12:0/3350063169 conn(0x555bca69f5b0 msgr2=0x555bca69e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff331f75740 1 -- 172.21.15.12:0/3350063169 shutdown_connections 2024-05-09T09:29:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff331f75740 1 -- 172.21.15.12:0/3350063169 wait complete. 2024-05-09T09:29:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.841+0000 7ff331f75740 20 client.17073 trim_cache size 0 max 16384 2024-05-09T09:29:30.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff331f75740 1 Processor -- start 2024-05-09T09:29:30.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff331f75740 1 -- start start 2024-05-09T09:29:30.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff331f75740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a3f40 0x555bca69df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff331f75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 0x555bca6b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff331f75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 0x555bca69e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff331f75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff3281aa580 con 0x555bca6a4930 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff331f75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3281ac300 con 0x555bca6a9aa0 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff331f75740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff3281aad10 con 0x555bca6a3f40 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff32d000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 0x555bca6b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff32d000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 0x555bca6b5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49748/0 (socket says 172.21.15.12:49748) 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff32d000640 1 -- 172.21.15.12:0/4073179046 learned_addr learned my addr 172.21.15.12:0/4073179046 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff327fff640 1 --2- 172.21.15.12:0/4073179046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a3f40 0x555bca69df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff327fff640 1 --2- 172.21.15.12:0/4073179046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 0x555bca69e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff32d000640 1 -- 172.21.15.12:0/4073179046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a3f40 msgr2=0x555bca69df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff32d000640 1 --2- 172.21.15.12:0/4073179046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a3f40 0x555bca69df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff32d000640 1 -- 172.21.15.12:0/4073179046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 msgr2=0x555bca69e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff32d000640 1 --2- 172.21.15.12:0/4073179046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 0x555bca69e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff32d000640 1 -- 172.21.15.12:0/4073179046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bca690d20 con 0x555bca6a4930 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.855+0000 7ff32d000640 1 --2- 172.21.15.12:0/4073179046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 0x555bca6b5880 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff31c046970 tx=0x7ff31c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95274b3438d9ff37 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.856+0000 7ff3127fc640 1 -- 172.21.15.12:0/4073179046 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c035030 con 0x555bca6a4930 2024-05-09T09:29:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.856+0000 7ff3127fc640 1 -- 172.21.15.12:0/4073179046 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff31c005850 con 0x555bca6a4930 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.856+0000 7ff3127fc640 1 -- 172.21.15.12:0/4073179046 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c000e60 con 0x555bca6a4930 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.856+0000 7ff331f75740 1 -- 172.21.15.12:0/4073179046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 msgr2=0x555bca6b5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.856+0000 7ff331f75740 1 --2- 172.21.15.12:0/4073179046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 0x555bca6b5880 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff31c046970 tx=0x7ff31c005c90 comp rx=0 tx=0).stop 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.856+0000 7ff331f75740 1 -- 172.21.15.12:0/4073179046 shutdown_connections 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.856+0000 7ff331f75740 1 --2- 172.21.15.12:0/4073179046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a9aa0 0x555bca69e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.856+0000 7ff331f75740 1 --2- 172.21.15.12:0/4073179046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a4930 0x555bca6b5880 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.856+0000 7ff331f75740 1 --2- 172.21.15.12:0/4073179046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a3f40 0x555bca69df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.856+0000 7ff331f75740 1 -- 172.21.15.12:0/4073179046 >> 172.21.15.12:0/4073179046 conn(0x555bca69f5b0 msgr2=0x555bca6a5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.856+0000 7ff331f75740 1 -- 172.21.15.12:0/4073179046 shutdown_connections 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.856+0000 7ff331f75740 1 -- 172.21.15.12:0/4073179046 wait complete. 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.856+0000 7ff331f75740 1 -- start start 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff331f75740 1 Processor -- start 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff331f75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 0x555bca6b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff331f75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a4930 0x555bca6b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff331f75740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a9aa0 0x555bca6b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff331f75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff3281abc30 con 0x555bca6a3f40 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff331f75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3281ab4a0 con 0x555bca6a4930 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff331f75740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff3281a8560 con 0x555bca6a9aa0 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff331f75740 10 client.0 _ll_register_callbacks cb 0x7ff31c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff327fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 0x555bca6b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff327fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 0x555bca6b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49754/0 (socket says 172.21.15.12:49754) 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff327fff640 1 -- 172.21.15.12:0/213093782 learned_addr learned my addr 172.21.15.12:0/213093782 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff32d000640 1 --2- 172.21.15.12:0/213093782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a4930 0x555bca6b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff327fff640 1 --2- 172.21.15.12:0/213093782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a9aa0 0x555bca6b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff327fff640 1 -- 172.21.15.12:0/213093782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a9aa0 msgr2=0x555bca6b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff327fff640 1 --2- 172.21.15.12:0/213093782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a9aa0 0x555bca6b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff327fff640 1 -- 172.21.15.12:0/213093782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a4930 msgr2=0x555bca6b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff327fff640 1 --2- 172.21.15.12:0/213093782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a4930 0x555bca6b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.857+0000 7ff327fff640 1 -- 172.21.15.12:0/213093782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bca6b48e0 con 0x555bca6a3f40 2024-05-09T09:29:30.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff327fff640 1 --2- 172.21.15.12:0/213093782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 0x555bca6b2df0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x555bca6a5190 tx=0x7ff31c031c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00d656dbeca2285 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff307fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff307fff640 1 -- 172.21.15.12:0/213093782 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31402b680 con 0x555bca6a3f40 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff307fff640 1 -- 172.21.15.12:0/213093782 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff31402b820 con 0x555bca6a3f40 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff331f75740 1 -- 172.21.15.12:0/213093782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff31c005850 con 0x555bca6a3f40 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff307fff640 1 -- 172.21.15.12:0/213093782 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff314005020 con 0x555bca6a3f40 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff331f75740 10 client.17079 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff331f75740 1 -- 172.21.15.12:0/213093782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555bca6b5be0 con 0x555bca6a3f40 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff307fff640 1 -- 172.21.15.12:0/213093782 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff31c005850 con 0x555bca6a3f40 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff307fff640 1 -- 172.21.15.12:0/213093782 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7ff314028040 con 0x555bca6a3f40 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff331f75740 10 client.17079 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff331f75740 10 client.17079 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff331f75740 20 client.17079 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff331f75740 1 --2- 172.21.15.12:0/213093782 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555bca720fe0 0x555bca723400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff331f75740 4 client.17079 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:29:30.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.858+0000 7ff331f75740 1 -- 172.21.15.12:0/213093782 --> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555bca6106c0 con 0x555bca720fe0 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.859+0000 7ff32d000640 1 --2- 172.21.15.12:0/213093782 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555bca720fe0 0x555bca723400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.859+0000 7ff307fff640 10 client.17079 ms_handle_connect on v2:172.21.15.16:6838/3932062083 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.859+0000 7ff32d000640 1 --2- 172.21.15.12:0/213093782 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555bca720fe0 0x555bca723400 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0b908f5e5158da3 server_cookie=d6654c4b4f561cbb in_seq=0 out_seq=0 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.859+0000 7ff307fff640 1 -- 172.21.15.12:0/213093782 <== mds.1 v2:172.21.15.16:6838/3932062083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555bca6106c0 con 0x555bca720fe0 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.859+0000 7ff307fff640 10 client.17079 handle_command_reply: tid=0 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.859+0000 7ff331f75740 1 client.17079 shutdown 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.859+0000 7ff331f75740 1 -- 172.21.15.12:0/213093782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 msgr2=0x555bca6b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.859+0000 7ff331f75740 1 --2- 172.21.15.12:0/213093782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 0x555bca6b2df0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x555bca6a5190 tx=0x7ff31c031c30 comp rx=0 tx=0).stop 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff331f75740 1 -- 172.21.15.12:0/213093782 shutdown_connections 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff331f75740 1 --2- 172.21.15.12:0/213093782 >> [v2:172.21.15.16:6838/3932062083,v1:172.21.15.16:6839/3932062083] conn(0x555bca720fe0 0x555bca723400 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff331f75740 1 --2- 172.21.15.12:0/213093782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bca6a9aa0 0x555bca6b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff331f75740 1 --2- 172.21.15.12:0/213093782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bca6a4930 0x555bca6b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff307fff640 0 client.17079 ms_handle_reset on v2:172.21.15.16:6838/3932062083 2024-05-09T09:29:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff331f75740 1 --2- 172.21.15.12:0/213093782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bca6a3f40 0x555bca6b2df0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff331f75740 1 -- 172.21.15.12:0/213093782 >> 172.21.15.12:0/213093782 conn(0x555bca69f5b0 msgr2=0x555bca6a2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff331f75740 1 -- 172.21.15.12:0/213093782 shutdown_connections 2024-05-09T09:29:30.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff331f75740 1 -- 172.21.15.12:0/213093782 wait complete. 2024-05-09T09:29:30.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff331f75740 20 client.17079 trim_cache size 0 max 16384 2024-05-09T09:29:30.867 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:29:30.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff32f264640 1 -- 172.21.15.12:0/1214702350 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff3180a2b90 msgr2=0x7ff3180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff32f264640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff3180a2b90 0x7ff3180a5050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff320097e60 tx=0x7ff320084f90 comp rx=0 tx=0).stop 2024-05-09T09:29:30.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff32f264640 1 -- 172.21.15.12:0/1214702350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3281075d0 msgr2=0x7ff32819f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff32f264640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3281075d0 0x7ff32819f110 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff32000eaa0 tx=0x7ff32000eea0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff32f264640 1 -- 172.21.15.12:0/1214702350 shutdown_connections 2024-05-09T09:29:30.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff32f264640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7ff3180a2b90 0x7ff3180a5050 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff32f264640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3281075d0 0x7ff32819f110 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff32f264640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff328104ca0 0x7ff3281980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff32f264640 1 --2- 172.21.15.12:0/1214702350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff328102370 0x7ff328197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff32f264640 1 -- 172.21.15.12:0/1214702350 >> 172.21.15.12:0/1214702350 conn(0x7ff3280fbd20 msgr2=0x7ff328102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff32f264640 1 -- 172.21.15.12:0/1214702350 shutdown_connections 2024-05-09T09:29:30.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.860+0000 7ff32f264640 1 -- 172.21.15.12:0/1214702350 wait complete. 2024-05-09T09:29:30.871 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:29:30.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 -- 172.21.15.12:0/3379960688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc3c102320 msgr2=0x7fcc3c108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 --2- 172.21.15.12:0/3379960688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc3c102320 0x7fcc3c108bb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009900 tx=0x7fcc3402a410 comp rx=0 tx=0).stop 2024-05-09T09:29:30.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 -- 172.21.15.12:0/3379960688 shutdown_connections 2024-05-09T09:29:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 --2- 172.21.15.12:0/3379960688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc3c102320 0x7fcc3c108bb0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 --2- 172.21.15.12:0/3379960688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3c101960 0x7fcc3c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 --2- 172.21.15.12:0/3379960688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc3c100770 0x7fcc3c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 -- 172.21.15.12:0/3379960688 >> 172.21.15.12:0/3379960688 conn(0x7fcc3c0fbce0 msgr2=0x7fcc3c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 -- 172.21.15.12:0/3379960688 shutdown_connections 2024-05-09T09:29:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 -- 172.21.15.12:0/3379960688 wait complete. 2024-05-09T09:29:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 Processor -- start 2024-05-09T09:29:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 -- start start 2024-05-09T09:29:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc3c100770 0x7fcc3c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3c101960 0x7fcc3c19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc3c102320 0x7fcc3c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c10c3c0 con 0x7fcc3c100770 2024-05-09T09:29:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c10c240 con 0x7fcc3c101960 2024-05-09T09:29:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.979+0000 7fcc42bb0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c10c540 con 0x7fcc3c102320 2024-05-09T09:29:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc4094c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc3c100770 0x7fcc3c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc4094c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc3c100770 0x7fcc3c069530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49770/0 (socket says 172.21.15.12:49770) 2024-05-09T09:29:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc4094c640 1 -- 172.21.15.12:0/3474507433 learned_addr learned my addr 172.21.15.12:0/3474507433 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc4114d640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc3c102320 0x7fcc3c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc3bfff640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3c101960 0x7fcc3c19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc4094c640 1 -- 172.21.15.12:0/3474507433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc3c102320 msgr2=0x7fcc3c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc4094c640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc3c102320 0x7fcc3c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc4094c640 1 -- 172.21.15.12:0/3474507433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3c101960 msgr2=0x7fcc3c19c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc4094c640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3c101960 0x7fcc3c19c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc4094c640 1 -- 172.21.15.12:0/3474507433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c19f7e0 con 0x7fcc3c100770 2024-05-09T09:29:30.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc3bfff640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3c101960 0x7fcc3c19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:29:30.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc4094c640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc3c100770 0x7fcc3c069530 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400d770 tx=0x7fcc2400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc39ffb640 1 -- 172.21.15.12:0/3474507433 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24017070 con 0x7fcc3c100770 2024-05-09T09:29:30.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc39ffb640 1 -- 172.21.15.12:0/3474507433 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcc24009c30 con 0x7fcc3c100770 2024-05-09T09:29:30.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc42bb0640 1 -- 172.21.15.12:0/3474507433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3c19fa70 con 0x7fcc3c100770 2024-05-09T09:29:30.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc39ffb640 1 -- 172.21.15.12:0/3474507433 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24003e10 con 0x7fcc3c100770 2024-05-09T09:29:30.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.980+0000 7fcc42bb0640 1 -- 172.21.15.12:0/3474507433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3c19ffc0 con 0x7fcc3c100770 2024-05-09T09:29:30.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc39ffb640 1 -- 172.21.15.12:0/3474507433 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcc24004020 con 0x7fcc3c100770 2024-05-09T09:29:30.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc4599e740 1 Processor -- start 2024-05-09T09:29:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc4599e740 1 -- start start 2024-05-09T09:29:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc4599e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c2f40 0x5636b39c3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc4599e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc4599e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 0x5636b39caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc4599e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5636b3927c40 con 0x5636b39c3930 2024-05-09T09:29:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc4599e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5636b38cf580 con 0x5636b39c2f40 2024-05-09T09:29:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc4599e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5636b3923c00 con 0x5636b39c8aa0 2024-05-09T09:29:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc4094c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc4094c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49774/0 (socket says 172.21.15.12:49774) 2024-05-09T09:29:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc4094c640 1 -- 172.21.15.12:0/3894877831 learned_addr learned my addr 172.21.15.12:0/3894877831 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc4114d640 1 --2- 172.21.15.12:0/3894877831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c2f40 0x5636b39c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.983+0000 7fcc3bfff640 1 --2- 172.21.15.12:0/3894877831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 0x5636b39caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.984+0000 7fcc4094c640 1 -- 172.21.15.12:0/3894877831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 msgr2=0x5636b39caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.984+0000 7fcc4094c640 1 --2- 172.21.15.12:0/3894877831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 0x5636b39caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.984+0000 7fcc39ffb640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcc280a2b90 0x7fcc280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.984+0000 7fcc4094c640 1 -- 172.21.15.12:0/3894877831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c2f40 msgr2=0x5636b39c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.984+0000 7fcc4094c640 1 --2- 172.21.15.12:0/3894877831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c2f40 0x5636b39c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.984+0000 7fcc4094c640 1 -- 172.21.15.12:0/3894877831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b363b3f0 con 0x5636b39c3930 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.984+0000 7fcc4114d640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcc280a2b90 0x7fcc280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.984+0000 7fcc39ffb640 1 -- 172.21.15.12:0/3474507433 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fcc240cf230 con 0x7fcc3c100770 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.984+0000 7fcc4094c640 1 --2- 172.21.15.12:0/3894877831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39c8430 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcc240a4490 tx=0x7fcc240d2f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e43268f7ceb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.984+0000 7fcc32ffd640 1 -- 172.21.15.12:0/3894877831 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24020070 con 0x5636b39c3930 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.984+0000 7fcc32ffd640 1 -- 172.21.15.12:0/3894877831 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcc240a42d0 con 0x5636b39c3930 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.984+0000 7fcc32ffd640 1 -- 172.21.15.12:0/3894877831 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2409b1a0 con 0x5636b39c3930 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.985+0000 7fcc4599e740 1 -- 172.21.15.12:0/3894877831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 msgr2=0x5636b39c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.985+0000 7fcc4114d640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcc280a2b90 0x7fcc280a5050 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x5636b39c4190 tx=0x7fcc34009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.985+0000 7fcc4599e740 1 --2- 172.21.15.12:0/3894877831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39c8430 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcc240a4490 tx=0x7fcc240d2f80 comp rx=0 tx=0).stop 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.985+0000 7fcc4599e740 1 -- 172.21.15.12:0/3894877831 shutdown_connections 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.985+0000 7fcc4599e740 1 --2- 172.21.15.12:0/3894877831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 0x5636b39caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.985+0000 7fcc4599e740 1 --2- 172.21.15.12:0/3894877831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39c8430 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.985+0000 7fcc4599e740 1 --2- 172.21.15.12:0/3894877831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c2f40 0x5636b39c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.985+0000 7fcc4599e740 1 -- 172.21.15.12:0/3894877831 >> 172.21.15.12:0/3894877831 conn(0x5636b39be5b0 msgr2=0x5636b39be9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.985+0000 7fcc4599e740 1 -- 172.21.15.12:0/3894877831 shutdown_connections 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.985+0000 7fcc4599e740 1 -- 172.21.15.12:0/3894877831 wait complete. 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.985+0000 7fcc4599e740 1 -- start start 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.985+0000 7fcc4599e740 1 Processor -- start 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc4599e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c2f40 0x5636b39cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc4599e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc4599e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 0x5636b39d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc4599e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5636b39cb990 con 0x5636b39c3930 2024-05-09T09:29:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc4599e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5636b38e2c40 con 0x5636b39c2f40 2024-05-09T09:29:30.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc4599e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5636b39cbb10 con 0x5636b39c8aa0 2024-05-09T09:29:30.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc4599e740 10 client.0 _ll_register_callbacks cb 0x5636b363abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:30.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49782/0 (socket says 172.21.15.12:49782) 2024-05-09T09:29:30.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc3bfff640 1 -- 172.21.15.12:0/817387873 learned_addr learned my addr 172.21.15.12:0/817387873 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:30.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc4114d640 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 0x5636b39d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc4094c640 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c2f40 0x5636b39cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:30.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc3bfff640 1 -- 172.21.15.12:0/817387873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 msgr2=0x5636b39d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc3bfff640 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 0x5636b39d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc3bfff640 1 -- 172.21.15.12:0/817387873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c2f40 msgr2=0x5636b39cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:30.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc3bfff640 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c2f40 0x5636b39cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:30.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc3bfff640 1 -- 172.21.15.12:0/817387873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b37057d0 con 0x5636b39c3930 2024-05-09T09:29:30.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc4094c640 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c2f40 0x5636b39cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc3bfff640 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39cd7b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c011710 tx=0x7fcc2c011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef683c8d8b4c42c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.986+0000 7fcc1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc1bfff640 1 -- 172.21.15.12:0/817387873 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c00fae0 con 0x5636b39c3930 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc1bfff640 1 -- 172.21.15.12:0/817387873 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fcc2c00fc80 con 0x5636b39c3930 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc4599e740 1 -- 172.21.15.12:0/817387873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5636b3935370 con 0x5636b39c3930 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc1bfff640 1 -- 172.21.15.12:0/817387873 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fcc2c01d480 con 0x5636b39c3930 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc4599e740 10 client.17091 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc4599e740 1 -- 172.21.15.12:0/817387873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5636b39d7c80 con 0x5636b39c3930 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc1bfff640 1 -- 172.21.15.12:0/817387873 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x5636b3935370 con 0x5636b39c3930 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc1bfff640 1 -- 172.21.15.12:0/817387873 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fcc2c0126e0 con 0x5636b39c3930 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc4599e740 10 client.17091 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc4599e740 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc4599e740 20 client.17091 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:30.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc4599e740 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5636b39d93d0 0x5636b39db890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc4599e740 4 client.17091 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc4599e740 1 -- 172.21.15.12:0/817387873 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636b392f6c0 con 0x5636b39d93d0 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.987+0000 7fcc4094c640 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5636b39d93d0 0x5636b39db890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.988+0000 7fcc4094c640 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5636b39d93d0 0x5636b39db890 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b7863546ba34e88 server_cookie=3c0a3e2318c2e1a6 in_seq=0 out_seq=0 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.988+0000 7fcc1bfff640 10 client.17091 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.988+0000 7fcc1bfff640 1 -- 172.21.15.12:0/817387873 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636b392f6c0 con 0x5636b39d93d0 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.988+0000 7fcc1bfff640 10 client.17091 handle_command_reply: tid=0 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.989+0000 7fcc4599e740 1 client.17091 shutdown 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.989+0000 7fcc4599e740 1 -- 172.21.15.12:0/817387873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 msgr2=0x5636b39cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.989+0000 7fcc4599e740 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39cd7b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c011710 tx=0x7fcc2c011bd0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.989+0000 7fcc4599e740 1 -- 172.21.15.12:0/817387873 shutdown_connections 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.989+0000 7fcc4599e740 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5636b39d93d0 0x5636b39db890 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.989+0000 7fcc4599e740 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 0x5636b39d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.989+0000 7fcc1bfff640 0 client.17091 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:29:31.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.989+0000 7fcc4599e740 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39cd7b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.989+0000 7fcc4599e740 1 --2- 172.21.15.12:0/817387873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c2f40 0x5636b39cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.989+0000 7fcc4599e740 1 -- 172.21.15.12:0/817387873 >> 172.21.15.12:0/817387873 conn(0x5636b39be5b0 msgr2=0x5636b39bd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:31.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.989+0000 7fcc4599e740 1 -- 172.21.15.12:0/817387873 shutdown_connections 2024-05-09T09:29:31.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.989+0000 7fcc4599e740 1 -- 172.21.15.12:0/817387873 wait complete. 2024-05-09T09:29:31.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:30.989+0000 7fcc4599e740 20 client.17091 trim_cache size 0 max 16384 2024-05-09T09:29:31.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.002+0000 7fcc4599e740 1 Processor -- start 2024-05-09T09:29:31.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc4599e740 1 -- start start 2024-05-09T09:29:31.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc4599e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c2f40 0x5636b39bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:31.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc4599e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:31.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc4599e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c8aa0 0x5636b39bd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:31.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc4599e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1aa620 con 0x5636b39c3930 2024-05-09T09:29:31.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc4599e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1ac3a0 con 0x5636b39c8aa0 2024-05-09T09:29:31.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc4599e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1aadb0 con 0x5636b39c2f40 2024-05-09T09:29:31.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:31.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39d47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49792/0 (socket says 172.21.15.12:49792) 2024-05-09T09:29:31.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc3bfff640 1 -- 172.21.15.12:0/3966919090 learned_addr learned my addr 172.21.15.12:0/3966919090 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:31.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc3bfff640 1 -- 172.21.15.12:0/3966919090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c2f40 msgr2=0x5636b39bcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:31.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc4094c640 1 --2- 172.21.15.12:0/3966919090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c2f40 0x5636b39bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:31.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc3bfff640 1 --2- 172.21.15.12:0/3966919090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c2f40 0x5636b39bcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc3bfff640 1 -- 172.21.15.12:0/3966919090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c8aa0 msgr2=0x5636b39bd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:31.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc3bfff640 1 --2- 172.21.15.12:0/3966919090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c8aa0 0x5636b39bd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc3bfff640 1 -- 172.21.15.12:0/3966919090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b39bf320 con 0x5636b39c3930 2024-05-09T09:29:31.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc3bfff640 1 --2- 172.21.15.12:0/3966919090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39d47b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c012b50 tx=0x7fcc2c00ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293d4ee3cb17e5dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:31.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc327fc640 1 -- 172.21.15.12:0/3966919090 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c004be0 con 0x5636b39c3930 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc327fc640 1 -- 172.21.15.12:0/3966919090 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcc2c012920 con 0x5636b39c3930 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.003+0000 7fcc327fc640 1 -- 172.21.15.12:0/3966919090 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c00fe50 con 0x5636b39c3930 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.004+0000 7fcc4599e740 1 -- 172.21.15.12:0/3966919090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 msgr2=0x5636b39d47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.004+0000 7fcc4599e740 1 --2- 172.21.15.12:0/3966919090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39d47b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c012b50 tx=0x7fcc2c00ae90 comp rx=0 tx=0).stop 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.004+0000 7fcc4599e740 1 -- 172.21.15.12:0/3966919090 shutdown_connections 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.004+0000 7fcc4599e740 1 --2- 172.21.15.12:0/3966919090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c8aa0 0x5636b39bd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.004+0000 7fcc4599e740 1 --2- 172.21.15.12:0/3966919090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c3930 0x5636b39d47b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.004+0000 7fcc4599e740 1 --2- 172.21.15.12:0/3966919090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c2f40 0x5636b39bcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.004+0000 7fcc4599e740 1 -- 172.21.15.12:0/3966919090 >> 172.21.15.12:0/3966919090 conn(0x5636b39be5b0 msgr2=0x5636b39c4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.004+0000 7fcc4599e740 1 -- 172.21.15.12:0/3966919090 shutdown_connections 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.004+0000 7fcc4599e740 1 -- 172.21.15.12:0/3966919090 wait complete. 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.004+0000 7fcc4599e740 1 -- start start 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4599e740 1 Processor -- start 2024-05-09T09:29:31.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4599e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c2f40 0x5636b39d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4599e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c3930 0x5636b39d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4599e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 0x5636b39d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4599e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1abcd0 con 0x5636b39c2f40 2024-05-09T09:29:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4599e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1ab540 con 0x5636b39c3930 2024-05-09T09:29:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4599e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1a8600 con 0x5636b39c8aa0 2024-05-09T09:29:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4599e740 10 client.0 _ll_register_callbacks cb 0x7fcc2c00a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:29:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4094c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c2f40 0x5636b39d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4094c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c2f40 0x5636b39d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49798/0 (socket says 172.21.15.12:49798) 2024-05-09T09:29:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4094c640 1 -- 172.21.15.12:0/115564999 learned_addr learned my addr 172.21.15.12:0/115564999 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4094c640 1 -- 172.21.15.12:0/115564999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 msgr2=0x5636b39d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4094c640 1 --2- 172.21.15.12:0/115564999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 0x5636b39d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4094c640 1 -- 172.21.15.12:0/115564999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c3930 msgr2=0x5636b39d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:29:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc3bfff640 1 --2- 172.21.15.12:0/115564999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c3930 0x5636b39d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4094c640 1 --2- 172.21.15.12:0/115564999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c3930 0x5636b39d24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4094c640 1 -- 172.21.15.12:0/115564999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b39d3aa0 con 0x5636b39c2f40 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc4094c640 1 --2- 172.21.15.12:0/115564999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c2f40 0x5636b39d1f60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcc2409b360 tx=0x7fcc2400df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d405a80c2eb407b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.005+0000 7fcc1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc1bfff640 1 -- 172.21.15.12:0/115564999 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24020040 con 0x5636b39c2f40 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc1bfff640 1 -- 172.21.15.12:0/115564999 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fcc24099700 con 0x5636b39c2f40 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc4599e740 1 -- 172.21.15.12:0/115564999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc2c012920 con 0x5636b39c2f40 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc1bfff640 1 -- 172.21.15.12:0/115564999 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fcc240d2d60 con 0x5636b39c2f40 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc4599e740 10 client.17097 fetch_fsmap learned FSMap version 42 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc4599e740 1 -- 172.21.15.12:0/115564999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5636b3a3fae0 con 0x5636b39c2f40 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc1bfff640 1 -- 172.21.15.12:0/115564999 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcc2c012920 con 0x5636b39c2f40 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc1bfff640 1 -- 172.21.15.12:0/115564999 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fcc24004310 con 0x5636b39c2f40 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc4599e740 10 client.17097 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:29:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc4599e740 10 client.17097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T09:29:31.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc4599e740 20 client.17097 populate_metadata read hostname 'smithi012' 2024-05-09T09:29:31.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc4599e740 1 --2- 172.21.15.12:0/115564999 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5636b3a40100 0x5636b3a42510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:31.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc4599e740 4 client.17097 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:29:31.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc4599e740 1 -- 172.21.15.12:0/115564999 --> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636b38ac820 con 0x5636b3a40100 2024-05-09T09:29:31.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.006+0000 7fcc3bfff640 1 --2- 172.21.15.12:0/115564999 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5636b3a40100 0x5636b3a42510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:31.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc3bfff640 1 --2- 172.21.15.12:0/115564999 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5636b3a40100 0x5636b3a42510 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=263c7e0901fbb41f server_cookie=8fe489400a4dcba4 in_seq=0 out_seq=0 2024-05-09T09:29:31.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc1bfff640 10 client.17097 ms_handle_connect on v2:172.21.15.170:6838/3731005466 2024-05-09T09:29:31.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc1bfff640 1 -- 172.21.15.12:0/115564999 <== mds.0 v2:172.21.15.170:6838/3731005466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636b38ac820 con 0x5636b3a40100 2024-05-09T09:29:31.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc1bfff640 10 client.17097 handle_command_reply: tid=0 2024-05-09T09:29:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc4599e740 1 client.17097 shutdown 2024-05-09T09:29:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc4599e740 1 -- 172.21.15.12:0/115564999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c2f40 msgr2=0x5636b39d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc4599e740 1 --2- 172.21.15.12:0/115564999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c2f40 0x5636b39d1f60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcc2409b360 tx=0x7fcc2400df10 comp rx=0 tx=0).stop 2024-05-09T09:29:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc4599e740 1 -- 172.21.15.12:0/115564999 shutdown_connections 2024-05-09T09:29:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc4599e740 1 --2- 172.21.15.12:0/115564999 >> [v2:172.21.15.170:6838/3731005466,v1:172.21.15.170:6839/3731005466] conn(0x5636b3a40100 0x5636b3a42510 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc4599e740 1 --2- 172.21.15.12:0/115564999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5636b39c8aa0 0x5636b39d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc4599e740 1 --2- 172.21.15.12:0/115564999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636b39c3930 0x5636b39d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc1bfff640 0 client.17097 ms_handle_reset on v2:172.21.15.170:6838/3731005466 2024-05-09T09:29:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc4599e740 1 --2- 172.21.15.12:0/115564999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5636b39c2f40 0x5636b39d1f60 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc4599e740 1 -- 172.21.15.12:0/115564999 >> 172.21.15.12:0/115564999 conn(0x5636b39be5b0 msgr2=0x5636b39bcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc4599e740 1 -- 172.21.15.12:0/115564999 shutdown_connections 2024-05-09T09:29:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc4599e740 1 -- 172.21.15.12:0/115564999 wait complete. 2024-05-09T09:29:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.007+0000 7fcc4599e740 20 client.17097 trim_cache size 0 max 16384 2024-05-09T09:29:31.016 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T09:29:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.008+0000 7fcc42bb0640 1 -- 172.21.15.12:0/3474507433 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcc280a2b90 msgr2=0x7fcc280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.008+0000 7fcc42bb0640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcc280a2b90 0x7fcc280a5050 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x5636b39c4190 tx=0x7fcc34009260 comp rx=0 tx=0).stop 2024-05-09T09:29:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.008+0000 7fcc42bb0640 1 -- 172.21.15.12:0/3474507433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc3c100770 msgr2=0x7fcc3c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.008+0000 7fcc42bb0640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc3c100770 0x7fcc3c069530 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400d770 tx=0x7fcc2400dc30 comp rx=0 tx=0).stop 2024-05-09T09:29:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.008+0000 7fcc42bb0640 1 -- 172.21.15.12:0/3474507433 shutdown_connections 2024-05-09T09:29:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.008+0000 7fcc42bb0640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fcc280a2b90 0x7fcc280a5050 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.008+0000 7fcc42bb0640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc3c102320 0x7fcc3c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.008+0000 7fcc42bb0640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3c101960 0x7fcc3c19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.008+0000 7fcc42bb0640 1 --2- 172.21.15.12:0/3474507433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc3c100770 0x7fcc3c069530 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.008+0000 7fcc42bb0640 1 -- 172.21.15.12:0/3474507433 >> 172.21.15.12:0/3474507433 conn(0x7fcc3c0fbce0 msgr2=0x7fcc3c104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.008+0000 7fcc42bb0640 1 -- 172.21.15.12:0/3474507433 shutdown_connections 2024-05-09T09:29:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:31.008+0000 7fcc42bb0640 1 -- 172.21.15.12:0/3474507433 wait complete. 2024-05-09T09:29:31.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:31 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2743219745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:29:31.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:31 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2743219745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:29:31.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:31 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2743219745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:29:32.019 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-05-09T09:29:32.020 INFO:tasks.fwd_scrub:done joining 2024-05-09T09:29:32.020 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-09T09:29:32.031 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:29:32.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.151+0000 7f94913b0640 1 -- 172.21.15.12:0/3089519088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948c102320 msgr2=0x7f948c108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.151+0000 7f94913b0640 1 --2- 172.21.15.12:0/3089519088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948c102320 0x7f948c108bb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9484009900 tx=0x7f948402a410 comp rx=0 tx=0).stop 2024-05-09T09:29:32.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.151+0000 7f94913b0640 1 -- 172.21.15.12:0/3089519088 shutdown_connections 2024-05-09T09:29:32.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.151+0000 7f94913b0640 1 --2- 172.21.15.12:0/3089519088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948c102320 0x7f948c108bb0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.151+0000 7f94913b0640 1 --2- 172.21.15.12:0/3089519088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c101960 0x7f948c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.151+0000 7f94913b0640 1 --2- 172.21.15.12:0/3089519088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f948c100770 0x7f948c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.151+0000 7f94913b0640 1 -- 172.21.15.12:0/3089519088 >> 172.21.15.12:0/3089519088 conn(0x7f948c0fbce0 msgr2=0x7f948c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:32.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.151+0000 7f94913b0640 1 -- 172.21.15.12:0/3089519088 shutdown_connections 2024-05-09T09:29:32.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.151+0000 7f94913b0640 1 -- 172.21.15.12:0/3089519088 wait complete. 2024-05-09T09:29:32.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.151+0000 7f94913b0640 1 Processor -- start 2024-05-09T09:29:32.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f94913b0640 1 -- start start 2024-05-09T09:29:32.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f94913b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948c100770 0x7f948c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f94913b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c101960 0x7f948c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f94913b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f948c102320 0x7f948c1a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f94913b0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f948c10c420 con 0x7f948c100770 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f94913b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f948c10c2a0 con 0x7f948c102320 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f94913b0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f948c10c5a0 con 0x7f948c101960 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f948affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948c100770 0x7f948c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f948affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948c100770 0x7f948c069530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34858/0 (socket says 172.21.15.12:34858) 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f948affd640 1 -- 172.21.15.12:0/164540081 learned_addr learned my addr 172.21.15.12:0/164540081 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f948a7fc640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c101960 0x7f948c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f948affd640 1 -- 172.21.15.12:0/164540081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c101960 msgr2=0x7f948c19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f948b7fe640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f948c102320 0x7f948c1a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f948affd640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c101960 0x7f948c19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f948affd640 1 -- 172.21.15.12:0/164540081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f948c102320 msgr2=0x7f948c1a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f948affd640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f948c102320 0x7f948c1a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f948affd640 1 -- 172.21.15.12:0/164540081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948c1a19c0 con 0x7f948c100770 2024-05-09T09:29:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.152+0000 7f948a7fc640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c101960 0x7f948c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:29:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.153+0000 7f948affd640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948c100770 0x7f948c069530 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f947c00ea80 tx=0x7f947c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.153+0000 7f9473fff640 1 -- 172.21.15.12:0/164540081 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c00ce40 con 0x7f948c100770 2024-05-09T09:29:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.153+0000 7f94913b0640 1 -- 172.21.15.12:0/164540081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948c1a1c50 con 0x7f948c100770 2024-05-09T09:29:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.153+0000 7f9473fff640 1 -- 172.21.15.12:0/164540081 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f947c011940 con 0x7f948c100770 2024-05-09T09:29:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.153+0000 7f9473fff640 1 -- 172.21.15.12:0/164540081 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c01ab30 con 0x7f948c100770 2024-05-09T09:29:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.153+0000 7f94913b0640 1 -- 172.21.15.12:0/164540081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948c1a21a0 con 0x7f948c100770 2024-05-09T09:29:32.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.154+0000 7f94913b0640 1 -- 172.21.15.12:0/164540081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f948c107a30 con 0x7f948c100770 2024-05-09T09:29:32.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.155+0000 7f9473fff640 1 -- 172.21.15.12:0/164540081 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f947c01acd0 con 0x7f948c100770 2024-05-09T09:29:32.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.156+0000 7f9473fff640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f94740a2bc0 0x7f94740a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:32.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.156+0000 7f9473fff640 1 -- 172.21.15.12:0/164540081 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f947c0cfc10 con 0x7f948c100770 2024-05-09T09:29:32.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.156+0000 7f948a7fc640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f94740a2bc0 0x7f94740a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:32.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.156+0000 7f948a7fc640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f94740a2bc0 0x7f94740a5080 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f948c0692a0 tx=0x7f9480005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:32.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.159+0000 7f9473fff640 1 -- 172.21.15.12:0/164540081 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f947c016030 con 0x7f948c100770 2024-05-09T09:29:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.348+0000 7f94913b0640 1 -- 172.21.15.12:0/164540081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f948c02d470 con 0x7f948c100770 2024-05-09T09:29:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.349+0000 7f9473fff640 1 -- 172.21.15.12:0/164540081 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24762 (secure 0 0 0) 0x7f947c09c7a0 con 0x7f948c100770 2024-05-09T09:29:32.349 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:29:32.350 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":94,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:27:40.951852+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:29:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.352+0000 7f94913b0640 1 -- 172.21.15.12:0/164540081 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f94740a2bc0 msgr2=0x7f94740a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.352+0000 7f94913b0640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f94740a2bc0 0x7f94740a5080 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f948c0692a0 tx=0x7f9480005f90 comp rx=0 tx=0).stop 2024-05-09T09:29:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.352+0000 7f94913b0640 1 -- 172.21.15.12:0/164540081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948c100770 msgr2=0x7f948c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.352+0000 7f94913b0640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948c100770 0x7f948c069530 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f947c00ea80 tx=0x7f947c00ef40 comp rx=0 tx=0).stop 2024-05-09T09:29:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.352+0000 7f94913b0640 1 -- 172.21.15.12:0/164540081 shutdown_connections 2024-05-09T09:29:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.352+0000 7f94913b0640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f94740a2bc0 0x7f94740a5080 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.352+0000 7f94913b0640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f948c102320 0x7f948c1a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.352+0000 7f94913b0640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c101960 0x7f948c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.352+0000 7f94913b0640 1 --2- 172.21.15.12:0/164540081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948c100770 0x7f948c069530 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.352+0000 7f94913b0640 1 -- 172.21.15.12:0/164540081 >> 172.21.15.12:0/164540081 conn(0x7f948c0fbce0 msgr2=0x7f948c104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.352+0000 7f94913b0640 1 -- 172.21.15.12:0/164540081 shutdown_connections 2024-05-09T09:29:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.352+0000 7f94913b0640 1 -- 172.21.15.12:0/164540081 wait complete. 2024-05-09T09:29:32.362 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:32 smithi012 ceph-mon[30140]: pgmap v487: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-09T09:29:32.363 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:23:08.167107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:23:09.065670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:23:12.773640+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:29:32.364 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:29:32.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.472+0000 7efd81215640 1 -- 172.21.15.12:0/2005841976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd7c103440 msgr2=0x7efd7c109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.472+0000 7efd81215640 1 --2- 172.21.15.12:0/2005841976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd7c103440 0x7efd7c109cd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7efd74009ca0 tx=0x7efd7402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.472+0000 7efd81215640 1 -- 172.21.15.12:0/2005841976 shutdown_connections 2024-05-09T09:29:32.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.472+0000 7efd81215640 1 --2- 172.21.15.12:0/2005841976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd7c103440 0x7efd7c109cd0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.472+0000 7efd81215640 1 --2- 172.21.15.12:0/2005841976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c102a80 0x7efd7c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.472+0000 7efd81215640 1 --2- 172.21.15.12:0/2005841976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c101890 0x7efd7c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.472+0000 7efd81215640 1 -- 172.21.15.12:0/2005841976 >> 172.21.15.12:0/2005841976 conn(0x7efd7c0fce00 msgr2=0x7efd7c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.472+0000 7efd81215640 1 -- 172.21.15.12:0/2005841976 shutdown_connections 2024-05-09T09:29:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.472+0000 7efd81215640 1 -- 172.21.15.12:0/2005841976 wait complete. 2024-05-09T09:29:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.473+0000 7efd81215640 1 Processor -- start 2024-05-09T09:29:32.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.473+0000 7efd81215640 1 -- start start 2024-05-09T09:29:32.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.473+0000 7efd81215640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c101890 0x7efd7c197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:32.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.473+0000 7efd81215640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c102a80 0x7efd7c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.473+0000 7efd81215640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd7c103440 0x7efd7c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.473+0000 7efd81215640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c10d570 con 0x7efd7c103440 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.473+0000 7efd81215640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c10d3f0 con 0x7efd7c102a80 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.473+0000 7efd81215640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c10d6f0 con 0x7efd7c101890 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.473+0000 7efd7b59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd7c103440 0x7efd7c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.473+0000 7efd7b59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd7c103440 0x7efd7c19f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34876/0 (socket says 172.21.15.12:34876) 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.473+0000 7efd7b59e640 1 -- 172.21.15.12:0/2960192445 learned_addr learned my addr 172.21.15.12:0/2960192445 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.473+0000 7efd7259c640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c102a80 0x7efd7c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.474+0000 7efd7ad9d640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c101890 0x7efd7c197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.474+0000 7efd7b59e640 1 -- 172.21.15.12:0/2960192445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c101890 msgr2=0x7efd7c197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.474+0000 7efd7b59e640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c101890 0x7efd7c197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.474+0000 7efd7b59e640 1 -- 172.21.15.12:0/2960192445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c102a80 msgr2=0x7efd7c198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.474+0000 7efd7b59e640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c102a80 0x7efd7c198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.474+0000 7efd7b59e640 1 -- 172.21.15.12:0/2960192445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c19f810 con 0x7efd7c103440 2024-05-09T09:29:32.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.474+0000 7efd7ad9d640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c101890 0x7efd7c197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:29:32.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.474+0000 7efd7b59e640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd7c103440 0x7efd7c19f1a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7efd74004ad0 tx=0x7efd7402f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:32.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.474+0000 7efd78d99640 1 -- 172.21.15.12:0/2960192445 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74003cf0 con 0x7efd7c103440 2024-05-09T09:29:32.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.474+0000 7efd81215640 1 -- 172.21.15.12:0/2960192445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c19faa0 con 0x7efd7c103440 2024-05-09T09:29:32.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.474+0000 7efd78d99640 1 -- 172.21.15.12:0/2960192445 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efd74004510 con 0x7efd7c103440 2024-05-09T09:29:32.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.474+0000 7efd78d99640 1 -- 172.21.15.12:0/2960192445 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7402fd10 con 0x7efd7c103440 2024-05-09T09:29:32.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.474+0000 7efd81215640 1 -- 172.21.15.12:0/2960192445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c1a0040 con 0x7efd7c103440 2024-05-09T09:29:32.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.475+0000 7efd81215640 1 -- 172.21.15.12:0/2960192445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7c108bd0 con 0x7efd7c103440 2024-05-09T09:29:32.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.476+0000 7efd78d99640 1 -- 172.21.15.12:0/2960192445 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efd74004040 con 0x7efd7c103440 2024-05-09T09:29:32.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.477+0000 7efd78d99640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efd680a2b90 0x7efd680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:32.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.477+0000 7efd7ad9d640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efd680a2b90 0x7efd680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:32.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.477+0000 7efd78d99640 1 -- 172.21.15.12:0/2960192445 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7efd740f0dd0 con 0x7efd7c103440 2024-05-09T09:29:32.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.477+0000 7efd7ad9d640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efd680a2b90 0x7efd680a5050 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efd6c006fd0 tx=0x7efd6c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:32.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.480+0000 7efd78d99640 1 -- 172.21.15.12:0/2960192445 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efd74004330 con 0x7efd7c103440 2024-05-09T09:29:32.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:32 smithi016 ceph-mon[33335]: pgmap v487: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-09T09:29:32.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.665+0000 7efd81215640 1 -- 172.21.15.12:0/2960192445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd7c02d3c0 con 0x7efd7c103440 2024-05-09T09:29:32.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.666+0000 7efd78d99640 1 -- 172.21.15.12:0/2960192445 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7efd740bd900 con 0x7efd7c103440 2024-05-09T09:29:32.666 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:29:32.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.667+0000 7efd81215640 1 -- 172.21.15.12:0/2960192445 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efd680a2b90 msgr2=0x7efd680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.668+0000 7efd81215640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efd680a2b90 0x7efd680a5050 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efd6c006fd0 tx=0x7efd6c008040 comp rx=0 tx=0).stop 2024-05-09T09:29:32.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.668+0000 7efd81215640 1 -- 172.21.15.12:0/2960192445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd7c103440 msgr2=0x7efd7c19f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.668+0000 7efd81215640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd7c103440 0x7efd7c19f1a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7efd74004ad0 tx=0x7efd7402f450 comp rx=0 tx=0).stop 2024-05-09T09:29:32.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.668+0000 7efd81215640 1 -- 172.21.15.12:0/2960192445 shutdown_connections 2024-05-09T09:29:32.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.668+0000 7efd81215640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efd680a2b90 0x7efd680a5050 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.668+0000 7efd81215640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd7c103440 0x7efd7c19f1a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.668+0000 7efd81215640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c102a80 0x7efd7c198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.668+0000 7efd81215640 1 --2- 172.21.15.12:0/2960192445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c101890 0x7efd7c197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.668+0000 7efd81215640 1 -- 172.21.15.12:0/2960192445 >> 172.21.15.12:0/2960192445 conn(0x7efd7c0fce00 msgr2=0x7efd7c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:32.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.668+0000 7efd81215640 1 -- 172.21.15.12:0/2960192445 shutdown_connections 2024-05-09T09:29:32.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.668+0000 7efd81215640 1 -- 172.21.15.12:0/2960192445 wait complete. 2024-05-09T09:29:32.679 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:29:32.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:32 smithi170 ceph-mon[33632]: pgmap v487: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-09T09:29:32.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.788+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/2983677550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b08102320 msgr2=0x7f8b08108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.788+0000 7f8b0e6a5640 1 --2- 172.21.15.12:0/2983677550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b08102320 0x7f8b08108bb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009ca0 tx=0x7f8b0002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.788+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/2983677550 shutdown_connections 2024-05-09T09:29:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.788+0000 7f8b0e6a5640 1 --2- 172.21.15.12:0/2983677550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b08102320 0x7f8b08108bb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.788+0000 7f8b0e6a5640 1 --2- 172.21.15.12:0/2983677550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b08101960 0x7f8b08101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.788+0000 7f8b0e6a5640 1 --2- 172.21.15.12:0/2983677550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b08100770 0x7f8b08100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.788+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/2983677550 >> 172.21.15.12:0/2983677550 conn(0x7f8b080fbce0 msgr2=0x7f8b080fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.788+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/2983677550 shutdown_connections 2024-05-09T09:29:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.788+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/2983677550 wait complete. 2024-05-09T09:29:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.788+0000 7f8b0e6a5640 1 Processor -- start 2024-05-09T09:29:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.788+0000 7f8b0e6a5640 1 -- start start 2024-05-09T09:29:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b0e6a5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b08100770 0x7f8b08197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b0e6a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b08101960 0x7f8b081980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b0e6a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b08102320 0x7f8b0819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b0e6a5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0810c3d0 con 0x7f8b08100770 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b0e6a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0810c250 con 0x7f8b08101960 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b0e6a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0810c550 con 0x7f8b08102320 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b07fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b08100770 0x7f8b08197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b07fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b08100770 0x7f8b08197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34896/0 (socket says 172.21.15.12:34896) 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b07fff640 1 -- 172.21.15.12:0/916059618 learned_addr learned my addr 172.21.15.12:0/916059618 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b0cc42640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b08102320 0x7f8b0819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b077fe640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b08101960 0x7f8b081980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b07fff640 1 -- 172.21.15.12:0/916059618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b08102320 msgr2=0x7f8b0819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b07fff640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b08102320 0x7f8b0819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b07fff640 1 -- 172.21.15.12:0/916059618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b08101960 msgr2=0x7f8b081980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b07fff640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b08101960 0x7f8b081980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b07fff640 1 -- 172.21.15.12:0/916059618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0819f7c0 con 0x7f8b08100770 2024-05-09T09:29:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b07fff640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b08100770 0x7f8b08197ba0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8af800e9c0 tx=0x7f8af800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b057fa640 1 -- 172.21.15.12:0/916059618 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af800cd80 con 0x7f8b08100770 2024-05-09T09:29:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.789+0000 7f8b057fa640 1 -- 172.21.15.12:0/916059618 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8af80035f0 con 0x7f8b08100770 2024-05-09T09:29:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.790+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/916059618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0819fab0 con 0x7f8b08100770 2024-05-09T09:29:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.790+0000 7f8b057fa640 1 -- 172.21.15.12:0/916059618 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8010690 con 0x7f8b08100770 2024-05-09T09:29:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.790+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/916059618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0819ffb0 con 0x7f8b08100770 2024-05-09T09:29:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.791+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/916059618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b081a0240 con 0x7f8b08100770 2024-05-09T09:29:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.792+0000 7f8b057fa640 1 -- 172.21.15.12:0/916059618 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8af80037a0 con 0x7f8b08100770 2024-05-09T09:29:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.793+0000 7f8b057fa640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8af00a2b90 0x7f8af00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.793+0000 7f8b077fe640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8af00a2b90 0x7f8af00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.793+0000 7f8b057fa640 1 -- 172.21.15.12:0/916059618 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8af80cf7c0 con 0x7f8b08100770 2024-05-09T09:29:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.793+0000 7f8b077fe640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8af00a2b90 0x7f8af00a5050 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b081990b0 tx=0x7f8afc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.797+0000 7f8b057fa640 1 -- 172.21.15.12:0/916059618 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8af809c350 con 0x7f8b08100770 2024-05-09T09:29:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.986+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/916059618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8b081a0410 con 0x7f8b08100770 2024-05-09T09:29:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.986+0000 7f8b057fa640 1 -- 172.21.15.12:0/916059618 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f8af809bc50 con 0x7f8b08100770 2024-05-09T09:29:32.986 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:29:32.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.988+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/916059618 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8af00a2b90 msgr2=0x7f8af00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.988+0000 7f8b0e6a5640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8af00a2b90 0x7f8af00a5050 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b081990b0 tx=0x7f8afc00a2d0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.988+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/916059618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b08100770 msgr2=0x7f8b08197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:32.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.988+0000 7f8b0e6a5640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b08100770 0x7f8b08197ba0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8af800e9c0 tx=0x7f8af800ee80 comp rx=0 tx=0).stop 2024-05-09T09:29:32.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.988+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/916059618 shutdown_connections 2024-05-09T09:29:32.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.988+0000 7f8b0e6a5640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f8af00a2b90 0x7f8af00a5050 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.989+0000 7f8b0e6a5640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b08102320 0x7f8b0819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.989+0000 7f8b0e6a5640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b08101960 0x7f8b081980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.989+0000 7f8b0e6a5640 1 --2- 172.21.15.12:0/916059618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b08100770 0x7f8b08197ba0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:32.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.989+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/916059618 >> 172.21.15.12:0/916059618 conn(0x7f8b080fbce0 msgr2=0x7f8b080fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:32.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.989+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/916059618 shutdown_connections 2024-05-09T09:29:32.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:32.989+0000 7f8b0e6a5640 1 -- 172.21.15.12:0/916059618 wait complete. 2024-05-09T09:29:33.000 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:29:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.109+0000 7f96b419e640 1 -- 172.21.15.12:0/3978665216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac102310 msgr2=0x7f96ac108ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.109+0000 7f96b419e640 1 --2- 172.21.15.12:0/3978665216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac102310 0x7f96ac108ba0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009ca0 tx=0x7f96a402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.109+0000 7f96b419e640 1 -- 172.21.15.12:0/3978665216 shutdown_connections 2024-05-09T09:29:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.109+0000 7f96b419e640 1 --2- 172.21.15.12:0/3978665216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac102310 0x7f96ac108ba0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.109+0000 7f96b419e640 1 --2- 172.21.15.12:0/3978665216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96ac101950 0x7f96ac101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.109+0000 7f96b419e640 1 --2- 172.21.15.12:0/3978665216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96ac100760 0x7f96ac100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.109+0000 7f96b419e640 1 -- 172.21.15.12:0/3978665216 >> 172.21.15.12:0/3978665216 conn(0x7f96ac0fbcd0 msgr2=0x7f96ac0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.109+0000 7f96b419e640 1 -- 172.21.15.12:0/3978665216 shutdown_connections 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.109+0000 7f96b419e640 1 -- 172.21.15.12:0/3978665216 wait complete. 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b419e640 1 Processor -- start 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b419e640 1 -- start start 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b419e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac100760 0x7f96ac1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b419e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96ac101950 0x7f96ac196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b419e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96ac102310 0x7f96ac19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b419e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac10c3b0 con 0x7f96ac100760 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b419e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac10c230 con 0x7f96ac102310 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b419e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac10c530 con 0x7f96ac101950 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b1f3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac100760 0x7f96ac1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b1f3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac100760 0x7f96ac1969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34910/0 (socket says 172.21.15.12:34910) 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b1f3a640 1 -- 172.21.15.12:0/279587187 learned_addr learned my addr 172.21.15.12:0/279587187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b273b640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96ac102310 0x7f96ac19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b1f3a640 1 -- 172.21.15.12:0/279587187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96ac101950 msgr2=0x7f96ac196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b1f3a640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96ac101950 0x7f96ac196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b1f3a640 1 -- 172.21.15.12:0/279587187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96ac102310 msgr2=0x7f96ac19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b1f3a640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96ac102310 0x7f96ac19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b1f3a640 1 -- 172.21.15.12:0/279587187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac19e610 con 0x7f96ac100760 2024-05-09T09:29:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b273b640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96ac102310 0x7f96ac19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:29:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.110+0000 7f96b1f3a640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac100760 0x7f96ac1969f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f96a800d950 tx=0x7f96a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.111+0000 7f96a2ffd640 1 -- 172.21.15.12:0/279587187 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8017020 con 0x7f96ac100760 2024-05-09T09:29:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.111+0000 7f96a2ffd640 1 -- 172.21.15.12:0/279587187 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f96a8009c30 con 0x7f96ac100760 2024-05-09T09:29:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.111+0000 7f96a2ffd640 1 -- 172.21.15.12:0/279587187 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a800f740 con 0x7f96ac100760 2024-05-09T09:29:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.111+0000 7f96b419e640 1 -- 172.21.15.12:0/279587187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ac19e900 con 0x7f96ac100760 2024-05-09T09:29:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.111+0000 7f96b419e640 1 -- 172.21.15.12:0/279587187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ac19ee50 con 0x7f96ac100760 2024-05-09T09:29:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.112+0000 7f96b419e640 1 -- 172.21.15.12:0/279587187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96ac107a80 con 0x7f96ac100760 2024-05-09T09:29:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.113+0000 7f96a2ffd640 1 -- 172.21.15.12:0/279587187 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f96a801a020 con 0x7f96ac100760 2024-05-09T09:29:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.114+0000 7f96a2ffd640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f96980a2b90 0x7f96980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.114+0000 7f96b1739640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f96980a2b90 0x7f96980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.114+0000 7f96a2ffd640 1 -- 172.21.15.12:0/279587187 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f96a80cf840 con 0x7f96ac100760 2024-05-09T09:29:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.114+0000 7f96b1739640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f96980a2b90 0x7f96980a5050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f969c005d80 tx=0x7f969c005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:33.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.117+0000 7f96a2ffd640 1 -- 172.21.15.12:0/279587187 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f96a8021020 con 0x7f96ac100760 2024-05-09T09:29:33.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.304+0000 7f96b419e640 1 -- 172.21.15.12:0/279587187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f96ac02d370 con 0x7f96ac100760 2024-05-09T09:29:33.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.304+0000 7f96a2ffd640 1 -- 172.21.15.12:0/279587187 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f96a800e350 con 0x7f96ac100760 2024-05-09T09:29:33.304 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T09:29:33.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.306+0000 7f96b419e640 1 -- 172.21.15.12:0/279587187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f96980a2b90 msgr2=0x7f96980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.306+0000 7f96b419e640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f96980a2b90 0x7f96980a5050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f969c005d80 tx=0x7f969c005c90 comp rx=0 tx=0).stop 2024-05-09T09:29:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.306+0000 7f96b419e640 1 -- 172.21.15.12:0/279587187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac100760 msgr2=0x7f96ac1969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.306+0000 7f96b419e640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac100760 0x7f96ac1969f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f96a800d950 tx=0x7f96a800de10 comp rx=0 tx=0).stop 2024-05-09T09:29:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.306+0000 7f96b419e640 1 -- 172.21.15.12:0/279587187 shutdown_connections 2024-05-09T09:29:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.306+0000 7f96b419e640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f96980a2b90 0x7f96980a5050 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.306+0000 7f96b419e640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96ac102310 0x7f96ac19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.306+0000 7f96b419e640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96ac101950 0x7f96ac196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.306+0000 7f96b419e640 1 --2- 172.21.15.12:0/279587187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac100760 0x7f96ac1969f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.306+0000 7f96b419e640 1 -- 172.21.15.12:0/279587187 >> 172.21.15.12:0/279587187 conn(0x7f96ac0fbcd0 msgr2=0x7f96ac104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.307+0000 7f96b419e640 1 -- 172.21.15.12:0/279587187 shutdown_connections 2024-05-09T09:29:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.307+0000 7f96b419e640 1 -- 172.21.15.12:0/279587187 wait complete. 2024-05-09T09:29:33.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:33 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/164540081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:29:33.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:33 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2960192445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:29:33.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:33 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/916059618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:29:33.318 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:29:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.427+0000 7f5e96b48640 1 -- 172.21.15.12:0/3910853278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e90101980 msgr2=0x7f5e90101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.427+0000 7f5e96b48640 1 --2- 172.21.15.12:0/3910853278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e90101980 0x7f5e90101e00 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009900 tx=0x7f5e8402a410 comp rx=0 tx=0).stop 2024-05-09T09:29:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.427+0000 7f5e96b48640 1 -- 172.21.15.12:0/3910853278 shutdown_connections 2024-05-09T09:29:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.427+0000 7f5e96b48640 1 --2- 172.21.15.12:0/3910853278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e90102340 0x7f5e90108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.427+0000 7f5e96b48640 1 --2- 172.21.15.12:0/3910853278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e90101980 0x7f5e90101e00 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.427+0000 7f5e96b48640 1 --2- 172.21.15.12:0/3910853278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90100790 0x7f5e90100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.427+0000 7f5e96b48640 1 -- 172.21.15.12:0/3910853278 >> 172.21.15.12:0/3910853278 conn(0x7f5e900fbd20 msgr2=0x7f5e900fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.427+0000 7f5e96b48640 1 -- 172.21.15.12:0/3910853278 shutdown_connections 2024-05-09T09:29:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.427+0000 7f5e96b48640 1 -- 172.21.15.12:0/3910853278 wait complete. 2024-05-09T09:29:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.427+0000 7f5e96b48640 1 Processor -- start 2024-05-09T09:29:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.427+0000 7f5e96b48640 1 -- start start 2024-05-09T09:29:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e96b48640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e90100790 0x7f5e901047f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e96b48640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e90101980 0x7f5e90102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e96b48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90102340 0x7f5e90103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e96b48640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9010c3f0 con 0x7f5e90100790 2024-05-09T09:29:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e96b48640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9010c270 con 0x7f5e90102340 2024-05-09T09:29:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e96b48640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9010c570 con 0x7f5e90101980 2024-05-09T09:29:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e948e4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e90100790 0x7f5e901047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e948e4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e90100790 0x7f5e901047f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34936/0 (socket says 172.21.15.12:34936) 2024-05-09T09:29:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e948e4640 1 -- 172.21.15.12:0/1866234820 learned_addr learned my addr 172.21.15.12:0/1866234820 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e8ffff640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e90101980 0x7f5e90102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e950e5640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90102340 0x7f5e90103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e948e4640 1 -- 172.21.15.12:0/1866234820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e90101980 msgr2=0x7f5e90102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e948e4640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e90101980 0x7f5e90102e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e948e4640 1 -- 172.21.15.12:0/1866234820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90102340 msgr2=0x7f5e90103370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e948e4640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90102340 0x7f5e90103370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e948e4640 1 -- 172.21.15.12:0/1866234820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e90103c00 con 0x7f5e90100790 2024-05-09T09:29:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e950e5640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90102340 0x7f5e90103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:29:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e948e4640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e90100790 0x7f5e901047f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000d950 tx=0x7f5e8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e8dffb640 1 -- 172.21.15.12:0/1866234820 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80017020 con 0x7f5e90100790 2024-05-09T09:29:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e8dffb640 1 -- 172.21.15.12:0/1866234820 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5e80009c30 con 0x7f5e90100790 2024-05-09T09:29:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.428+0000 7f5e96b48640 1 -- 172.21.15.12:0/1866234820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e901a1ce0 con 0x7f5e90100790 2024-05-09T09:29:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.429+0000 7f5e8dffb640 1 -- 172.21.15.12:0/1866234820 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8000f740 con 0x7f5e90100790 2024-05-09T09:29:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.429+0000 7f5e96b48640 1 -- 172.21.15.12:0/1866234820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e901a21b0 con 0x7f5e90100790 2024-05-09T09:29:33.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.430+0000 7f5e96b48640 1 -- 172.21.15.12:0/1866234820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e90068aa0 con 0x7f5e90100790 2024-05-09T09:29:33.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.431+0000 7f5e8dffb640 1 -- 172.21.15.12:0/1866234820 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5e8001a020 con 0x7f5e90100790 2024-05-09T09:29:33.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.432+0000 7f5e8dffb640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5e780a2b90 0x7f5e780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:33.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.432+0000 7f5e8ffff640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5e780a2b90 0x7f5e780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:33.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.432+0000 7f5e8dffb640 1 -- 172.21.15.12:0/1866234820 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f5e800cee70 con 0x7f5e90100790 2024-05-09T09:29:33.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.432+0000 7f5e8ffff640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5e780a2b90 0x7f5e780a5050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5e84000c00 tx=0x7f5e84031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.436+0000 7f5e8dffb640 1 -- 172.21.15.12:0/1866234820 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5e80021070 con 0x7f5e90100790 2024-05-09T09:29:33.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:33 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/164540081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:29:33.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:33 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2960192445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:29:33.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:33 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/916059618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:29:33.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.621+0000 7f5e96b48640 1 -- 172.21.15.12:0/1866234820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5e901a2380 con 0x7f5e90100790 2024-05-09T09:29:33.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.622+0000 7f5e8dffb640 1 -- 172.21.15.12:0/1866234820 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f5e80009de0 con 0x7f5e90100790 2024-05-09T09:29:33.622 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:29:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.624+0000 7f5e96b48640 1 -- 172.21.15.12:0/1866234820 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5e780a2b90 msgr2=0x7f5e780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.624+0000 7f5e96b48640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5e780a2b90 0x7f5e780a5050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5e84000c00 tx=0x7f5e84031510 comp rx=0 tx=0).stop 2024-05-09T09:29:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.624+0000 7f5e96b48640 1 -- 172.21.15.12:0/1866234820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e90100790 msgr2=0x7f5e901047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.624+0000 7f5e96b48640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e90100790 0x7f5e901047f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000d950 tx=0x7f5e8000de10 comp rx=0 tx=0).stop 2024-05-09T09:29:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.624+0000 7f5e96b48640 1 -- 172.21.15.12:0/1866234820 shutdown_connections 2024-05-09T09:29:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.624+0000 7f5e96b48640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5e780a2b90 0x7f5e780a5050 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.624+0000 7f5e96b48640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90102340 0x7f5e90103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.624+0000 7f5e96b48640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e90101980 0x7f5e90102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.624+0000 7f5e96b48640 1 --2- 172.21.15.12:0/1866234820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e90100790 0x7f5e901047f0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.624+0000 7f5e96b48640 1 -- 172.21.15.12:0/1866234820 >> 172.21.15.12:0/1866234820 conn(0x7f5e900fbd20 msgr2=0x7f5e90069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.624+0000 7f5e96b48640 1 -- 172.21.15.12:0/1866234820 shutdown_connections 2024-05-09T09:29:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.624+0000 7f5e96b48640 1 -- 172.21.15.12:0/1866234820 wait complete. 2024-05-09T09:29:33.635 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-09T09:29:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.741+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2370560244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d24100770 msgr2=0x7f7d24100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.741+0000 7f7d2a2a7640 1 --2- 172.21.15.12:0/2370560244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d24100770 0x7f7d24100b90 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009900 tx=0x7f7d1402a410 comp rx=0 tx=0).stop 2024-05-09T09:29:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.741+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2370560244 shutdown_connections 2024-05-09T09:29:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.741+0000 7f7d2a2a7640 1 --2- 172.21.15.12:0/2370560244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d24102320 0x7f7d24108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.741+0000 7f7d2a2a7640 1 --2- 172.21.15.12:0/2370560244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d24101960 0x7f7d24101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.741+0000 7f7d2a2a7640 1 --2- 172.21.15.12:0/2370560244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d24100770 0x7f7d24100b90 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.741+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2370560244 >> 172.21.15.12:0/2370560244 conn(0x7f7d240fbce0 msgr2=0x7f7d240fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.741+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2370560244 shutdown_connections 2024-05-09T09:29:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.741+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2370560244 wait complete. 2024-05-09T09:29:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.741+0000 7f7d2a2a7640 1 Processor -- start 2024-05-09T09:29:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d2a2a7640 1 -- start start 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d2a2a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d24100770 0x7f7d24197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d2a2a7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d24101960 0x7f7d24198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d2a2a7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d24102320 0x7f7d2419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d2a2a7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410c3d0 con 0x7f7d24102320 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d2a2a7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410c250 con 0x7f7d24100770 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d2a2a7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410c550 con 0x7f7d24101960 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d28844640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d24102320 0x7f7d2419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d28844640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d24102320 0x7f7d2419f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34944/0 (socket says 172.21.15.12:34944) 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d28844640 1 -- 172.21.15.12:0/2946094322 learned_addr learned my addr 172.21.15.12:0/2946094322 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d23fff640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d24100770 0x7f7d24197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d1bfff640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d24101960 0x7f7d24198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d28844640 1 -- 172.21.15.12:0/2946094322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d24101960 msgr2=0x7f7d24198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d28844640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d24101960 0x7f7d24198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d28844640 1 -- 172.21.15.12:0/2946094322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d24100770 msgr2=0x7f7d24197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d28844640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d24100770 0x7f7d24197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d28844640 1 -- 172.21.15.12:0/2946094322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2419f7f0 con 0x7f7d24102320 2024-05-09T09:29:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d28844640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d24102320 0x7f7d2419f180 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00eb10 tx=0x7f7d1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d21ffb640 1 -- 172.21.15.12:0/2946094322 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c004b20 con 0x7f7d24102320 2024-05-09T09:29:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2946094322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2419fae0 con 0x7f7d24102320 2024-05-09T09:29:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.742+0000 7f7d21ffb640 1 -- 172.21.15.12:0/2946094322 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7d1c004cc0 con 0x7f7d24102320 2024-05-09T09:29:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.743+0000 7f7d21ffb640 1 -- 172.21.15.12:0/2946094322 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c010700 con 0x7f7d24102320 2024-05-09T09:29:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.743+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2946094322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2419ffe0 con 0x7f7d24102320 2024-05-09T09:29:33.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:33 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/164540081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:29:33.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:33 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2960192445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:29:33.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:33 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/916059618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:29:33.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.744+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2946094322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d241a0270 con 0x7f7d24102320 2024-05-09T09:29:33.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.745+0000 7f7d21ffb640 1 -- 172.21.15.12:0/2946094322 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7d1c004e70 con 0x7f7d24102320 2024-05-09T09:29:33.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.746+0000 7f7d21ffb640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7d100a2b90 0x7f7d100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:29:33.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.746+0000 7f7d23fff640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7d100a2b90 0x7f7d100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:29:33.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.746+0000 7f7d21ffb640 1 -- 172.21.15.12:0/2946094322 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f7d1c0cfc30 con 0x7f7d24102320 2024-05-09T09:29:33.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.746+0000 7f7d23fff640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7d100a2b90 0x7f7d100a5050 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7d14006010 tx=0x7f7d14031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:29:33.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.749+0000 7f7d21ffb640 1 -- 172.21.15.12:0/2946094322 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7d1c053740 con 0x7f7d24102320 2024-05-09T09:29:33.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.985+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2946094322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f7d241a0440 con 0x7f7d24102320 2024-05-09T09:29:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.994+0000 7f7d21ffb640 1 -- 172.21.15.12:0/2946094322 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740370 (secure 0 0 0) 0x7f7d1c09c7c0 con 0x7f7d24102320 2024-05-09T09:29:34.004 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:29:34.004 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:29:34.004 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 18, 2024-05-09T09:29:34.004 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.004 INFO:teuthology.orchestra.run.smithi012.stdout: "active_gid": 14150, 2024-05-09T09:29:34.004 INFO:teuthology.orchestra.run.smithi012.stdout: "active_name": "x", 2024-05-09T09:29:34.004 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addrs": { 2024-05-09T09:29:34.004 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3809515815 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3809515815 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addr": "172.21.15.12:6801/3809515815", 2024-05-09T09:29:34.005 INFO:teuthology.orchestra.run.smithi012.stdout: "active_change": "2024-05-09T09:16:04.467042+0000", 2024-05-09T09:29:34.006 INFO:teuthology.orchestra.run.smithi012.stdout: "active_mgr_features": 4540701547738038271, 2024-05-09T09:29:34.006 INFO:teuthology.orchestra.run.smithi012.stdout: "available": true, 2024-05-09T09:29:34.006 INFO:teuthology.orchestra.run.smithi012.stdout: "standbys": [ 2024-05-09T09:29:34.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.006 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 24109, 2024-05-09T09:29:34.006 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "z", 2024-05-09T09:29:34.006 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540701547738038271, 2024-05-09T09:29:34.006 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-05-09T09:29:34.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.006 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-05-09T09:29:34.006 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.006 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.006 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.007 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:34.007 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:34.007 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.007 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.007 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.007 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:34.007 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.007 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.007 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.007 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T09:29:34.007 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.007 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.007 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.008 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.009 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.010 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.010 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.010 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.010 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.010 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.010 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.010 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.010 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.010 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.010 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.011 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.012 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-05-09T09:29:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-05-09T09:29:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-05-09T09:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-05-09T09:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-05-09T09:29:34.016 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.017 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.017 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.017 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-05-09T09:29:34.017 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-05-09T09:29:34.017 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.017 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.017 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.017 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-05-09T09:29:34.017 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.017 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.017 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.017 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.018 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-05-09T09:29:34.019 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.019 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.019 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-05-09T09:29:34.019 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-05-09T09:29:34.019 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.019 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.019 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.019 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.019 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.019 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.020 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.021 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.021 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-05-09T09:29:34.021 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.021 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.021 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-05-09T09:29:34.021 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.022 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.022 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.022 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-05-09T09:29:34.022 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-05-09T09:29:34.022 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.022 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.022 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.022 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.022 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.022 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.022 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.022 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.023 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T09:29:34.024 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T09:29:34.024 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.024 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.024 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-05-09T09:29:34.024 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-05-09T09:29:34.024 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.024 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.024 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:34.024 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-09T09:29:34.024 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-09T09:29:34.024 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.025 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T09:29:34.025 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T09:29:34.025 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.025 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.025 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-05-09T09:29:34.025 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-05-09T09:29:34.025 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.025 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.025 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.025 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-05-09T09:29:34.025 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T09:29:34.025 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.026 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-05-09T09:29:34.027 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.028 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-09T09:29:34.029 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-09T09:29:34.030 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.030 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T09:29:34.030 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T09:29:34.030 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.030 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.030 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.030 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.030 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.030 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.030 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.030 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.031 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.032 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.032 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.032 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.032 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.032 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.032 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.032 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.032 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.032 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.032 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.032 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.032 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.033 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.034 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.034 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.034 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.034 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.034 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.034 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.034 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.034 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.034 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.034 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.034 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.034 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.035 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.036 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.037 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.038 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-05-09T09:29:34.038 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-05-09T09:29:34.038 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.038 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.038 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.038 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.038 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.038 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.038 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.038 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T09:29:34.038 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.038 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.039 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.040 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.040 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-05-09T09:29:34.040 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-05-09T09:29:34.040 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.040 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.040 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.040 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T09:29:34.040 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T09:29:34.040 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.040 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.040 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T09:29:34.040 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T09:29:34.041 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.042 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.043 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.044 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.045 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.045 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-05-09T09:29:34.045 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.045 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.045 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.045 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T09:29:34.045 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.045 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.045 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.045 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-05-09T09:29:34.045 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-05-09T09:29:34.046 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.046 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.046 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.046 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.046 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.046 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.046 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.046 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T09:29:34.046 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T09:29:34.046 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.046 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.046 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.047 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-05-09T09:29:34.048 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T09:29:34.048 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.048 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.048 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.048 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-05-09T09:29:34.048 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.048 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.048 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.048 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T09:29:34.048 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.048 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T09:29:34.049 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.050 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.050 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.050 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-05-09T09:29:34.050 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-05-09T09:29:34.050 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.050 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.050 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.050 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.050 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.050 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.050 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.050 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T09:29:34.051 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T09:29:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T09:29:34.053 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-05-09T09:29:34.054 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-05-09T09:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T09:29:34.056 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.057 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.057 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.057 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-05-09T09:29:34.057 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-05-09T09:29:34.057 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.057 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.057 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.057 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T09:29:34.057 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.057 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.057 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.057 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-05-09T09:29:34.058 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.058 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.058 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.058 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-05-09T09:29:34.058 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-05-09T09:29:34.058 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.058 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.058 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.058 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T09:29:34.058 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.058 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.058 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.059 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.061 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-05-09T09:29:34.061 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.061 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.061 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.061 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-05-09T09:29:34.061 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-05-09T09:29:34.061 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.061 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.061 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.061 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T09:29:34.061 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.061 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.062 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.063 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.064 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.065 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.066 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.067 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.068 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.069 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-05-09T09:29:34.069 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.069 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.069 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.069 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-05-09T09:29:34.069 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-05-09T09:29:34.069 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.069 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.069 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.069 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.069 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.069 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.070 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.071 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.073 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.074 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.075 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.076 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.077 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.078 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.079 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.080 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.081 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.081 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.081 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.081 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.081 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.081 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.081 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.081 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.081 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.081 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.081 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.081 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-05-09T09:29:34.082 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-05-09T09:29:34.082 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.082 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.082 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.082 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.082 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.082 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.082 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.082 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T09:29:34.082 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.082 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.082 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.083 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.084 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.085 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.086 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.086 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.086 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.086 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.086 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.086 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.086 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T09:29:34.086 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T09:29:34.086 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.086 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.086 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.086 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.087 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-05-09T09:29:34.088 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.088 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.088 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.088 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T09:29:34.088 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.088 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.088 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.088 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.089 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-05-09T09:29:34.090 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-05-09T09:29:34.090 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.090 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.090 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.090 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.090 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.090 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.090 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.090 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T09:29:34.090 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.090 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.090 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.091 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-05-09T09:29:34.091 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-05-09T09:29:34.091 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.091 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.091 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.091 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.091 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.091 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.091 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.091 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-05-09T09:29:34.091 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.091 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-05-09T09:29:34.092 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.093 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.094 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T09:29:34.095 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.095 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.095 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.095 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-05-09T09:29:34.095 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-05-09T09:29:34.095 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.095 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.095 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.095 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.095 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.095 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.096 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.097 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.098 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.099 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.099 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T09:29:34.099 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.099 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.099 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.099 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-05-09T09:29:34.099 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-05-09T09:29:34.099 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.099 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.099 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.099 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.099 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-05-09T09:29:34.100 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.101 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.102 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.103 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.103 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.103 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T09:29:34.103 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.103 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.103 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.104 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.105 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.106 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-05-09T09:29:34.106 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-05-09T09:29:34.106 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.106 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.106 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.106 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.106 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.106 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.106 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.106 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T09:29:34.106 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.106 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.107 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.108 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.108 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.108 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-05-09T09:29:34.108 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.108 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.108 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.108 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.108 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.108 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.108 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.109 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.110 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.110 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.110 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.110 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.110 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.110 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.110 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.110 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.110 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.110 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.110 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.110 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.111 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.112 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.113 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.114 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-05-09T09:29:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.116 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.117 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.118 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.119 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-05-09T09:29:34.119 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T09:29:34.119 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.119 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.119 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.119 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.119 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.119 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.119 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.119 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.119 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.119 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.121 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T09:29:34.121 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T09:29:34.121 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.121 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.121 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.121 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.121 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.121 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.121 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.121 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.121 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.122 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.122 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.122 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T09:29:34.122 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T09:29:34.122 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.122 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.122 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.122 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.122 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.122 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.122 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.122 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.123 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.124 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.125 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.126 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T09:29:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T09:29:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.128 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.129 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.130 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.132 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.133 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.134 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.135 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T09:29:34.135 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T09:29:34.135 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.135 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.135 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.135 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.135 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.135 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.135 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.135 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.135 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.135 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.136 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.137 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.138 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T09:29:34.138 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T09:29:34.138 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.138 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.138 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.138 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.138 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.138 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.138 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.138 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.138 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.138 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.139 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.140 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.141 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.142 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.145 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.146 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T09:29:34.147 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T09:29:34.147 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.147 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.147 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.147 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.147 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.147 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.147 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.147 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.147 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.147 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.147 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.153 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.154 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.155 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T09:29:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T09:29:34.157 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-05-09T09:29:34.158 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-05-09T09:29:34.159 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.160 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.160 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.160 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.160 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.160 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.160 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.160 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.160 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.160 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.160 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.160 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-05-09T09:29:34.160 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-05-09T09:29:34.161 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-09T09:29:34.162 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.164 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.165 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.166 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.167 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.168 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.168 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.168 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.168 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.168 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.168 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.168 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.168 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.168 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.169 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.169 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-05-09T09:29:34.169 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-05-09T09:29:34.169 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.169 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.169 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.169 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.170 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.171 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.171 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.171 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.171 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.171 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.171 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.171 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.171 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T09:29:34.171 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T09:29:34.171 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.171 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.172 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.173 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.174 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.175 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.175 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-05-09T09:29:34.175 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.175 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.175 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.175 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.175 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.175 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.175 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.175 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-09T09:29:34.175 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.176 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.177 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.178 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-05-09T09:29:34.178 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-05-09T09:29:34.178 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.178 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.178 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.178 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.178 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.178 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.178 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.178 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.178 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.178 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.179 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.180 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.181 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.182 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.182 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.182 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.182 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.182 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.182 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.182 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.182 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.182 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.182 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.182 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.182 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.183 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.184 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-05-09T09:29:34.185 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-05-09T09:29:34.186 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-05-09T09:29:34.187 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.188 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.188 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.188 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-05-09T09:29:34.188 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.188 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.188 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.188 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-05-09T09:29:34.188 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-05-09T09:29:34.188 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.189 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.189 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.189 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-09T09:29:34.189 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.189 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.189 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.189 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T09:29:34.189 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.189 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.189 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.189 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:34.191 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:34.191 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.191 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.191 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.191 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:34.191 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.191 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.191 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.191 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T09:29:34.191 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.191 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.192 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.192 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-05-09T09:29:34.192 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-05-09T09:29:34.192 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.192 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.192 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.192 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-05-09T09:29:34.192 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.192 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.192 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.192 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T09:29:34.192 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.193 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.194 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.195 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.196 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.198 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.199 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.200 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-05-09T09:29:34.201 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.203 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-05-09T09:29:34.203 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-05-09T09:29:34.203 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.203 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.203 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.203 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-05-09T09:29:34.203 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.203 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.203 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.203 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T09:29:34.203 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.203 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.204 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.205 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.206 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.206 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.206 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.206 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.206 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.206 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.206 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.206 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.206 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.206 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.207 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.208 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.209 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.210 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.211 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.216 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-05-09T09:29:34.216 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-05-09T09:29:34.216 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.216 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.216 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-05-09T09:29:34.216 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.216 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.216 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.216 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T09:29:34.216 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.216 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.216 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.217 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.218 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.219 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.219 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.219 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.219 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.219 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.219 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.219 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.219 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.219 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.219 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.219 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.219 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.220 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.221 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-05-09T09:29:34.222 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.223 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.224 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.225 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.226 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.227 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-05-09T09:29:34.228 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.229 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-05-09T09:29:34.230 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.230 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.230 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T09:29:34.230 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.230 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.230 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.230 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.230 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.230 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.230 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.230 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.230 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.231 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.231 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.231 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.231 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.231 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.231 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.231 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.231 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.231 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.231 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.231 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.231 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.231 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.232 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.233 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.234 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.234 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.234 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.234 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.234 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.234 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.234 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.234 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.234 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.234 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.234 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.234 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-05-09T09:29:34.235 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.236 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.237 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.238 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.239 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-05-09T09:29:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-05-09T09:29:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-05-09T09:29:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-05-09T09:29:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-05-09T09:29:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-05-09T09:29:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-05-09T09:29:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T09:29:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T09:29:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-05-09T09:29:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.257 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.257 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.257 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.257 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.257 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.257 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.257 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.257 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.257 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.257 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.257 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.257 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.258 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.259 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.260 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.261 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.262 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.262 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.262 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.262 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.262 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.262 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.262 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.262 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.262 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.262 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.262 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.262 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.263 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T09:29:34.264 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.265 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.266 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.268 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.269 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-05-09T09:29:34.270 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.271 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-05-09T09:29:34.271 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.271 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.271 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.271 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.271 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-05-09T09:29:34.271 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.271 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.271 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.271 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.272 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.273 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.274 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.275 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.276 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.277 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.278 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.279 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.280 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.281 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.282 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.283 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.284 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.285 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.285 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.285 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.285 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.285 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.285 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.285 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.285 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.285 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.285 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.286 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:34.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-05-09T09:29:34.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.290 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-05-09T09:29:34.290 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.290 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.290 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.290 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-05-09T09:29:34.290 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-05-09T09:29:34.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.290 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.290 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.290 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.290 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.290 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.290 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.292 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.293 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.295 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.296 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-05-09T09:29:34.297 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.298 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.298 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.298 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:34.298 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:34.298 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.298 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.298 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.298 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T09:29:34.298 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.298 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.298 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.298 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-05-09T09:29:34.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-05-09T09:29:34.300 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.301 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.302 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.304 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.305 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.306 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.306 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.306 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.306 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.306 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.306 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.306 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.306 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.306 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.306 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-05-09T09:29:34.306 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-05-09T09:29:34.306 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.307 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.308 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-05-09T09:29:34.308 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.308 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.308 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.308 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.308 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.308 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.308 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.308 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-05-09T09:29:34.308 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T09:29:34.309 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T09:29:34.310 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-05-09T09:29:34.311 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-05-09T09:29:34.312 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.312 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.312 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.312 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-05-09T09:29:34.312 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.312 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.312 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.312 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.312 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.312 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.312 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.312 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.313 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.314 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.314 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-09T09:29:34.314 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-09T09:29:34.314 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.314 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.314 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.314 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-09T09:29:34.314 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-09T09:29:34.314 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-09T09:29:34.314 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.314 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.314 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.315 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.316 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.317 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.318 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.319 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.320 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.321 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.322 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-05-09T09:29:34.323 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.324 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.325 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-09T09:29:34.325 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-09T09:29:34.325 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.325 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.325 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.325 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.325 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.325 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.325 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.325 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.325 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.326 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.326 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.326 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.326 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.326 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.326 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.326 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.326 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.326 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.326 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.329 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.329 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.329 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.329 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.329 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.329 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.329 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.329 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.329 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.329 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.329 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.329 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.329 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.331 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.332 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.333 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.334 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.334 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.334 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.334 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.334 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.334 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.334 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.334 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.334 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.334 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.334 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.334 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.334 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.335 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.336 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.338 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.339 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.342 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.342 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.342 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.342 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.342 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.342 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.342 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.342 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.342 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.342 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.342 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.342 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.343 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.344 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-05-09T09:29:34.345 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.345 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.345 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.345 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T09:29:34.345 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.345 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.345 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.345 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.345 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.345 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-05-09T09:29:34.345 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.346 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.346 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.346 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.346 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.346 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.346 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.346 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.346 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.346 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.346 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.346 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.346 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.346 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.347 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.348 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.349 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.350 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.350 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.350 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.350 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.350 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.350 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.350 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.350 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.350 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.350 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.350 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.350 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.351 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.352 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.353 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.354 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.354 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-05-09T09:29:34.354 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-05-09T09:29:34.354 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.354 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.354 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.354 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.354 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.354 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.354 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.354 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.354 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.355 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.356 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.357 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.357 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.357 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-05-09T09:29:34.357 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-05-09T09:29:34.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.357 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.357 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.357 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.357 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.357 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.357 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.358 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.358 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.358 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.358 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.358 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-05-09T09:29:34.358 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-05-09T09:29:34.358 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.358 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.358 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.359 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.359 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.359 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.359 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.359 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.359 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.359 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.359 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.359 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-05-09T09:29:34.360 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-05-09T09:29:34.360 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.360 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.360 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.360 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.360 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T09:29:34.360 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-05-09T09:29:34.360 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.360 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.360 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.360 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.361 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.361 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-05-09T09:29:34.361 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-05-09T09:29:34.361 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.361 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.361 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.361 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.361 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.361 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.361 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.362 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.362 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.362 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.362 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-05-09T09:29:34.362 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-05-09T09:29:34.362 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.362 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.362 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.362 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.362 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.362 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.363 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.363 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.363 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.363 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.363 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.363 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-05-09T09:29:34.363 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-05-09T09:29:34.363 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.363 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.363 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.363 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.363 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.364 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.365 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.366 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.367 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.368 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.369 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.369 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.369 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.369 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.369 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.369 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.369 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.369 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.369 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.369 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.369 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.369 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.370 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.370 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.370 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.370 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.370 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.370 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.370 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.370 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.370 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.370 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.370 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.370 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.371 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.371 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.373 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.373 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.373 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.373 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.373 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.373 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.373 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.373 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.373 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.374 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.374 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.374 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.374 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.374 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-05-09T09:29:34.374 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.374 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.374 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.374 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.374 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.374 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.375 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.377 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.379 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.379 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.379 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.379 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.379 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.379 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.379 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.379 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.379 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.379 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.379 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.379 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.380 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.381 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.382 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.383 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.384 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-05-09T09:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-05-09T09:29:34.386 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.386 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.386 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.386 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T09:29:34.386 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.386 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.386 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.386 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.386 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.386 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.386 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.387 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:34.387 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:34.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.387 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.387 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.387 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-05-09T09:29:34.387 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.387 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.387 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.387 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.387 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.387 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.388 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.389 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.389 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.389 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.389 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.389 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.389 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.389 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.389 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.389 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.389 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.389 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.390 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.390 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.390 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.390 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.390 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.390 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.390 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.390 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.390 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.390 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.390 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.390 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.391 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.391 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.391 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.391 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.391 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.391 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.391 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.391 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.391 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.391 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.392 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.392 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.392 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.392 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.392 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.392 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.392 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.392 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.392 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.392 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.392 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.392 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.394 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.394 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-05-09T09:29:34.394 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-05-09T09:29:34.394 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.394 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.394 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.394 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.394 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.394 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.394 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.394 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T09:29:34.394 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.394 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.395 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.396 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.397 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.397 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-05-09T09:29:34.397 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-05-09T09:29:34.397 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.397 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.397 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.397 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.397 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.397 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.397 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.397 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T09:29:34.397 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T09:29:34.398 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.399 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.399 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.399 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-05-09T09:29:34.399 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-05-09T09:29:34.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.399 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.399 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.399 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.399 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.399 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.399 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.400 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.400 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.400 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.400 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-05-09T09:29:34.400 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-05-09T09:29:34.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.400 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.400 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.400 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.401 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.401 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.401 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.401 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.401 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.401 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.401 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.401 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-05-09T09:29:34.401 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-05-09T09:29:34.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.401 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.402 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.402 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T09:29:34.402 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.402 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.402 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.402 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.402 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.402 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.402 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.402 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-09T09:29:34.402 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-09T09:29:34.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:34.403 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-05-09T09:29:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-05-09T09:29:34.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-05-09T09:29:34.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.408 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.409 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.410 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.410 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.410 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.410 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.410 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.410 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.410 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.410 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.410 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.410 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.410 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.411 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-05-09T09:29:34.412 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-05-09T09:29:34.413 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-05-09T09:29:34.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.414 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.414 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.414 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.414 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.414 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.414 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.414 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.414 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.414 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.415 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.416 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.416 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-05-09T09:29:34.416 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.416 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.416 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.416 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.416 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.416 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.416 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.416 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.417 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.418 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.418 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.418 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.418 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.418 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.418 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.418 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.418 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.418 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.418 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.419 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.420 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-05-09T09:29:34.421 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.422 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.423 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.425 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.425 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.425 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.425 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.425 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.425 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.425 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.425 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.425 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.425 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.425 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.425 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.425 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.426 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.427 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.428 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.428 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-05-09T09:29:34.428 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-05-09T09:29:34.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.428 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.428 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.428 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.428 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.428 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.428 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-05-09T09:29:34.428 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.429 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.429 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.429 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-05-09T09:29:34.429 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-05-09T09:29:34.429 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.429 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.429 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.429 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:34.429 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.429 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.429 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.429 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T09:29:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-05-09T09:29:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T09:29:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-05-09T09:29:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-05-09T09:29:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.436 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.437 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.439 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.440 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.441 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.442 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T09:29:34.442 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.442 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.442 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.442 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.442 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.442 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.442 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.442 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.442 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.442 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.442 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 24113, 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "y", 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540701547738038271, 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.443 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.444 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.445 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.447 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.452 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.453 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.454 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.455 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.456 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.456 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T09:29:34.456 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.456 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.456 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.456 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-05-09T09:29:34.456 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-05-09T09:29:34.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.456 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.456 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.457 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.458 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.458 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-05-09T09:29:34.458 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-05-09T09:29:34.458 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.458 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.458 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.458 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.458 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.458 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.458 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.458 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T09:29:34.458 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.458 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T09:29:34.459 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T09:29:34.460 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.461 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T09:29:34.462 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T09:29:34.463 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T09:29:34.464 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T09:29:34.465 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.466 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.466 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.466 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.466 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.466 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.466 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.466 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.466 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.466 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.466 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.466 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.466 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.468 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.468 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.468 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.468 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.468 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.468 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.468 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.468 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.468 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.468 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.468 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.468 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.469 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.469 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.469 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.469 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.469 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.469 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.469 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.469 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.469 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.469 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.469 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.469 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.469 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.470 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.470 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.470 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.470 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.470 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.470 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.470 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.470 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.470 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.470 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.471 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.472 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-05-09T09:29:34.473 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-05-09T09:29:34.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.474 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.474 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.474 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.474 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.474 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.474 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T09:29:34.474 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.474 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.474 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.474 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.475 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.476 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.477 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.478 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.479 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.481 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T09:29:34.481 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.481 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.481 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.481 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-05-09T09:29:34.481 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-05-09T09:29:34.481 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.481 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.481 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.481 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.481 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.482 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.482 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T09:29:34.482 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T09:29:34.482 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.482 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.482 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-05-09T09:29:34.482 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-05-09T09:29:34.482 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.482 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.482 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.482 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:34.482 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-05-09T09:29:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.484 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-05-09T09:29:34.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-05-09T09:29:34.486 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-05-09T09:29:34.487 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.488 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.489 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.490 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.491 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.492 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.493 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T09:29:34.493 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.493 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.493 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.493 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-05-09T09:29:34.493 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.493 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.493 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.493 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-05-09T09:29:34.493 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-05-09T09:29:34.493 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.493 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.494 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.495 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.496 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.496 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T09:29:34.496 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.496 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.496 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.496 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-05-09T09:29:34.496 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.496 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.496 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.496 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-05-09T09:29:34.496 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-05-09T09:29:34.496 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-05-09T09:29:34.497 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.498 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.498 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.498 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T09:29:34.498 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.498 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.498 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.498 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-05-09T09:29:34.498 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.498 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.498 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.498 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-05-09T09:29:34.498 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-05-09T09:29:34.499 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-05-09T09:29:34.500 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.501 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.502 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.502 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T09:29:34.502 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.502 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.502 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.502 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-05-09T09:29:34.502 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.502 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.502 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.502 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-05-09T09:29:34.502 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-05-09T09:29:34.503 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-05-09T09:29:34.504 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.505 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.506 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.506 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:34.506 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.506 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.506 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.506 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T09:29:34.506 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.506 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.506 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-05-09T09:29:34.506 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-05-09T09:29:34.507 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-05-09T09:29:34.508 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-05-09T09:29:34.509 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-05-09T09:29:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-05-09T09:29:34.511 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-05-09T09:29:34.512 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-05-09T09:29:34.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.513 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.513 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.513 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:34.513 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.513 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.513 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.513 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-05-09T09:29:34.513 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.513 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.513 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.514 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.515 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.516 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.517 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T09:29:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.519 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.520 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.521 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T09:29:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T09:29:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T09:29:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.523 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-05-09T09:29:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-05-09T09:29:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-05-09T09:29:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.524 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-05-09T09:29:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-05-09T09:29:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-05-09T09:29:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.526 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T09:29:34.527 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T09:29:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.529 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.530 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.530 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-05-09T09:29:34.530 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-05-09T09:29:34.530 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.530 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.530 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.530 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.530 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.530 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.530 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.530 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T09:29:34.530 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.531 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.531 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.531 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-05-09T09:29:34.531 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-05-09T09:29:34.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.531 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.531 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.531 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.531 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.531 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.531 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.532 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.533 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.534 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.535 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T09:29:34.536 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T09:29:34.537 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T09:29:34.538 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T09:29:34.539 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T09:29:34.540 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.542 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.543 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.543 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.543 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.543 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.543 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.543 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.543 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.543 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.543 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.543 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.543 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.544 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.545 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.546 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.546 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.546 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.546 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.546 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.546 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.546 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.546 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.546 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.546 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.546 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.546 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.547 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-05-09T09:29:34.548 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-05-09T09:29:34.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.548 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.548 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.548 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-05-09T09:29:34.548 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.548 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.548 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.548 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T09:29:34.548 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.548 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.549 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.550 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.551 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.552 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.553 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.554 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.555 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.556 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.557 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.557 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.557 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.557 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.557 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T09:29:34.557 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T09:29:34.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.557 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.557 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.557 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.557 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.557 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.558 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.558 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.558 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.558 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.558 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.558 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T09:29:34.558 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T09:29:34.558 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.558 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.558 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.558 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.558 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.559 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.560 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.561 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.562 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.562 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.562 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.562 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.562 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.562 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T09:29:34.562 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T09:29:34.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.562 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.562 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.562 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.563 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.564 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.565 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.566 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T09:29:34.567 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.569 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.570 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.570 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.570 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.570 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.570 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.570 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.570 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T09:29:34.570 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T09:29:34.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.570 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.570 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.571 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.571 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.571 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.571 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.571 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.571 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.571 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T09:29:34.571 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T09:29:34.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.571 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.572 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.573 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.574 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.574 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.574 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.574 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.574 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.574 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.574 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.574 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.574 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T09:29:34.574 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T09:29:34.574 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.574 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.575 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.576 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.577 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.578 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.579 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.580 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.581 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.581 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.581 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.581 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.581 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.581 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.581 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.581 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.581 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T09:29:34.581 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T09:29:34.581 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.581 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.582 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.583 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.583 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T09:29:34.583 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.583 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.583 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.583 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.583 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.583 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.583 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T09:29:34.583 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T09:29:34.583 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T09:29:34.584 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.585 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.586 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.588 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.589 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.590 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.591 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.593 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.594 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.594 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.594 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.594 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.594 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.594 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.594 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.594 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-05-09T09:29:34.594 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-05-09T09:29:34.594 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.594 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.594 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.595 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.596 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.596 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.596 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.596 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-05-09T09:29:34.596 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.596 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.596 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.596 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-05-09T09:29:34.596 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-05-09T09:29:34.596 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.596 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.596 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.597 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.598 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.598 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.598 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.598 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.598 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.598 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.598 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.598 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.598 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.598 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.599 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.600 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.601 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.601 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.601 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.601 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.601 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.601 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.601 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.601 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.601 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.601 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.601 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.602 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-05-09T09:29:34.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T09:29:34.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T09:29:34.605 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T09:29:34.606 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-09T09:29:34.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-05-09T09:29:34.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.609 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-05-09T09:29:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-05-09T09:29:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.611 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.612 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.613 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.614 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.615 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.616 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.617 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.618 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-05-09T09:29:34.619 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-05-09T09:29:34.620 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-05-09T09:29:34.621 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-09T09:29:34.622 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.623 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.623 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.623 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T09:29:34.623 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.623 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.623 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.623 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-05-09T09:29:34.623 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-05-09T09:29:34.623 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.623 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.623 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.623 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.624 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.625 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.626 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.627 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.628 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.629 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.630 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.630 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.630 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.630 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.630 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.630 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.630 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.630 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.630 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.630 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.630 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.630 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.630 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-05-09T09:29:34.635 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-05-09T09:29:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.637 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.637 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.637 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T09:29:34.637 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.637 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.637 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.637 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-05-09T09:29:34.637 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-05-09T09:29:34.637 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.637 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.637 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.637 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.638 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.638 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.638 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.638 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-05-09T09:29:34.638 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.638 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.638 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.638 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:34.638 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:34.638 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.638 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.638 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.639 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.640 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.640 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.640 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.640 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.640 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.640 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.640 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.640 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.640 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.640 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.640 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.640 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.641 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.641 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.641 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.641 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.641 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.641 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.641 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.641 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.641 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.641 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.641 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.641 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.642 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.643 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-05-09T09:29:34.644 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-05-09T09:29:34.645 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.645 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.645 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.645 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.645 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.645 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.645 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.645 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-05-09T09:29:34.645 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.645 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.645 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.645 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.646 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.647 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.648 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-05-09T09:29:34.649 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-05-09T09:29:34.650 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.650 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.650 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.650 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-05-09T09:29:34.650 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.650 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.650 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.650 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T09:29:34.650 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.650 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.650 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.650 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.651 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.654 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.655 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.655 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.655 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.655 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.655 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.655 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.655 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.655 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.655 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.655 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.655 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.655 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.656 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.657 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.658 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.658 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.658 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.658 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.658 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.658 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.658 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.658 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.658 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.658 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.658 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.658 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.659 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.660 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.660 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.660 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.660 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.660 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.660 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.660 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.660 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.660 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.660 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.660 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.660 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.660 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.662 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.664 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.665 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.665 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.665 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.665 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.665 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.665 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.665 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.665 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.665 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.665 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.665 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.665 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.665 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.666 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.667 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.668 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.669 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-05-09T09:29:34.670 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.671 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.672 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.673 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.674 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.674 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.674 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.674 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.674 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.674 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.674 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.674 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.674 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.674 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.674 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.674 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.674 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T09:29:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-05-09T09:29:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-05-09T09:29:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-05-09T09:29:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.679 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-05-09T09:29:34.680 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.681 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.682 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.683 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.684 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.686 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.686 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.686 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.686 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.686 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.686 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.686 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.686 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.686 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.686 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.686 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.686 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.686 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.687 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.690 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.691 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.692 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.693 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.693 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.693 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.693 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.693 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.693 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.693 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.693 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.693 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.693 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.693 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.693 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.694 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.695 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.696 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.697 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-09T09:29:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-05-09T09:29:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-05-09T09:29:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.699 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.700 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.701 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.702 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.704 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.705 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-05-09T09:29:34.706 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.706 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.706 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.706 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.706 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.706 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.706 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.706 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.706 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.706 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.706 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.706 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.707 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.708 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.708 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.708 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.708 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.708 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.708 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.708 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.708 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.708 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.708 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.708 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.708 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.709 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.710 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.711 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.711 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.711 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.711 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.711 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.711 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.711 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-05-09T09:29:34.711 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.711 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.711 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.711 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.712 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.713 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.714 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.717 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.718 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.720 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.720 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.720 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.720 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.720 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.720 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.720 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.720 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.720 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.720 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.720 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.720 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.720 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.721 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.722 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.723 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.723 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.723 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-05-09T09:29:34.723 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-05-09T09:29:34.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.723 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.723 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.723 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-09T09:29:34.723 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-05-09T09:29:34.723 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.723 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-05-09T09:29:34.724 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-09T09:29:34.725 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.726 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.729 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-05-09T09:29:34.731 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:34.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.733 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-05-09T09:29:34.734 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.734 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.734 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.734 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-05-09T09:29:34.734 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-05-09T09:29:34.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.734 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.734 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.734 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.734 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.734 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.734 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.735 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.736 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.737 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.738 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.738 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.738 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.738 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.738 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.738 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.738 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.738 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.738 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.738 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.739 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.740 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.741 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.742 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.743 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.744 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.745 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.746 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.747 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.747 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.747 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-09T09:29:34.747 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-09T09:29:34.747 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.747 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.747 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.747 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-05-09T09:29:34.747 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.747 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.747 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.748 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.749 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.750 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.750 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.750 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.750 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.750 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.750 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.750 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.750 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.750 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.750 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.750 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.750 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.751 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.752 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.753 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.755 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.756 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.757 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.759 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.761 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.761 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.761 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.761 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.761 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.761 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.761 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.761 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.761 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.761 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.761 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.761 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.761 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.763 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.764 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.765 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.766 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-05-09T09:29:34.767 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.768 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.769 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.770 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.771 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.773 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.775 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.775 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.775 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.775 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.775 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.775 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.775 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.775 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.775 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.775 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.775 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.776 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.778 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.778 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.778 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.778 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.778 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.778 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.778 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-05-09T09:29:34.778 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-05-09T09:29:34.778 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.778 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.778 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.778 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.779 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.780 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.781 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.781 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.781 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.781 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.781 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.781 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.781 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.781 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.781 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.781 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.781 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.781 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.782 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.783 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.784 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.785 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.785 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.785 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.785 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.785 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.785 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.785 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.786 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.786 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.786 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.786 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.786 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.786 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.786 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-05-09T09:29:34.786 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-05-09T09:29:34.786 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.786 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.786 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.786 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-05-09T09:29:34.787 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.788 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.790 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.790 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.790 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.790 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.790 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.790 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-05-09T09:29:34.790 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-05-09T09:29:34.790 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.790 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.790 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.790 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.790 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.791 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.792 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.793 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.794 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.794 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.794 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.794 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.794 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.794 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-05-09T09:29:34.794 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-05-09T09:29:34.794 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.794 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.794 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.795 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.796 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.797 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.798 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.801 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.802 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.803 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.804 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.804 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.804 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.804 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.804 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.804 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.804 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.804 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.804 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.804 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.804 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.804 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.805 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.805 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.805 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.805 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.805 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.805 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.805 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.805 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.805 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.805 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.805 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.805 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.805 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.806 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.807 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.808 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.809 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.810 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.811 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.812 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.813 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.814 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.816 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T09:29:34.817 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.818 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.818 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.818 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.818 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.818 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.818 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.818 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:34.818 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:34.818 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.818 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.818 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.818 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.819 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.820 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.821 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.822 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.823 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.824 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.825 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.826 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.827 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.829 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-05-09T09:29:34.830 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-05-09T09:29:34.830 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.830 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.830 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.830 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.830 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.830 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.830 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.830 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.830 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.830 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.830 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.831 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-05-09T09:29:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-05-09T09:29:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T09:29:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.834 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:34.834 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:34.834 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.834 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.834 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.834 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-05-09T09:29:34.834 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-05-09T09:29:34.834 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.834 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.834 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.834 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.834 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.834 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.837 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.838 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.839 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.840 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.845 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.846 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.846 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.846 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-05-09T09:29:34.846 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.846 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.846 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.846 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.846 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.846 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.846 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.846 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.846 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.847 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.848 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.848 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.848 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.848 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.848 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.848 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.848 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.848 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.848 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.848 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.848 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.848 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.852 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.853 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.853 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.853 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.853 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.853 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.853 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.853 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.853 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.853 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.853 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.853 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.853 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.854 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.855 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.856 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.857 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.858 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.859 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.860 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-05-09T09:29:34.861 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-05-09T09:29:34.862 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.862 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.862 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.862 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-05-09T09:29:34.862 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.862 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.862 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.862 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.862 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.862 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.862 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.862 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.863 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.864 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.865 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.866 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.867 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.868 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.868 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.868 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.868 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.868 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.868 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.868 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.868 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.868 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.868 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.868 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.868 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.869 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.869 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.869 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.869 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.869 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.869 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.869 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.869 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.869 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.869 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.869 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.869 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.870 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-05-09T09:29:34.870 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-05-09T09:29:34.870 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.870 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.870 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.870 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.870 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.870 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.870 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.871 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.871 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.871 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.871 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.871 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-05-09T09:29:34.871 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-05-09T09:29:34.871 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.871 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.871 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.871 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-05-09T09:29:34.871 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.871 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.872 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.872 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.872 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.872 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.872 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.872 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-05-09T09:29:34.873 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-05-09T09:29:34.873 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.873 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.873 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.873 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T09:29:34.873 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.873 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.873 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.874 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.874 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.874 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.874 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.874 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.874 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.874 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.874 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T09:29:34.874 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.874 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.874 INFO:teuthology.orchestra.run.smithi012.stdout: "modules": [ 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard", 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: "iostat", 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: "nfs", 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: "restful" 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:34.875 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.876 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.876 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.876 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:34.876 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.876 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.876 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.876 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T09:29:34.876 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.876 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.876 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.876 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.877 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.878 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.879 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.880 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.881 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-05-09T09:29:34.882 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-05-09T09:29:34.882 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.882 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.882 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.882 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.882 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.882 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.882 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.882 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-05-09T09:29:34.882 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.882 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.882 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.885 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-05-09T09:29:34.886 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-05-09T09:29:34.886 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.886 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.886 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.886 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-05-09T09:29:34.886 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.886 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.886 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.886 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-05-09T09:29:34.886 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.886 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.886 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.887 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-05-09T09:29:34.887 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-05-09T09:29:34.887 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.887 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.887 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.887 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.887 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.887 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.887 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.887 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-05-09T09:29:34.887 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.887 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.887 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.888 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.889 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.890 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.891 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-05-09T09:29:34.892 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.892 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.892 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.892 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T09:29:34.892 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T09:29:34.892 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.892 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.892 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-05-09T09:29:34.892 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-05-09T09:29:34.892 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.892 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.892 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.893 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.894 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.895 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.896 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.897 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.898 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:34.898 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-09T09:29:34.898 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-09T09:29:34.898 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.898 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T09:29:34.898 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T09:29:34.898 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.898 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.898 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.898 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.898 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.898 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.899 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.901 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.902 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.903 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.904 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.907 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-05-09T09:29:34.908 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-05-09T09:29:34.908 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.908 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.908 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.908 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T09:29:34.908 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T09:29:34.908 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.908 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.908 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T09:29:34.908 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T09:29:34.908 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.908 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.909 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-09T09:29:34.910 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-05-09T09:29:34.911 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-05-09T09:29:34.912 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.913 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.914 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.914 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T09:29:34.914 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T09:29:34.914 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.914 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.914 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-05-09T09:29:34.914 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-05-09T09:29:34.914 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.914 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.914 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.914 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.915 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-05-09T09:29:34.916 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.916 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.916 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.916 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T09:29:34.916 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.916 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.916 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-05-09T09:29:34.916 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-05-09T09:29:34.916 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.916 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.917 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.917 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.917 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.917 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.917 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.917 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-05-09T09:29:34.917 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-05-09T09:29:34.917 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-05-09T09:29:34.917 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.917 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T09:29:34.918 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.918 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.918 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.918 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-05-09T09:29:34.918 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-05-09T09:29:34.918 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.918 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.918 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.918 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.919 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.919 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.919 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.919 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T09:29:34.919 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.919 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.919 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.920 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-05-09T09:29:34.920 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-05-09T09:29:34.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.920 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.920 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.920 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.920 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.920 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.920 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.921 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.922 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.924 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.925 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.926 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T09:29:34.926 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.926 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.926 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.926 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-05-09T09:29:34.926 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-05-09T09:29:34.926 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.926 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.926 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.926 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T09:29:34.926 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.926 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-05-09T09:29:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-05-09T09:29:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-05-09T09:29:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T09:29:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T09:29:34.928 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T09:29:34.929 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.930 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.930 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.930 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-05-09T09:29:34.930 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.930 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.930 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.930 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-05-09T09:29:34.930 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-05-09T09:29:34.930 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.930 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.931 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.931 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T09:29:34.931 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.931 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.931 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.931 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-05-09T09:29:34.931 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.931 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.931 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.931 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-05-09T09:29:34.932 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-05-09T09:29:34.932 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.932 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.932 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.932 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T09:29:34.932 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.932 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.932 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.932 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-05-09T09:29:34.932 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.932 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.933 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.933 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-05-09T09:29:34.933 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-05-09T09:29:34.933 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.933 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.933 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.933 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T09:29:34.933 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.933 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.934 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.934 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T09:29:34.934 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.934 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.934 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.934 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-05-09T09:29:34.934 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-05-09T09:29:34.934 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.935 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.935 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.935 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T09:29:34.935 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.935 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.935 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.935 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-05-09T09:29:34.935 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.935 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.935 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.936 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-05-09T09:29:34.936 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-05-09T09:29:34.936 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.936 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.936 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.936 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T09:29:34.936 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.936 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.936 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.936 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T09:29:34.937 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.937 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.937 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.937 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-05-09T09:29:34.937 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-05-09T09:29:34.937 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.937 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.937 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.938 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T09:29:34.938 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.938 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.938 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.938 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-05-09T09:29:34.938 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.938 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.938 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.938 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-05-09T09:29:34.939 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-05-09T09:29:34.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.939 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.939 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T09:29:34.939 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.939 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.939 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.939 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-05-09T09:29:34.939 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.940 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.940 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-05-09T09:29:34.940 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-05-09T09:29:34.940 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.940 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.940 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T09:29:34.940 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.940 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.941 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.941 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-05-09T09:29:34.941 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.941 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.941 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.941 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-05-09T09:29:34.941 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-05-09T09:29:34.941 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.942 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.942 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.942 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.942 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.942 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.942 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.942 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T09:29:34.942 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.942 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.942 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.943 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-05-09T09:29:34.943 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-05-09T09:29:34.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.943 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.943 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.943 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:34.943 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.943 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.943 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.944 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T09:29:34.944 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.944 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.944 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.944 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-05-09T09:29:34.944 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T09:29:34.944 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.944 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.944 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.945 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-05-09T09:29:34.945 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.945 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.945 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.945 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T09:29:34.945 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.945 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.945 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.945 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-05-09T09:29:34.946 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-05-09T09:29:34.946 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.946 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.946 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.946 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-05-09T09:29:34.946 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.946 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.946 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.946 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T09:29:34.947 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.947 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.947 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.947 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-05-09T09:29:34.947 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-05-09T09:29:34.947 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.947 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.947 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.947 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-05-09T09:29:34.948 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.948 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.948 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.948 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-05-09T09:29:34.948 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.948 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.948 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.948 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-05-09T09:29:34.948 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-05-09T09:29:34.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.949 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.949 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.949 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.949 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.949 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.949 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T09:29:34.949 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.950 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.950 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-05-09T09:29:34.950 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-05-09T09:29:34.950 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.950 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.950 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.950 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:34.950 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.951 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.951 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.951 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-05-09T09:29:34.951 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.951 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.951 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.951 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-05-09T09:29:34.951 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-05-09T09:29:34.951 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.951 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.951 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.952 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T09:29:34.952 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.952 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.952 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.952 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T09:29:34.952 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.952 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.952 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.952 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-05-09T09:29:34.952 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-05-09T09:29:34.952 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.952 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.953 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:34.953 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.953 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.953 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.953 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-05-09T09:29:34.953 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.953 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.953 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.953 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-05-09T09:29:34.953 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-05-09T09:29:34.953 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-05-09T09:29:34.954 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.955 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.955 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.955 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.955 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.955 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.955 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.955 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T09:29:34.955 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.955 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.955 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.956 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.956 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.956 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.956 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.956 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.956 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.956 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.956 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.956 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.956 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.956 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.957 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.958 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.958 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.958 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.958 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.958 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T09:29:34.958 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.958 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.958 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.958 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.958 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.959 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.960 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.960 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.960 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.960 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.960 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.960 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.960 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.960 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.960 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.960 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.960 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.960 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.960 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.961 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.962 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.962 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.962 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.962 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.962 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T09:29:34.962 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T09:29:34.962 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.962 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.962 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.962 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.962 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.962 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.964 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.965 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.966 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.967 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.968 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.969 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.970 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.971 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.972 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.973 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.974 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-05-09T09:29:34.975 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.976 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.977 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.978 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.979 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.981 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:34.982 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.983 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.984 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:34.985 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:34.986 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-05-09T09:29:34.987 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:34.988 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-05-09T09:29:34.989 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:34.990 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:34.990 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:34.990 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T09:29:34.990 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T09:29:34.990 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:34.990 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.990 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.990 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:34.990 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.990 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.990 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.990 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.991 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.992 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.993 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.994 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.994 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.994 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T09:29:34.994 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T09:29:34.994 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.994 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.994 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.994 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.994 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.994 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.994 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.994 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.995 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.995 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.995 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.995 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T09:29:34.995 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T09:29:34.995 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.995 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.995 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.995 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.995 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.995 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.995 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.996 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.996 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.996 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.996 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.996 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T09:29:34.996 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T09:29:34.996 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.996 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.996 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.996 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.996 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.996 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.997 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.998 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:34.999 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.000 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.001 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.002 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.003 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.004 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.005 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.006 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.006 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-05-09T09:29:35.006 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-05-09T09:29:35.006 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.006 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.006 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.006 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.006 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.006 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.006 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.006 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.007 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.008 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.009 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.010 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.011 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.011 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.011 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T09:29:35.011 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T09:29:35.011 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.011 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.011 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.011 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.011 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.011 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.011 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.011 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.012 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.013 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.015 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.016 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.017 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.018 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.020 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.021 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T09:29:35.022 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T09:29:35.022 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.022 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.022 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.022 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T09:29:35.022 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.022 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.022 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.022 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.022 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.022 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.022 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.028 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.029 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.030 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.031 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-05-09T09:29:35.032 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-05-09T09:29:35.033 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-05-09T09:29:35.034 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-05-09T09:29:35.035 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-09T09:29:35.036 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.037 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.038 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.038 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.038 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.038 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.038 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.038 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.038 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.038 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.038 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.038 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.038 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.038 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.039 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.039 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.039 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.039 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.039 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.039 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.039 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.039 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.039 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.039 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.040 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.040 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.040 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.040 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.040 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.040 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.040 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.040 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.040 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.040 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.041 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.042 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.043 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.044 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.046 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.047 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.048 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.049 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.050 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-05-09T09:29:35.050 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.050 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.050 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.050 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.050 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.050 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-09T09:29:35.050 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-09T09:29:35.050 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.051 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.051 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.051 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-09T09:29:35.051 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-09T09:29:35.051 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-09T09:29:35.051 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.051 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.051 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.051 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.051 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.051 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-05-09T09:29:35.051 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.053 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.054 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.055 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.056 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.056 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.056 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.056 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.056 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.056 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.056 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.056 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.056 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.056 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.056 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.057 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.058 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.059 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.060 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.061 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.062 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.063 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.064 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.065 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.066 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.067 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.068 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.069 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.071 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.072 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.072 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-09T09:29:35.072 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.072 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.072 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.072 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.072 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.072 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.072 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.074 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-05-09T09:29:35.075 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-05-09T09:29:35.076 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.077 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-05-09T09:29:35.078 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.079 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.080 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.081 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.082 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.083 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.084 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-05-09T09:29:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-05-09T09:29:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-05-09T09:29:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-05-09T09:29:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-05-09T09:29:35.087 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.088 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T09:29:35.088 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.088 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.088 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.088 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-05-09T09:29:35.088 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-05-09T09:29:35.088 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.088 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.088 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.088 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.088 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.088 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.089 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.090 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.090 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T09:29:35.090 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.090 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.090 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.090 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.090 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.090 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-05-09T09:29:35.090 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.090 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.090 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.092 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.093 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.095 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.096 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.099 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.100 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-05-09T09:29:35.101 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-05-09T09:29:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.103 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.104 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.105 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.105 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.105 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.105 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.105 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.105 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.105 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.105 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.105 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.105 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.105 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.105 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.106 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.106 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.106 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.106 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.106 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.106 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.106 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.106 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.106 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.106 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.106 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.106 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.107 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.109 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.110 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.111 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.112 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.113 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.114 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T09:29:35.115 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.116 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.120 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.120 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.120 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-05-09T09:29:35.120 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.120 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.120 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.120 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.120 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.120 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.120 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.120 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.121 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.122 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.122 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.122 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.122 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.122 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.122 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.122 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.122 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.122 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.122 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.122 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.122 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.123 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.124 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.124 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.124 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.124 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.124 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.124 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.124 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.124 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.125 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.125 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.125 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.125 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.125 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.125 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.125 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.126 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.126 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.126 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.126 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.126 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.126 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.126 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.126 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.127 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-05-09T09:29:35.127 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.127 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.127 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.127 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.127 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.128 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.128 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.128 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.128 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.128 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.128 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.128 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.128 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.129 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.130 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.130 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.130 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.130 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.130 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.130 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.130 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.130 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.130 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.130 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.130 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.131 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.132 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.133 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.134 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.135 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.136 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.137 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.138 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.139 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.140 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.142 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.145 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.145 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.145 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-05-09T09:29:35.145 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-09T09:29:35.145 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.145 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.145 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.145 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.145 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.145 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.145 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.145 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-05-09T09:29:35.146 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-05-09T09:29:35.146 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-05-09T09:29:35.146 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.146 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-05-09T09:29:35.146 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.146 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.146 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.146 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.147 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.149 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.150 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.152 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.153 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.154 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.156 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.157 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-05-09T09:29:35.158 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.158 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.158 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.158 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.158 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.158 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.158 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.158 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.158 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.158 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.158 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.158 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.158 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.159 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.160 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.161 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.162 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.163 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-05-09T09:29:35.164 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.166 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.167 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.168 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.169 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.170 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.171 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.172 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.174 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T09:29:35.174 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.174 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.174 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.174 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-05-09T09:29:35.174 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.174 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.174 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.174 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.174 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.174 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.175 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T09:29:35.176 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.177 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.178 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.179 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.180 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.181 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.182 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.183 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.184 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.186 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.187 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.188 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.189 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.190 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.191 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.192 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.193 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.194 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.194 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.194 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.194 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.194 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.194 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.194 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.194 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.194 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.194 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.194 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.194 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.195 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.195 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.195 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.195 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.195 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.195 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.195 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.195 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.195 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.195 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.195 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-05-09T09:29:35.195 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T09:29:35.196 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-05-09T09:29:35.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.199 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.200 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.202 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.203 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.204 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.205 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.206 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.207 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.208 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.211 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.212 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.213 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.215 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.216 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.217 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-05-09T09:29:35.218 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.220 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.221 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.222 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.224 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.225 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.226 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-05-09T09:29:35.226 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-05-09T09:29:35.226 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.226 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.226 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.226 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-05-09T09:29:35.226 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.226 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.226 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.226 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.226 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.226 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.227 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.228 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.230 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-05-09T09:29:35.230 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-05-09T09:29:35.230 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.230 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.230 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.230 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.230 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.230 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.230 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.230 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.230 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.230 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.230 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.234 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-05-09T09:29:35.234 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-05-09T09:29:35.234 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.234 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.234 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.234 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.234 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.234 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.234 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.234 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.234 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.234 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.234 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.236 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.237 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.238 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.239 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.240 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.241 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.242 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.242 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.242 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.242 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.242 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.242 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.242 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.242 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.242 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.242 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.242 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.242 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.242 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.243 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.244 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.246 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.247 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.247 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.247 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.247 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.247 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.247 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.247 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.247 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.247 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.247 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.247 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.247 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.248 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.250 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.251 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.252 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.254 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.255 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:35.256 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.257 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.258 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.259 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.260 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-05-09T09:29:35.262 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.263 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.263 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.263 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-05-09T09:29:35.263 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-05-09T09:29:35.263 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.263 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.263 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.263 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.263 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.263 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.263 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.263 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T09:29:35.263 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.264 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.264 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.264 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-05-09T09:29:35.264 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-05-09T09:29:35.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.264 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.264 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.264 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.264 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.264 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.264 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T09:29:35.265 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T09:29:35.266 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T09:29:35.267 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.268 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.269 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.270 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.272 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.273 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.274 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.275 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.275 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-05-09T09:29:35.275 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-05-09T09:29:35.275 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.275 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.275 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.275 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.275 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.275 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.275 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.275 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.275 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.276 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.278 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.278 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.278 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.278 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.278 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.278 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.278 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.278 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.278 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.278 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.278 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.279 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.280 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.281 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.281 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.281 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.281 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-05-09T09:29:35.281 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-05-09T09:29:35.281 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.281 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.281 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.281 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.281 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.281 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.282 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.283 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.285 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.286 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.287 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.290 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.292 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.293 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.294 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.294 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.294 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.294 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.294 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.294 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.294 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.294 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.294 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.294 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.294 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.296 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.297 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.297 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-05-09T09:29:35.297 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-05-09T09:29:35.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.297 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.297 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.297 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T09:29:35.297 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.298 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.298 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.298 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T09:29:35.298 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.298 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.298 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.298 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-05-09T09:29:35.298 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T09:29:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T09:29:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T09:29:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.300 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.301 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.302 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.303 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T09:29:35.304 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T09:29:35.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.306 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.307 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.308 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.309 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.310 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.310 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-05-09T09:29:35.310 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-05-09T09:29:35.310 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T09:29:35.310 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T09:29:35.310 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T09:29:35.310 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T09:29:35.310 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T09:29:35.310 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T09:29:35.310 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T09:29:35.310 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T09:29:35.310 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: "services": { 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard": "https://172.21.15.12:8443/" 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: "always_on_modules": { 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: "octopus": [ 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-09T09:29:35.311 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "pacific": [ 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-09T09:29:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "quincy": [ 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-09T09:29:35.313 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: "reef": [ 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T09:29:35.314 INFO:teuthology.orchestra.run.smithi012.stdout: "squid": [ 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: "last_failure_osd_epoch": 3, 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: "active_clients": [ 2024-05-09T09:29:35.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2129538570 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "libcephsqlite", 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-09T09:29:35.316 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3115254556 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 278654935 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T09:29:35.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:35.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.318 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-05-09T09:29:35.318 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-09T09:29:35.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T09:29:35.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-09T09:29:35.318 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-09T09:29:35.318 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3709764060 2024-05-09T09:29:35.318 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.318 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T09:29:35.318 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:35.318 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T09:29:35.318 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:29:35.320 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:34 smithi170 ceph-mon[33632]: pgmap v488: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T09:29:35.320 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:34 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/279587187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:29:35.320 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:34 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1866234820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:29:35.320 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:34 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2946094322' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T09:29:35.320 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:34 smithi016 ceph-mon[33335]: pgmap v488: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T09:29:35.320 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:34 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/279587187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:29:35.320 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:34 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1866234820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:29:35.320 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:34 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2946094322' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T09:29:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.999+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2946094322 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7d100a2b90 msgr2=0x7f7d100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.999+0000 7f7d2a2a7640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7d100a2b90 0x7f7d100a5050 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7d14006010 tx=0x7f7d14031510 comp rx=0 tx=0).stop 2024-05-09T09:29:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.999+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2946094322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d24102320 msgr2=0x7f7d2419f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:29:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.999+0000 7f7d2a2a7640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d24102320 0x7f7d2419f180 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00eb10 tx=0x7f7d1c00c2d0 comp rx=0 tx=0).stop 2024-05-09T09:29:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.999+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2946094322 shutdown_connections 2024-05-09T09:29:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.999+0000 7f7d2a2a7640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f7d100a2b90 0x7f7d100a5050 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.999+0000 7f7d2a2a7640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d24102320 0x7f7d2419f180 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.999+0000 7f7d2a2a7640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d24101960 0x7f7d24198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.999+0000 7f7d2a2a7640 1 --2- 172.21.15.12:0/2946094322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d24100770 0x7f7d24197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:29:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.999+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2946094322 >> 172.21.15.12:0/2946094322 conn(0x7f7d240fbce0 msgr2=0x7f7d240689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:29:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.999+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2946094322 shutdown_connections 2024-05-09T09:29:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:29:33.999+0000 7f7d2a2a7640 1 -- 172.21.15.12:0/2946094322 wait complete. 2024-05-09T09:29:35.327 DEBUG:tasks.check_counter:Getting stats from a 2024-05-09T09:29:35.328 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-09T09:29:35.328 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:34 smithi012 ceph-mon[30140]: pgmap v488: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T09:29:35.328 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:34 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/279587187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:29:35.328 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:34 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1866234820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:29:35.328 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:34 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2946094322' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T09:29:35.586 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:29:36.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:36 smithi012 ceph-mon[30140]: pgmap v489: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-09T09:29:36.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:36 smithi170 ceph-mon[33632]: pgmap v489: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-09T09:29:36.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:36 smithi016 ceph-mon[33335]: pgmap v489: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-09T09:29:37.454 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:29:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:29:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 10703, 2024-05-09T09:29:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 14652, 2024-05-09T09:29:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 3789464, 2024-05-09T09:29:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 10738974, 2024-05-09T09:29:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 16, 2024-05-09T09:29:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 6, 2024-05-09T09:29:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.862507785, 2024-05-09T09:29:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.407186243, 2024-05-09T09:29:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.452951554, 2024-05-09T09:29:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.061928781, 2024-05-09T09:29:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 14647, 2024-05-09T09:29:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.496351337, 2024-05-09T09:29:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000033887 2024-05-09T09:29:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 19943, 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.003513391, 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000176 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 21243, 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 17169, 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 8524261, 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 4510174, 2024-05-09T09:29:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 9, 2024-05-09T09:29:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 6, 2024-05-09T09:29:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 1.131228203, 2024-05-09T09:29:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.453565146, 2024-05-09T09:29:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.777174349, 2024-05-09T09:29:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.067150516, 2024-05-09T09:29:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 17172, 2024-05-09T09:29:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.544469204, 2024-05-09T09:29:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000031706 2024-05-09T09:29:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 34959, 2024-05-09T09:29:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.004905063, 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000140 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 578416, 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 76288 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 4798, 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 4363, 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 1579504, 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 5235125, 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 25, 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 7, 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.299495395, 2024-05-09T09:29:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.110120491, 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.160011801, 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.044053701, 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 4361, 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.217157144, 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000049795 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 6175, 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000962812, 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000155 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T09:29:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 219536 2024-05-09T09:29:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-05-09T09:29:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 1, 2024-05-09T09:29:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-05-09T09:29:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-MDSRank": { 2024-05-09T09:29:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-05-09T09:29:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-05-09T09:29:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 2319, 2024-05-09T09:29:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 1.943484637, 2024-05-09T09:29:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000838070 2024-05-09T09:29:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-PurgeQueue": { 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 2747, 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.303550724, 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000110502 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "request": 3499, 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "reply": 3172, 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "reply_latency": { 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 3172, 2024-05-09T09:29:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 5.539668917, 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.001746427 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_reply": 0, 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "forward": 327, 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_complete": 0, 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_keys": 0, 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_commit": 11, 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_split": 0, 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_merge": 0, 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes": 32, 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_top": 9, 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_bottom": 10, 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pin_tail": 13, 2024-05-09T09:29:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pinned": 16, 2024-05-09T09:29:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_expired": 1223, 2024-05-09T09:29:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_with_caps": 0, 2024-05-09T09:29:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": 0, 2024-05-09T09:29:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "subtrees": 6, 2024-05-09T09:29:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse": 8937, 2024-05-09T09:29:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_hit": 5146, 2024-05-09T09:29:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_forward": 327, 2024-05-09T09:29:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_discover": 603, 2024-05-09T09:29:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:29:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_remote_ino": 0, 2024-05-09T09:29:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_lock": 0, 2024-05-09T09:29:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "load_cent": 2000, 2024-05-09T09:29:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "q": 2, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "exported": 25, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "exported_inodes": 129, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "imported": 25, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "imported_inodes": 1075, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_dir_fetch": 0, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_backtrace_fetch": 462, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_peer_discover": 0, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rfiles": 618, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rbytes": 3930405, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rsnaps": 0, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_fetch": 462, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_set_tag": 0, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:29:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_inodes": 13, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dirfrag_rstats": 13, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_file_inodes": 449, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_inode_file_caps": 26, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_revoke": 421, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_grant": 2070, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps": 1693, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps_dirty": 656, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_cap_release": 87, 2024-05-09T09:29:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "process_request_cap_release": 3293 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache": { 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update": 335, 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update_receipt": 2034, 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_try_discover": 1310, 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_send_discover": 1416, 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_handle_discover": 551, 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays": 3, 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_delayed": 0, 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_created": 1435, 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_enqueued": 1432, 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_reintegrated": 0, 2024-05-09T09:29:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_migrated": 0, 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_processing": 0, 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_started": 0, 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_completed": 0, 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_exportdir": 26, 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_flush": 0, 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragmentdir": 0, 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragstats": 0, 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_inodestats": 0 2024-05-09T09:29:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log": { 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "evlrg": 0, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "evadd": 4877, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "evex": 0, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "evtrm": 0, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "ev": 4877, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "evexg": 0, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "evexd": 0, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "segadd": 6, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "segex": 0, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "segtrm": 0, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "seg": 6, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "segmjr": 2, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "segexg": 0, 2024-05-09T09:29:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "segexd": 0, 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "expos": 4194304, 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "wrpos": 11800885, 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "rdpos": 0, 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "jlat": { 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 951, 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 1.902225417, 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.002000237 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "replayed": 0 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_mem": { 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "ino": 35, 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "ino+": 2077, 2024-05-09T09:29:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "ino-": 2042, 2024-05-09T09:29:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "dir": 34, 2024-05-09T09:29:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "dir+": 864, 2024-05-09T09:29:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "dir-": 830, 2024-05-09T09:29:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "dn": 32, 2024-05-09T09:29:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "dn+": 3627, 2024-05-09T09:29:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "dn-": 3595, 2024-05-09T09:29:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "cap": 0, 2024-05-09T09:29:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "cap+": 1520, 2024-05-09T09:29:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "cap-": 1520, 2024-05-09T09:29:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "rss": 66932, 2024-05-09T09:29:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "heap": 223612 2024-05-09T09:29:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_server": { 2024-05-09T09:29:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_client_request": 6948, 2024-05-09T09:29:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_server_request": 109, 2024-05-09T09:29:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_request": 3499, 2024-05-09T09:29:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_session": 5, 2024-05-09T09:29:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_peer_request": 1648, 2024-05-09T09:29:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "req_create_latency": { 2024-05-09T09:29:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 423, 2024-05-09T09:29:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.844592386, 2024-05-09T09:29:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.001996672 2024-05-09T09:29:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getattr_latency": { 2024-05-09T09:29:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getfilelock_latency": { 2024-05-09T09:29:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "req_link_latency": { 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookup_latency": { 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 1184, 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.605737957, 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000511603 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookuphash_latency": { 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupino_latency": { 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupname_latency": { 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupparent_latency": { 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupsnap_latency": { 2024-05-09T09:29:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lssnap_latency": { 2024-05-09T09:29:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mkdir_latency": { 2024-05-09T09:29:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mknod_latency": { 2024-05-09T09:29:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mksnap_latency": { 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "req_open_latency": { 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "req_readdir_latency": { 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 130, 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.140512955, 2024-05-09T09:29:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.001080868 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rename_latency": { 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "req_renamesnap_latency": { 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "req_snapdiff_latency": { 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmdir_latency": { 2024-05-09T09:29:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 60, 2024-05-09T09:29:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 1.099617439, 2024-05-09T09:29:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.018326957 2024-05-09T09:29:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmsnap_latency": { 2024-05-09T09:29:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmxattr_latency": { 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setattr_latency": { 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setdirlayout_latency": { 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setfilelock_latency": { 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setlayout_latency": { 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setxattr_latency": { 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "req_symlink_latency": { 2024-05-09T09:29:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "req_unlink_latency": { 2024-05-09T09:29:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 1375, 2024-05-09T09:29:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 2.849208180, 2024-05-09T09:29:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.002072151 2024-05-09T09:29:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:29:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getvxattr_latency": { 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_sessions": { 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "session_count": 1, 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "session_add": 1, 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "session_remove": 0, 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_open": 1, 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_stale": 0, 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "total_load": 1871, 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "average_load": 1871, 2024-05-09T09:29:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "avg_session_uptime": 247, 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "mdthresh_evicted": 0 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:29:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:29:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:29:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:29:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:29:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:29:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-05-09T09:29:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:29:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-05-09T09:29:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:29:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:29:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-05-09T09:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-05-09T09:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-05-09T09:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-05-09T09:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:29:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:29:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:29:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:29:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 284007, 2024-05-09T09:29:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 114, 2024-05-09T09:29:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:29:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-05-09T09:29:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-05-09T09:29:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-05-09T09:29:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:29:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-05-09T09:29:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:29:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-05-09T09:29:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 9080, 2024-05-09T09:29:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 174, 2024-05-09T09:29:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:29:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:29:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-05-09T09:29:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-05-09T09:29:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 146006, 2024-05-09T09:29:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 1027, 2024-05-09T09:29:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-05-09T09:29:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-05-09T09:29:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-05-09T09:29:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-05-09T09:29:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter": { 2024-05-09T09:29:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "op_active": 0, 2024-05-09T09:29:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "op_laggy": 0, 2024-05-09T09:29:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send": 4557, 2024-05-09T09:29:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send_bytes": 7850970, 2024-05-09T09:29:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "op_resend": 33, 2024-05-09T09:29:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "op_reply": 4557, 2024-05-09T09:29:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "op_latency": { 2024-05-09T09:29:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 4557, 2024-05-09T09:29:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 13.324887343, 2024-05-09T09:29:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.002924048 2024-05-09T09:29:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "op_inflight": 0, 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "oplen_avg": { 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 4557, 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 4681 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "op": 4557, 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "op_r": 551, 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "op_w": 4006, 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "op_rmw": 0, 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "op_pg": 0, 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_stat": 0, 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_create": 10, 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_read": 89, 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_write": 1040, 2024-05-09T09:29:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writefull": 52, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writesame": 0, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_append": 0, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_zero": 0, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_truncate": 0, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_delete": 2820, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_mapext": 0, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_sparse_read": 0, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_clonerange": 0, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_getxattr": 462, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_setxattr": 20, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_rmxattr": 0, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:29:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_call": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_watch": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_notify": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_other": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_active": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_send": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_resend": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_ping": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_active": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_send": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_resend": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_active": 0, 2024-05-09T09:29:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_send": 0, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_resend": 0, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_active": 0, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_send": 0, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_resend": 0, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "command_active": 0, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "command_send": 0, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "command_resend": 0, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "map_epoch": 94, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "map_full": 0, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "map_inc": 5, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_sessions": 12, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_open": 12, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_close": 0, 2024-05-09T09:29:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_laggy": 0, 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_wr": 143, 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_rd": 0, 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_del": 45 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "oft": { 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_objs": 1, 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_updates": 0, 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_removes": 0 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "osd-slow-ops": { 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_ops_count": 0 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "purge_queue": { 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops": 0, 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops_high_water": 10, 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing": 0, 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_high_water": 9, 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed_ops": 2807, 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed": 1432, 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_item_in_journal": 0 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 36740, 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 10575175, 2024-05-09T09:29:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 36740, 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 36740, 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 10575175, 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_bytes": { 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T09:29:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 4557, 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 8006602, 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 4557, 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 8006602, 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_ops": { 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 1024, 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 4557, 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 4557, 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 4557, 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 4557, 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T09:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 1432, 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 155632, 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 1432, 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-09T09:29:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 89, 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 155632, 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle-0x55eb0d954500": { 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 4877, 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 7606581, 2024-05-09T09:29:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 4877, 2024-05-09T09:29:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-09T09:29:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-09T09:29:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 951, 2024-05-09T09:29:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 7606581, 2024-05-09T09:29:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T09:29:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:37.501 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:29:38.172 INFO:tasks.check_counter:Daemon mds.a mds.exported=25 2024-05-09T09:29:38.172 INFO:tasks.check_counter:Daemon mds.a mds.imported=25 2024-05-09T09:29:38.172 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=335 2024-05-09T09:29:38.172 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=2034 2024-05-09T09:29:38.172 DEBUG:tasks.check_counter:Getting stats from d 2024-05-09T09:29:38.173 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-09T09:29:38.433 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:29:38.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:38 smithi012 ceph-mon[30140]: pgmap v490: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:29:38.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:38 smithi170 ceph-mon[33632]: pgmap v490: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:29:38.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:38 smithi016 ceph-mon[33335]: pgmap v490: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:29:40.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:40 smithi012 ceph-mon[30140]: pgmap v491: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:29:40.476 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:29:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:29:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 4, 2024-05-09T09:29:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 4, 2024-05-09T09:29:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 2992, 2024-05-09T09:29:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 448, 2024-05-09T09:29:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001828135, 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000048142, 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000757025, 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 3, 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000763242, 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000254414 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-09T09:29:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T09:29:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:29:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 1, 2024-05-09T09:29:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 87, 2024-05-09T09:29:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 160, 2024-05-09T09:29:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 92928, 2024-05-09T09:29:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 3, 2024-05-09T09:29:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-05-09T09:29:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.006477543, 2024-05-09T09:29:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.004616282, 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001092182, 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 85, 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.003668563, 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000043159 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T09:29:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 92928 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 118, 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 115, 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 449296, 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 69808, 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.017691211, 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.006593955, 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.010659497, 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.002156546, 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 114, 2024-05-09T09:29:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.004423241, 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000038800 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 449296, 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 69808 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-05-09T09:29:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:29:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-05-09T09:29:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:29:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:29:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:29:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:29:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:29:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:29:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:29:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:29:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:29:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:29:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:29:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:29:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:29:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-05-09T09:29:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-05-09T09:29:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-05-09T09:29:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:29:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:29:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:29:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:29:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148903, 2024-05-09T09:29:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 29, 2024-05-09T09:29:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:29:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-05-09T09:29:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-05-09T09:29:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-05-09T09:29:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:29:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-05-09T09:29:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:29:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-05-09T09:29:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-05-09T09:29:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-05-09T09:29:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:29:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:29:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-05-09T09:29:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-05-09T09:29:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-05-09T09:29:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-05-09T09:29:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-05-09T09:29:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-05-09T09:29:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-05-09T09:29:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-05-09T09:29:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:29:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T09:29:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-09T09:29:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T09:29:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 119, 2024-05-09T09:29:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 432658, 2024-05-09T09:29:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 119, 2024-05-09T09:29:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-09T09:29:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-09T09:29:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 119, 2024-05-09T09:29:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 432658, 2024-05-09T09:29:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T09:29:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:40.488 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:29:40.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:40 smithi170 ceph-mon[33632]: pgmap v491: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:29:40.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:40 smithi016 ceph-mon[33335]: pgmap v491: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:29:41.292 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-09T09:29:41.292 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-09T09:29:41.292 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-09T09:29:41.292 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-09T09:29:41.292 DEBUG:tasks.check_counter:Getting stats from g 2024-05-09T09:29:41.293 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-09T09:29:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:29:42.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:42 smithi012 ceph-mon[30140]: pgmap v492: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:29:42.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:42 smithi170 ceph-mon[33632]: pgmap v492: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:29:42.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:42 smithi016 ceph-mon[33335]: pgmap v492: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T09:29:43.524 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:29:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:29:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 507, 2024-05-09T09:29:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 490, 2024-05-09T09:29:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 685184, 2024-05-09T09:29:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 186624, 2024-05-09T09:29:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 5, 2024-05-09T09:29:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.055485601, 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.021165464, 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.019836105, 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.012417137, 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 489, 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.019655007, 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000040194 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 561536, 2024-05-09T09:29:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 69600 2024-05-09T09:29:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:29:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 673, 2024-05-09T09:29:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 756, 2024-05-09T09:29:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 7778181, 2024-05-09T09:29:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 433401, 2024-05-09T09:29:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 5, 2024-05-09T09:29:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 4, 2024-05-09T09:29:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.053372936, 2024-05-09T09:29:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.015819042, 2024-05-09T09:29:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.018622315, 2024-05-09T09:29:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.015229077, 2024-05-09T09:29:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 754, 2024-05-09T09:29:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.020251113, 2024-05-09T09:29:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000026858 2024-05-09T09:29:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 219648 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 8, 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 8, 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 4621, 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 1732, 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 5, 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 4, 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001537365, 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000094426, 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001304518, 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000107285, 2024-05-09T09:29:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 7, 2024-05-09T09:29:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.002525338, 2024-05-09T09:29:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000360762 2024-05-09T09:29:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 1, 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-MDSRank": { 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 1811, 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.042392816, 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000023408 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-PurgeQueue": { 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "request": 0, 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "reply": 0, 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "reply_latency": { 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_reply": 0, 2024-05-09T09:29:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "forward": 0, 2024-05-09T09:29:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_complete": 0, 2024-05-09T09:29:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_keys": 0, 2024-05-09T09:29:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_commit": 0, 2024-05-09T09:29:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_split": 0, 2024-05-09T09:29:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_merge": 0, 2024-05-09T09:29:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes": 1487, 2024-05-09T09:29:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_top": 0, 2024-05-09T09:29:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_bottom": 1487, 2024-05-09T09:29:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pin_tail": 0, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pinned": 1487, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_expired": 0, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_with_caps": 0, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": 0, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "subtrees": 1, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse": 0, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_hit": 0, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_forward": 0, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_discover": 0, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_remote_ino": 0, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_lock": 0, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "load_cent": 0, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "q": 0, 2024-05-09T09:29:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "exported": 0, 2024-05-09T09:29:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "exported_inodes": 0, 2024-05-09T09:29:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "imported": 0, 2024-05-09T09:29:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "imported_inodes": 0, 2024-05-09T09:29:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_dir_fetch": 0, 2024-05-09T09:29:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_backtrace_fetch": 0, 2024-05-09T09:29:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_peer_discover": 0, 2024-05-09T09:29:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rfiles": 0, 2024-05-09T09:29:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rbytes": 0, 2024-05-09T09:29:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rsnaps": 0, 2024-05-09T09:29:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T09:29:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_set_tag": 0, 2024-05-09T09:29:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:29:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:29:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_inodes": 0, 2024-05-09T09:29:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:29:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T09:29:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_file_inodes": 0, 2024-05-09T09:29:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_inode_file_caps": 0, 2024-05-09T09:29:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T09:29:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_grant": 0, 2024-05-09T09:29:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:29:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:29:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:29:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps": 0, 2024-05-09T09:29:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps_dirty": 0, 2024-05-09T09:29:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_cap_release": 0, 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "process_request_cap_release": 0 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache": { 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update": 0, 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update_receipt": 0, 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_try_discover": 0, 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_send_discover": 0, 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_handle_discover": 0, 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays": 0, 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_delayed": 0, 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_created": 0, 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_enqueued": 0, 2024-05-09T09:29:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_reintegrated": 0, 2024-05-09T09:29:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_migrated": 0, 2024-05-09T09:29:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_processing": 0, 2024-05-09T09:29:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:29:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:29:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_started": 0, 2024-05-09T09:29:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_completed": 0, 2024-05-09T09:29:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:29:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:29:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:29:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_exportdir": 0, 2024-05-09T09:29:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_flush": 0, 2024-05-09T09:29:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragmentdir": 0, 2024-05-09T09:29:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragstats": 0, 2024-05-09T09:29:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_inodestats": 0 2024-05-09T09:29:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log": { 2024-05-09T09:29:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "evlrg": 0, 2024-05-09T09:29:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "evadd": 0, 2024-05-09T09:29:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "evex": 0, 2024-05-09T09:29:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "evtrm": 0, 2024-05-09T09:29:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "ev": 4877, 2024-05-09T09:29:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "evexg": 0, 2024-05-09T09:29:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "evexd": 0, 2024-05-09T09:29:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "segadd": 0, 2024-05-09T09:29:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "segex": 0, 2024-05-09T09:29:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "segtrm": 0, 2024-05-09T09:29:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "seg": 6, 2024-05-09T09:29:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "segmjr": 2, 2024-05-09T09:29:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "segexg": 0, 2024-05-09T09:29:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "segexd": 0, 2024-05-09T09:29:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "expos": 4194304, 2024-05-09T09:29:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "wrpos": 11800885, 2024-05-09T09:29:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "rdpos": 11799747, 2024-05-09T09:29:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "jlat": { 2024-05-09T09:29:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "replayed": 4877 2024-05-09T09:29:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_mem": { 2024-05-09T09:29:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "ino": 11, 2024-05-09T09:29:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "ino+": 1525, 2024-05-09T09:29:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "ino-": 1514, 2024-05-09T09:29:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "dir": 9, 2024-05-09T09:29:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "dir+": 164, 2024-05-09T09:29:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "dir-": 155, 2024-05-09T09:29:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "dn": 1487, 2024-05-09T09:29:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "dn+": 2605, 2024-05-09T09:29:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "dn-": 1118, 2024-05-09T09:29:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "cap": 0, 2024-05-09T09:29:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "cap+": 0, 2024-05-09T09:29:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "cap-": 0, 2024-05-09T09:29:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "rss": 52532, 2024-05-09T09:29:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "heap": 215420 2024-05-09T09:29:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_server": { 2024-05-09T09:29:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_client_request": 0, 2024-05-09T09:29:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_server_request": 0, 2024-05-09T09:29:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_request": 0, 2024-05-09T09:29:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_session": 0, 2024-05-09T09:29:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_peer_request": 0, 2024-05-09T09:29:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "req_create_latency": { 2024-05-09T09:29:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getattr_latency": { 2024-05-09T09:29:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getfilelock_latency": { 2024-05-09T09:29:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "req_link_latency": { 2024-05-09T09:29:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookup_latency": { 2024-05-09T09:29:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookuphash_latency": { 2024-05-09T09:29:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupino_latency": { 2024-05-09T09:29:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupname_latency": { 2024-05-09T09:29:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupparent_latency": { 2024-05-09T09:29:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupsnap_latency": { 2024-05-09T09:29:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lssnap_latency": { 2024-05-09T09:29:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mkdir_latency": { 2024-05-09T09:29:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mknod_latency": { 2024-05-09T09:29:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mksnap_latency": { 2024-05-09T09:29:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "req_open_latency": { 2024-05-09T09:29:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "req_readdir_latency": { 2024-05-09T09:29:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rename_latency": { 2024-05-09T09:29:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "req_renamesnap_latency": { 2024-05-09T09:29:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "req_snapdiff_latency": { 2024-05-09T09:29:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmdir_latency": { 2024-05-09T09:29:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmsnap_latency": { 2024-05-09T09:29:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmxattr_latency": { 2024-05-09T09:29:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setattr_latency": { 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setdirlayout_latency": { 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setfilelock_latency": { 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setlayout_latency": { 2024-05-09T09:29:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setxattr_latency": { 2024-05-09T09:29:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "req_symlink_latency": { 2024-05-09T09:29:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "req_unlink_latency": { 2024-05-09T09:29:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:29:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:29:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getvxattr_latency": { 2024-05-09T09:29:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_sessions": { 2024-05-09T09:29:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "session_count": 1, 2024-05-09T09:29:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "session_add": 1, 2024-05-09T09:29:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "session_remove": 0, 2024-05-09T09:29:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_open": 1, 2024-05-09T09:29:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_stale": 0, 2024-05-09T09:29:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "total_load": 0, 2024-05-09T09:29:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "average_load": 0, 2024-05-09T09:29:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "avg_session_uptime": 1374, 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mdthresh_evicted": 0 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:29:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:29:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:29:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:29:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:29:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:29:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-05-09T09:29:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:29:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-05-09T09:29:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:29:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:29:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:29:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:29:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:29:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 197336, 2024-05-09T09:29:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 53, 2024-05-09T09:29:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:29:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-05-09T09:29:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-05-09T09:29:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-05-09T09:29:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:29:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-05-09T09:29:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:29:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-05-09T09:29:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 9080, 2024-05-09T09:29:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 174, 2024-05-09T09:29:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:29:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:29:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 1083825, 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 6623, 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter": { 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "op_active": 0, 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "op_laggy": 0, 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send": 1044, 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send_bytes": 21, 2024-05-09T09:29:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "op_resend": 3, 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "op_reply": 1044, 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "op_latency": { 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 1044, 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 1.299989036, 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.001245200 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "op_inflight": 0, 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "oplen_avg": { 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 1044, 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 1045 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "op": 1044, 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "op_r": 1044, 2024-05-09T09:29:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "op_w": 0, 2024-05-09T09:29:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "op_rmw": 0, 2024-05-09T09:29:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "op_pg": 0, 2024-05-09T09:29:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_stat": 621, 2024-05-09T09:29:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_create": 0, 2024-05-09T09:29:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_read": 421, 2024-05-09T09:29:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_write": 0, 2024-05-09T09:29:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writefull": 0, 2024-05-09T09:29:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writesame": 0, 2024-05-09T09:29:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_append": 0, 2024-05-09T09:29:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_zero": 0, 2024-05-09T09:29:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_truncate": 0, 2024-05-09T09:29:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_delete": 0, 2024-05-09T09:29:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_mapext": 0, 2024-05-09T09:29:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_sparse_read": 0, 2024-05-09T09:29:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_clonerange": 0, 2024-05-09T09:29:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_getxattr": 1, 2024-05-09T09:29:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_setxattr": 0, 2024-05-09T09:29:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:29:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_rmxattr": 0, 2024-05-09T09:29:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:29:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_call": 0, 2024-05-09T09:29:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_watch": 0, 2024-05-09T09:29:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_notify": 0, 2024-05-09T09:29:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:29:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls": 0, 2024-05-09T09:29:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_other": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_active": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_send": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_resend": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_ping": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_active": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_send": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_resend": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_active": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_send": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_resend": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_active": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_send": 0, 2024-05-09T09:29:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_resend": 0, 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "command_active": 0, 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "command_send": 0, 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "command_resend": 0, 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "map_epoch": 94, 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "map_full": 0, 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "map_inc": 5, 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_sessions": 6, 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_open": 6, 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_close": 0, 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_laggy": 0, 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_wr": 0, 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_rd": 2, 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_del": 0 2024-05-09T09:29:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "oft": { 2024-05-09T09:29:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_objs": 0, 2024-05-09T09:29:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:29:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_updates": 0, 2024-05-09T09:29:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_removes": 0 2024-05-09T09:29:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "osd-slow-ops": { 2024-05-09T09:29:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_ops_count": 0 2024-05-09T09:29:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "purge_queue": { 2024-05-09T09:29:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops": 0, 2024-05-09T09:29:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T09:29:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing": 0, 2024-05-09T09:29:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_high_water": 0, 2024-05-09T09:29:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed_ops": 0, 2024-05-09T09:29:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed": 0, 2024-05-09T09:29:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_item_in_journal": 0 2024-05-09T09:29:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:29:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T09:29:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-09T09:29:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T09:29:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 1184, 2024-05-09T09:29:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 8351416, 2024-05-09T09:29:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 1184, 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 1184, 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 8351416, 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_bytes": { 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-09T09:29:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T09:29:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-05-09T09:29:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-05-09T09:29:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 1044, 2024-05-09T09:29:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 7606586, 2024-05-09T09:29:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 1044, 2024-05-09T09:29:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 7606586, 2024-05-09T09:29:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T09:29:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_ops": { 2024-05-09T09:29:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T09:29:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 1024, 2024-05-09T09:29:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T09:29:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-05-09T09:29:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-05-09T09:29:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 1044, 2024-05-09T09:29:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 1044, 2024-05-09T09:29:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 1044, 2024-05-09T09:29:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 1044, 2024-05-09T09:29:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T09:29:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:29:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T09:29:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-05-09T09:29:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T09:29:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-05-09T09:29:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 0, 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 0, 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle-0x55d967df26e0": { 2024-05-09T09:29:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 0, 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 0, 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:43.577 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:29:44.439 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-09T09:29:44.439 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-09T09:29:44.439 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-05-09T09:29:44.439 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-05-09T09:29:44.439 DEBUG:tasks.check_counter:Getting stats from j 2024-05-09T09:29:44.440 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-09T09:29:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:29:44.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:44 smithi012 ceph-mon[30140]: pgmap v493: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:29:44.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:44 smithi170 ceph-mon[33632]: pgmap v493: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:29:44.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:44 smithi016 ceph-mon[33335]: pgmap v493: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T09:29:46.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:46 smithi012 ceph-mon[30140]: pgmap v494: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T09:29:46.570 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 4, 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 4, 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 2992, 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 448, 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001578064, 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000043121, 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000563584, 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 3, 2024-05-09T09:29:46.571 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000712695, 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000237565 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 1, 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 89, 2024-05-09T09:29:46.572 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 160, 2024-05-09T09:29:46.573 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 93280, 2024-05-09T09:29:46.573 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 3, 2024-05-09T09:29:46.573 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-05-09T09:29:46.573 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.007232387, 2024-05-09T09:29:46.573 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.005107489, 2024-05-09T09:29:46.573 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001254508, 2024-05-09T09:29:46.573 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:29:46.573 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:46.573 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 87, 2024-05-09T09:29:46.573 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.003749377, 2024-05-09T09:29:46.573 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000043096 2024-05-09T09:29:46.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:46.573 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:46.574 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:46.574 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:46.574 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:46.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:46.574 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T09:29:46.574 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 93280 2024-05-09T09:29:46.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:46.574 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:29:46.574 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 121, 2024-05-09T09:29:46.574 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 118, 2024-05-09T09:29:46.574 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 450112, 2024-05-09T09:29:46.574 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 71728, 2024-05-09T09:29:46.575 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-05-09T09:29:46.575 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-05-09T09:29:46.575 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.018081656, 2024-05-09T09:29:46.575 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.006641245, 2024-05-09T09:29:46.575 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.011133937, 2024-05-09T09:29:46.575 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.002226154, 2024-05-09T09:29:46.575 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:46.575 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 117, 2024-05-09T09:29:46.575 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.004647423, 2024-05-09T09:29:46.575 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000039721 2024-05-09T09:29:46.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:46.575 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:46.576 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:46.576 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:46.576 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:46.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:46.576 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 450112, 2024-05-09T09:29:46.576 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 71728 2024-05-09T09:29:46.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:46.576 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-05-09T09:29:46.576 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-05-09T09:29:46.576 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-05-09T09:29:46.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:46.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-05-09T09:29:46.577 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:29:46.577 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-05-09T09:29:46.577 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:29:46.577 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:29:46.577 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:29:46.577 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:29:46.577 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:29:46.577 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:29:46.577 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:29:46.577 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:29:46.577 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:29:46.577 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:29:46.577 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:29:46.578 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:29:46.578 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:29:46.578 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-05-09T09:29:46.578 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:29:46.578 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-05-09T09:29:46.578 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:29:46.578 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:29:46.578 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:29:46.578 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:29:46.578 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:29:46.578 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:29:46.578 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:29:46.579 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-05-09T09:29:46.579 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:29:46.579 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:29:46.579 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:29:46.579 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-05-09T09:29:46.579 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-05-09T09:29:46.579 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-05-09T09:29:46.579 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:29:46.579 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:29:46.579 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:29:46.579 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:29:46.579 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148863, 2024-05-09T09:29:46.579 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 28, 2024-05-09T09:29:46.580 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:29:46.580 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-05-09T09:29:46.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-05-09T09:29:46.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-05-09T09:29:46.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:29:46.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-05-09T09:29:46.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:29:46.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-05-09T09:29:46.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-05-09T09:29:46.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-05-09T09:29:46.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:29:46.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:29:46.580 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-05-09T09:29:46.581 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-05-09T09:29:46.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-05-09T09:29:46.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-05-09T09:29:46.581 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-05-09T09:29:46.581 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-05-09T09:29:46.581 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-05-09T09:29:46.581 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-05-09T09:29:46.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T09:29:46.581 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:29:46.581 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T09:29:46.581 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-09T09:29:46.581 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T09:29:46.582 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 122, 2024-05-09T09:29:46.582 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 433048, 2024-05-09T09:29:46.582 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:46.582 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 122, 2024-05-09T09:29:46.582 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-09T09:29:46.582 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-09T09:29:46.582 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 122, 2024-05-09T09:29:46.582 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 433048, 2024-05-09T09:29:46.582 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T09:29:46.582 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T09:29:46.582 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T09:29:46.582 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T09:29:46.582 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:46.583 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T09:29:46.583 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T09:29:46.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:46 smithi170 ceph-mon[33632]: pgmap v494: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T09:29:46.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:46 smithi016 ceph-mon[33335]: pgmap v494: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T09:29:47.418 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-09T09:29:47.418 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-09T09:29:47.418 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-09T09:29:47.418 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-09T09:29:47.419 DEBUG:tasks.check_counter:Getting stats from b 2024-05-09T09:29:47.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-09T09:29:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:29:48.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:48 smithi012 ceph-mon[30140]: pgmap v495: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T09:29:48.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:48 smithi016 ceph-mon[33335]: pgmap v495: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T09:29:48.545 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:48 smithi170 ceph-mon[33632]: pgmap v495: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T09:29:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-09T09:29:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:29:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 11389, 2024-05-09T09:29:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 18174, 2024-05-09T09:29:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3821762, 2024-05-09T09:29:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 19230473, 2024-05-09T09:29:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 25, 2024-05-09T09:29:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 6, 2024-05-09T09:29:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.773416776, 2024-05-09T09:29:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.301582615, 2024-05-09T09:29:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.443889229, 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.075147966, 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 18166, 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.513581024, 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000028271 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 21330, 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.003467323, 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000162 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:29:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 24763, 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 20913, 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 10444531, 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 6667915, 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 8, 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 5, 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 1.195460248, 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.457116901, 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.920331173, 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.030362464, 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 20919, 2024-05-09T09:29:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.427155252, 2024-05-09T09:29:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000020419 2024-05-09T09:29:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 42911, 2024-05-09T09:29:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.005905063, 2024-05-09T09:29:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000137 2024-05-09T09:29:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 600224, 2024-05-09T09:29:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 78528 2024-05-09T09:29:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:29:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 6949, 2024-05-09T09:29:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 7029, 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2544322, 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 3379147, 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 17, 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 6, 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.404251769, 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.145038106, 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.213411734, 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.064398863, 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 7027, 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.200056947, 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000028469 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 7877, 2024-05-09T09:29:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.001181569, 2024-05-09T09:29:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000150 2024-05-09T09:29:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T09:29:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 223072 2024-05-09T09:29:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-05-09T09:29:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-05-09T09:29:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-05-09T09:29:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-05-09T09:29:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-05-09T09:29:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-05-09T09:29:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2204, 2024-05-09T09:29:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 3.039626677, 2024-05-09T09:29:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001379140 2024-05-09T09:29:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-05-09T09:29:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-05-09T09:29:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-05-09T09:29:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3866, 2024-05-09T09:29:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.390818335, 2024-05-09T09:29:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000101091 2024-05-09T09:29:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-05-09T09:29:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 5085, 2024-05-09T09:29:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 4110, 2024-05-09T09:29:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-05-09T09:29:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 4110, 2024-05-09T09:29:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 6.922890454, 2024-05-09T09:29:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001684401 2024-05-09T09:29:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-05-09T09:29:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 975, 2024-05-09T09:29:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 20, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 1, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 29, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 8, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 11, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 10, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 14, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 1702, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 2, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 2, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 6, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 8391, 2024-05-09T09:29:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 5836, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 974, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 588, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 22, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 593, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 15, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 1701, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 1898, 2024-05-09T09:29:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 618, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 3930405, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 1898, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 108, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 108, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 1790, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 28, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 602, 2024-05-09T09:29:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 3230, 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 3384, 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 1262, 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 99, 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 4898 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 503, 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 1998, 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 1291, 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 1404, 2024-05-09T09:29:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 850, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 1, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 2025, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 2024, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:29:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 23, 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 1, 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 6997, 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 6997, 2024-05-09T09:29:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 10, 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 10, 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 17188891, 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 0, 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 414, 2024-05-09T09:29:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.512743269, 2024-05-09T09:29:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.003653969 2024-05-09T09:29:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 0 2024-05-09T09:29:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-05-09T09:29:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 34, 2024-05-09T09:29:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 3161, 2024-05-09T09:29:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 3127, 2024-05-09T09:29:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 33, 2024-05-09T09:29:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 894, 2024-05-09T09:29:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 861, 2024-05-09T09:29:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 29, 2024-05-09T09:29:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 4851, 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 4822, 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 2, 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 2645, 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 2643, 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 70396, 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 223612 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 6971, 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 138, 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 5085, 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 12, 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 865, 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-05-09T09:29:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 847, 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.995691702, 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001175551 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2, 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.001843158, 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000921579 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 960, 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.687859593, 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000716520 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-05-09T09:29:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-05-09T09:29:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-05-09T09:29:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-05-09T09:29:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-05-09T09:29:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-05-09T09:29:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-05-09T09:29:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 276, 2024-05-09T09:29:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.363407991, 2024-05-09T09:29:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001316695 2024-05-09T09:29:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-05-09T09:29:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-05-09T09:29:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 76, 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.496547367, 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.019691412 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-05-09T09:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1949, 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 3.377540643, 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001732960 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 1, 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 0, 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-05-09T09:29:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 2494, 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 2494, 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 260, 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:29:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:29:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 299991, 2024-05-09T09:29:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 117, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 9080, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 174, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-05-09T09:29:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 141368, 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 729, 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 6616, 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 13519672, 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 33, 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 6616, 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-05-09T09:29:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 6616, 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 17.929710927, 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.002710053 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 6616, 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 6747 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 6616, 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 1973, 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 4643, 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-05-09T09:29:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 0, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 10, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 75, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 489, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 44, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 1, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 4004, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1898, 2024-05-09T09:29:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 20, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-05-09T09:29:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 94, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 5, 2024-05-09T09:29:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 12, 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 12, 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 160, 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 0, 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 46 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 1, 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 17, 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 8, 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 3989, 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 2024, 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-09T09:29:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 43097, 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 12919927, 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 43097, 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 43097, 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 12919927, 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 6616, 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 13739704, 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 6616, 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 13739704, 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 6616, 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 6616, 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 6616, 2024-05-09T09:29:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 6616, 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 2024, 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 220032, 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 2024, 2024-05-09T09:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 75, 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 220032, 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x557dce60a500": { 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 6997, 2024-05-09T09:29:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 12994587, 2024-05-09T09:29:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 6997, 2024-05-09T09:29:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-09T09:29:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-09T09:29:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 414, 2024-05-09T09:29:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 12994587, 2024-05-09T09:29:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:49.625 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-09T09:29:50.369 INFO:tasks.check_counter:Daemon mds.b mds.exported=22 2024-05-09T09:29:50.369 INFO:tasks.check_counter:Daemon mds.b mds.imported=15 2024-05-09T09:29:50.370 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=503 2024-05-09T09:29:50.370 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=1998 2024-05-09T09:29:50.370 DEBUG:tasks.check_counter:Getting stats from e 2024-05-09T09:29:50.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-09T09:29:50.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:50 smithi016 ceph-mon[33335]: pgmap v496: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T09:29:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:29:50.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:50 smithi012 ceph-mon[30140]: pgmap v496: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T09:29:50.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:50 smithi170 ceph-mon[33632]: pgmap v496: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T09:29:51.590 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:51 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:29:51.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:51 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:29:51.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:51 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:29:52.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:52 smithi016 ceph-mon[33335]: pgmap v497: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T09:29:52.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:52 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:29:52.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:52 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:29:52.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:52 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:29:52.512 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 988, 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 990, 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 12866139, 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 314191, 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.084817646, 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.024611494, 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.028072508, 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.024978441, 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 988, 2024-05-09T09:29:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.029198351, 2024-05-09T09:29:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000029552 2024-05-09T09:29:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:29:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:29:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:29:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 241, 2024-05-09T09:29:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 219, 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1018561, 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 102252, 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.029505460, 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.009536123, 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.016548221, 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.005021576, 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 219, 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.012344726, 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000056368 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 584016, 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 73888 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 90, 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 428, 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 225694, 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-05-09T09:29:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.005929747, 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.003704352, 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001226738, 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000030104, 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 88, 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.005352615, 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000060825 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 225376 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1868, 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.043715133, 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000023402 2024-05-09T09:29:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-05-09T09:29:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-05-09T09:29:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-05-09T09:29:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-05-09T09:29:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-05-09T09:29:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-05-09T09:29:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 2048, 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 2048, 2024-05-09T09:29:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-05-09T09:29:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 2048, 2024-05-09T09:29:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-05-09T09:29:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-05-09T09:29:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-05-09T09:29:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-05-09T09:29:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-05-09T09:29:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-05-09T09:29:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-05-09T09:29:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-05-09T09:29:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:29:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-05-09T09:29:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-05-09T09:29:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-05-09T09:29:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-05-09T09:29:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-05-09T09:29:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-05-09T09:29:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-05-09T09:29:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-05-09T09:29:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-05-09T09:29:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-05-09T09:29:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-05-09T09:29:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-05-09T09:29:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-05-09T09:29:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-05-09T09:29:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T09:29:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-05-09T09:29:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:29:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:29:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-05-09T09:29:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:29:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T09:29:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-05-09T09:29:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-05-09T09:29:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T09:29:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-05-09T09:29:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:29:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:29:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-05-09T09:29:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-05-09T09:29:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-05-09T09:29:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-05-09T09:29:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-05-09T09:29:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-05-09T09:29:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-05-09T09:29:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-05-09T09:29:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-05-09T09:29:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 6997, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 10, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 17188891, 2024-05-09T09:29:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 17187753, 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 6997 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 13, 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 2606, 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 2593, 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 11, 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 202, 2024-05-09T09:29:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 191, 2024-05-09T09:29:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 2048, 2024-05-09T09:29:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 3782, 2024-05-09T09:29:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 1734, 2024-05-09T09:29:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-05-09T09:29:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-05-09T09:29:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-05-09T09:29:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 53656, 2024-05-09T09:29:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215420 2024-05-09T09:29:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-05-09T09:29:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.535 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-05-09T09:29:52.535 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.535 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.535 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-05-09T09:29:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-05-09T09:29:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-05-09T09:29:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-05-09T09:29:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-05-09T09:29:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-05-09T09:29:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-05-09T09:29:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-05-09T09:29:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-05-09T09:29:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-05-09T09:29:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-05-09T09:29:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:29:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:29:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-05-09T09:29:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-05-09T09:29:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-05-09T09:29:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 1, 2024-05-09T09:29:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 0, 2024-05-09T09:29:52.545 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2024-05-09T09:29:52.545 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-05-09T09:29:52.545 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-05-09T09:29:52.545 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-05-09T09:29:52.545 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 1365, 2024-05-09T09:29:52.545 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-05-09T09:29:52.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.545 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-05-09T09:29:52.545 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:29:52.545 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-05-09T09:29:52.545 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:29:52.545 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:29:52.545 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:29:52.546 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:29:52.546 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:29:52.546 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:29:52.546 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:29:52.546 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:29:52.546 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:29:52.546 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:29:52.546 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:29:52.546 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:29:52.546 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:29:52.547 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-05-09T09:29:52.547 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:29:52.547 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-05-09T09:29:52.547 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:29:52.547 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:29:52.547 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:29:52.547 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:29:52.547 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:29:52.547 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:29:52.547 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:29:52.547 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-05-09T09:29:52.547 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:29:52.547 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:29:52.548 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:29:52.548 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-05-09T09:29:52.548 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-05-09T09:29:52.548 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-05-09T09:29:52.548 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:29:52.548 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:29:52.548 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:29:52.548 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:29:52.548 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 190063, 2024-05-09T09:29:52.548 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 51, 2024-05-09T09:29:52.548 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:29:52.549 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-05-09T09:29:52.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-05-09T09:29:52.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-05-09T09:29:52.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:29:52.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-05-09T09:29:52.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:29:52.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-05-09T09:29:52.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 9080, 2024-05-09T09:29:52.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 174, 2024-05-09T09:29:52.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:29:52.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:29:52.550 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-05-09T09:29:52.550 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-05-09T09:29:52.550 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 1485347, 2024-05-09T09:29:52.550 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 8491, 2024-05-09T09:29:52.550 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-05-09T09:29:52.550 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-05-09T09:29:52.550 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-05-09T09:29:52.551 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-05-09T09:29:52.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.551 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-05-09T09:29:52.551 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-05-09T09:29:52.551 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-05-09T09:29:52.551 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 1078, 2024-05-09T09:29:52.551 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-05-09T09:29:52.551 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-05-09T09:29:52.551 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 1078, 2024-05-09T09:29:52.552 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-05-09T09:29:52.552 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1078, 2024-05-09T09:29:52.552 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.111982079, 2024-05-09T09:29:52.552 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001031523 2024-05-09T09:29:52.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.552 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-05-09T09:29:52.552 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-05-09T09:29:52.552 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1078, 2024-05-09T09:29:52.552 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1079 2024-05-09T09:29:52.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.553 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 1078, 2024-05-09T09:29:52.553 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 1078, 2024-05-09T09:29:52.553 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-05-09T09:29:52.553 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-05-09T09:29:52.553 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-05-09T09:29:52.553 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 640, 2024-05-09T09:29:52.553 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-05-09T09:29:52.553 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 436, 2024-05-09T09:29:52.553 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-05-09T09:29:52.553 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-05-09T09:29:52.554 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-05-09T09:29:52.554 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-05-09T09:29:52.554 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-05-09T09:29:52.554 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-05-09T09:29:52.554 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-05-09T09:29:52.554 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-05-09T09:29:52.554 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-05-09T09:29:52.554 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-05-09T09:29:52.554 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-05-09T09:29:52.555 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-05-09T09:29:52.555 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:29:52.555 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-05-09T09:29:52.555 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:29:52.555 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-05-09T09:29:52.555 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-05-09T09:29:52.555 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-05-09T09:29:52.555 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:29:52.555 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-05-09T09:29:52.555 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:29:52.555 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-05-09T09:29:52.556 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-05-09T09:29:52.556 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-05-09T09:29:52.556 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-05-09T09:29:52.556 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-05-09T09:29:52.556 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-05-09T09:29:52.556 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-05-09T09:29:52.556 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-05-09T09:29:52.556 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-05-09T09:29:52.557 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-05-09T09:29:52.557 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-05-09T09:29:52.557 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-05-09T09:29:52.557 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-05-09T09:29:52.557 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-05-09T09:29:52.557 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-05-09T09:29:52.557 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-05-09T09:29:52.557 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-05-09T09:29:52.557 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 94, 2024-05-09T09:29:52.558 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-05-09T09:29:52.558 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 5, 2024-05-09T09:29:52.558 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 5, 2024-05-09T09:29:52.559 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 5, 2024-05-09T09:29:52.559 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-05-09T09:29:52.559 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-05-09T09:29:52.559 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-05-09T09:29:52.559 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-05-09T09:29:52.559 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-05-09T09:29:52.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.560 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-05-09T09:29:52.560 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-05-09T09:29:52.560 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:29:52.560 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-05-09T09:29:52.560 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-05-09T09:29:52.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.560 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-05-09T09:29:52.560 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-05-09T09:29:52.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.561 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-05-09T09:29:52.561 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-05-09T09:29:52.561 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T09:29:52.561 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-05-09T09:29:52.561 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-05-09T09:29:52.561 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-05-09T09:29:52.561 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-05-09T09:29:52.561 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-05-09T09:29:52.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.561 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:29:52.561 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:52.562 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-09T09:29:52.562 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:52.562 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 1227, 2024-05-09T09:29:52.562 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 13764241, 2024-05-09T09:29:52.562 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:52.562 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 1227, 2024-05-09T09:29:52.562 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-09T09:29:52.562 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-09T09:29:52.562 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 1227, 2024-05-09T09:29:52.562 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 13764241, 2024-05-09T09:29:52.563 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:52.563 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.563 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.563 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.563 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:52.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.563 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-05-09T09:29:52.563 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:52.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-09T09:29:52.563 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:52.563 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:52.564 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:52.564 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:52.564 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:52.564 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 1078, 2024-05-09T09:29:52.564 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 12994592, 2024-05-09T09:29:52.564 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 1078, 2024-05-09T09:29:52.564 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 12994592, 2024-05-09T09:29:52.564 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:52.564 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.564 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.564 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.564 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-05-09T09:29:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-05-09T09:29:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 1078, 2024-05-09T09:29:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 1078, 2024-05-09T09:29:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 1078, 2024-05-09T09:29:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1078, 2024-05-09T09:29:52.566 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:52.566 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.566 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.566 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.566 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:52.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.566 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:29:52.566 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:52.567 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-05-09T09:29:52.567 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:52.567 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:52.567 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:52.567 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:52.567 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:52.567 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-09T09:29:52.567 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-09T09:29:52.567 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-05-09T09:29:52.567 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-05-09T09:29:52.567 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:52.567 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.568 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.568 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.568 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:52.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:52.568 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55c5914b46e0": { 2024-05-09T09:29:52.568 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:52.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-05-09T09:29:52.568 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:52.568 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:52.568 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:52.568 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:52.569 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:52.569 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-09T09:29:52.569 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-09T09:29:52.569 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-05-09T09:29:52.569 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-05-09T09:29:52.570 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:52.570 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:52.570 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:52.570 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:52.570 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:52.570 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:52.570 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-09T09:29:52.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:52 smithi012 ceph-mon[30140]: pgmap v497: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T09:29:52.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:52 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:29:52.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:52 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:29:52.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:52 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:29:52.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:52 smithi170 ceph-mon[33632]: pgmap v497: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T09:29:52.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:52 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:29:52.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:52 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:29:52.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:52 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' 2024-05-09T09:29:53.354 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-09T09:29:53.354 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-09T09:29:53.354 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-09T09:29:53.354 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-05-09T09:29:53.354 DEBUG:tasks.check_counter:Getting stats from h 2024-05-09T09:29:53.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-09T09:29:53.607 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:29:54.722 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:54 smithi016 ceph-mon[33335]: pgmap v498: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T09:29:54.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:54 smithi012 ceph-mon[30140]: pgmap v498: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T09:29:54.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:54 smithi170 ceph-mon[33632]: pgmap v498: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-09T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 193, 2024-05-09T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 182, 2024-05-09T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 271152, 2024-05-09T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 67432, 2024-05-09T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-05-09T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-05-09T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.021946069, 2024-05-09T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.007318270, 2024-05-09T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.009283152, 2024-05-09T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.004449652, 2024-05-09T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 182, 2024-05-09T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.009970713, 2024-05-09T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000054784 2024-05-09T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 225632, 2024-05-09T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 24816 2024-05-09T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 142, 2024-05-09T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 171, 2024-05-09T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 8352613, 2024-05-09T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 180712, 2024-05-09T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-05-09T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-05-09T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.015959087, 2024-05-09T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.003012674, 2024-05-09T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.009233984, 2024-05-09T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.002809141, 2024-05-09T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 169, 2024-05-09T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.006288876, 2024-05-09T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000037212 2024-05-09T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 135808 2024-05-09T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 219, 2024-05-09T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 221, 2024-05-09T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 16875103, 2024-05-09T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 69649, 2024-05-09T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-05-09T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-05-09T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.022736271, 2024-05-09T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.003932573, 2024-05-09T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.012677279, 2024-05-09T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.004972884, 2024-05-09T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 219, 2024-05-09T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.008046216, 2024-05-09T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000036740 2024-05-09T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-05-09T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 830, 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.047238941, 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000056914 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-05-09T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-05-09T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 2790, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 1927, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 863, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 37, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 2, 2024-05-09T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-05-09T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-05-09T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-05-09T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-05-09T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-05-09T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-05-09T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-05-09T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-05-09T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-05-09T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-05-09T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-05-09T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-05-09T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-05-09T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-05-09T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-05-09T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 4, 2024-05-09T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 234, 2024-05-09T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-05-09T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-05-09T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-05-09T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-05-09T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-05-09T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-05-09T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-05-09T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-05-09T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-05-09T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-05-09T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-05-09T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-05-09T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-05-09T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-05-09T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-05-09T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-05-09T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-05-09T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-05-09T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-05-09T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-05-09T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-05-09T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-05-09T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-05-09T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-05-09T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-05-09T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-05-09T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 12357, 2024-05-09T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-05-09T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-05-09T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-05-09T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-05-09T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-05-09T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-05-09T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 7, 2024-05-09T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-05-09T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-05-09T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 54471075, 2024-05-09T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 54566662, 2024-05-09T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 54564165, 2024-05-09T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-05-09T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 12357 2024-05-09T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-05-09T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 21, 2024-05-09T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 5196, 2024-05-09T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 5175, 2024-05-09T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 31, 2024-05-09T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 509, 2024-05-09T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 478, 2024-05-09T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 2790, 2024-05-09T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 7749, 2024-05-09T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 4959, 2024-05-09T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-05-09T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-05-09T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-05-09T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 55816, 2024-05-09T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215396 2024-05-09T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-05-09T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-05-09T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-05-09T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-05-09T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-05-09T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-05-09T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-05-09T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-05-09T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-05-09T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-05-09T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-05-09T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-05-09T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-05-09T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-05-09T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-05-09T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-05-09T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-05-09T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-05-09T09:29:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-05-09T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-05-09T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-05-09T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-05-09T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-05-09T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-05-09T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-05-09T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-05-09T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-05-09T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-05-09T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-05-09T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-05-09T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-05-09T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-05-09T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-05-09T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-05-09T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 1, 2024-05-09T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 0, 2024-05-09T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 0, 2024-05-09T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-05-09T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-05-09T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-05-09T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 1368, 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-05-09T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-05-09T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 197550, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 50, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 9080, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 174, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 2052289, 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 13350, 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 494, 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 26, 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 4, 2024-05-09T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 494, 2024-05-09T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-05-09T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 494, 2024-05-09T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.397993584, 2024-05-09T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000805655 2024-05-09T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-05-09T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-05-09T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 494, 2024-05-09T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 495 2024-05-09T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 494, 2024-05-09T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 494, 2024-05-09T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-05-09T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-05-09T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-05-09T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 268, 2024-05-09T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-05-09T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 223, 2024-05-09T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-05-09T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-05-09T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-05-09T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-05-09T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-05-09T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-05-09T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-05-09T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-05-09T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-05-09T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-05-09T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 2, 2024-05-09T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-05-09T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-05-09T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-05-09T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-05-09T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-05-09T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-05-09T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-05-09T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-05-09T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-05-09T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-05-09T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-05-09T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-05-09T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-05-09T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-05-09T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-05-09T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-05-09T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-05-09T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-05-09T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-05-09T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-05-09T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-05-09T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-05-09T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-05-09T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 0, 2024-05-09T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-05-09T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 0, 2024-05-09T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 6, 2024-05-09T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 6, 2024-05-09T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-05-09T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-05-09T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-05-09T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-05-09T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-05-09T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-05-09T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-05-09T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-05-09T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-05-09T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-05-09T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-05-09T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-05-09T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-05-09T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-05-09T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-05-09T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-05-09T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-05-09T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-05-09T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-09T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 550, 2024-05-09T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 25443610, 2024-05-09T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 550, 2024-05-09T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-09T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-09T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 550, 2024-05-09T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 25443610, 2024-05-09T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-05-09T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-09T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 494, 2024-05-09T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 25134693, 2024-05-09T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 494, 2024-05-09T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 25134693, 2024-05-09T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-05-09T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-05-09T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 494, 2024-05-09T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 494, 2024-05-09T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 494, 2024-05-09T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 494, 2024-05-09T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-05-09T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-09T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-09T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-05-09T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-05-09T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55991ea746e0": { 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-09T09:29:56.038 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-09T09:29:56.038 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-09T09:29:56.038 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-09T09:29:56.038 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-05-09T09:29:56.039 DEBUG:tasks.check_counter:Getting stats from k 2024-05-09T09:29:56.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-09T09:29:56.292 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:29:56.565 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:56 smithi016 ceph-mon[33335]: pgmap v499: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T09:29:56.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:56 smithi012 ceph-mon[30140]: pgmap v499: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T09:29:56.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:56 smithi170 ceph-mon[33632]: pgmap v499: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T09:29:58.093 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-09T09:29:58.094 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:29:58.094 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 744, 2024-05-09T09:29:58.094 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 744, 2024-05-09T09:29:58.094 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 4418129, 2024-05-09T09:29:58.094 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 235769, 2024-05-09T09:29:58.094 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 7, 2024-05-09T09:29:58.094 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-05-09T09:29:58.094 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.072465934, 2024-05-09T09:29:58.094 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.023339111, 2024-05-09T09:29:58.094 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.021493105, 2024-05-09T09:29:58.094 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.020817943, 2024-05-09T09:29:58.094 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 743, 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.027932635, 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000037594 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:29:58.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 452, 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 432, 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 8231167, 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 171146, 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.052358013, 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.013919885, 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.026430330, 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.011854597, 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 431, 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.020817199, 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000048299 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 581840, 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 75360 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 67, 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 158, 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 4211202, 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 248508, 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-05-09T09:29:58.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-05-09T09:29:58.098 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.013899286, 2024-05-09T09:29:58.098 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.004626840, 2024-05-09T09:29:58.098 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.006328172, 2024-05-09T09:29:58.098 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.001594292, 2024-05-09T09:29:58.098 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:29:58.098 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 156, 2024-05-09T09:29:58.098 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.007706555, 2024-05-09T09:29:58.098 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000049400 2024-05-09T09:29:58.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.098 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:29:58.098 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.098 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.098 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 227520 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1898, 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.048308272, 2024-05-09T09:29:58.099 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000025452 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-05-09T09:29:58.100 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-05-09T09:29:58.101 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.101 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.101 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.101 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-05-09T09:29:58.101 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-05-09T09:29:58.101 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-05-09T09:29:58.101 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-05-09T09:29:58.101 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-05-09T09:29:58.101 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-05-09T09:29:58.101 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-05-09T09:29:58.101 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 2740, 2024-05-09T09:29:58.101 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-05-09T09:29:58.103 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 2740, 2024-05-09T09:29:58.103 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-05-09T09:29:58.103 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 2740, 2024-05-09T09:29:58.103 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-05-09T09:29:58.103 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-05-09T09:29:58.103 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-05-09T09:29:58.103 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-05-09T09:29:58.104 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-05-09T09:29:58.106 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-05-09T09:29:58.106 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-05-09T09:29:58.106 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-05-09T09:29:58.107 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-05-09T09:29:58.107 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-05-09T09:29:58.107 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T09:29:58.107 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-05-09T09:29:58.107 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:29:58.107 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:29:58.107 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-05-09T09:29:58.107 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:29:58.107 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T09:29:58.107 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-05-09T09:29:58.107 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-05-09T09:29:58.108 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T09:29:58.108 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-05-09T09:29:58.108 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:29:58.108 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:29:58.108 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:29:58.108 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-05-09T09:29:58.108 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-05-09T09:29:58.108 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-05-09T09:29:58.108 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-05-09T09:29:58.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.109 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-05-09T09:29:58.109 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-05-09T09:29:58.109 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-05-09T09:29:58.109 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-05-09T09:29:58.109 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-05-09T09:29:58.109 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-05-09T09:29:58.109 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-05-09T09:29:58.109 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-05-09T09:29:58.109 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:29:58.110 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-05-09T09:29:58.110 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-05-09T09:29:58.110 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-05-09T09:29:58.110 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-05-09T09:29:58.110 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-05-09T09:29:58.110 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:29:58.110 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:29:58.110 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-05-09T09:29:58.110 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-05-09T09:29:58.111 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:29:58.111 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:29:58.111 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:29:58.111 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-05-09T09:29:58.111 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-05-09T09:29:58.111 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-05-09T09:29:58.111 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-05-09T09:29:58.111 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-05-09T09:29:58.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.112 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-05-09T09:29:58.112 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-05-09T09:29:58.112 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-05-09T09:29:58.112 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-05-09T09:29:58.112 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-05-09T09:29:58.112 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 9237, 2024-05-09T09:29:58.112 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-05-09T09:29:58.112 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-05-09T09:29:58.112 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-05-09T09:29:58.112 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-05-09T09:29:58.112 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-05-09T09:29:58.112 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 12, 2024-05-09T09:29:58.113 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-05-09T09:29:58.113 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-05-09T09:29:58.113 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-05-09T09:29:58.113 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-05-09T09:29:58.113 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 20155765, 2024-05-09T09:29:58.113 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 20154627, 2024-05-09T09:29:58.113 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-05-09T09:29:58.113 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.113 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.113 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.113 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 9237 2024-05-09T09:29:58.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.114 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-05-09T09:29:58.114 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 19, 2024-05-09T09:29:58.114 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 2716, 2024-05-09T09:29:58.114 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 2697, 2024-05-09T09:29:58.114 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 17, 2024-05-09T09:29:58.114 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 221, 2024-05-09T09:29:58.114 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 204, 2024-05-09T09:29:58.114 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 2740, 2024-05-09T09:29:58.114 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 4709, 2024-05-09T09:29:58.114 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 1969, 2024-05-09T09:29:58.114 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 54900, 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215420 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.117 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.117 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-05-09T09:29:58.117 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.117 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.117 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.117 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-05-09T09:29:58.117 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.117 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.117 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.118 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-05-09T09:29:58.118 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.118 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.118 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.118 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-05-09T09:29:58.118 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.118 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.118 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.118 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-05-09T09:29:58.119 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.119 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.119 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.119 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-05-09T09:29:58.119 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.119 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.119 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.119 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-05-09T09:29:58.120 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.120 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.120 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.120 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-05-09T09:29:58.121 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.121 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.121 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.122 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-05-09T09:29:58.122 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.122 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.122 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.122 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-05-09T09:29:58.122 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.122 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.122 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.123 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-05-09T09:29:58.123 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.123 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.123 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.123 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-05-09T09:29:58.123 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.123 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.123 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.124 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-05-09T09:29:58.124 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.124 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.124 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.124 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-05-09T09:29:58.124 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.124 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.124 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.124 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-05-09T09:29:58.125 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.125 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.125 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.125 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-05-09T09:29:58.125 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.125 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.125 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.125 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-05-09T09:29:58.125 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.126 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.126 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.126 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-05-09T09:29:58.126 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.126 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.126 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.126 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-05-09T09:29:58.126 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.126 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.126 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.127 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-05-09T09:29:58.127 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.127 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.127 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.127 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-05-09T09:29:58.127 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.127 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-05-09T09:29:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-05-09T09:29:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-05-09T09:29:58.129 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.129 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.129 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.129 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-05-09T09:29:58.129 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.129 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.129 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.129 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:29:58.129 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:29:58.130 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-05-09T09:29:58.130 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.130 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.130 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.130 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:58.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.130 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-05-09T09:29:58.130 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-05-09T09:29:58.131 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 1, 2024-05-09T09:29:58.131 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 0, 2024-05-09T09:29:58.131 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2024-05-09T09:29:58.131 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-05-09T09:29:58.131 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-05-09T09:29:58.131 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-05-09T09:29:58.131 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 1371, 2024-05-09T09:29:58.131 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-05-09T09:29:58.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.131 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-05-09T09:29:58.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:29:58.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-05-09T09:29:58.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:29:58.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:29:58.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:29:58.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:29:58.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:29:58.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:29:58.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:29:58.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:29:58.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:29:58.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:29:58.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:29:58.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:29:58.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:29:58.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-05-09T09:29:58.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:29:58.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-05-09T09:29:58.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:29:58.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:29:58.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:29:58.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:29:58.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:29:58.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:29:58.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:29:58.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-05-09T09:29:58.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:29:58.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:29:58.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:29:58.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-05-09T09:29:58.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-05-09T09:29:58.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-05-09T09:29:58.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:29:58.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:29:58.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:29:58.135 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:29:58.135 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 206183, 2024-05-09T09:29:58.135 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 58, 2024-05-09T09:29:58.135 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:29:58.135 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-05-09T09:29:58.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-05-09T09:29:58.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-05-09T09:29:58.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:29:58.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-05-09T09:29:58.136 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:29:58.136 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-05-09T09:29:58.136 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 9080, 2024-05-09T09:29:58.136 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 174, 2024-05-09T09:29:58.136 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:29:58.136 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:29:58.136 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-05-09T09:29:58.136 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-05-09T09:29:58.136 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 1994538, 2024-05-09T09:29:58.136 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 12420, 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 1107, 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 1107, 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-05-09T09:29:58.137 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1107, 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.954984596, 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000862678 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1107, 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1108 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 1107, 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 1107, 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-05-09T09:29:58.138 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 651, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 454, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-05-09T09:29:58.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-05-09T09:29:58.140 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 94, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-05-09T09:29:58.141 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 5, 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 7, 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 7, 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.142 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-05-09T09:29:58.143 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-05-09T09:29:58.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.143 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-05-09T09:29:58.143 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-05-09T09:29:58.143 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T09:29:58.143 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-05-09T09:29:58.143 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-05-09T09:29:58.143 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-05-09T09:29:58.144 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-05-09T09:29:58.144 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-05-09T09:29:58.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.144 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:29:58.144 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:58.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-09T09:29:58.144 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:58.144 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 1259, 2024-05-09T09:29:58.144 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 16736575, 2024-05-09T09:29:58.144 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:58.144 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 1259, 2024-05-09T09:29:58.144 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-09T09:29:58.144 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 1259, 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 16736575, 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:58.146 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:58.146 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:58.146 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 1107, 2024-05-09T09:29:58.146 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 15961466, 2024-05-09T09:29:58.146 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 1107, 2024-05-09T09:29:58.146 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 15961466, 2024-05-09T09:29:58.146 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:58.146 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.146 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.146 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.146 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:58.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.146 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 1107, 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 1107, 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 1107, 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1107, 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.147 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.148 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.148 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:58.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.148 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:29:58.148 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:58.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-05-09T09:29:58.148 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:58.148 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:58.148 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:58.148 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:58.148 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:58.148 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-09T09:29:58.148 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55725ccc86e0": { 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-05-09T09:29:58.149 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-09T09:29:58.150 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-05-09T09:29:58.150 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-09T09:29:58.150 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-09T09:29:58.150 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-05-09T09:29:58.150 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-05-09T09:29:58.150 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-09T09:29:58.150 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-09T09:29:58.150 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-09T09:29:58.150 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-09T09:29:58.150 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:58.150 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-09T09:29:58.150 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-09T09:29:58.503 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:29:58 smithi016 ceph-mon[33335]: pgmap v500: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:29:58.551 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:29:58 smithi012 ceph-mon[30140]: pgmap v500: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:29:58.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:29:58 smithi170 ceph-mon[33632]: pgmap v500: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:29:58.892 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-09T09:29:58.892 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-09T09:29:58.892 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-05-09T09:29:58.892 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-05-09T09:29:58.892 DEBUG:tasks.check_counter:Getting stats from c 2024-05-09T09:29:58.892 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-09T09:29:59.148 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:30:00.621 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:00 smithi170 ceph-mon[33632]: pgmap v501: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:00.622 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:00 smithi170 ceph-mon[33632]: overall HEALTH_OK 2024-05-09T09:30:00.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:00 smithi012 ceph-mon[30140]: pgmap v501: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:00.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:00 smithi012 ceph-mon[30140]: overall HEALTH_OK 2024-05-09T09:30:00.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:00 smithi016 ceph-mon[33335]: pgmap v501: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:00.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:00 smithi016 ceph-mon[33335]: overall HEALTH_OK 2024-05-09T09:30:00.907 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-05-09T09:30:00.908 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:30:00.908 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 26520, 2024-05-09T09:30:00.908 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 22822, 2024-05-09T09:30:00.908 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 11036387, 2024-05-09T09:30:00.908 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 10967629, 2024-05-09T09:30:00.908 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 9, 2024-05-09T09:30:00.908 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2024-05-09T09:30:00.908 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.485031701, 2024-05-09T09:30:00.908 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.660657174, 2024-05-09T09:30:00.908 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.973891053, 2024-05-09T09:30:00.908 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.063562700, 2024-05-09T09:30:00.908 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:30:00.909 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 22828, 2024-05-09T09:30:00.909 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.596781260, 2024-05-09T09:30:00.909 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000026142 2024-05-09T09:30:00.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.909 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:30:00.909 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 44334, 2024-05-09T09:30:00.909 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.006894040, 2024-05-09T09:30:00.909 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000155 2024-05-09T09:30:00.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.909 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 597680, 2024-05-09T09:30:00.909 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 81552 2024-05-09T09:30:00.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.910 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:30:00.910 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 6959, 2024-05-09T09:30:00.910 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 7051, 2024-05-09T09:30:00.910 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 2228772, 2024-05-09T09:30:00.910 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 10016427, 2024-05-09T09:30:00.910 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 10, 2024-05-09T09:30:00.910 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2024-05-09T09:30:00.910 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.457016141, 2024-05-09T09:30:00.910 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.193936785, 2024-05-09T09:30:00.910 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.200122232, 2024-05-09T09:30:00.910 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.071266215, 2024-05-09T09:30:00.910 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:30:00.911 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 7049, 2024-05-09T09:30:00.911 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.298640431, 2024-05-09T09:30:00.911 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000042366 2024-05-09T09:30:00.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.911 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:30:00.911 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 7849, 2024-05-09T09:30:00.911 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.001446429, 2024-05-09T09:30:00.911 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000184 2024-05-09T09:30:00.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.911 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T09:30:00.911 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 229040 2024-05-09T09:30:00.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.912 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:30:00.912 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 13000, 2024-05-09T09:30:00.912 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 20283, 2024-05-09T09:30:00.912 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 4674467, 2024-05-09T09:30:00.912 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 14336077, 2024-05-09T09:30:00.912 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 32, 2024-05-09T09:30:00.912 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 9, 2024-05-09T09:30:00.912 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.787783515, 2024-05-09T09:30:00.912 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.327632632, 2024-05-09T09:30:00.912 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.424918040, 2024-05-09T09:30:00.912 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.076737436, 2024-05-09T09:30:00.912 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 20275, 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.569088050, 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000028068 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 24202, 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.003534756, 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000146 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1936, 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 1584 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-05-09T09:30:00.913 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2559, 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3.753218593, 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001466673 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-05-09T09:30:00.914 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4914, 2024-05-09T09:30:00.915 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.484563575, 2024-05-09T09:30:00.915 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000098608 2024-05-09T09:30:00.915 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:00.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.915 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-05-09T09:30:00.915 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 5660, 2024-05-09T09:30:00.915 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 5085, 2024-05-09T09:30:00.915 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-05-09T09:30:00.915 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5085, 2024-05-09T09:30:00.915 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 12.843027361, 2024-05-09T09:30:00.915 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002525669 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 575, 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 20, 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 1, 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 36, 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 13, 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 10, 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 13, 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 16, 2024-05-09T09:30:00.916 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 1238, 2024-05-09T09:30:00.917 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 1, 2024-05-09T09:30:00.917 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 1, 2024-05-09T09:30:00.917 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 6, 2024-05-09T09:30:00.917 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 9998, 2024-05-09T09:30:00.917 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 7214, 2024-05-09T09:30:00.917 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 566, 2024-05-09T09:30:00.917 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 547, 2024-05-09T09:30:00.918 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:30:00.918 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-05-09T09:30:00.918 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-05-09T09:30:00.918 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-05-09T09:30:00.918 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-05-09T09:30:00.918 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 16, 2024-05-09T09:30:00.918 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 165, 2024-05-09T09:30:00.918 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 17, 2024-05-09T09:30:00.918 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 1788, 2024-05-09T09:30:00.918 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-05-09T09:30:00.918 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 2057, 2024-05-09T09:30:00.918 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-05-09T09:30:00.918 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 618, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 3930405, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 2057, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 129, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 129, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 1928, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 46, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 551, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 3748, 2024-05-09T09:30:00.919 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:30:00.920 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:30:00.920 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:30:00.920 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 4172, 2024-05-09T09:30:00.920 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 1837, 2024-05-09T09:30:00.920 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 84, 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 5994 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 570, 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 1877, 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 1208, 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 1328, 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 1018, 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 3, 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 2555, 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 2552, 2024-05-09T09:30:00.921 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 16, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 1, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-05-09T09:30:00.922 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 8869, 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 8869, 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 12, 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 12, 2024-05-09T09:30:00.923 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 2, 2024-05-09T09:30:00.924 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-05-09T09:30:00.924 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-05-09T09:30:00.924 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-05-09T09:30:00.924 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 20273854, 2024-05-09T09:30:00.924 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-05-09T09:30:00.924 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-05-09T09:30:00.924 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 631, 2024-05-09T09:30:00.924 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.255156290, 2024-05-09T09:30:00.924 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003573940 2024-05-09T09:30:00.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.924 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-05-09T09:30:00.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.924 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 36, 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 3255, 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 3219, 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 42, 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 860, 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 818, 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 36, 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 5438, 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 5402, 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 1, 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 2728, 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 2727, 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 73808, 2024-05-09T09:30:00.925 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 223612 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 8184, 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 199, 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 5660, 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 11, 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 1359, 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 879, 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.106020382, 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001258271 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-05-09T09:30:00.926 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3, 2024-05-09T09:30:00.927 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000709331, 2024-05-09T09:30:00.927 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000236443 2024-05-09T09:30:00.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.927 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-05-09T09:30:00.927 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.927 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.927 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.927 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1395, 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.058714496, 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000758935 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.930 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-05-09T09:30:00.930 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.930 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.930 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.931 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-05-09T09:30:00.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.931 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.931 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-05-09T09:30:00.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.931 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.931 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-05-09T09:30:00.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.932 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 252, 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.009603017, 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.015911123 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 88, 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.844943141, 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.020965262 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.934 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.001722664, 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001722664 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.935 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.936 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-05-09T09:30:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-05-09T09:30:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2467, 2024-05-09T09:30:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.821314330, 2024-05-09T09:30:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001954322 2024-05-09T09:30:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 2, 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 1, 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 1, 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-05-09T09:30:00.938 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 3105, 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 3105, 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 270, 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:30:00.939 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:30:00.940 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 290760, 2024-05-09T09:30:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 113, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9080, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 174, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-05-09T09:30:00.942 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-05-09T09:30:00.943 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 163511, 2024-05-09T09:30:00.943 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 1137, 2024-05-09T09:30:00.943 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-05-09T09:30:00.943 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-05-09T09:30:00.943 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-05-09T09:30:00.943 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-05-09T09:30:00.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.943 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-05-09T09:30:00.943 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-05-09T09:30:00.943 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-05-09T09:30:00.943 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 8138, 2024-05-09T09:30:00.943 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 16691035, 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 27, 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 8138, 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8138, 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 24.512422382, 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003012094 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8138, 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 8289 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 8138, 2024-05-09T09:30:00.944 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 2166, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 5972, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 109, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 740, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 64, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 1, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 5050, 2024-05-09T09:30:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 2057, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:30:00.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-05-09T09:30:00.947 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 94, 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 5, 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 175, 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 0, 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 63 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.948 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-05-09T09:30:00.949 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-05-09T09:30:00.949 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:30:00.949 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-05-09T09:30:00.949 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-05-09T09:30:00.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.949 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-05-09T09:30:00.949 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-05-09T09:30:00.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.949 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-05-09T09:30:00.949 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-05-09T09:30:00.949 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 10, 2024-05-09T09:30:00.949 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-05-09T09:30:00.949 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 7, 2024-05-09T09:30:00.950 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 5035, 2024-05-09T09:30:00.950 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 2552, 2024-05-09T09:30:00.950 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-05-09T09:30:00.950 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.950 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:30:00.950 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:00.950 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-05-09T09:30:00.950 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:00.950 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 46475, 2024-05-09T09:30:00.950 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 13744522, 2024-05-09T09:30:00.950 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 46475, 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 46475, 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 13744522, 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-05-09T09:30:00.951 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:00.952 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-05-09T09:30:00.952 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-05-09T09:30:00.952 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:00.952 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-05-09T09:30:00.952 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 8138, 2024-05-09T09:30:00.952 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 16968539, 2024-05-09T09:30:00.952 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 8138, 2024-05-09T09:30:00.952 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 16968539, 2024-05-09T09:30:00.952 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:00.952 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.952 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.952 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.953 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:00.953 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.953 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-05-09T09:30:00.953 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:00.953 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-05-09T09:30:00.953 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:00.953 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-05-09T09:30:00.953 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-05-09T09:30:00.953 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:00.953 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-05-09T09:30:00.953 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 8138, 2024-05-09T09:30:00.953 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 8138, 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 8138, 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 8138, 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 2552, 2024-05-09T09:30:00.954 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 277504, 2024-05-09T09:30:00.955 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:00.955 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 2552, 2024-05-09T09:30:00.955 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-05-09T09:30:00.955 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-05-09T09:30:00.955 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 109, 2024-05-09T09:30:00.955 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 277504, 2024-05-09T09:30:00.955 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:00.955 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.955 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.955 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.955 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:00.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:00.955 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x564d007ca500": { 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 8869, 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 16079550, 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 8869, 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 631, 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 16079550, 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:00.956 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:00.957 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:00.957 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:00.957 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:00.957 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-05-09T09:30:01.723 INFO:tasks.check_counter:Daemon mds.c mds.exported=16 2024-05-09T09:30:01.723 INFO:tasks.check_counter:Daemon mds.c mds.imported=17 2024-05-09T09:30:01.723 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=570 2024-05-09T09:30:01.723 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=1877 2024-05-09T09:30:01.723 DEBUG:tasks.check_counter:Getting stats from f 2024-05-09T09:30:01.723 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-09T09:30:01.980 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:30:02.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:02 smithi012 ceph-mon[30140]: pgmap v502: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T09:30:02.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:02 smithi170 ceph-mon[33632]: pgmap v502: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T09:30:02.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:02 smithi016 ceph-mon[33335]: pgmap v502: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T09:30:03.762 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-05-09T09:30:03.763 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:30:03.763 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 15613, 2024-05-09T09:30:03.763 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 24146, 2024-05-09T09:30:03.763 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 5400235, 2024-05-09T09:30:03.763 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 20214769, 2024-05-09T09:30:03.763 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 16, 2024-05-09T09:30:03.763 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2024-05-09T09:30:03.763 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.969901813, 2024-05-09T09:30:03.763 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.404928751, 2024-05-09T09:30:03.763 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.530188430, 2024-05-09T09:30:03.763 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.085502161, 2024-05-09T09:30:03.763 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:30:03.763 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 24144, 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.812617816, 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000033657 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 30269, 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.004870273, 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000160 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 7995, 2024-05-09T09:30:03.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 8423, 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 2540861, 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 7238180, 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 18, 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.485243835, 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.181613325, 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.224710375, 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.089427685, 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8421, 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.402263721, 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000047769 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8733, 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.001445633, 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000165 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 229152 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 23738, 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 21118, 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 10136555, 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 9553052, 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 16, 2024-05-09T09:30:03.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2024-05-09T09:30:03.767 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.415208183, 2024-05-09T09:30:03.767 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.656351266, 2024-05-09T09:30:03.767 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.895300672, 2024-05-09T09:30:03.767 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.057507929, 2024-05-09T09:30:03.767 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:30:03.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 21118, 2024-05-09T09:30:03.767 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.605829807, 2024-05-09T09:30:03.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000028687 2024-05-09T09:30:03.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.767 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:30:03.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 40215, 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.006227524, 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000154 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 593056, 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 82512 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-05-09T09:30:03.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3259, 2024-05-09T09:30:03.769 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.295096238, 2024-05-09T09:30:03.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001317918 2024-05-09T09:30:03.769 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:03.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.769 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-05-09T09:30:03.769 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-05-09T09:30:03.769 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-05-09T09:30:03.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4942, 2024-05-09T09:30:03.769 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.716981207, 2024-05-09T09:30:03.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000145079 2024-05-09T09:30:03.770 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:03.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.770 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 5729, 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 5356, 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5356, 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 9.116675003, 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001702142 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 373, 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 11, 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-05-09T09:30:03.771 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-05-09T09:30:03.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 37, 2024-05-09T09:30:03.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 11, 2024-05-09T09:30:03.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 10, 2024-05-09T09:30:03.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 16, 2024-05-09T09:30:03.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 19, 2024-05-09T09:30:03.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 1187, 2024-05-09T09:30:03.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2024-05-09T09:30:03.772 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2024-05-09T09:30:03.772 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 5, 2024-05-09T09:30:03.772 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 11022, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 7741, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 370, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 552, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 31, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 139, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 32, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 1864, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 2235, 2024-05-09T09:30:03.773 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-05-09T09:30:03.774 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 618, 2024-05-09T09:30:03.774 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 3930405, 2024-05-09T09:30:03.774 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2024-05-09T09:30:03.774 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 2235, 2024-05-09T09:30:03.774 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-05-09T09:30:03.774 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 142, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 142, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 2093, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 28, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 614, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 3719, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 3465, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 1643, 2024-05-09T09:30:03.775 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 86, 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 6028 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 638, 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 1868, 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 1206, 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 1297, 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 1087, 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 6, 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 2630, 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 2624, 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-05-09T09:30:03.776 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 31, 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-05-09T09:30:03.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 9237, 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 9237, 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 12, 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 12, 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 2, 2024-05-09T09:30:03.778 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-05-09T09:30:03.779 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-05-09T09:30:03.779 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-05-09T09:30:03.779 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 20155765, 2024-05-09T09:30:03.779 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-05-09T09:30:03.779 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-05-09T09:30:03.779 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 951, 2024-05-09T09:30:03.779 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.963297498, 2024-05-09T09:30:03.779 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003115980 2024-05-09T09:30:03.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.780 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-05-09T09:30:03.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.780 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-05-09T09:30:03.780 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 31, 2024-05-09T09:30:03.780 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 3224, 2024-05-09T09:30:03.780 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 3193, 2024-05-09T09:30:03.780 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 29, 2024-05-09T09:30:03.780 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 860, 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 831, 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 37, 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 5670, 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 5633, 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 2711, 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 2711, 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 69976, 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 223612 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 8951, 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 302, 2024-05-09T09:30:03.781 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 5729, 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 5, 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 1871, 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 757, 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.122291735, 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001482551 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-05-09T09:30:03.782 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1706, 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.891012140, 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000522281 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-05-09T09:30:03.783 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-05-09T09:30:03.784 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.785 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.785 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.785 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-05-09T09:30:03.785 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.785 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.785 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.785 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-05-09T09:30:03.786 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.786 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.786 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.786 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-05-09T09:30:03.786 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 11, 2024-05-09T09:30:03.786 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.023045725, 2024-05-09T09:30:03.786 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002095065 2024-05-09T09:30:03.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.786 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-05-09T09:30:03.786 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.787 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 251, 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.328405548, 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001308388 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.788 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 107, 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.254721316, 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.021072161 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.789 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-05-09T09:30:03.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.790 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.790 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-05-09T09:30:03.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2024-05-09T09:30:03.790 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.001257865, 2024-05-09T09:30:03.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001257865 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.791 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-05-09T09:30:03.792 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2523, 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.495940674, 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001781982 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 1, 2024-05-09T09:30:03.793 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 0, 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 1, 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 3426, 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 3426, 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 272, 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:30:03.794 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:30:03.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:30:03.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:30:03.797 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 315407, 2024-05-09T09:30:03.797 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 121, 2024-05-09T09:30:03.797 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:30:03.797 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-05-09T09:30:03.797 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-05-09T09:30:03.797 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-05-09T09:30:03.797 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:30:03.797 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-05-09T09:30:03.797 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:30:03.797 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-05-09T09:30:03.797 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9080, 2024-05-09T09:30:03.797 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 174, 2024-05-09T09:30:03.797 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 136301, 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 1477, 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 8759, 2024-05-09T09:30:03.798 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 16432148, 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 28, 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 8759, 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8759, 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 24.032434007, 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002743741 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8759, 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 8896 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 8759, 2024-05-09T09:30:03.799 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 2348, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 6411, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 113, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 1064, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 67, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 5178, 2024-05-09T09:30:03.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-05-09T09:30:03.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-05-09T09:30:03.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-05-09T09:30:03.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 2235, 2024-05-09T09:30:03.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2024-05-09T09:30:03.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:30:03.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-05-09T09:30:03.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:30:03.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-05-09T09:30:03.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-05-09T09:30:03.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-05-09T09:30:03.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:30:03.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-05-09T09:30:03.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:30:03.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-05-09T09:30:03.802 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-05-09T09:30:03.802 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-05-09T09:30:03.802 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-05-09T09:30:03.802 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-05-09T09:30:03.802 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 94, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 5, 2024-05-09T09:30:03.803 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 149, 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 0, 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 60 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.804 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-05-09T09:30:03.805 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-05-09T09:30:03.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.805 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-05-09T09:30:03.805 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-05-09T09:30:03.805 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 10, 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 8, 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 5163, 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 2624, 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 47342, 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 13804777, 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:03.806 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 47342, 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 47342, 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 13804777, 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:03.807 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-05-09T09:30:03.808 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-05-09T09:30:03.808 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:03.808 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-05-09T09:30:03.808 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 8759, 2024-05-09T09:30:03.808 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 16717372, 2024-05-09T09:30:03.808 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 8759, 2024-05-09T09:30:03.808 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 16717372, 2024-05-09T09:30:03.808 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:03.808 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.808 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 8759, 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 8759, 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 8759, 2024-05-09T09:30:03.809 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 8759, 2024-05-09T09:30:03.810 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:03.810 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.810 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.810 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.810 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:03.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.810 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:30:03.810 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:03.810 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-05-09T09:30:03.810 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:03.810 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 2624, 2024-05-09T09:30:03.811 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 285224, 2024-05-09T09:30:03.811 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:03.811 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 2624, 2024-05-09T09:30:03.811 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-05-09T09:30:03.811 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-05-09T09:30:03.811 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 113, 2024-05-09T09:30:03.811 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 285224, 2024-05-09T09:30:03.811 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:03.811 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55930537e500": { 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 9237, 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 15961461, 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 9237, 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-05-09T09:30:03.812 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 951, 2024-05-09T09:30:03.813 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 15961461, 2024-05-09T09:30:03.813 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:03.813 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:03.813 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:03.813 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:03.813 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:03.813 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:03.813 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-05-09T09:30:04.394 INFO:tasks.check_counter:Daemon mds.f mds.exported=31 2024-05-09T09:30:04.394 INFO:tasks.check_counter:Daemon mds.f mds.imported=32 2024-05-09T09:30:04.394 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=638 2024-05-09T09:30:04.394 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=1868 2024-05-09T09:30:04.395 DEBUG:tasks.check_counter:Getting stats from i 2024-05-09T09:30:04.395 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-09T09:30:04.638 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:30:04.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:04 smithi012 ceph-mon[30140]: pgmap v503: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:04.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:04 smithi170 ceph-mon[33632]: pgmap v503: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:04.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:04 smithi016 ceph-mon[33335]: pgmap v503: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:06.319 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:06 smithi170 ceph-mon[33632]: pgmap v504: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:30:06.357 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-05-09T09:30:06.358 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:30:06.358 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 32910, 2024-05-09T09:30:06.358 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 39658, 2024-05-09T09:30:06.358 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 10338233, 2024-05-09T09:30:06.358 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 28048510, 2024-05-09T09:30:06.358 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 9, 2024-05-09T09:30:06.358 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-05-09T09:30:06.359 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 2.002237134, 2024-05-09T09:30:06.359 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.888938986, 2024-05-09T09:30:06.359 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 1.104795378, 2024-05-09T09:30:06.359 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.177873268, 2024-05-09T09:30:06.359 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:30:06.359 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 39661, 2024-05-09T09:30:06.359 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 5.575957670, 2024-05-09T09:30:06.359 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000140590 2024-05-09T09:30:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:30:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 59239, 2024-05-09T09:30:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.010069763, 2024-05-09T09:30:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000169 2024-05-09T09:30:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 615680, 2024-05-09T09:30:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 90800 2024-05-09T09:30:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:30:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 50390, 2024-05-09T09:30:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 81634, 2024-05-09T09:30:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 16207892, 2024-05-09T09:30:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 56808772, 2024-05-09T09:30:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 15, 2024-05-09T09:30:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2024-05-09T09:30:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 3.048635308, 2024-05-09T09:30:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 1.272914505, 2024-05-09T09:30:06.362 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 1.698725151, 2024-05-09T09:30:06.362 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.270764340, 2024-05-09T09:30:06.362 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:30:06.362 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 81632, 2024-05-09T09:30:06.362 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 7.596595975, 2024-05-09T09:30:06.362 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000093059 2024-05-09T09:30:06.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.362 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:30:06.363 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 111166, 2024-05-09T09:30:06.363 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.017200738, 2024-05-09T09:30:06.363 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000154 2024-05-09T09:30:06.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.363 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 44864, 2024-05-09T09:30:06.363 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 270160 2024-05-09T09:30:06.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.363 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:30:06.363 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 22564, 2024-05-09T09:30:06.364 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 26384, 2024-05-09T09:30:06.364 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 6654675, 2024-05-09T09:30:06.364 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 38923766, 2024-05-09T09:30:06.364 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 106, 2024-05-09T09:30:06.364 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2024-05-09T09:30:06.364 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.639995210, 2024-05-09T09:30:06.364 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.700765854, 2024-05-09T09:30:06.364 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.696425385, 2024-05-09T09:30:06.364 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.273300439, 2024-05-09T09:30:06.365 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:30:06.365 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 26379, 2024-05-09T09:30:06.365 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 9.099254005, 2024-05-09T09:30:06.365 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000344943 2024-05-09T09:30:06.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.365 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:30:06.365 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 28737, 2024-05-09T09:30:06.365 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.005829271, 2024-05-09T09:30:06.366 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000202 2024-05-09T09:30:06.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.366 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:30:06.366 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:30:06.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.366 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-05-09T09:30:06.366 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-05-09T09:30:06.366 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-05-09T09:30:06.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.367 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-05-09T09:30:06.367 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-05-09T09:30:06.367 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-05-09T09:30:06.367 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 6758, 2024-05-09T09:30:06.367 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 15.216125827, 2024-05-09T09:30:06.367 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002251572 2024-05-09T09:30:06.367 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:06.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.367 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-05-09T09:30:06.368 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-05-09T09:30:06.368 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-05-09T09:30:06.368 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 6120, 2024-05-09T09:30:06.368 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.722987944, 2024-05-09T09:30:06.368 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000118135 2024-05-09T09:30:06.368 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:06.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.368 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-05-09T09:30:06.369 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 16660, 2024-05-09T09:30:06.369 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 14948, 2024-05-09T09:30:06.369 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-05-09T09:30:06.369 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 14948, 2024-05-09T09:30:06.369 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 40.016912455, 2024-05-09T09:30:06.369 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002677074 2024-05-09T09:30:06.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.369 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-05-09T09:30:06.369 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 1712, 2024-05-09T09:30:06.370 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 9, 2024-05-09T09:30:06.370 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 3, 2024-05-09T09:30:06.370 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 872, 2024-05-09T09:30:06.370 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 10, 2024-05-09T09:30:06.370 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 3, 2024-05-09T09:30:06.370 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 89, 2024-05-09T09:30:06.370 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 18, 2024-05-09T09:30:06.371 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 58, 2024-05-09T09:30:06.371 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 13, 2024-05-09T09:30:06.371 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 37, 2024-05-09T09:30:06.371 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 6954, 2024-05-09T09:30:06.371 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 2, 2024-05-09T09:30:06.371 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 2, 2024-05-09T09:30:06.371 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 6, 2024-05-09T09:30:06.372 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 21019, 2024-05-09T09:30:06.372 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 18050, 2024-05-09T09:30:06.372 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 1548, 2024-05-09T09:30:06.372 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 361, 2024-05-09T09:30:06.372 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:30:06.372 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-05-09T09:30:06.372 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-05-09T09:30:06.373 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-05-09T09:30:06.373 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-05-09T09:30:06.373 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 82, 2024-05-09T09:30:06.373 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 6280, 2024-05-09T09:30:06.373 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 87, 2024-05-09T09:30:06.373 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 878, 2024-05-09T09:30:06.373 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-05-09T09:30:06.374 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 2154, 2024-05-09T09:30:06.374 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-05-09T09:30:06.374 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 4, 2024-05-09T09:30:06.374 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 234, 2024-05-09T09:30:06.374 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2024-05-09T09:30:06.375 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 2154, 2024-05-09T09:30:06.375 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-05-09T09:30:06.375 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:30:06.375 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:30:06.375 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 333, 2024-05-09T09:30:06.375 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 8, 2024-05-09T09:30:06.376 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 325, 2024-05-09T09:30:06.376 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 1821, 2024-05-09T09:30:06.376 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 8, 2024-05-09T09:30:06.376 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 3286, 2024-05-09T09:30:06.376 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 12035, 2024-05-09T09:30:06.376 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:30:06.376 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:30:06.377 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:30:06.377 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 12492, 2024-05-09T09:30:06.377 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 6803, 2024-05-09T09:30:06.377 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 130, 2024-05-09T09:30:06.377 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 15165 2024-05-09T09:30:06.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.377 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-05-09T09:30:06.378 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 1128, 2024-05-09T09:30:06.378 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 1185, 2024-05-09T09:30:06.378 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 773, 2024-05-09T09:30:06.378 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 775, 2024-05-09T09:30:06.378 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 2714, 2024-05-09T09:30:06.378 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 7, 2024-05-09T09:30:06.379 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-05-09T09:30:06.379 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:30:06.379 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 3116, 2024-05-09T09:30:06.379 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 3109, 2024-05-09T09:30:06.379 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-05-09T09:30:06.379 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-05-09T09:30:06.379 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-05-09T09:30:06.380 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:30:06.380 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:30:06.380 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-05-09T09:30:06.380 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-05-09T09:30:06.381 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:30:06.381 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:30:06.381 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 8, 2024-05-09T09:30:06.381 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 83, 2024-05-09T09:30:06.381 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-05-09T09:30:06.381 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 13, 2024-05-09T09:30:06.382 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-05-09T09:30:06.382 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-05-09T09:30:06.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.382 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-05-09T09:30:06.382 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-05-09T09:30:06.382 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 25357, 2024-05-09T09:30:06.383 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 25303, 2024-05-09T09:30:06.383 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 25303, 2024-05-09T09:30:06.383 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 54, 2024-05-09T09:30:06.383 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-05-09T09:30:06.383 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 2321, 2024-05-09T09:30:06.383 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 46, 2024-05-09T09:30:06.384 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 44, 2024-05-09T09:30:06.384 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 44, 2024-05-09T09:30:06.384 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 2, 2024-05-09T09:30:06.384 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 1, 2024-05-09T09:30:06.384 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-05-09T09:30:06.384 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 4, 2024-05-09T09:30:06.384 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 54471075, 2024-05-09T09:30:06.385 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 54566662, 2024-05-09T09:30:06.385 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-05-09T09:30:06.385 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-05-09T09:30:06.385 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2684, 2024-05-09T09:30:06.385 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 46.121769966, 2024-05-09T09:30:06.385 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.017183967 2024-05-09T09:30:06.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.386 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-05-09T09:30:06.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.386 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-05-09T09:30:06.386 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 42, 2024-05-09T09:30:06.386 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 9765, 2024-05-09T09:30:06.386 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 9723, 2024-05-09T09:30:06.387 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 48, 2024-05-09T09:30:06.387 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 1160, 2024-05-09T09:30:06.387 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 1112, 2024-05-09T09:30:06.387 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 89, 2024-05-09T09:30:06.387 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 12612, 2024-05-09T09:30:06.387 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 12523, 2024-05-09T09:30:06.387 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 2, 2024-05-09T09:30:06.388 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 9476, 2024-05-09T09:30:06.388 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 9474, 2024-05-09T09:30:06.388 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 95144, 2024-05-09T09:30:06.388 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 231804 2024-05-09T09:30:06.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.388 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-05-09T09:30:06.389 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 21627, 2024-05-09T09:30:06.389 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 3821, 2024-05-09T09:30:06.389 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 16660, 2024-05-09T09:30:06.389 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 96, 2024-05-09T09:30:06.389 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 8224, 2024-05-09T09:30:06.389 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-05-09T09:30:06.389 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8242, 2024-05-09T09:30:06.390 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 17.474183681, 2024-05-09T09:30:06.390 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002120138 2024-05-09T09:30:06.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.390 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-05-09T09:30:06.390 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 52, 2024-05-09T09:30:06.390 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.028821302, 2024-05-09T09:30:06.390 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000554255 2024-05-09T09:30:06.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.391 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-05-09T09:30:06.391 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.391 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.391 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.391 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-05-09T09:30:06.392 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.392 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.392 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.392 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-05-09T09:30:06.392 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2521, 2024-05-09T09:30:06.392 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 12.491965297, 2024-05-09T09:30:06.393 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004955162 2024-05-09T09:30:06.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.393 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-05-09T09:30:06.393 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.393 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.393 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.394 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-05-09T09:30:06.394 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.394 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.394 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.395 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-05-09T09:30:06.395 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.395 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.395 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.395 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-05-09T09:30:06.396 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.396 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.396 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.396 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-05-09T09:30:06.396 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.396 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.397 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.398 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-05-09T09:30:06.398 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.398 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.398 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.399 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-05-09T09:30:06.399 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 612, 2024-05-09T09:30:06.399 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.792407395, 2024-05-09T09:30:06.399 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001294783 2024-05-09T09:30:06.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.399 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-05-09T09:30:06.400 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.400 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.400 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.400 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-05-09T09:30:06.400 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.400 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.401 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.401 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-05-09T09:30:06.401 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2024-05-09T09:30:06.401 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000437637, 2024-05-09T09:30:06.401 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000437637 2024-05-09T09:30:06.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.402 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-05-09T09:30:06.402 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 371, 2024-05-09T09:30:06.403 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.522764478, 2024-05-09T09:30:06.403 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001409068 2024-05-09T09:30:06.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.403 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-05-09T09:30:06.403 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2, 2024-05-09T09:30:06.403 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.002070606, 2024-05-09T09:30:06.404 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001035303 2024-05-09T09:30:06.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.404 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-05-09T09:30:06.404 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.404 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.404 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.405 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-05-09T09:30:06.405 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.405 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.405 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.406 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-05-09T09:30:06.406 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 286, 2024-05-09T09:30:06.406 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3.836671157, 2024-05-09T09:30:06.406 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.013414934 2024-05-09T09:30:06.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.406 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-05-09T09:30:06.407 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.407 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.407 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.408 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-05-09T09:30:06.408 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.408 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.408 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.408 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-05-09T09:30:06.409 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 23, 2024-05-09T09:30:06.409 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.029483258, 2024-05-09T09:30:06.409 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001281880 2024-05-09T09:30:06.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.409 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-05-09T09:30:06.409 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.410 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.410 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.410 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-05-09T09:30:06.410 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.410 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.411 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.411 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-05-09T09:30:06.411 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.411 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.411 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.412 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-05-09T09:30:06.412 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8, 2024-05-09T09:30:06.412 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.020250988, 2024-05-09T09:30:06.412 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002531373 2024-05-09T09:30:06.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.416 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-05-09T09:30:06.416 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.416 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.416 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.416 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-05-09T09:30:06.416 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2830, 2024-05-09T09:30:06.416 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.817856656, 2024-05-09T09:30:06.417 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001702422 2024-05-09T09:30:06.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.417 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:30:06.417 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:30:06.417 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-05-09T09:30:06.417 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.417 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.417 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.417 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:06.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.417 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-05-09T09:30:06.417 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2024-05-09T09:30:06.417 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 3, 2024-05-09T09:30:06.418 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 2, 2024-05-09T09:30:06.418 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 1, 2024-05-09T09:30:06.418 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-05-09T09:30:06.418 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 7442, 2024-05-09T09:30:06.418 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 7442, 2024-05-09T09:30:06.418 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 276, 2024-05-09T09:30:06.418 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-05-09T09:30:06.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.418 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-05-09T09:30:06.418 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:30:06.418 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-05-09T09:30:06.418 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:30:06.418 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:30:06.419 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:30:06.419 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:30:06.419 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:30:06.419 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:30:06.419 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:30:06.419 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:30:06.420 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:30:06.420 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:30:06.420 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:30:06.420 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:30:06.420 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:30:06.420 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-05-09T09:30:06.420 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:30:06.421 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-05-09T09:30:06.421 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:30:06.421 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:30:06.421 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:30:06.421 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:30:06.421 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:30:06.421 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:30:06.421 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:30:06.421 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-05-09T09:30:06.421 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:30:06.421 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:30:06.421 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:30:06.422 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-05-09T09:30:06.422 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-05-09T09:30:06.422 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-05-09T09:30:06.422 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:30:06.422 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:30:06.422 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:30:06.422 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:30:06.422 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 312927, 2024-05-09T09:30:06.422 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 113, 2024-05-09T09:30:06.422 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:30:06.422 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-05-09T09:30:06.422 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-05-09T09:30:06.423 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-05-09T09:30:06.423 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:30:06.423 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-05-09T09:30:06.423 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:30:06.423 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-05-09T09:30:06.423 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9080, 2024-05-09T09:30:06.423 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 174, 2024-05-09T09:30:06.423 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:30:06.423 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:30:06.423 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-05-09T09:30:06.423 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-05-09T09:30:06.423 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 229464, 2024-05-09T09:30:06.424 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 2433, 2024-05-09T09:30:06.424 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-05-09T09:30:06.424 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-05-09T09:30:06.424 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-05-09T09:30:06.424 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-05-09T09:30:06.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.424 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-05-09T09:30:06.424 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-05-09T09:30:06.424 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-05-09T09:30:06.424 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 24228, 2024-05-09T09:30:06.424 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 62285514, 2024-05-09T09:30:06.424 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 37, 2024-05-09T09:30:06.424 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 24228, 2024-05-09T09:30:06.425 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-05-09T09:30:06.425 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 24228, 2024-05-09T09:30:06.425 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3858.147389636, 2024-05-09T09:30:06.425 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.159243329 2024-05-09T09:30:06.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.426 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-05-09T09:30:06.426 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-05-09T09:30:06.426 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 24228, 2024-05-09T09:30:06.426 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 42935 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 24228, 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 2351, 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 21877, 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 313, 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 11306, 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 177, 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 2853, 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 162, 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-05-09T09:30:06.427 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 10, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 6528, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 2174, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 17308, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-05-09T09:30:06.428 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-05-09T09:30:06.429 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 94, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 10, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 1958, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 24, 2024-05-09T09:30:06.430 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 122 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 21, 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-05-09T09:30:06.431 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 9, 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 5987, 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 3109, 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 105860, 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 23655004, 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 105860, 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-05-09T09:30:06.432 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 105860, 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 23655004, 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-05-09T09:30:06.433 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-05-09T09:30:06.434 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:06.434 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-05-09T09:30:06.434 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 24228, 2024-05-09T09:30:06.434 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 62622031, 2024-05-09T09:30:06.434 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 24228, 2024-05-09T09:30:06.434 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 62622031, 2024-05-09T09:30:06.434 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:06.434 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.436 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.436 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.436 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:06.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.436 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-05-09T09:30:06.436 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:06.436 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-05-09T09:30:06.436 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 24228, 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 24228, 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 24228, 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 24228, 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.437 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:30:06.438 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:06.438 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-05-09T09:30:06.438 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:06.438 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 3109, 2024-05-09T09:30:06.438 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 336673, 2024-05-09T09:30:06.438 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:06.438 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 3109, 2024-05-09T09:30:06.438 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-05-09T09:30:06.438 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-05-09T09:30:06.438 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 169, 2024-05-09T09:30:06.438 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 336673, 2024-05-09T09:30:06.438 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:06.439 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.439 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.439 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.439 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:06.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:06.439 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x556d6ccb0500": { 2024-05-09T09:30:06.439 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:06.439 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-05-09T09:30:06.439 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:06.439 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 25357, 2024-05-09T09:30:06.439 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 50372358, 2024-05-09T09:30:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 25357, 2024-05-09T09:30:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-05-09T09:30:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-05-09T09:30:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2684, 2024-05-09T09:30:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 50372358, 2024-05-09T09:30:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:06.440 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-05-09T09:30:06.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:06 smithi012 ceph-mon[30140]: pgmap v504: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:30:06.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:06 smithi016 ceph-mon[33335]: pgmap v504: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:30:07.098 INFO:tasks.check_counter:Daemon mds.i mds.exported=82 2024-05-09T09:30:07.098 INFO:tasks.check_counter:Daemon mds.i mds.imported=87 2024-05-09T09:30:07.098 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=1128 2024-05-09T09:30:07.098 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=1185 2024-05-09T09:30:07.098 DEBUG:tasks.check_counter:Getting stats from l 2024-05-09T09:30:07.099 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-09T09:30:07.343 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:30:08.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:08 smithi170 ceph-mon[33632]: pgmap v505: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:08.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:08 smithi012 ceph-mon[30140]: pgmap v505: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:08.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:08 smithi016 ceph-mon[33335]: pgmap v505: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:09.069 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-05-09T09:30:09.069 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:30:09.069 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 238, 2024-05-09T09:30:09.069 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 218, 2024-05-09T09:30:09.069 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 603934, 2024-05-09T09:30:09.069 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 104320, 2024-05-09T09:30:09.069 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 6, 2024-05-09T09:30:09.069 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 2, 2024-05-09T09:30:09.069 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.029522401, 2024-05-09T09:30:09.070 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.010028740, 2024-05-09T09:30:09.070 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.014328377, 2024-05-09T09:30:09.070 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.005374515, 2024-05-09T09:30:09.070 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:30:09.070 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 217, 2024-05-09T09:30:09.070 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.013154777, 2024-05-09T09:30:09.070 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000060621 2024-05-09T09:30:09.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.070 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:30:09.071 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.071 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.071 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.071 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 582928, 2024-05-09T09:30:09.071 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 77920 2024-05-09T09:30:09.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.071 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:30:09.071 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 366, 2024-05-09T09:30:09.072 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 458, 2024-05-09T09:30:09.072 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 7784570, 2024-05-09T09:30:09.072 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 346531, 2024-05-09T09:30:09.072 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2024-05-09T09:30:09.072 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-05-09T09:30:09.072 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.039677915, 2024-05-09T09:30:09.072 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.012792574, 2024-05-09T09:30:09.072 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.012816697, 2024-05-09T09:30:09.073 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.010489789, 2024-05-09T09:30:09.073 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:30:09.073 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 457, 2024-05-09T09:30:09.073 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.016839086, 2024-05-09T09:30:09.073 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000036847 2024-05-09T09:30:09.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.073 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:30:09.073 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.073 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.074 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.074 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-09T09:30:09.074 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 231728 2024-05-09T09:30:09.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.074 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:30:09.074 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 754, 2024-05-09T09:30:09.074 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 756, 2024-05-09T09:30:09.074 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 8617745, 2024-05-09T09:30:09.074 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 239830, 2024-05-09T09:30:09.074 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2024-05-09T09:30:09.074 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-05-09T09:30:09.074 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.072043729, 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.020004395, 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.024037678, 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.022005830, 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 754, 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.028534606, 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000037844 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.075 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2030, 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.053567625, 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000026387 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:09.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 0, 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 0, 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.077 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 0, 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 0, 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 2718, 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 0, 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 2718, 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 0, 2024-05-09T09:30:09.078 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 2718, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 0, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 1, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 0, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 0, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2024-05-09T09:30:09.079 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:30:09.080 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 0, 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 0, 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 0, 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 0 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-05-09T09:30:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:30:09.082 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:30:09.083 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-05-09T09:30:09.083 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-05-09T09:30:09.083 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:30:09.083 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:30:09.083 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:30:09.083 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2024-05-09T09:30:09.083 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-05-09T09:30:09.083 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-05-09T09:30:09.083 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-05-09T09:30:09.083 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-05-09T09:30:09.084 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.084 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-05-09T09:30:09.084 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-05-09T09:30:09.084 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 0, 2024-05-09T09:30:09.084 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-05-09T09:30:09.084 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-05-09T09:30:09.084 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 8869, 2024-05-09T09:30:09.084 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-05-09T09:30:09.085 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-05-09T09:30:09.085 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 0, 2024-05-09T09:30:09.085 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-05-09T09:30:09.085 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-05-09T09:30:09.085 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 12, 2024-05-09T09:30:09.085 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 2, 2024-05-09T09:30:09.085 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-05-09T09:30:09.085 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-05-09T09:30:09.085 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-05-09T09:30:09.085 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 20273854, 2024-05-09T09:30:09.086 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 20272716, 2024-05-09T09:30:09.086 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-05-09T09:30:09.086 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.086 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.086 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.086 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 8869 2024-05-09T09:30:09.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.086 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-05-09T09:30:09.086 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 15, 2024-05-09T09:30:09.086 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 2716, 2024-05-09T09:30:09.086 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 2701, 2024-05-09T09:30:09.086 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 20, 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 186, 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 166, 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 2718, 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 4405, 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 1687, 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 0, 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 0, 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 54468, 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 215420 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 0, 2024-05-09T09:30:09.087 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2024-05-09T09:30:09.088 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 0, 2024-05-09T09:30:09.088 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 0, 2024-05-09T09:30:09.088 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 0, 2024-05-09T09:30:09.088 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-05-09T09:30:09.088 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.088 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.088 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.088 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-05-09T09:30:09.088 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.088 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.088 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.090 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-05-09T09:30:09.090 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.090 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.090 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.090 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-05-09T09:30:09.090 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.090 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.090 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.090 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-05-09T09:30:09.091 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.091 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.091 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.091 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-05-09T09:30:09.091 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.091 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.091 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.091 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-05-09T09:30:09.091 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.092 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.092 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.092 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-05-09T09:30:09.092 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.092 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.092 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.092 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-05-09T09:30:09.093 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.093 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.093 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.094 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-05-09T09:30:09.094 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.094 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.094 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.094 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-05-09T09:30:09.094 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.094 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.094 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.094 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-05-09T09:30:09.094 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.094 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-05-09T09:30:09.095 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.096 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.096 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.096 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-05-09T09:30:09.096 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.096 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.096 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.096 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-05-09T09:30:09.096 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.096 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.096 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.097 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-05-09T09:30:09.097 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.097 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.097 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.097 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-05-09T09:30:09.097 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.097 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.098 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.098 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-05-09T09:30:09.098 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.098 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.098 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.098 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-05-09T09:30:09.098 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.098 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.098 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.099 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-05-09T09:30:09.099 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.099 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.099 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.099 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-05-09T09:30:09.099 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.099 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.099 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.100 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 2, 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 1, 2024-05-09T09:30:09.101 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 1, 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 0, 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 0, 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 1371, 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:30:09.102 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-05-09T09:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-05-09T09:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-05-09T09:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-05-09T09:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:30:09.105 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:30:09.105 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:30:09.105 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 206263, 2024-05-09T09:30:09.105 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 60, 2024-05-09T09:30:09.105 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:30:09.105 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-05-09T09:30:09.105 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-05-09T09:30:09.105 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-05-09T09:30:09.105 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:30:09.105 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-05-09T09:30:09.105 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:30:09.105 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-05-09T09:30:09.105 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9080, 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 174, 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 1974586, 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 11899, 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-05-09T09:30:09.106 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-05-09T09:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 1198, 2024-05-09T09:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 21, 2024-05-09T09:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 3, 2024-05-09T09:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 1198, 2024-05-09T09:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-05-09T09:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1198, 2024-05-09T09:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.735982726, 2024-05-09T09:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000614342 2024-05-09T09:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-05-09T09:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-05-09T09:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1198, 2024-05-09T09:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1199 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 1198, 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 1198, 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 0, 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 727, 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 0, 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 469, 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 0, 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 0, 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-05-09T09:30:09.108 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 0, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 1, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 0, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-05-09T09:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-05-09T09:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 94, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 5, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 7, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 7, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 0, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2024-05-09T09:30:09.111 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 0 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 0, 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T09:30:09.112 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 0, 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 0, 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 0, 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 1354, 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 16873568, 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 1354, 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-05-09T09:30:09.113 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-05-09T09:30:09.114 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1354, 2024-05-09T09:30:09.114 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 16873568, 2024-05-09T09:30:09.114 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:09.114 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.114 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.114 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.114 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:09.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.114 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-05-09T09:30:09.114 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 1198, 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 16082801, 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1198, 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 16082801, 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 1198, 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 1198, 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1198, 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1198, 2024-05-09T09:30:09.116 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-05-09T09:30:09.117 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x5574727da6e0": { 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-05-09T09:30:09.118 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-05-09T09:30:09.119 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-05-09T09:30:09.119 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-05-09T09:30:09.119 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-05-09T09:30:09.119 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-05-09T09:30:09.119 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-05-09T09:30:09.119 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-05-09T09:30:09.119 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-05-09T09:30:09.119 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-05-09T09:30:09.119 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-05-09T09:30:09.119 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:09.119 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-05-09T09:30:09.119 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-05-09T09:30:09.833 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-09T09:30:09.833 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-09T09:30:09.833 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-05-09T09:30:09.834 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-05-09T09:30:09.834 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-09T09:30:10.037 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-09T09:30:10.037 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:30:10.037 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T09:30:10.059 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:30:10.060 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T09:30:10.121 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:30:10.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.280+0000 7efe92ad4640 1 -- 172.21.15.12:0/3068313413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe8c101980 msgr2=0x7efe8c101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:10.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.280+0000 7efe92ad4640 1 --2- 172.21.15.12:0/3068313413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe8c101980 0x7efe8c101e00 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009ca0 tx=0x7efe7c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.280+0000 7efe92ad4640 1 -- 172.21.15.12:0/3068313413 shutdown_connections 2024-05-09T09:30:10.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.280+0000 7efe92ad4640 1 --2- 172.21.15.12:0/3068313413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c102340 0x7efe8c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.280+0000 7efe92ad4640 1 --2- 172.21.15.12:0/3068313413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe8c101980 0x7efe8c101e00 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.280+0000 7efe92ad4640 1 --2- 172.21.15.12:0/3068313413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe8c100790 0x7efe8c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.280+0000 7efe92ad4640 1 -- 172.21.15.12:0/3068313413 >> 172.21.15.12:0/3068313413 conn(0x7efe8c0fbd20 msgr2=0x7efe8c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:10.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.280+0000 7efe92ad4640 1 -- 172.21.15.12:0/3068313413 shutdown_connections 2024-05-09T09:30:10.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.280+0000 7efe92ad4640 1 -- 172.21.15.12:0/3068313413 wait complete. 2024-05-09T09:30:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe92ad4640 1 Processor -- start 2024-05-09T09:30:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe92ad4640 1 -- start start 2024-05-09T09:30:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe92ad4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe8c100790 0x7efe8c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe92ad4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe8c101980 0x7efe8c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe92ad4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c102340 0x7efe8c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe92ad4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c10d5a0 con 0x7efe8c101980 2024-05-09T09:30:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe92ad4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c10d420 con 0x7efe8c102340 2024-05-09T09:30:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe92ad4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c10d720 con 0x7efe8c100790 2024-05-09T09:30:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe8c101980 0x7efe8c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe8c101980 0x7efe8c198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36598/0 (socket says 172.21.15.12:36598) 2024-05-09T09:30:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe8bfff640 1 -- 172.21.15.12:0/1007500187 learned_addr learned my addr 172.21.15.12:0/1007500187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:10.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe91071640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c102340 0x7efe8c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:10.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe90870640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe8c100790 0x7efe8c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:10.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe8bfff640 1 -- 172.21.15.12:0/1007500187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe8c100790 msgr2=0x7efe8c197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:10.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe8bfff640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe8c100790 0x7efe8c197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe8bfff640 1 -- 172.21.15.12:0/1007500187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c102340 msgr2=0x7efe8c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:10.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe8bfff640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c102340 0x7efe8c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe8bfff640 1 -- 172.21.15.12:0/1007500187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8c19f770 con 0x7efe8c101980 2024-05-09T09:30:10.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.281+0000 7efe91071640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c102340 0x7efe8c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:30:10.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.282+0000 7efe8bfff640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe8c101980 0x7efe8c198090 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7efe7c000f80 tx=0x7efe7c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:10.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.282+0000 7efe89ffb640 1 -- 172.21.15.12:0/1007500187 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c004040 con 0x7efe8c101980 2024-05-09T09:30:10.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.282+0000 7efe89ffb640 1 -- 172.21.15.12:0/1007500187 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efe7c0046b0 con 0x7efe8c101980 2024-05-09T09:30:10.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.282+0000 7efe92ad4640 1 -- 172.21.15.12:0/1007500187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8c19fa00 con 0x7efe8c101980 2024-05-09T09:30:10.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.282+0000 7efe89ffb640 1 -- 172.21.15.12:0/1007500187 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0305c0 con 0x7efe8c101980 2024-05-09T09:30:10.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.282+0000 7efe92ad4640 1 -- 172.21.15.12:0/1007500187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8c1a0020 con 0x7efe8c101980 2024-05-09T09:30:10.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.283+0000 7efe92ad4640 1 -- 172.21.15.12:0/1007500187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8c107ab0 con 0x7efe8c101980 2024-05-09T09:30:10.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.284+0000 7efe89ffb640 1 -- 172.21.15.12:0/1007500187 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efe7c02f6a0 con 0x7efe8c101980 2024-05-09T09:30:10.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.285+0000 7efe89ffb640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efe780a2c10 0x7efe780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:10.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.285+0000 7efe89ffb640 1 -- 172.21.15.12:0/1007500187 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7efe7c0f0e80 con 0x7efe8c101980 2024-05-09T09:30:10.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.285+0000 7efe90870640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efe780a2c10 0x7efe780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:10.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.286+0000 7efe90870640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efe780a2c10 0x7efe780a50d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efe740058c0 tx=0x7efe74005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:10.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.289+0000 7efe89ffb640 1 -- 172.21.15.12:0/1007500187 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe7c0443d0 con 0x7efe8c101980 2024-05-09T09:30:10.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.481+0000 7efe92ad4640 1 -- 172.21.15.12:0/1007500187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe8c02d410 con 0x7efe8c101980 2024-05-09T09:30:10.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.482+0000 7efe89ffb640 1 -- 172.21.15.12:0/1007500187 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24762 (secure 0 0 0) 0x7efe7c0b7e20 con 0x7efe8c101980 2024-05-09T09:30:10.484 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:30:10.484 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":94,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:27:40.951852+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:30:10.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:10 smithi012 ceph-mon[30140]: pgmap v506: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:10.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.485+0000 7efe92ad4640 1 -- 172.21.15.12:0/1007500187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efe780a2c10 msgr2=0x7efe780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.485+0000 7efe92ad4640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efe780a2c10 0x7efe780a50d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efe740058c0 tx=0x7efe74005850 comp rx=0 tx=0).stop 2024-05-09T09:30:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.485+0000 7efe92ad4640 1 -- 172.21.15.12:0/1007500187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe8c101980 msgr2=0x7efe8c198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.485+0000 7efe92ad4640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe8c101980 0x7efe8c198090 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7efe7c000f80 tx=0x7efe7c003ba0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.485+0000 7efe92ad4640 1 -- 172.21.15.12:0/1007500187 shutdown_connections 2024-05-09T09:30:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.485+0000 7efe92ad4640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7efe780a2c10 0x7efe780a50d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.485+0000 7efe92ad4640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c102340 0x7efe8c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.485+0000 7efe92ad4640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe8c101980 0x7efe8c198090 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.485+0000 7efe92ad4640 1 --2- 172.21.15.12:0/1007500187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe8c100790 0x7efe8c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.485+0000 7efe92ad4640 1 -- 172.21.15.12:0/1007500187 >> 172.21.15.12:0/1007500187 conn(0x7efe8c0fbd20 msgr2=0x7efe8c104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.485+0000 7efe92ad4640 1 -- 172.21.15.12:0/1007500187 shutdown_connections 2024-05-09T09:30:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.485+0000 7efe92ad4640 1 -- 172.21.15.12:0/1007500187 wait complete. 2024-05-09T09:30:10.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:23:08.167107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:23:09.065670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:23:12.773640+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:30:10.497 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:30:10.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.606+0000 7f87e7e22640 1 -- 172.21.15.12:0/2007049234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87e0100180 msgr2=0x7f87e01005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:10.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.606+0000 7f87e7e22640 1 --2- 172.21.15.12:0/2007049234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87e0100180 0x7f87e01005a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009900 tx=0x7f87dc02a480 comp rx=0 tx=0).stop 2024-05-09T09:30:10.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.606+0000 7f87e7e22640 1 -- 172.21.15.12:0/2007049234 shutdown_connections 2024-05-09T09:30:10.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.606+0000 7f87e7e22640 1 --2- 172.21.15.12:0/2007049234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e0101480 0x7f87e0101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.606+0000 7f87e7e22640 1 --2- 172.21.15.12:0/2007049234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e0100ae0 0x7f87e0100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.606+0000 7f87e7e22640 1 --2- 172.21.15.12:0/2007049234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87e0100180 0x7f87e01005a0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.606+0000 7f87e7e22640 1 -- 172.21.15.12:0/2007049234 >> 172.21.15.12:0/2007049234 conn(0x7f87e00fbd20 msgr2=0x7f87e00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:10.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.607+0000 7f87e7e22640 1 -- 172.21.15.12:0/2007049234 shutdown_connections 2024-05-09T09:30:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.607+0000 7f87e7e22640 1 -- 172.21.15.12:0/2007049234 wait complete. 2024-05-09T09:30:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.607+0000 7f87e7e22640 1 Processor -- start 2024-05-09T09:30:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.607+0000 7f87e7e22640 1 -- start start 2024-05-09T09:30:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.607+0000 7f87e7e22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e0100180 0x7f87e0199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.607+0000 7f87e7e22640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87e0100ae0 0x7f87e019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.607+0000 7f87e7e22640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e0101480 0x7f87e01a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.607+0000 7f87e7e22640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f87e010f6e0 con 0x7f87e0100ae0 2024-05-09T09:30:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.607+0000 7f87e7e22640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87e010f560 con 0x7f87e0100180 2024-05-09T09:30:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.607+0000 7f87e7e22640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f87e010f860 con 0x7f87e0101480 2024-05-09T09:30:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.607+0000 7f87e53bd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87e0100ae0 0x7f87e019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.607+0000 7f87e53bd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87e0100ae0 0x7f87e019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36628/0 (socket says 172.21.15.12:36628) 2024-05-09T09:30:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.607+0000 7f87e53bd640 1 -- 172.21.15.12:0/1206480840 learned_addr learned my addr 172.21.15.12:0/1206480840 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.608+0000 7f87e5bbe640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e0100180 0x7f87e0199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.608+0000 7f87e63bf640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e0101480 0x7f87e01a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.608+0000 7f87e53bd640 1 -- 172.21.15.12:0/1206480840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e0101480 msgr2=0x7f87e01a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.608+0000 7f87e53bd640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e0101480 0x7f87e01a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.608+0000 7f87e53bd640 1 -- 172.21.15.12:0/1206480840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e0100180 msgr2=0x7f87e0199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.608+0000 7f87e53bd640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e0100180 0x7f87e0199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.608+0000 7f87e53bd640 1 -- 172.21.15.12:0/1206480840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e01a1950 con 0x7f87e0100ae0 2024-05-09T09:30:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.608+0000 7f87e53bd640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87e0100ae0 0x7f87e019a280 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f87d000c920 tx=0x7f87d000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.608+0000 7f87d6ffd640 1 -- 172.21.15.12:0/1206480840 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0007c70 con 0x7f87e0100ae0 2024-05-09T09:30:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.608+0000 7f87d6ffd640 1 -- 172.21.15.12:0/1206480840 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f87d0007e10 con 0x7f87e0100ae0 2024-05-09T09:30:10.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.608+0000 7f87d6ffd640 1 -- 172.21.15.12:0/1206480840 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0018850 con 0x7f87e0100ae0 2024-05-09T09:30:10.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.608+0000 7f87e7e22640 1 -- 172.21.15.12:0/1206480840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e01a1c40 con 0x7f87e0100ae0 2024-05-09T09:30:10.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.608+0000 7f87e7e22640 1 -- 172.21.15.12:0/1206480840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e01a2190 con 0x7f87e0100ae0 2024-05-09T09:30:10.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.609+0000 7f87e7e22640 1 -- 172.21.15.12:0/1206480840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e010aed0 con 0x7f87e0100ae0 2024-05-09T09:30:10.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.611+0000 7f87d6ffd640 1 -- 172.21.15.12:0/1206480840 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87d0018a30 con 0x7f87e0100ae0 2024-05-09T09:30:10.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.611+0000 7f87d6ffd640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f87cc0a2c10 0x7f87cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:10.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.611+0000 7f87e5bbe640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f87cc0a2c10 0x7f87cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:10.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.611+0000 7f87d6ffd640 1 -- 172.21.15.12:0/1206480840 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f87d00cf4e0 con 0x7f87e0100ae0 2024-05-09T09:30:10.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.612+0000 7f87e5bbe640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f87cc0a2c10 0x7f87cc0a50d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f87dc006010 tx=0x7f87dc031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:10.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.616+0000 7f87d6ffd640 1 -- 172.21.15.12:0/1206480840 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f87d0014030 con 0x7f87e0100ae0 2024-05-09T09:30:10.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:10 smithi170 ceph-mon[33632]: pgmap v506: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:10.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:10 smithi016 ceph-mon[33335]: pgmap v506: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:10.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.802+0000 7f87e7e22640 1 -- 172.21.15.12:0/1206480840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87e00624d0 con 0x7f87e0100ae0 2024-05-09T09:30:10.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.802+0000 7f87d6ffd640 1 -- 172.21.15.12:0/1206480840 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f87d009c070 con 0x7f87e0100ae0 2024-05-09T09:30:10.803 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:30:10.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.804+0000 7f87e7e22640 1 -- 172.21.15.12:0/1206480840 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f87cc0a2c10 msgr2=0x7f87cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:10.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.804+0000 7f87e7e22640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f87cc0a2c10 0x7f87cc0a50d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f87dc006010 tx=0x7f87dc031510 comp rx=0 tx=0).stop 2024-05-09T09:30:10.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.804+0000 7f87e7e22640 1 -- 172.21.15.12:0/1206480840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87e0100ae0 msgr2=0x7f87e019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:10.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.804+0000 7f87e7e22640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87e0100ae0 0x7f87e019a280 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f87d000c920 tx=0x7f87d000cde0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.804+0000 7f87e7e22640 1 -- 172.21.15.12:0/1206480840 shutdown_connections 2024-05-09T09:30:10.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.804+0000 7f87e7e22640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f87cc0a2c10 0x7f87cc0a50d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.804+0000 7f87e7e22640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e0101480 0x7f87e01a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.804+0000 7f87e7e22640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87e0100ae0 0x7f87e019a280 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.804+0000 7f87e7e22640 1 --2- 172.21.15.12:0/1206480840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87e0100180 0x7f87e0199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.804+0000 7f87e7e22640 1 -- 172.21.15.12:0/1206480840 >> 172.21.15.12:0/1206480840 conn(0x7f87e00fbd20 msgr2=0x7f87e00fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:10.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.805+0000 7f87e7e22640 1 -- 172.21.15.12:0/1206480840 shutdown_connections 2024-05-09T09:30:10.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.805+0000 7f87e7e22640 1 -- 172.21.15.12:0/1206480840 wait complete. 2024-05-09T09:30:10.816 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:30:10.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 -- 172.21.15.12:0/2836197555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f469c107610 msgr2=0x7f469c109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:10.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 --2- 172.21.15.12:0/2836197555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f469c107610 0x7f469c109a00 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4694009ca0 tx=0x7f469402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 -- 172.21.15.12:0/2836197555 shutdown_connections 2024-05-09T09:30:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 --2- 172.21.15.12:0/2836197555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f469c107610 0x7f469c109a00 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 --2- 172.21.15.12:0/2836197555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f469c104ce0 0x7f469c1070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 --2- 172.21.15.12:0/2836197555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f469c1023b0 0x7f469c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 -- 172.21.15.12:0/2836197555 >> 172.21.15.12:0/2836197555 conn(0x7f469c0fbce0 msgr2=0x7f469c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 -- 172.21.15.12:0/2836197555 shutdown_connections 2024-05-09T09:30:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 -- 172.21.15.12:0/2836197555 wait complete. 2024-05-09T09:30:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 Processor -- start 2024-05-09T09:30:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 -- start start 2024-05-09T09:30:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f469c1023b0 0x7f469c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f469c104ce0 0x7f469c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f469c107610 0x7f469c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f469c10d4e0 con 0x7f469c107610 2024-05-09T09:30:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f469c10d360 con 0x7f469c1023b0 2024-05-09T09:30:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.925+0000 7f46a28ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f469c10d660 con 0x7f469c104ce0 2024-05-09T09:30:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f46a0e8b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f469c107610 0x7f469c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f46a0e8b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f469c107610 0x7f469c19f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36654/0 (socket says 172.21.15.12:36654) 2024-05-09T09:30:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f46a0e8b640 1 -- 172.21.15.12:0/3875835402 learned_addr learned my addr 172.21.15.12:0/3875835402 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f469bfff640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f469c1023b0 0x7f469c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f4693fff640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f469c104ce0 0x7f469c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f46a0e8b640 1 -- 172.21.15.12:0/3875835402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f469c104ce0 msgr2=0x7f469c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f46a0e8b640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f469c104ce0 0x7f469c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f46a0e8b640 1 -- 172.21.15.12:0/3875835402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f469c1023b0 msgr2=0x7f469c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f46a0e8b640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f469c1023b0 0x7f469c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f46a0e8b640 1 -- 172.21.15.12:0/3875835402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469c19f7b0 con 0x7f469c107610 2024-05-09T09:30:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f46a0e8b640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f469c107610 0x7f469c19f140 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f469402f500 tx=0x7f4694003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f4699ffb640 1 -- 172.21.15.12:0/3875835402 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694003e20 con 0x7f469c107610 2024-05-09T09:30:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f46a28ee640 1 -- 172.21.15.12:0/3875835402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469c19fa40 con 0x7f469c107610 2024-05-09T09:30:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f4699ffb640 1 -- 172.21.15.12:0/3875835402 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4694004510 con 0x7f469c107610 2024-05-09T09:30:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f4699ffb640 1 -- 172.21.15.12:0/3875835402 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469402fe50 con 0x7f469c107610 2024-05-09T09:30:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.926+0000 7f46a28ee640 1 -- 172.21.15.12:0/3875835402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f469c19ff90 con 0x7f469c107610 2024-05-09T09:30:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.927+0000 7f46a28ee640 1 -- 172.21.15.12:0/3875835402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469c0fd7d0 con 0x7f469c107610 2024-05-09T09:30:10.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.929+0000 7f4699ffb640 1 -- 172.21.15.12:0/3875835402 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4694004040 con 0x7f469c107610 2024-05-09T09:30:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.929+0000 7f4699ffb640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46880a2b90 0x7f46880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.929+0000 7f469bfff640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46880a2b90 0x7f46880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.929+0000 7f4699ffb640 1 -- 172.21.15.12:0/3875835402 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f46940f0cb0 con 0x7f469c107610 2024-05-09T09:30:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.930+0000 7f469bfff640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46880a2b90 0x7f46880a5050 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f468c0058c0 tx=0x7f468c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:10.934+0000 7f4699ffb640 1 -- 172.21.15.12:0/3875835402 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4694004330 con 0x7f469c107610 2024-05-09T09:30:11.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.119+0000 7f46a28ee640 1 -- 172.21.15.12:0/3875835402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f469c062430 con 0x7f469c107610 2024-05-09T09:30:11.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.120+0000 7f4699ffb640 1 -- 172.21.15.12:0/3875835402 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f46940bd840 con 0x7f469c107610 2024-05-09T09:30:11.120 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:30:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.122+0000 7f46a28ee640 1 -- 172.21.15.12:0/3875835402 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46880a2b90 msgr2=0x7f46880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.122+0000 7f46a28ee640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46880a2b90 0x7f46880a5050 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f468c0058c0 tx=0x7f468c00a2d0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.122+0000 7f46a28ee640 1 -- 172.21.15.12:0/3875835402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f469c107610 msgr2=0x7f469c19f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.122+0000 7f46a28ee640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f469c107610 0x7f469c19f140 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f469402f500 tx=0x7f4694003970 comp rx=0 tx=0).stop 2024-05-09T09:30:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.122+0000 7f46a28ee640 1 -- 172.21.15.12:0/3875835402 shutdown_connections 2024-05-09T09:30:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.122+0000 7f46a28ee640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f46880a2b90 0x7f46880a5050 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.122+0000 7f46a28ee640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f469c107610 0x7f469c19f140 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.122+0000 7f46a28ee640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f469c104ce0 0x7f469c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.122+0000 7f46a28ee640 1 --2- 172.21.15.12:0/3875835402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f469c1023b0 0x7f469c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.122+0000 7f46a28ee640 1 -- 172.21.15.12:0/3875835402 >> 172.21.15.12:0/3875835402 conn(0x7f469c0fbce0 msgr2=0x7f469c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.122+0000 7f46a28ee640 1 -- 172.21.15.12:0/3875835402 shutdown_connections 2024-05-09T09:30:11.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.122+0000 7f46a28ee640 1 -- 172.21.15.12:0/3875835402 wait complete. 2024-05-09T09:30:11.134 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:30:11.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.242+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1538002599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ad8100790 msgr2=0x7f6ad8100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.242+0000 7f6adfdf9640 1 --2- 172.21.15.12:0/1538002599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ad8100790 0x7f6ad8100bb0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009900 tx=0x7f6ad402a410 comp rx=0 tx=0).stop 2024-05-09T09:30:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.242+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1538002599 shutdown_connections 2024-05-09T09:30:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.242+0000 7f6adfdf9640 1 --2- 172.21.15.12:0/1538002599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad8102340 0x7f6ad8108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.242+0000 7f6adfdf9640 1 --2- 172.21.15.12:0/1538002599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ad8101980 0x7f6ad8101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.242+0000 7f6adfdf9640 1 --2- 172.21.15.12:0/1538002599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ad8100790 0x7f6ad8100bb0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.242+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1538002599 >> 172.21.15.12:0/1538002599 conn(0x7f6ad80fbd20 msgr2=0x7f6ad80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.242+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1538002599 shutdown_connections 2024-05-09T09:30:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.242+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1538002599 wait complete. 2024-05-09T09:30:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6adfdf9640 1 Processor -- start 2024-05-09T09:30:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6adfdf9640 1 -- start start 2024-05-09T09:30:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6adfdf9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ad8100790 0x7f6ad8196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6adfdf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad8101980 0x7f6ad8197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6adfdf9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ad8102340 0x7f6ad819e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6adfdf9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad810c3f0 con 0x7f6ad8102340 2024-05-09T09:30:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6adfdf9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad810c270 con 0x7f6ad8101980 2024-05-09T09:30:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6adfdf9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad810c570 con 0x7f6ad8100790 2024-05-09T09:30:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6ade396640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ad8102340 0x7f6ad819e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6ade396640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ad8102340 0x7f6ad819e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36670/0 (socket says 172.21.15.12:36670) 2024-05-09T09:30:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6ade396640 1 -- 172.21.15.12:0/1713993180 learned_addr learned my addr 172.21.15.12:0/1713993180 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6add394640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad8101980 0x7f6ad8197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6addb95640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ad8100790 0x7f6ad8196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6ade396640 1 -- 172.21.15.12:0/1713993180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ad8100790 msgr2=0x7f6ad8196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6ade396640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ad8100790 0x7f6ad8196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6ade396640 1 -- 172.21.15.12:0/1713993180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad8101980 msgr2=0x7f6ad8197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6ade396640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad8101980 0x7f6ad8197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6ade396640 1 -- 172.21.15.12:0/1713993180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad819e730 con 0x7f6ad8102340 2024-05-09T09:30:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.243+0000 7f6add394640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad8101980 0x7f6ad8197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:30:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.244+0000 7f6ade396640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ad8102340 0x7f6ad819e0c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000eb10 tx=0x7f6ad000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.244+0000 7f6aceffd640 1 -- 172.21.15.12:0/1713993180 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0004b20 con 0x7f6ad8102340 2024-05-09T09:30:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.244+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1713993180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad819ea20 con 0x7f6ad8102340 2024-05-09T09:30:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.244+0000 7f6aceffd640 1 -- 172.21.15.12:0/1713993180 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6ad0004cc0 con 0x7f6ad8102340 2024-05-09T09:30:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.244+0000 7f6aceffd640 1 -- 172.21.15.12:0/1713993180 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0010700 con 0x7f6ad8102340 2024-05-09T09:30:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.244+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1713993180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad819ef70 con 0x7f6ad8102340 2024-05-09T09:30:11.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.245+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1713993180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad8107bd0 con 0x7f6ad8102340 2024-05-09T09:30:11.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.246+0000 7f6aceffd640 1 -- 172.21.15.12:0/1713993180 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6ad0004e70 con 0x7f6ad8102340 2024-05-09T09:30:11.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.247+0000 7f6aceffd640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6ac40a2b90 0x7f6ac40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:11.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.247+0000 7f6addb95640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6ac40a2b90 0x7f6ac40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:11.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.247+0000 7f6aceffd640 1 -- 172.21.15.12:0/1713993180 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6ad00cfb10 con 0x7f6ad8102340 2024-05-09T09:30:11.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.247+0000 7f6addb95640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6ac40a2b90 0x7f6ac40a5050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4000c00 tx=0x7f6ad4031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:11.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.251+0000 7f6aceffd640 1 -- 172.21.15.12:0/1713993180 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6ad009c6a0 con 0x7f6ad8102340 2024-05-09T09:30:11.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.437+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1713993180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6ad8062430 con 0x7f6ad8102340 2024-05-09T09:30:11.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.438+0000 7f6aceffd640 1 -- 172.21.15.12:0/1713993180 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f6ad009bfa0 con 0x7f6ad8102340 2024-05-09T09:30:11.437 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T09:30:11.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.439+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1713993180 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6ac40a2b90 msgr2=0x7f6ac40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.439+0000 7f6adfdf9640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6ac40a2b90 0x7f6ac40a5050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4000c00 tx=0x7f6ad4031510 comp rx=0 tx=0).stop 2024-05-09T09:30:11.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.439+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1713993180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ad8102340 msgr2=0x7f6ad819e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.439+0000 7f6adfdf9640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ad8102340 0x7f6ad819e0c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000eb10 tx=0x7f6ad000c2d0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.440+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1713993180 shutdown_connections 2024-05-09T09:30:11.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.440+0000 7f6adfdf9640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f6ac40a2b90 0x7f6ac40a5050 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.440+0000 7f6adfdf9640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ad8102340 0x7f6ad819e0c0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.440+0000 7f6adfdf9640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad8101980 0x7f6ad8197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.440+0000 7f6adfdf9640 1 --2- 172.21.15.12:0/1713993180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ad8100790 0x7f6ad8196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.440+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1713993180 >> 172.21.15.12:0/1713993180 conn(0x7f6ad80fbd20 msgr2=0x7f6ad8104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:11.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.440+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1713993180 shutdown_connections 2024-05-09T09:30:11.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.440+0000 7f6adfdf9640 1 -- 172.21.15.12:0/1713993180 wait complete. 2024-05-09T09:30:11.449 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:11 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1007500187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:30:11.449 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:11 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1206480840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:30:11.450 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:11 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3875835402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:30:11.451 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:30:11.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.559+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/3116926350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d0104ca0 msgr2=0x7fe8d0107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.559+0000 7fe8d6b5a640 1 --2- 172.21.15.12:0/3116926350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d0104ca0 0x7fe8d0107090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009ca0 tx=0x7fe8b802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.559+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/3116926350 shutdown_connections 2024-05-09T09:30:11.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.559+0000 7fe8d6b5a640 1 --2- 172.21.15.12:0/3116926350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8d01075d0 0x7fe8d01099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.559+0000 7fe8d6b5a640 1 --2- 172.21.15.12:0/3116926350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d0104ca0 0x7fe8d0107090 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.559+0000 7fe8d6b5a640 1 --2- 172.21.15.12:0/3116926350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d0102370 0x7fe8d0104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.559+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/3116926350 >> 172.21.15.12:0/3116926350 conn(0x7fe8d00fbd20 msgr2=0x7fe8d00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:11.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.559+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/3116926350 shutdown_connections 2024-05-09T09:30:11.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.559+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/3116926350 wait complete. 2024-05-09T09:30:11.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d6b5a640 1 Processor -- start 2024-05-09T09:30:11.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d6b5a640 1 -- start start 2024-05-09T09:30:11.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d6b5a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d0102370 0x7fe8d0101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:11.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d6b5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d0104ca0 0x7fe8d0100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:11.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d6b5a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8d01075d0 0x7fe8d0100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:11.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d6b5a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d010d4f0 con 0x7fe8d0102370 2024-05-09T09:30:11.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d6b5a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d010d370 con 0x7fe8d0104ca0 2024-05-09T09:30:11.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d6b5a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d010d670 con 0x7fe8d01075d0 2024-05-09T09:30:11.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d48f6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d0102370 0x7fe8d0101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:11.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d48f6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d0102370 0x7fe8d0101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36686/0 (socket says 172.21.15.12:36686) 2024-05-09T09:30:11.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d48f6640 1 -- 172.21.15.12:0/942161778 learned_addr learned my addr 172.21.15.12:0/942161778 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:11.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8cffff640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d0104ca0 0x7fe8d0100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:11.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d48f6640 1 -- 172.21.15.12:0/942161778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8d01075d0 msgr2=0x7fe8d0100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d48f6640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8d01075d0 0x7fe8d0100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d48f6640 1 -- 172.21.15.12:0/942161778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d0104ca0 msgr2=0x7fe8d0100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d48f6640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d0104ca0 0x7fe8d0100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d48f6640 1 -- 172.21.15.12:0/942161778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d01010f0 con 0x7fe8d0102370 2024-05-09T09:30:11.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8cffff640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d0104ca0 0x7fe8d0100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:30:11.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.560+0000 7fe8d48f6640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d0102370 0x7fe8d0101ce0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000c890 tx=0x7fe8c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:11.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.561+0000 7fe8cdffb640 1 -- 172.21.15.12:0/942161778 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0004220 con 0x7fe8d0102370 2024-05-09T09:30:11.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.561+0000 7fe8cdffb640 1 -- 172.21.15.12:0/942161778 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe8c00043c0 con 0x7fe8d0102370 2024-05-09T09:30:11.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.561+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/942161778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d01a1cf0 con 0x7fe8d0102370 2024-05-09T09:30:11.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.561+0000 7fe8cdffb640 1 -- 172.21.15.12:0/942161778 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0004a00 con 0x7fe8d0102370 2024-05-09T09:30:11.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.561+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/942161778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d01a2190 con 0x7fe8d0102370 2024-05-09T09:30:11.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.562+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/942161778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d00fd810 con 0x7fe8d0102370 2024-05-09T09:30:11.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.563+0000 7fe8cdffb640 1 -- 172.21.15.12:0/942161778 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe8c0004ba0 con 0x7fe8d0102370 2024-05-09T09:30:11.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.564+0000 7fe8cdffb640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe8bc0a2b90 0x7fe8bc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:11.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.564+0000 7fe8cdffb640 1 -- 172.21.15.12:0/942161778 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe8c00cf280 con 0x7fe8d0102370 2024-05-09T09:30:11.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.564+0000 7fe8cffff640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe8bc0a2b90 0x7fe8bc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:11.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.564+0000 7fe8cffff640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe8bc0a2b90 0x7fe8bc0a5050 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8006010 tx=0x7fe8b8000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:11.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.568+0000 7fe8cdffb640 1 -- 172.21.15.12:0/942161778 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe8c0014030 con 0x7fe8d0102370 2024-05-09T09:30:11.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:11 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1007500187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:30:11.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:11 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1206480840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:30:11.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:11 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3875835402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:30:11.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:11 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1007500187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:30:11.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:11 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1206480840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:30:11.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:11 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3875835402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:30:11.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.755+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/942161778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe8d002d1c0 con 0x7fe8d0102370 2024-05-09T09:30:11.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.756+0000 7fe8cdffb640 1 -- 172.21.15.12:0/942161778 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fe8d002d1c0 con 0x7fe8d0102370 2024-05-09T09:30:11.755 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:30:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.758+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/942161778 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe8bc0a2b90 msgr2=0x7fe8bc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.758+0000 7fe8d6b5a640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe8bc0a2b90 0x7fe8bc0a5050 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8006010 tx=0x7fe8b8000f40 comp rx=0 tx=0).stop 2024-05-09T09:30:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.758+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/942161778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d0102370 msgr2=0x7fe8d0101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.758+0000 7fe8d6b5a640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d0102370 0x7fe8d0101ce0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000c890 tx=0x7fe8c000cd50 comp rx=0 tx=0).stop 2024-05-09T09:30:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.758+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/942161778 shutdown_connections 2024-05-09T09:30:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.758+0000 7fe8d6b5a640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fe8bc0a2b90 0x7fe8bc0a5050 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.758+0000 7fe8d6b5a640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8d01075d0 0x7fe8d0100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.758+0000 7fe8d6b5a640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d0104ca0 0x7fe8d0100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.758+0000 7fe8d6b5a640 1 --2- 172.21.15.12:0/942161778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d0102370 0x7fe8d0101ce0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.758+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/942161778 >> 172.21.15.12:0/942161778 conn(0x7fe8d00fbd20 msgr2=0x7fe8d0102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:11.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.758+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/942161778 shutdown_connections 2024-05-09T09:30:11.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.758+0000 7fe8d6b5a640 1 -- 172.21.15.12:0/942161778 wait complete. 2024-05-09T09:30:11.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T09:30:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.877+0000 7fd46444c640 1 -- 172.21.15.12:0/3721609451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd45c103440 msgr2=0x7fd45c109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.877+0000 7fd46444c640 1 --2- 172.21.15.12:0/3721609451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd45c103440 0x7fd45c109cd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd454009ca0 tx=0x7fd45402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.877+0000 7fd46444c640 1 -- 172.21.15.12:0/3721609451 shutdown_connections 2024-05-09T09:30:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.877+0000 7fd46444c640 1 --2- 172.21.15.12:0/3721609451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd45c103440 0x7fd45c109cd0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.877+0000 7fd46444c640 1 --2- 172.21.15.12:0/3721609451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd45c102a80 0x7fd45c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.877+0000 7fd46444c640 1 --2- 172.21.15.12:0/3721609451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45c101890 0x7fd45c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.877+0000 7fd46444c640 1 -- 172.21.15.12:0/3721609451 >> 172.21.15.12:0/3721609451 conn(0x7fd45c0fce00 msgr2=0x7fd45c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.877+0000 7fd46444c640 1 -- 172.21.15.12:0/3721609451 shutdown_connections 2024-05-09T09:30:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.877+0000 7fd46444c640 1 -- 172.21.15.12:0/3721609451 wait complete. 2024-05-09T09:30:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.878+0000 7fd46444c640 1 Processor -- start 2024-05-09T09:30:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.878+0000 7fd46444c640 1 -- start start 2024-05-09T09:30:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.878+0000 7fd46444c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd45c101890 0x7fd45c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.878+0000 7fd46444c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd45c102a80 0x7fd45c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.878+0000 7fd46444c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45c103440 0x7fd45c19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.878+0000 7fd46444c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c10d4f0 con 0x7fd45c102a80 2024-05-09T09:30:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.878+0000 7fd46444c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c10d370 con 0x7fd45c103440 2024-05-09T09:30:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.878+0000 7fd46444c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c10d670 con 0x7fd45c101890 2024-05-09T09:30:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.878+0000 7fd4619e7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd45c102a80 0x7fd45c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.878+0000 7fd4619e7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd45c102a80 0x7fd45c197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58700/0 (socket says 172.21.15.12:58700) 2024-05-09T09:30:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.878+0000 7fd4619e7640 1 -- 172.21.15.12:0/2137341469 learned_addr learned my addr 172.21.15.12:0/2137341469 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.878+0000 7fd4621e8640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd45c101890 0x7fd45c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.878+0000 7fd4629e9640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45c103440 0x7fd45c19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.879+0000 7fd4619e7640 1 -- 172.21.15.12:0/2137341469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd45c101890 msgr2=0x7fd45c197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.879+0000 7fd4619e7640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd45c101890 0x7fd45c197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.879+0000 7fd4619e7640 1 -- 172.21.15.12:0/2137341469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45c103440 msgr2=0x7fd45c19f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.879+0000 7fd4619e7640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45c103440 0x7fd45c19f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:11.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.879+0000 7fd4619e7640 1 -- 172.21.15.12:0/2137341469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45c19f6a0 con 0x7fd45c102a80 2024-05-09T09:30:11.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.879+0000 7fd4619e7640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd45c102a80 0x7fd45c197fc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e9f0 tx=0x7fd44c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:11.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.879+0000 7fd4537fe640 1 -- 172.21.15.12:0/2137341469 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00cd80 con 0x7fd45c102a80 2024-05-09T09:30:11.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.879+0000 7fd4537fe640 1 -- 172.21.15.12:0/2137341469 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd44c0054f0 con 0x7fd45c102a80 2024-05-09T09:30:11.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.879+0000 7fd46444c640 1 -- 172.21.15.12:0/2137341469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45c19f990 con 0x7fd45c102a80 2024-05-09T09:30:11.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.879+0000 7fd4537fe640 1 -- 172.21.15.12:0/2137341469 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c010690 con 0x7fd45c102a80 2024-05-09T09:30:11.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.879+0000 7fd46444c640 1 -- 172.21.15.12:0/2137341469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd45c19fee0 con 0x7fd45c102a80 2024-05-09T09:30:11.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.880+0000 7fd46444c640 1 -- 172.21.15.12:0/2137341469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd45c108ae0 con 0x7fd45c102a80 2024-05-09T09:30:11.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.881+0000 7fd4537fe640 1 -- 172.21.15.12:0/2137341469 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd44c004f20 con 0x7fd45c102a80 2024-05-09T09:30:11.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.882+0000 7fd4537fe640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4480a2c10 0x7fd4480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:11.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.882+0000 7fd4621e8640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4480a2c10 0x7fd4480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:11.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.882+0000 7fd4537fe640 1 -- 172.21.15.12:0/2137341469 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd44c0cf820 con 0x7fd45c102a80 2024-05-09T09:30:11.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.883+0000 7fd4621e8640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4480a2c10 0x7fd4480a50d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd45c1028e0 tx=0x7fd458005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:11.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:11.887+0000 7fd4537fe640 1 -- 172.21.15.12:0/2137341469 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd44c09c350 con 0x7fd45c102a80 2024-05-09T09:30:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.085+0000 7fd46444c640 1 -- 172.21.15.12:0/2137341469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd45c02d1c0 con 0x7fd45c102a80 2024-05-09T09:30:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.086+0000 7fd4537fe640 1 -- 172.21.15.12:0/2137341469 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+591 (secure 0 0 0) 0x7fd44c09bc50 con 0x7fd45c102a80 2024-05-09T09:30:12.086 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1454579683 2024-05-10T09:16:04.466876+0000 2024-05-09T09:30:12.086 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/2168557623 2024-05-10T09:16:04.466876+0000 2024-05-09T09:30:12.086 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/2168557623 2024-05-10T09:16:04.466876+0000 2024-05-09T09:30:12.086 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/545109421 2024-05-10T09:15:36.182637+0000 2024-05-09T09:30:12.086 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1153421462 2024-05-10T09:16:04.466876+0000 2024-05-09T09:30:12.086 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1731765200 2024-05-10T09:15:36.182637+0000 2024-05-09T09:30:12.086 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2508497448 2024-05-10T09:15:36.182637+0000 2024-05-09T09:30:12.086 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1876427478 2024-05-10T09:15:36.182637+0000 2024-05-09T09:30:12.086 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3479121957 2024-05-10T09:16:04.466876+0000 2024-05-09T09:30:12.087 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1876427478 2024-05-10T09:15:36.182637+0000 2024-05-09T09:30:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:listed 10 entries 2024-05-09T09:30:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.088+0000 7fd46444c640 1 -- 172.21.15.12:0/2137341469 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4480a2c10 msgr2=0x7fd4480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.088+0000 7fd46444c640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4480a2c10 0x7fd4480a50d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd45c1028e0 tx=0x7fd458005f90 comp rx=0 tx=0).stop 2024-05-09T09:30:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.088+0000 7fd46444c640 1 -- 172.21.15.12:0/2137341469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd45c102a80 msgr2=0x7fd45c197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.088+0000 7fd46444c640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd45c102a80 0x7fd45c197fc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e9f0 tx=0x7fd44c00edf0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.088+0000 7fd46444c640 1 -- 172.21.15.12:0/2137341469 shutdown_connections 2024-05-09T09:30:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.088+0000 7fd46444c640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fd4480a2c10 0x7fd4480a50d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.088+0000 7fd46444c640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45c103440 0x7fd45c19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.088+0000 7fd46444c640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd45c102a80 0x7fd45c197fc0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.088+0000 7fd46444c640 1 --2- 172.21.15.12:0/2137341469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd45c101890 0x7fd45c197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.088+0000 7fd46444c640 1 -- 172.21.15.12:0/2137341469 >> 172.21.15.12:0/2137341469 conn(0x7fd45c0fce00 msgr2=0x7fd45c105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:12.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.088+0000 7fd46444c640 1 -- 172.21.15.12:0/2137341469 shutdown_connections 2024-05-09T09:30:12.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.088+0000 7fd46444c640 1 -- 172.21.15.12:0/2137341469 wait complete. 2024-05-09T09:30:12.099 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T09:30:12.099 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T09:30:12.118 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:30:12.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.263+0000 7f850e390640 1 -- 172.21.15.12:0/511073408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8508104c40 msgr2=0x7f8508107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.263+0000 7f850e390640 1 --2- 172.21.15.12:0/511073408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8508104c40 0x7f8508107030 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009900 tx=0x7f84f002a410 comp rx=0 tx=0).stop 2024-05-09T09:30:12.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.263+0000 7f850e390640 1 -- 172.21.15.12:0/511073408 shutdown_connections 2024-05-09T09:30:12.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 --2- 172.21.15.12:0/511073408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8508107570 0x7f85081099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 --2- 172.21.15.12:0/511073408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8508104c40 0x7f8508107030 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 --2- 172.21.15.12:0/511073408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8508102310 0x7f8508104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 -- 172.21.15.12:0/511073408 >> 172.21.15.12:0/511073408 conn(0x7f85080fbd20 msgr2=0x7f85080fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:12.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 -- 172.21.15.12:0/511073408 shutdown_connections 2024-05-09T09:30:12.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 -- 172.21.15.12:0/511073408 wait complete. 2024-05-09T09:30:12.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 Processor -- start 2024-05-09T09:30:12.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 -- start start 2024-05-09T09:30:12.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8508102310 0x7f8508197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:12.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8508104c40 0x7f8508198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:12.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8508107570 0x7f850819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:12.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f850810d4f0 con 0x7f8508104c40 2024-05-09T09:30:12.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f850810d370 con 0x7f8508107570 2024-05-09T09:30:12.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f850e390640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f850810d670 con 0x7f8508102310 2024-05-09T09:30:12.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f84fffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8508104c40 0x7f8508198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:12.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.264+0000 7f84fffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8508104c40 0x7f8508198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58712/0 (socket says 172.21.15.12:58712) 2024-05-09T09:30:12.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f84fffff640 1 -- 172.21.15.12:0/3304288349 learned_addr learned my addr 172.21.15.12:0/3304288349 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:12.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f8507fff640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8508102310 0x7f8508197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:12.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f850c92d640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8508107570 0x7f850819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:12.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f84fffff640 1 -- 172.21.15.12:0/3304288349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8508102310 msgr2=0x7f8508197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f84fffff640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8508102310 0x7f8508197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f84fffff640 1 -- 172.21.15.12:0/3304288349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8508107570 msgr2=0x7f850819f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f84fffff640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8508107570 0x7f850819f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f84fffff640 1 -- 172.21.15.12:0/3304288349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850819f7e0 con 0x7f8508104c40 2024-05-09T09:30:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f84fffff640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8508104c40 0x7f8508198100 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f84f0005ab0 tx=0x7f84f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f8505ffb640 1 -- 172.21.15.12:0/3304288349 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f003f030 con 0x7f8508104c40 2024-05-09T09:30:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f850e390640 1 -- 172.21.15.12:0/3304288349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850819fa70 con 0x7f8508104c40 2024-05-09T09:30:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f8505ffb640 1 -- 172.21.15.12:0/3304288349 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f84f0003ac0 con 0x7f8508104c40 2024-05-09T09:30:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f8505ffb640 1 -- 172.21.15.12:0/3304288349 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0035070 con 0x7f8508104c40 2024-05-09T09:30:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.265+0000 7f850e390640 1 -- 172.21.15.12:0/3304288349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850819ffb0 con 0x7f8508104c40 2024-05-09T09:30:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.266+0000 7f850e390640 1 -- 172.21.15.12:0/3304288349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85080fd810 con 0x7f8508104c40 2024-05-09T09:30:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.267+0000 7f8505ffb640 1 -- 172.21.15.12:0/3304288349 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f84f0042030 con 0x7f8508104c40 2024-05-09T09:30:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.268+0000 7f8505ffb640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f84f40a2c10 0x7f84f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.268+0000 7f8507fff640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f84f40a2c10 0x7f84f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.268+0000 7f8505ffb640 1 -- 172.21.15.12:0/3304288349 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f84f00efa50 con 0x7f8508104c40 2024-05-09T09:30:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.269+0000 7f8507fff640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f84f40a2c10 0x7f84f40a50d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009bb0 tx=0x7f84f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:12.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.272+0000 7f8505ffb640 1 -- 172.21.15.12:0/3304288349 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f84f00bc580 con 0x7f8508104c40 2024-05-09T09:30:12.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.460+0000 7f850e390640 1 -- 172.21.15.12:0/3304288349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8508062430 con 0x7f8508104c40 2024-05-09T09:30:12.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.462+0000 7f8505ffb640 1 -- 172.21.15.12:0/3304288349 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24762 (secure 0 0 0) 0x7f84f00bbe80 con 0x7f8508104c40 2024-05-09T09:30:12.462 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:30:12.462 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":94,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:27:40.951852+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:30:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.464+0000 7f850e390640 1 -- 172.21.15.12:0/3304288349 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f84f40a2c10 msgr2=0x7f84f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.464+0000 7f850e390640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f84f40a2c10 0x7f84f40a50d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009bb0 tx=0x7f84f8005f90 comp rx=0 tx=0).stop 2024-05-09T09:30:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.464+0000 7f850e390640 1 -- 172.21.15.12:0/3304288349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8508104c40 msgr2=0x7f8508198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.464+0000 7f850e390640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8508104c40 0x7f8508198100 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f84f0005ab0 tx=0x7f84f00042f0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.464+0000 7f850e390640 1 -- 172.21.15.12:0/3304288349 shutdown_connections 2024-05-09T09:30:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.464+0000 7f850e390640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f84f40a2c10 0x7f84f40a50d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.464+0000 7f850e390640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8508107570 0x7f850819f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.464+0000 7f850e390640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8508104c40 0x7f8508198100 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.464+0000 7f850e390640 1 --2- 172.21.15.12:0/3304288349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8508102310 0x7f8508197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.464+0000 7f850e390640 1 -- 172.21.15.12:0/3304288349 >> 172.21.15.12:0/3304288349 conn(0x7f85080fbd20 msgr2=0x7f8508102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.464+0000 7f850e390640 1 -- 172.21.15.12:0/3304288349 shutdown_connections 2024-05-09T09:30:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.464+0000 7f850e390640 1 -- 172.21.15.12:0/3304288349 wait complete. 2024-05-09T09:30:12.474 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:12 smithi012 ceph-mon[30140]: pgmap v507: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T09:30:12.474 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:12 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/1713993180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:30:12.474 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:12 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/942161778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:30:12.474 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:12 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2137341469' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:30:12.476 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:18:31.001259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:23:08.167107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:23:09.065670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:23:12.773640+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:30:12.476 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:30:12.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.584+0000 7f806e59e640 1 -- 172.21.15.12:0/3389344239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8068103430 msgr2=0x7f8068109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.584+0000 7f806e59e640 1 --2- 172.21.15.12:0/3389344239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8068103430 0x7f8068109cc0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8060009ca0 tx=0x7f806002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.584+0000 7f806e59e640 1 -- 172.21.15.12:0/3389344239 shutdown_connections 2024-05-09T09:30:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.584+0000 7f806e59e640 1 --2- 172.21.15.12:0/3389344239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8068103430 0x7f8068109cc0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.584+0000 7f806e59e640 1 --2- 172.21.15.12:0/3389344239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8068102a70 0x7f8068102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.584+0000 7f806e59e640 1 --2- 172.21.15.12:0/3389344239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8068101880 0x7f8068101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.584+0000 7f806e59e640 1 -- 172.21.15.12:0/3389344239 >> 172.21.15.12:0/3389344239 conn(0x7f80680fce10 msgr2=0x7f80680ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.584+0000 7f806e59e640 1 -- 172.21.15.12:0/3389344239 shutdown_connections 2024-05-09T09:30:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.585+0000 7f806e59e640 1 -- 172.21.15.12:0/3389344239 wait complete. 2024-05-09T09:30:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.585+0000 7f806e59e640 1 Processor -- start 2024-05-09T09:30:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.585+0000 7f806e59e640 1 -- start start 2024-05-09T09:30:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.585+0000 7f806e59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8068101880 0x7f8068197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.585+0000 7f806e59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8068102a70 0x7f8068198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:12.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.585+0000 7f806e59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8068103430 0x7f806819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.585+0000 7f806e59e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f806810d560 con 0x7f8068101880 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.585+0000 7f806e59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f806810d3e0 con 0x7f8068103430 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.585+0000 7f806e59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f806810d6e0 con 0x7f8068102a70 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.585+0000 7f8067fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8068101880 0x7f8068197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f8067fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8068101880 0x7f8068197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58740/0 (socket says 172.21.15.12:58740) 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f8067fff640 1 -- 172.21.15.12:0/393894640 learned_addr learned my addr 172.21.15.12:0/393894640 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f806cb3b640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8068103430 0x7f806819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f80677fe640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8068102a70 0x7f8068198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f8067fff640 1 -- 172.21.15.12:0/393894640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8068102a70 msgr2=0x7f8068198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f8067fff640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8068102a70 0x7f8068198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f8067fff640 1 -- 172.21.15.12:0/393894640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8068103430 msgr2=0x7f806819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f8067fff640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8068103430 0x7f806819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f8067fff640 1 -- 172.21.15.12:0/393894640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806819f740 con 0x7f8068101880 2024-05-09T09:30:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f8067fff640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8068101880 0x7f8068197b20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f805800b450 tx=0x7f805800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f80657fa640 1 -- 172.21.15.12:0/393894640 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058015020 con 0x7f8068101880 2024-05-09T09:30:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f80657fa640 1 -- 172.21.15.12:0/393894640 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8058011d60 con 0x7f8068101880 2024-05-09T09:30:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f806e59e640 1 -- 172.21.15.12:0/393894640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806819fa30 con 0x7f8068101880 2024-05-09T09:30:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f80657fa640 1 -- 172.21.15.12:0/393894640 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805800f340 con 0x7f8068101880 2024-05-09T09:30:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.586+0000 7f806e59e640 1 -- 172.21.15.12:0/393894640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f806819ffb0 con 0x7f8068101880 2024-05-09T09:30:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.587+0000 7f806e59e640 1 -- 172.21.15.12:0/393894640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8068108ba0 con 0x7f8068101880 2024-05-09T09:30:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.588+0000 7f80657fa640 1 -- 172.21.15.12:0/393894640 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8058018020 con 0x7f8068101880 2024-05-09T09:30:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.589+0000 7f80657fa640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f80500a2b90 0x7f80500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.589+0000 7f80677fe640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f80500a2b90 0x7f80500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.589+0000 7f80657fa640 1 -- 172.21.15.12:0/393894640 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f80580cefc0 con 0x7f8068101880 2024-05-09T09:30:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.590+0000 7f80677fe640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f80500a2b90 0x7f80500a5050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f805c005e10 tx=0x7f805c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:12.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.594+0000 7f80657fa640 1 -- 172.21.15.12:0/393894640 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f805801f020 con 0x7f8068101880 2024-05-09T09:30:12.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:12 smithi170 ceph-mon[33632]: pgmap v507: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T09:30:12.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:12 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/1713993180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:30:12.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:12 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/942161778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:30:12.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:12 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2137341469' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:30:12.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:12 smithi016 ceph-mon[33335]: pgmap v507: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T09:30:12.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:12 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/1713993180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:30:12.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:12 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/942161778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:30:12.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:12 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2137341469' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:30:12.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.782+0000 7f806e59e640 1 -- 172.21.15.12:0/393894640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8068068650 con 0x7f8068101880 2024-05-09T09:30:12.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.783+0000 7f80657fa640 1 -- 172.21.15.12:0/393894640 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f805800d040 con 0x7f8068101880 2024-05-09T09:30:12.783 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T09:30:12.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.784+0000 7f806e59e640 1 -- 172.21.15.12:0/393894640 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f80500a2b90 msgr2=0x7f80500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.784+0000 7f806e59e640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f80500a2b90 0x7f80500a5050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f805c005e10 tx=0x7f805c005da0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.784+0000 7f806e59e640 1 -- 172.21.15.12:0/393894640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8068101880 msgr2=0x7f8068197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.784+0000 7f806e59e640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8068101880 0x7f8068197b20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f805800b450 tx=0x7f805800b910 comp rx=0 tx=0).stop 2024-05-09T09:30:12.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.785+0000 7f806e59e640 1 -- 172.21.15.12:0/393894640 shutdown_connections 2024-05-09T09:30:12.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.785+0000 7f806e59e640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f80500a2b90 0x7f80500a5050 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.785+0000 7f806e59e640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8068103430 0x7f806819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.785+0000 7f806e59e640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8068102a70 0x7f8068198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.785+0000 7f806e59e640 1 --2- 172.21.15.12:0/393894640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8068101880 0x7f8068197b20 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.785+0000 7f806e59e640 1 -- 172.21.15.12:0/393894640 >> 172.21.15.12:0/393894640 conn(0x7f80680fce10 msgr2=0x7f8068105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:12.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.785+0000 7f806e59e640 1 -- 172.21.15.12:0/393894640 shutdown_connections 2024-05-09T09:30:12.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.785+0000 7f806e59e640 1 -- 172.21.15.12:0/393894640 wait complete. 2024-05-09T09:30:12.796 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:30:12.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 -- 172.21.15.12:0/2604668112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b0101960 msgr2=0x7f29b0101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 --2- 172.21.15.12:0/2604668112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b0101960 0x7f29b0101de0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009900 tx=0x7f29a402a410 comp rx=0 tx=0).stop 2024-05-09T09:30:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 -- 172.21.15.12:0/2604668112 shutdown_connections 2024-05-09T09:30:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 --2- 172.21.15.12:0/2604668112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29b0102320 0x7f29b0108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 --2- 172.21.15.12:0/2604668112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b0101960 0x7f29b0101de0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 --2- 172.21.15.12:0/2604668112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b0100770 0x7f29b0100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 -- 172.21.15.12:0/2604668112 >> 172.21.15.12:0/2604668112 conn(0x7f29b00fbce0 msgr2=0x7f29b00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 -- 172.21.15.12:0/2604668112 shutdown_connections 2024-05-09T09:30:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 -- 172.21.15.12:0/2604668112 wait complete. 2024-05-09T09:30:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 Processor -- start 2024-05-09T09:30:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 -- start start 2024-05-09T09:30:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b0100770 0x7f29b0069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b0101960 0x7f29b019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:12.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29b0102320 0x7f29b019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:12.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f29b010c430 con 0x7f29b0101960 2024-05-09T09:30:12.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29b010c2b0 con 0x7f29b0100770 2024-05-09T09:30:12.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.905+0000 7f29b7ba2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f29b010c5b0 con 0x7f29b0102320 2024-05-09T09:30:12.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b513d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b0101960 0x7f29b019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:12.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b513d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b0101960 0x7f29b019c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58764/0 (socket says 172.21.15.12:58764) 2024-05-09T09:30:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b513d640 1 -- 172.21.15.12:0/4105183488 learned_addr learned my addr 172.21.15.12:0/4105183488 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b593e640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b0100770 0x7f29b0069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b613f640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29b0102320 0x7f29b019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b513d640 1 -- 172.21.15.12:0/4105183488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29b0102320 msgr2=0x7f29b019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b513d640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29b0102320 0x7f29b019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b513d640 1 -- 172.21.15.12:0/4105183488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b0100770 msgr2=0x7f29b0069550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b513d640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b0100770 0x7f29b0069550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b513d640 1 -- 172.21.15.12:0/4105183488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b019f7d0 con 0x7f29b0101960 2024-05-09T09:30:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b613f640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29b0102320 0x7f29b019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:30:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b513d640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b0101960 0x7f29b019c8d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009230 tx=0x7f29a4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29aeffd640 1 -- 172.21.15.12:0/4105183488 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a403f030 con 0x7f29b0101960 2024-05-09T09:30:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29aeffd640 1 -- 172.21.15.12:0/4105183488 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f29a4003ac0 con 0x7f29b0101960 2024-05-09T09:30:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b7ba2640 1 -- 172.21.15.12:0/4105183488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b019fa60 con 0x7f29b0101960 2024-05-09T09:30:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29aeffd640 1 -- 172.21.15.12:0/4105183488 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4035070 con 0x7f29b0101960 2024-05-09T09:30:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.906+0000 7f29b7ba2640 1 -- 172.21.15.12:0/4105183488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b019ffd0 con 0x7f29b0101960 2024-05-09T09:30:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.907+0000 7f29b7ba2640 1 -- 172.21.15.12:0/4105183488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b0107a90 con 0x7f29b0101960 2024-05-09T09:30:12.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.908+0000 7f29aeffd640 1 -- 172.21.15.12:0/4105183488 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f29a4042030 con 0x7f29b0101960 2024-05-09T09:30:12.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.909+0000 7f29aeffd640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f29980a2b90 0x7f29980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:12.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.909+0000 7f29b593e640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f29980a2b90 0x7f29980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:12.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.909+0000 7f29aeffd640 1 -- 172.21.15.12:0/4105183488 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f29a40efc20 con 0x7f29b0101960 2024-05-09T09:30:12.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.910+0000 7f29b593e640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f29980a2b90 0x7f29980a5050 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f29b01017c0 tx=0x7f29a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:12.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:12.914+0000 7f29aeffd640 1 -- 172.21.15.12:0/4105183488 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f29a40bc750 con 0x7f29b0101960 2024-05-09T09:30:13.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.104+0000 7f29b7ba2640 1 -- 172.21.15.12:0/4105183488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f29b00624d0 con 0x7f29b0101960 2024-05-09T09:30:13.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.104+0000 7f29aeffd640 1 -- 172.21.15.12:0/4105183488 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f29a4003c70 con 0x7f29b0101960 2024-05-09T09:30:13.104 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:30:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.107+0000 7f29b7ba2640 1 -- 172.21.15.12:0/4105183488 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f29980a2b90 msgr2=0x7f29980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.107+0000 7f29b7ba2640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f29980a2b90 0x7f29980a5050 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f29b01017c0 tx=0x7f29a8005f90 comp rx=0 tx=0).stop 2024-05-09T09:30:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.107+0000 7f29b7ba2640 1 -- 172.21.15.12:0/4105183488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b0101960 msgr2=0x7f29b019c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.107+0000 7f29b7ba2640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b0101960 0x7f29b019c8d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009230 tx=0x7f29a4004360 comp rx=0 tx=0).stop 2024-05-09T09:30:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.107+0000 7f29b7ba2640 1 -- 172.21.15.12:0/4105183488 shutdown_connections 2024-05-09T09:30:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.107+0000 7f29b7ba2640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f29980a2b90 0x7f29980a5050 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.107+0000 7f29b7ba2640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29b0102320 0x7f29b019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.107+0000 7f29b7ba2640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b0101960 0x7f29b019c8d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.107+0000 7f29b7ba2640 1 --2- 172.21.15.12:0/4105183488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b0100770 0x7f29b0069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.107+0000 7f29b7ba2640 1 -- 172.21.15.12:0/4105183488 >> 172.21.15.12:0/4105183488 conn(0x7f29b00fbce0 msgr2=0x7f29b0104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.107+0000 7f29b7ba2640 1 -- 172.21.15.12:0/4105183488 shutdown_connections 2024-05-09T09:30:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.107+0000 7f29b7ba2640 1 -- 172.21.15.12:0/4105183488 wait complete. 2024-05-09T09:30:13.118 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:30:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.227+0000 7fecb6977640 1 -- 172.21.15.12:0/4094151136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fecb0101890 msgr2=0x7fecb0101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.227+0000 7fecb6977640 1 --2- 172.21.15.12:0/4094151136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fecb0101890 0x7fecb0101cb0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7feca0009900 tx=0x7feca002a410 comp rx=0 tx=0).stop 2024-05-09T09:30:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.227+0000 7fecb6977640 1 -- 172.21.15.12:0/4094151136 shutdown_connections 2024-05-09T09:30:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.227+0000 7fecb6977640 1 --2- 172.21.15.12:0/4094151136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0103440 0x7fecb0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.227+0000 7fecb6977640 1 --2- 172.21.15.12:0/4094151136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb0102a80 0x7fecb0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.227+0000 7fecb6977640 1 --2- 172.21.15.12:0/4094151136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fecb0101890 0x7fecb0101cb0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.227+0000 7fecb6977640 1 -- 172.21.15.12:0/4094151136 >> 172.21.15.12:0/4094151136 conn(0x7fecb00fce00 msgr2=0x7fecb00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.227+0000 7fecb6977640 1 -- 172.21.15.12:0/4094151136 shutdown_connections 2024-05-09T09:30:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.227+0000 7fecb6977640 1 -- 172.21.15.12:0/4094151136 wait complete. 2024-05-09T09:30:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.227+0000 7fecb6977640 1 Processor -- start 2024-05-09T09:30:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecb6977640 1 -- start start 2024-05-09T09:30:13.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecb6977640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0101890 0x7fecb0199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:13.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecb6977640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fecb0102a80 0x7fecb019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:13.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecb6977640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb0103440 0x7fecb01a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecb6977640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fecb010d4f0 con 0x7fecb0102a80 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecb6977640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fecb010d370 con 0x7fecb0101890 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecb6977640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fecb010d670 con 0x7fecb0103440 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecaf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fecb0102a80 0x7fecb019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecaf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fecb0102a80 0x7fecb019a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58784/0 (socket says 172.21.15.12:58784) 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecaf7fe640 1 -- 172.21.15.12:0/2073664620 learned_addr learned my addr 172.21.15.12:0/2073664620 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecaffff640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0101890 0x7fecb0199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecb4f14640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb0103440 0x7fecb01a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecaf7fe640 1 -- 172.21.15.12:0/2073664620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb0103440 msgr2=0x7fecb01a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecaf7fe640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb0103440 0x7fecb01a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecaf7fe640 1 -- 172.21.15.12:0/2073664620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0101890 msgr2=0x7fecb0199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecaf7fe640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0101890 0x7fecb0199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.228+0000 7fecaf7fe640 1 -- 172.21.15.12:0/2073664620 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb01a1970 con 0x7fecb0102a80 2024-05-09T09:30:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.229+0000 7fecaf7fe640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fecb0102a80 0x7fecb019a330 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7feca400d950 tx=0x7feca400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:13.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.229+0000 7fecad7fa640 1 -- 172.21.15.12:0/2073664620 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4017070 con 0x7fecb0102a80 2024-05-09T09:30:13.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.229+0000 7fecb6977640 1 -- 172.21.15.12:0/2073664620 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb01a1c60 con 0x7fecb0102a80 2024-05-09T09:30:13.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.229+0000 7fecad7fa640 1 -- 172.21.15.12:0/2073664620 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7feca4009c30 con 0x7fecb0102a80 2024-05-09T09:30:13.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.229+0000 7fecad7fa640 1 -- 172.21.15.12:0/2073664620 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca400f6b0 con 0x7fecb0102a80 2024-05-09T09:30:13.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.229+0000 7fecb6977640 1 -- 172.21.15.12:0/2073664620 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb01a21b0 con 0x7fecb0102a80 2024-05-09T09:30:13.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.230+0000 7fecb6977640 1 -- 172.21.15.12:0/2073664620 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb01a23e0 con 0x7fecb0102a80 2024-05-09T09:30:13.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.231+0000 7fecad7fa640 1 -- 172.21.15.12:0/2073664620 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feca4010a50 con 0x7fecb0102a80 2024-05-09T09:30:13.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.232+0000 7fecad7fa640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fec980a2b90 0x7fec980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:13.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.232+0000 7fecaffff640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fec980a2b90 0x7fec980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:13.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.232+0000 7fecad7fa640 1 -- 172.21.15.12:0/2073664620 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7feca40cf080 con 0x7fecb0102a80 2024-05-09T09:30:13.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.232+0000 7fecaffff640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fec980a2b90 0x7fec980a5050 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7feca0000c00 tx=0x7feca0031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:13.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.235+0000 7fecad7fa640 1 -- 172.21.15.12:0/2073664620 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feca4009de0 con 0x7fecb0102a80 2024-05-09T09:30:13.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.424+0000 7fecb6977640 1 -- 172.21.15.12:0/2073664620 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fecb01a25b0 con 0x7fecb0102a80 2024-05-09T09:30:13.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.424+0000 7fecad7fa640 1 -- 172.21.15.12:0/2073664620 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7feca409bc10 con 0x7fecb0102a80 2024-05-09T09:30:13.424 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T09:30:13.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.426+0000 7fecb6977640 1 -- 172.21.15.12:0/2073664620 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fec980a2b90 msgr2=0x7fec980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.426+0000 7fecb6977640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fec980a2b90 0x7fec980a5050 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7feca0000c00 tx=0x7feca0031510 comp rx=0 tx=0).stop 2024-05-09T09:30:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.426+0000 7fecb6977640 1 -- 172.21.15.12:0/2073664620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fecb0102a80 msgr2=0x7fecb019a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.426+0000 7fecb6977640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fecb0102a80 0x7fecb019a330 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7feca400d950 tx=0x7feca400de10 comp rx=0 tx=0).stop 2024-05-09T09:30:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.426+0000 7fecb6977640 1 -- 172.21.15.12:0/2073664620 shutdown_connections 2024-05-09T09:30:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.426+0000 7fecb6977640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fec980a2b90 0x7fec980a5050 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.426+0000 7fecb6977640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb0103440 0x7fecb01a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.426+0000 7fecb6977640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fecb0102a80 0x7fecb019a330 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.426+0000 7fecb6977640 1 --2- 172.21.15.12:0/2073664620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0101890 0x7fecb0199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.426+0000 7fecb6977640 1 -- 172.21.15.12:0/2073664620 >> 172.21.15.12:0/2073664620 conn(0x7fecb00fce00 msgr2=0x7fecb0073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.426+0000 7fecb6977640 1 -- 172.21.15.12:0/2073664620 shutdown_connections 2024-05-09T09:30:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.426+0000 7fecb6977640 1 -- 172.21.15.12:0/2073664620 wait complete. 2024-05-09T09:30:13.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:13 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3304288349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:30:13.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:13 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/393894640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:30:13.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:13 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4105183488' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:30:13.439 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:30:13.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.548+0000 7f37c182a640 1 -- 172.21.15.12:0/4109861652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37bc102340 msgr2=0x7f37bc108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.548+0000 7f37c182a640 1 --2- 172.21.15.12:0/4109861652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37bc102340 0x7f37bc108bd0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009ca0 tx=0x7f37b402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.548+0000 7f37c182a640 1 -- 172.21.15.12:0/4109861652 shutdown_connections 2024-05-09T09:30:13.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.548+0000 7f37c182a640 1 --2- 172.21.15.12:0/4109861652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37bc102340 0x7f37bc108bd0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.548+0000 7f37c182a640 1 --2- 172.21.15.12:0/4109861652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc101980 0x7f37bc101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.548+0000 7f37c182a640 1 --2- 172.21.15.12:0/4109861652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37bc100790 0x7f37bc100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.548+0000 7f37c182a640 1 -- 172.21.15.12:0/4109861652 >> 172.21.15.12:0/4109861652 conn(0x7f37bc0fbd20 msgr2=0x7f37bc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.548+0000 7f37c182a640 1 -- 172.21.15.12:0/4109861652 shutdown_connections 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.548+0000 7f37c182a640 1 -- 172.21.15.12:0/4109861652 wait complete. 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37c182a640 1 Processor -- start 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37c182a640 1 -- start start 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37c182a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37bc100790 0x7f37bc06a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37c182a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc101980 0x7f37bc19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37c182a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37bc102340 0x7f37bc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37c182a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc10c470 con 0x7f37bc100790 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37c182a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc10c2f0 con 0x7f37bc101980 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37c182a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc10c5f0 con 0x7f37bc102340 2024-05-09T09:30:13.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37baffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37bc100790 0x7f37bc06a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:13.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37baffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37bc100790 0x7f37bc06a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58804/0 (socket says 172.21.15.12:58804) 2024-05-09T09:30:13.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37baffd640 1 -- 172.21.15.12:0/4045756737 learned_addr learned my addr 172.21.15.12:0/4045756737 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:13.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37ba7fc640 1 --2- 172.21.15.12:0/4045756737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc101980 0x7f37bc19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:13.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37baffd640 1 -- 172.21.15.12:0/4045756737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37bc102340 msgr2=0x7f37bc19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37baffd640 1 --2- 172.21.15.12:0/4045756737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37bc102340 0x7f37bc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37baffd640 1 -- 172.21.15.12:0/4045756737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc101980 msgr2=0x7f37bc19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37baffd640 1 --2- 172.21.15.12:0/4045756737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc101980 0x7f37bc19c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37baffd640 1 -- 172.21.15.12:0/4045756737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc19f7f0 con 0x7f37bc100790 2024-05-09T09:30:13.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.549+0000 7f37baffd640 1 --2- 172.21.15.12:0/4045756737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37bc100790 0x7f37bc06a9c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00b450 tx=0x7f37ac00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:13.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.550+0000 7f37c0828640 1 -- 172.21.15.12:0/4045756737 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac015020 con 0x7f37bc100790 2024-05-09T09:30:13.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.550+0000 7f37c0828640 1 -- 172.21.15.12:0/4045756737 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f37ac011d60 con 0x7f37bc100790 2024-05-09T09:30:13.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.550+0000 7f37c182a640 1 -- 172.21.15.12:0/4045756737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37bc19fae0 con 0x7f37bc100790 2024-05-09T09:30:13.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.550+0000 7f37c0828640 1 -- 172.21.15.12:0/4045756737 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00f340 con 0x7f37bc100790 2024-05-09T09:30:13.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.550+0000 7f37c182a640 1 -- 172.21.15.12:0/4045756737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37bc1a0060 con 0x7f37bc100790 2024-05-09T09:30:13.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.551+0000 7f37c182a640 1 -- 172.21.15.12:0/4045756737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37bc107ab0 con 0x7f37bc100790 2024-05-09T09:30:13.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.552+0000 7f37c0828640 1 -- 172.21.15.12:0/4045756737 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f37ac018020 con 0x7f37bc100790 2024-05-09T09:30:13.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.553+0000 7f37c0828640 1 --2- 172.21.15.12:0/4045756737 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37a40a2b90 0x7f37a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:13.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.553+0000 7f37ba7fc640 1 --2- 172.21.15.12:0/4045756737 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37a40a2b90 0x7f37a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:13.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.553+0000 7f37c0828640 1 -- 172.21.15.12:0/4045756737 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f37ac0cf010 con 0x7f37bc100790 2024-05-09T09:30:13.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.553+0000 7f37ba7fc640 1 --2- 172.21.15.12:0/4045756737 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37a40a2b90 0x7f37a40a5050 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f37b0005fd0 tx=0x7f37b0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:13.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.557+0000 7f37c0828640 1 -- 172.21.15.12:0/4045756737 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f37ac01f020 con 0x7f37bc100790 2024-05-09T09:30:13.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.743+0000 7f37c182a640 1 -- 172.21.15.12:0/4045756737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f37bc0624d0 con 0x7f37bc100790 2024-05-09T09:30:13.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.743+0000 7f37c0828640 1 -- 172.21.15.12:0/4045756737 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f37ac00d040 con 0x7f37bc100790 2024-05-09T09:30:13.743 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T09:30:13.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:13 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3304288349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:30:13.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:13 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/393894640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:30:13.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:13 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/4105183488' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:30:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.745+0000 7f37c182a640 1 -- 172.21.15.12:0/4045756737 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37a40a2b90 msgr2=0x7f37a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.745+0000 7f37c182a640 1 --2- 172.21.15.12:0/4045756737 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37a40a2b90 0x7f37a40a5050 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f37b0005fd0 tx=0x7f37b0005850 comp rx=0 tx=0).stop 2024-05-09T09:30:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.745+0000 7f37c182a640 1 -- 172.21.15.12:0/4045756737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37bc100790 msgr2=0x7f37bc06a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.745+0000 7f37c182a640 1 --2- 172.21.15.12:0/4045756737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37bc100790 0x7f37bc06a9c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00b450 tx=0x7f37ac00b910 comp rx=0 tx=0).stop 2024-05-09T09:30:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.745+0000 7f37c182a640 1 -- 172.21.15.12:0/4045756737 shutdown_connections 2024-05-09T09:30:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.745+0000 7f37c182a640 1 --2- 172.21.15.12:0/4045756737 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f37a40a2b90 0x7f37a40a5050 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.745+0000 7f37c182a640 1 --2- 172.21.15.12:0/4045756737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37bc102340 0x7f37bc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.745+0000 7f37c182a640 1 --2- 172.21.15.12:0/4045756737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37bc101980 0x7f37bc19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.745+0000 7f37c182a640 1 --2- 172.21.15.12:0/4045756737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37bc100790 0x7f37bc06a9c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.745+0000 7f37c182a640 1 -- 172.21.15.12:0/4045756737 >> 172.21.15.12:0/4045756737 conn(0x7f37bc0fbd20 msgr2=0x7f37bc104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.745+0000 7f37c182a640 1 -- 172.21.15.12:0/4045756737 shutdown_connections 2024-05-09T09:30:13.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.745+0000 7f37c182a640 1 -- 172.21.15.12:0/4045756737 wait complete. 2024-05-09T09:30:13.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:13 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3304288349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:30:13.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:13 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/393894640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:30:13.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:13 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/4105183488' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:30:13.757 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T09:30:13.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.864+0000 7f62515e7640 1 -- 172.21.15.12:0/2582593124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f624c103440 msgr2=0x7f624c109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.864+0000 7f62515e7640 1 --2- 172.21.15.12:0/2582593124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f624c103440 0x7f624c109cd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6244009ca0 tx=0x7f624402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.864+0000 7f62515e7640 1 -- 172.21.15.12:0/2582593124 shutdown_connections 2024-05-09T09:30:13.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.864+0000 7f62515e7640 1 --2- 172.21.15.12:0/2582593124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f624c103440 0x7f624c109cd0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.864+0000 7f62515e7640 1 --2- 172.21.15.12:0/2582593124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624c102a80 0x7f624c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.864+0000 7f62515e7640 1 --2- 172.21.15.12:0/2582593124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c101890 0x7f624c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.864+0000 7f62515e7640 1 -- 172.21.15.12:0/2582593124 >> 172.21.15.12:0/2582593124 conn(0x7f624c0fce00 msgr2=0x7f624c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.864+0000 7f62515e7640 1 -- 172.21.15.12:0/2582593124 shutdown_connections 2024-05-09T09:30:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.864+0000 7f62515e7640 1 -- 172.21.15.12:0/2582593124 wait complete. 2024-05-09T09:30:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.865+0000 7f62515e7640 1 Processor -- start 2024-05-09T09:30:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.865+0000 7f62515e7640 1 -- start start 2024-05-09T09:30:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.865+0000 7f62515e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c101890 0x7f624c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.865+0000 7f62515e7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f624c102a80 0x7f624c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.865+0000 7f62515e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624c103440 0x7f624c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.865+0000 7f62515e7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f624c10d4f0 con 0x7f624c102a80 2024-05-09T09:30:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.865+0000 7f62515e7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f624c10d370 con 0x7f624c103440 2024-05-09T09:30:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.865+0000 7f62515e7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f624c10d670 con 0x7f624c101890 2024-05-09T09:30:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.865+0000 7f6243fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f624c102a80 0x7f624c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.865+0000 7f6243fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f624c102a80 0x7f624c198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58820/0 (socket says 172.21.15.12:58820) 2024-05-09T09:30:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.865+0000 7f6243fff640 1 -- 172.21.15.12:0/3381003539 learned_addr learned my addr 172.21.15.12:0/3381003539 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.865+0000 7f624b7fe640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624c103440 0x7f624c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.865+0000 7f624affd640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c101890 0x7f624c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.866+0000 7f6243fff640 1 -- 172.21.15.12:0/3381003539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c101890 msgr2=0x7f624c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.866+0000 7f6243fff640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c101890 0x7f624c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.866+0000 7f6243fff640 1 -- 172.21.15.12:0/3381003539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624c103440 msgr2=0x7f624c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.866+0000 7f6243fff640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624c103440 0x7f624c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.866+0000 7f6243fff640 1 -- 172.21.15.12:0/3381003539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624c19f710 con 0x7f624c102a80 2024-05-09T09:30:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.866+0000 7f624affd640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c101890 0x7f624c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:30:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.866+0000 7f6243fff640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f624c102a80 0x7f624c198030 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f623400d950 tx=0x7f623400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.866+0000 7f6248ff9640 1 -- 172.21.15.12:0/3381003539 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234017030 con 0x7f624c102a80 2024-05-09T09:30:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.866+0000 7f6248ff9640 1 -- 172.21.15.12:0/3381003539 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6234004020 con 0x7f624c102a80 2024-05-09T09:30:13.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.866+0000 7f6248ff9640 1 -- 172.21.15.12:0/3381003539 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62340047a0 con 0x7f624c102a80 2024-05-09T09:30:13.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.866+0000 7f62515e7640 1 -- 172.21.15.12:0/3381003539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624c19fa00 con 0x7f624c102a80 2024-05-09T09:30:13.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.866+0000 7f62515e7640 1 -- 172.21.15.12:0/3381003539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624c19ff50 con 0x7f624c102a80 2024-05-09T09:30:13.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.867+0000 7f62515e7640 1 -- 172.21.15.12:0/3381003539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f624c108b50 con 0x7f624c102a80 2024-05-09T09:30:13.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.868+0000 7f6248ff9640 1 -- 172.21.15.12:0/3381003539 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f62340041d0 con 0x7f624c102a80 2024-05-09T09:30:13.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.869+0000 7f6248ff9640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f62380a2b90 0x7f62380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:13.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.869+0000 7f6248ff9640 1 -- 172.21.15.12:0/3381003539 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f62340cf4f0 con 0x7f624c102a80 2024-05-09T09:30:13.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.869+0000 7f624affd640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f62380a2b90 0x7f62380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:13.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.870+0000 7f624affd640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f62380a2b90 0x7f62380a5050 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f624c1028e0 tx=0x7f623c005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:13.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:13.872+0000 7f6248ff9640 1 -- 172.21.15.12:0/3381003539 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f623409c080 con 0x7f624c102a80 2024-05-09T09:30:14.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.073+0000 7f62515e7640 1 -- 172.21.15.12:0/3381003539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f624c02d3f0 con 0x7f624c102a80 2024-05-09T09:30:14.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.073+0000 7f6248ff9640 1 -- 172.21.15.12:0/3381003539 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+591 (secure 0 0 0) 0x7f623409b980 con 0x7f624c102a80 2024-05-09T09:30:14.073 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1454579683 2024-05-10T09:16:04.466876+0000 2024-05-09T09:30:14.073 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/2168557623 2024-05-10T09:16:04.466876+0000 2024-05-09T09:30:14.073 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/2168557623 2024-05-10T09:16:04.466876+0000 2024-05-09T09:30:14.074 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/545109421 2024-05-10T09:15:36.182637+0000 2024-05-09T09:30:14.074 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1153421462 2024-05-10T09:16:04.466876+0000 2024-05-09T09:30:14.074 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1731765200 2024-05-10T09:15:36.182637+0000 2024-05-09T09:30:14.074 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2508497448 2024-05-10T09:15:36.182637+0000 2024-05-09T09:30:14.074 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1876427478 2024-05-10T09:15:36.182637+0000 2024-05-09T09:30:14.074 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3479121957 2024-05-10T09:16:04.466876+0000 2024-05-09T09:30:14.074 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1876427478 2024-05-10T09:15:36.182637+0000 2024-05-09T09:30:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:listed 10 entries 2024-05-09T09:30:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.076+0000 7f62515e7640 1 -- 172.21.15.12:0/3381003539 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f62380a2b90 msgr2=0x7f62380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.076+0000 7f62515e7640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f62380a2b90 0x7f62380a5050 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f624c1028e0 tx=0x7f623c005db0 comp rx=0 tx=0).stop 2024-05-09T09:30:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.076+0000 7f62515e7640 1 -- 172.21.15.12:0/3381003539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f624c102a80 msgr2=0x7f624c198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.076+0000 7f62515e7640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f624c102a80 0x7f624c198030 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f623400d950 tx=0x7f623400de10 comp rx=0 tx=0).stop 2024-05-09T09:30:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.076+0000 7f62515e7640 1 -- 172.21.15.12:0/3381003539 shutdown_connections 2024-05-09T09:30:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.076+0000 7f62515e7640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f62380a2b90 0x7f62380a5050 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.076+0000 7f62515e7640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624c103440 0x7f624c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.076+0000 7f62515e7640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f624c102a80 0x7f624c198030 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.076+0000 7f62515e7640 1 --2- 172.21.15.12:0/3381003539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c101890 0x7f624c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.076+0000 7f62515e7640 1 -- 172.21.15.12:0/3381003539 >> 172.21.15.12:0/3381003539 conn(0x7f624c0fce00 msgr2=0x7f624c105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.076+0000 7f62515e7640 1 -- 172.21.15.12:0/3381003539 shutdown_connections 2024-05-09T09:30:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:14.076+0000 7f62515e7640 1 -- 172.21.15.12:0/3381003539 wait complete. 2024-05-09T09:30:14.087 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi012.front.sepia.ceph.com... 2024-05-09T09:30:14.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:30:14.087 DEBUG:teuthology.orchestra.run.smithi012:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T09:30:14.112 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi012.stderr:ceph-fuse[85012]: fuse finished with error 0 and tester_r 0 2024-05-09T09:30:14.113 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T09:30:14.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:14 smithi012 ceph-mon[30140]: pgmap v508: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:14.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:14 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2073664620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:30:14.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:14 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4045756737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:30:14.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:14 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3381003539' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:30:14.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:14 smithi170 ceph-mon[33632]: pgmap v508: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:14.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:14 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2073664620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:30:14.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:14 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/4045756737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:30:14.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:14 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3381003539' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:30:14.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:14 smithi016 ceph-mon[33335]: pgmap v508: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T09:30:14.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:14 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2073664620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:30:14.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:14 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/4045756737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:30:14.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:14 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3381003539' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:30:16.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:16 smithi012 ceph-mon[30140]: pgmap v509: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:30:16.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:16 smithi170 ceph-mon[33632]: pgmap v509: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:30:16.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:16 smithi016 ceph-mon[33335]: pgmap v509: 129 pgs: 129 active+clean; 56 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T09:30:18.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:18 smithi012 ceph-mon[30140]: pgmap v510: 129 pgs: 129 active+clean; 56 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:30:18.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:18 smithi170 ceph-mon[33632]: pgmap v510: 129 pgs: 129 active+clean; 56 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:30:18.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:18 smithi016 ceph-mon[33335]: pgmap v510: 129 pgs: 129 active+clean; 56 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T09:30:20.215 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-05-09T09:30:20.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:30:20.216 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T09:30:20.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:30:20.237 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-05-09T09:30:20.298 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T09:30:20.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:30:20.298 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T09:30:20.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:30:20.373 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-05-09T09:30:20.466 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-09T09:30:20.482 INFO:tasks.cephadm:Teardown begin 2024-05-09T09:30:20.482 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T09:30:20.483 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:30:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.595+0000 7f2416b90640 1 -- 172.21.15.12:0/755202107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2410103450 msgr2=0x7f2410109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.595+0000 7f2416b90640 1 --2- 172.21.15.12:0/755202107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2410103450 0x7f2410109ce0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2408009ca0 tx=0x7f240802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.595+0000 7f2416b90640 1 -- 172.21.15.12:0/755202107 shutdown_connections 2024-05-09T09:30:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 --2- 172.21.15.12:0/755202107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2410103450 0x7f2410109ce0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 --2- 172.21.15.12:0/755202107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2410102a90 0x7f2410102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 --2- 172.21.15.12:0/755202107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24101018a0 0x7f2410101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 -- 172.21.15.12:0/755202107 >> 172.21.15.12:0/755202107 conn(0x7f24100fce30 msgr2=0x7f24100ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:20.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 -- 172.21.15.12:0/755202107 shutdown_connections 2024-05-09T09:30:20.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 -- 172.21.15.12:0/755202107 wait complete. 2024-05-09T09:30:20.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 Processor -- start 2024-05-09T09:30:20.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 -- start start 2024-05-09T09:30:20.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24101018a0 0x7f2410197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:20.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2410102a90 0x7f2410198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:20.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2410103450 0x7f241019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:20.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f241010d500 con 0x7f2410103450 2024-05-09T09:30:20.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f241010d380 con 0x7f24101018a0 2024-05-09T09:30:20.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.596+0000 7f2416b90640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f241010d680 con 0x7f2410102a90 2024-05-09T09:30:20.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f241512d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2410103450 0x7f241019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:20.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f241512d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2410103450 0x7f241019f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58836/0 (socket says 172.21.15.12:58836) 2024-05-09T09:30:20.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f241512d640 1 -- 172.21.15.12:0/3231681062 learned_addr learned my addr 172.21.15.12:0/3231681062 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:20.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f240ffff640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2410102a90 0x7f2410198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:20.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f241492c640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24101018a0 0x7f2410197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:20.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f241512d640 1 -- 172.21.15.12:0/3231681062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2410102a90 msgr2=0x7f2410198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:20.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f241512d640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2410102a90 0x7f2410198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f241512d640 1 -- 172.21.15.12:0/3231681062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24101018a0 msgr2=0x7f2410197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:20.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f241512d640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24101018a0 0x7f2410197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f241512d640 1 -- 172.21.15.12:0/3231681062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241019f770 con 0x7f2410103450 2024-05-09T09:30:20.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f241512d640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2410103450 0x7f241019f100 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2408009430 tx=0x7f240802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:20.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f240dffb640 1 -- 172.21.15.12:0/3231681062 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408003bf0 con 0x7f2410103450 2024-05-09T09:30:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f240dffb640 1 -- 172.21.15.12:0/3231681062 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2408003d90 con 0x7f2410103450 2024-05-09T09:30:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f2416b90640 1 -- 172.21.15.12:0/3231681062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241019fa00 con 0x7f2410103450 2024-05-09T09:30:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.597+0000 7f240dffb640 1 -- 172.21.15.12:0/3231681062 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240802fc90 con 0x7f2410103450 2024-05-09T09:30:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.598+0000 7f2416b90640 1 -- 172.21.15.12:0/3231681062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241019ffa0 con 0x7f2410103450 2024-05-09T09:30:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.599+0000 7f2416b90640 1 -- 172.21.15.12:0/3231681062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2410108c90 con 0x7f2410103450 2024-05-09T09:30:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.601+0000 7f240dffb640 1 -- 172.21.15.12:0/3231681062 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2408004040 con 0x7f2410103450 2024-05-09T09:30:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.601+0000 7f240dffb640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f23f80a2c10 0x7f23f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.601+0000 7f240dffb640 1 -- 172.21.15.12:0/3231681062 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f24080f0c40 con 0x7f2410103450 2024-05-09T09:30:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.601+0000 7f241492c640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f23f80a2c10 0x7f23f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.602+0000 7f241492c640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f23f80a2c10 0x7f23f80a50d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f24101028f0 tx=0x7f2400005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.605+0000 7f240dffb640 1 -- 172.21.15.12:0/3231681062 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f240807f740 con 0x7f2410103450 2024-05-09T09:30:20.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:20 smithi012 ceph-mon[30140]: pgmap v511: 129 pgs: 129 active+clean; 56 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-09T09:30:20.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:20 smithi170 ceph-mon[33632]: pgmap v511: 129 pgs: 129 active+clean; 56 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-09T09:30:20.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:20 smithi016 ceph-mon[33335]: pgmap v511: 129 pgs: 129 active+clean; 56 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-09T09:30:20.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.831+0000 7f2416b90640 1 -- 172.21.15.12:0/3231681062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f241002d1c0 con 0x7f2410103450 2024-05-09T09:30:20.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.832+0000 7f240dffb640 1 -- 172.21.15.12:0/3231681062 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11966 (secure 0 0 0) 0x7f24080bd860 con 0x7f2410103450 2024-05-09T09:30:20.832 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:30:20.833 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:28:24.476138+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25883":{"gid":25883,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3067829157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3067829157},{"type":"v1","addr":"172.21.15.16:6837","nonce":3067829157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[24502,24523,24515,14628,24484]},"id":1}]} 2024-05-09T09:30:20.833 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 42 2024-05-09T09:30:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.834+0000 7f2416b90640 1 -- 172.21.15.12:0/3231681062 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f23f80a2c10 msgr2=0x7f23f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.834+0000 7f2416b90640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f23f80a2c10 0x7f23f80a50d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f24101028f0 tx=0x7f2400005f90 comp rx=0 tx=0).stop 2024-05-09T09:30:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.834+0000 7f2416b90640 1 -- 172.21.15.12:0/3231681062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2410103450 msgr2=0x7f241019f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.834+0000 7f2416b90640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2410103450 0x7f241019f100 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2408009430 tx=0x7f240802f380 comp rx=0 tx=0).stop 2024-05-09T09:30:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.834+0000 7f2416b90640 1 -- 172.21.15.12:0/3231681062 shutdown_connections 2024-05-09T09:30:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.834+0000 7f2416b90640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f23f80a2c10 0x7f23f80a50d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.834+0000 7f2416b90640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2410103450 0x7f241019f100 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.834+0000 7f2416b90640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2410102a90 0x7f2410198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.834+0000 7f2416b90640 1 --2- 172.21.15.12:0/3231681062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24101018a0 0x7f2410197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.834+0000 7f2416b90640 1 -- 172.21.15.12:0/3231681062 >> 172.21.15.12:0/3231681062 conn(0x7f24100fce30 msgr2=0x7f2410105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.834+0000 7f2416b90640 1 -- 172.21.15.12:0/3231681062 shutdown_connections 2024-05-09T09:30:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.834+0000 7f2416b90640 1 -- 172.21.15.12:0/3231681062 wait complete. 2024-05-09T09:30:20.846 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:30:20.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.951+0000 7f1aadf9e640 1 -- 172.21.15.12:0/2712784299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa8104c40 msgr2=0x7f1aa8107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:20.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.951+0000 7f1aadf9e640 1 --2- 172.21.15.12:0/2712784299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa8104c40 0x7f1aa8107030 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009900 tx=0x7f1a9c02a410 comp rx=0 tx=0).stop 2024-05-09T09:30:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.951+0000 7f1aadf9e640 1 -- 172.21.15.12:0/2712784299 shutdown_connections 2024-05-09T09:30:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.951+0000 7f1aadf9e640 1 --2- 172.21.15.12:0/2712784299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1aa8107570 0x7f1aa81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.951+0000 7f1aadf9e640 1 --2- 172.21.15.12:0/2712784299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa8104c40 0x7f1aa8107030 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.951+0000 7f1aadf9e640 1 --2- 172.21.15.12:0/2712784299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aa8102310 0x7f1aa8104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.951+0000 7f1aadf9e640 1 -- 172.21.15.12:0/2712784299 >> 172.21.15.12:0/2712784299 conn(0x7f1aa80fbd20 msgr2=0x7f1aa80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.951+0000 7f1aadf9e640 1 -- 172.21.15.12:0/2712784299 shutdown_connections 2024-05-09T09:30:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.951+0000 7f1aadf9e640 1 -- 172.21.15.12:0/2712784299 wait complete. 2024-05-09T09:30:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.952+0000 7f1aadf9e640 1 Processor -- start 2024-05-09T09:30:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.952+0000 7f1aadf9e640 1 -- start start 2024-05-09T09:30:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.952+0000 7f1aadf9e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa8102310 0x7f1aa8101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.952+0000 7f1aadf9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aa8104c40 0x7f1aa81002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.952+0000 7f1aadf9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1aa8107570 0x7f1aa8100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.952+0000 7f1aadf9e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa810d4f0 con 0x7f1aa8102310 2024-05-09T09:30:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.952+0000 7f1aadf9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa810d370 con 0x7f1aa8107570 2024-05-09T09:30:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.952+0000 7f1aadf9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa810d670 con 0x7f1aa8104c40 2024-05-09T09:30:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.952+0000 7f1aa77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa8102310 0x7f1aa8101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.952+0000 7f1aa77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa8102310 0x7f1aa8101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58850/0 (socket says 172.21.15.12:58850) 2024-05-09T09:30:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.952+0000 7f1aa77fe640 1 -- 172.21.15.12:0/4203575813 learned_addr learned my addr 172.21.15.12:0/4203575813 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.952+0000 7f1aa7fff640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1aa8107570 0x7f1aa8100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.952+0000 7f1aa6ffd640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aa8104c40 0x7f1aa81002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.953+0000 7f1aa77fe640 1 -- 172.21.15.12:0/4203575813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aa8104c40 msgr2=0x7f1aa81002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.953+0000 7f1aa77fe640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aa8104c40 0x7f1aa81002c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.953+0000 7f1aa77fe640 1 -- 172.21.15.12:0/4203575813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1aa8107570 msgr2=0x7f1aa8100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.953+0000 7f1aa77fe640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1aa8107570 0x7f1aa8100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.953+0000 7f1aa77fe640 1 -- 172.21.15.12:0/4203575813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa8101090 con 0x7f1aa8102310 2024-05-09T09:30:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.953+0000 7f1aa77fe640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa8102310 0x7f1aa8101c80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800e9f0 tx=0x7f1a9800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.953+0000 7f1aa4ff9640 1 -- 172.21.15.12:0/4203575813 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9800cd80 con 0x7f1aa8102310 2024-05-09T09:30:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.953+0000 7f1aadf9e640 1 -- 172.21.15.12:0/4203575813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa81a1d20 con 0x7f1aa8102310 2024-05-09T09:30:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.953+0000 7f1aa4ff9640 1 -- 172.21.15.12:0/4203575813 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1a980054f0 con 0x7f1aa8102310 2024-05-09T09:30:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.953+0000 7f1aa4ff9640 1 -- 172.21.15.12:0/4203575813 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98010690 con 0x7f1aa8102310 2024-05-09T09:30:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.953+0000 7f1aadf9e640 1 -- 172.21.15.12:0/4203575813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa81a2190 con 0x7f1aa8102310 2024-05-09T09:30:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.954+0000 7f1aadf9e640 1 -- 172.21.15.12:0/4203575813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa80fd810 con 0x7f1aa8102310 2024-05-09T09:30:20.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.955+0000 7f1aa4ff9640 1 -- 172.21.15.12:0/4203575813 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1a98004f20 con 0x7f1aa8102310 2024-05-09T09:30:20.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.956+0000 7f1aa4ff9640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1a900a2b90 0x7f1a900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:20.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.956+0000 7f1aa4ff9640 1 -- 172.21.15.12:0/4203575813 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f1a980cf940 con 0x7f1aa8102310 2024-05-09T09:30:20.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.956+0000 7f1aa6ffd640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1a900a2b90 0x7f1a900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:20.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.957+0000 7f1aa6ffd640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1a900a2b90 0x7f1a900a5050 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c000c00 tx=0x7f1a9c031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:20.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:20.961+0000 7f1aa4ff9640 1 -- 172.21.15.12:0/4203575813 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1a9809c4d0 con 0x7f1aa8102310 2024-05-09T09:30:21.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.188+0000 7f1aadf9e640 1 -- 172.21.15.12:0/4203575813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1aa8062430 con 0x7f1aa8102310 2024-05-09T09:30:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.189+0000 7f1aa4ff9640 1 -- 172.21.15.12:0/4203575813 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11966 (secure 0 0 0) 0x7f1a9809bdd0 con 0x7f1aa8102310 2024-05-09T09:30:21.189 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:30:21.189 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2151845805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2151845805},{"type":"v1","addr":"172.21.15.12:6835","nonce":2151845805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3490188582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3490188582},{"type":"v1","addr":"172.21.15.12:6837","nonce":3490188582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:23:09.656296+0000","modified":"2024-05-09T09:28:24.476138+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24523,"mds_1":24515,"mds_2":24502,"mds_3":24484,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/901613691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":901613691},{"type":"v1","addr":"172.21.15.12:6839","nonce":901613691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":10,"addr":"172.21.15.12:6841/1434721532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1434721532},{"type":"v1","addr":"172.21.15.12:6841","nonce":1434721532}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6833/3659922754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3659922754},{"type":"v1","addr":"172.21.15.16:6833","nonce":3659922754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/4010027838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4010027838},{"type":"v1","addr":"172.21.15.170:6833","nonce":4010027838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1098165328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1098165328},{"type":"v1","addr":"172.21.15.16:6835","nonce":1098165328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6835/277363225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":277363225},{"type":"v1","addr":"172.21.15.170:6835","nonce":277363225}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/2973369023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2973369023},{"type":"v1","addr":"172.21.15.170:6837","nonce":2973369023}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6839/3932062083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3932062083},{"type":"v1","addr":"172.21.15.16:6839","nonce":3932062083}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.170:6839/3731005466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3731005466},{"type":"v1","addr":"172.21.15.170:6839","nonce":3731005466}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25883":{"gid":25883,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/3067829157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3067829157},{"type":"v1","addr":"172.21.15.16:6837","nonce":3067829157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24523,"qdb_cluster":[24502,24523,24515,14628,24484]},"id":1}]} 2024-05-09T09:30:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 42 2024-05-09T09:30:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.191+0000 7f1aadf9e640 1 -- 172.21.15.12:0/4203575813 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1a900a2b90 msgr2=0x7f1a900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.191+0000 7f1aadf9e640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1a900a2b90 0x7f1a900a5050 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c000c00 tx=0x7f1a9c031510 comp rx=0 tx=0).stop 2024-05-09T09:30:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.191+0000 7f1aadf9e640 1 -- 172.21.15.12:0/4203575813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa8102310 msgr2=0x7f1aa8101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.191+0000 7f1aadf9e640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa8102310 0x7f1aa8101c80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800e9f0 tx=0x7f1a9800edf0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.191+0000 7f1aadf9e640 1 -- 172.21.15.12:0/4203575813 shutdown_connections 2024-05-09T09:30:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.191+0000 7f1aadf9e640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f1a900a2b90 0x7f1a900a5050 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.191+0000 7f1aadf9e640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1aa8107570 0x7f1aa8100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.191+0000 7f1aadf9e640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aa8104c40 0x7f1aa81002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.191+0000 7f1aadf9e640 1 --2- 172.21.15.12:0/4203575813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa8102310 0x7f1aa8101c80 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.191+0000 7f1aadf9e640 1 -- 172.21.15.12:0/4203575813 >> 172.21.15.12:0/4203575813 conn(0x7f1aa80fbd20 msgr2=0x7f1aa8102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.191+0000 7f1aadf9e640 1 -- 172.21.15.12:0/4203575813 shutdown_connections 2024-05-09T09:30:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.191+0000 7f1aadf9e640 1 -- 172.21.15.12:0/4203575813 wait complete. 2024-05-09T09:30:21.204 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:30:21.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.308+0000 7f936ce38640 1 -- 172.21.15.12:0/485174615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9368102a90 msgr2=0x7f9368102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:21.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.308+0000 7f936ce38640 1 --2- 172.21.15.12:0/485174615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9368102a90 0x7f9368102f10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9350009900 tx=0x7f935002a410 comp rx=0 tx=0).stop 2024-05-09T09:30:21.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.308+0000 7f936ce38640 1 -- 172.21.15.12:0/485174615 shutdown_connections 2024-05-09T09:30:21.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.308+0000 7f936ce38640 1 --2- 172.21.15.12:0/485174615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9368103450 0x7f9368109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.308+0000 7f936ce38640 1 --2- 172.21.15.12:0/485174615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9368102a90 0x7f9368102f10 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.308+0000 7f936ce38640 1 --2- 172.21.15.12:0/485174615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93681018a0 0x7f9368101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.308+0000 7f936ce38640 1 -- 172.21.15.12:0/485174615 >> 172.21.15.12:0/485174615 conn(0x7f93680fce30 msgr2=0x7f93680ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.308+0000 7f936ce38640 1 -- 172.21.15.12:0/485174615 shutdown_connections 2024-05-09T09:30:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.308+0000 7f936ce38640 1 -- 172.21.15.12:0/485174615 wait complete. 2024-05-09T09:30:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f936ce38640 1 Processor -- start 2024-05-09T09:30:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f936ce38640 1 -- start start 2024-05-09T09:30:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f936ce38640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93681018a0 0x7f93680755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f936ce38640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9368102a90 0x7f9368073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f936ce38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9368103450 0x7f9368074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f936ce38640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f936810d500 con 0x7f9368102a90 2024-05-09T09:30:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f936ce38640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f936810d380 con 0x7f9368103450 2024-05-09T09:30:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f936ce38640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f936810d680 con 0x7f93681018a0 2024-05-09T09:30:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f935ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9368102a90 0x7f9368073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f935ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9368102a90 0x7f9368073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58864/0 (socket says 172.21.15.12:58864) 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f935ffff640 1 -- 172.21.15.12:0/4244002486 learned_addr learned my addr 172.21.15.12:0/4244002486 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f9366d9d640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9368103450 0x7f9368074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f936659c640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93681018a0 0x7f93680755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f935ffff640 1 -- 172.21.15.12:0/4244002486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93681018a0 msgr2=0x7f93680755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f935ffff640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93681018a0 0x7f93680755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f935ffff640 1 -- 172.21.15.12:0/4244002486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9368103450 msgr2=0x7f9368074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f935ffff640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9368103450 0x7f9368074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.309+0000 7f935ffff640 1 -- 172.21.15.12:0/4244002486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93680749f0 con 0x7f9368102a90 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.310+0000 7f935ffff640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9368102a90 0x7f9368073c20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9350009230 tx=0x7f9350031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.310+0000 7f935f7fe640 1 -- 172.21.15.12:0/4244002486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935003f030 con 0x7f9368102a90 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.310+0000 7f936ce38640 1 -- 172.21.15.12:0/4244002486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93681a1c60 con 0x7f9368102a90 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.310+0000 7f935f7fe640 1 -- 172.21.15.12:0/4244002486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9350003ac0 con 0x7f9368102a90 2024-05-09T09:30:21.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.310+0000 7f935f7fe640 1 -- 172.21.15.12:0/4244002486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350035070 con 0x7f9368102a90 2024-05-09T09:30:21.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.310+0000 7f936ce38640 1 -- 172.21.15.12:0/4244002486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93681a21b0 con 0x7f9368102a90 2024-05-09T09:30:21.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.311+0000 7f936ce38640 1 -- 172.21.15.12:0/4244002486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9368108bc0 con 0x7f9368102a90 2024-05-09T09:30:21.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.313+0000 7f935f7fe640 1 -- 172.21.15.12:0/4244002486 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9350042030 con 0x7f9368102a90 2024-05-09T09:30:21.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.313+0000 7f935f7fe640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f93540a2b90 0x7f93540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:21.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.314+0000 7f936659c640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f93540a2b90 0x7f93540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:21.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.314+0000 7f935f7fe640 1 -- 172.21.15.12:0/4244002486 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f93500efd40 con 0x7f9368102a90 2024-05-09T09:30:21.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.314+0000 7f936659c640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f93540a2b90 0x7f93540a5050 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f93681028f0 tx=0x7f9358005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:21.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.317+0000 7f935f7fe640 1 -- 172.21.15.12:0/4244002486 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f93500bc280 con 0x7f9368102a90 2024-05-09T09:30:21.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:21 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3231681062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:30:21.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:21 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4203575813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:30:21.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.506+0000 7f936ce38640 1 -- 172.21.15.12:0/4244002486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f936802d430 con 0x7f9368102a90 2024-05-09T09:30:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.508+0000 7f935f7fe640 1 -- 172.21.15.12:0/4244002486 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24762 (secure 0 0 0) 0x7f93500c0580 con 0x7f9368102a90 2024-05-09T09:30:21.509 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T09:30:21.510 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":94,"fsid":"770e12d0-0de4-11ef-bc98-c7b262605968","created":"2024-05-09T09:15:02.891661+0000","modified":"2024-05-09T09:27:40.951852+0000","last_up_change":"2024-05-09T09:22:19.559302+0000","last_in_change":"2024-05-09T09:21:59.588476+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:18:31.001259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:23:08.167107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:23:09.065670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:23:12.773640+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3abc7d81-df98-4a60-8798-58a765426230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6803","nonce":2958883281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6805","nonce":2958883281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6809","nonce":2958883281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2958883281},{"type":"v1","addr":"172.21.15.12:6807","nonce":2958883281}]},"public_addr":"172.21.15.12:6803/2958883281","cluster_addr":"172.21.15.12:6805/2958883281","heartbeat_back_addr":"172.21.15.12:6809/2958883281","heartbeat_front_addr":"172.21.15.12:6807/2958883281","state":["exists","up"]},{"osd":1,"uuid":"a4a1fa5a-7444-4090-a12d-c626ed8abdf6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6801","nonce":2346791948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6803","nonce":2346791948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6807","nonce":2346791948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2346791948},{"type":"v1","addr":"172.21.15.16:6805","nonce":2346791948}]},"public_addr":"172.21.15.16:6801/2346791948","cluster_addr":"172.21.15.16:6803/2346791948","heartbeat_back_addr":"172.21.15.16:6807/2346791948","heartbeat_front_addr":"172.21.15.16:6805/2346791948","state":["exists","up"]},{"osd":2,"uuid":"cc35c021-d74c-4b11-8e03-c54002bf99e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6801","nonce":4106675859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6803","nonce":4106675859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6807","nonce":4106675859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4106675859},{"type":"v1","addr":"172.21.15.170:6805","nonce":4106675859}]},"public_addr":"172.21.15.170:6801/4106675859","cluster_addr":"172.21.15.170:6803/4106675859","heartbeat_back_addr":"172.21.15.170:6807/4106675859","heartbeat_front_addr":"172.21.15.170:6805/4106675859","state":["exists","up"]},{"osd":3,"uuid":"d122daa0-3729-41b6-b07d-3c2db6f9d514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6811","nonce":2152501762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6813","nonce":2152501762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6817","nonce":2152501762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2152501762},{"type":"v1","addr":"172.21.15.12:6815","nonce":2152501762}]},"public_addr":"172.21.15.12:6811/2152501762","cluster_addr":"172.21.15.12:6813/2152501762","heartbeat_back_addr":"172.21.15.12:6817/2152501762","heartbeat_front_addr":"172.21.15.12:6815/2152501762","state":["exists","up"]},{"osd":4,"uuid":"d05507ed-2594-4791-93e2-0bf32b70942a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6809","nonce":587973120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6811","nonce":587973120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6815","nonce":587973120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":587973120},{"type":"v1","addr":"172.21.15.16:6813","nonce":587973120}]},"public_addr":"172.21.15.16:6809/587973120","cluster_addr":"172.21.15.16:6811/587973120","heartbeat_back_addr":"172.21.15.16:6815/587973120","heartbeat_front_addr":"172.21.15.16:6813/587973120","state":["exists","up"]},{"osd":5,"uuid":"40f06cd7-6010-43cb-9dab-71ea07e02745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6809","nonce":2002026273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6811","nonce":2002026273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6815","nonce":2002026273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2002026273},{"type":"v1","addr":"172.21.15.170:6813","nonce":2002026273}]},"public_addr":"172.21.15.170:6809/2002026273","cluster_addr":"172.21.15.170:6811/2002026273","heartbeat_back_addr":"172.21.15.170:6815/2002026273","heartbeat_front_addr":"172.21.15.170:6813/2002026273","state":["exists","up"]},{"osd":6,"uuid":"580dac7a-d03d-4830-9996-84ed2c9e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6819","nonce":3412258494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6821","nonce":3412258494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6825","nonce":3412258494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3412258494},{"type":"v1","addr":"172.21.15.12:6823","nonce":3412258494}]},"public_addr":"172.21.15.12:6819/3412258494","cluster_addr":"172.21.15.12:6821/3412258494","heartbeat_back_addr":"172.21.15.12:6825/3412258494","heartbeat_front_addr":"172.21.15.12:6823/3412258494","state":["exists","up"]},{"osd":7,"uuid":"0c7f5786-3123-417c-b8eb-9c44e5284858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6817","nonce":1916732888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6819","nonce":1916732888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6823","nonce":1916732888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1916732888},{"type":"v1","addr":"172.21.15.16:6821","nonce":1916732888}]},"public_addr":"172.21.15.16:6817/1916732888","cluster_addr":"172.21.15.16:6819/1916732888","heartbeat_back_addr":"172.21.15.16:6823/1916732888","heartbeat_front_addr":"172.21.15.16:6821/1916732888","state":["exists","up"]},{"osd":8,"uuid":"d78f2036-c1f3-4efc-a4bc-d173ed8cd846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6817","nonce":4075084847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6819","nonce":4075084847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6823","nonce":4075084847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4075084847},{"type":"v1","addr":"172.21.15.170:6821","nonce":4075084847}]},"public_addr":"172.21.15.170:6817/4075084847","cluster_addr":"172.21.15.170:6819/4075084847","heartbeat_back_addr":"172.21.15.170:6823/4075084847","heartbeat_front_addr":"172.21.15.170:6821/4075084847","state":["exists","up"]},{"osd":9,"uuid":"be6068d1-eaf5-4694-a49f-e7baa8613fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6827","nonce":1889064834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6829","nonce":1889064834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6833","nonce":1889064834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1889064834},{"type":"v1","addr":"172.21.15.12:6831","nonce":1889064834}]},"public_addr":"172.21.15.12:6827/1889064834","cluster_addr":"172.21.15.12:6829/1889064834","heartbeat_back_addr":"172.21.15.12:6833/1889064834","heartbeat_front_addr":"172.21.15.12:6831/1889064834","state":["exists","up"]},{"osd":10,"uuid":"caa08b1c-2f8b-4853-accc-f84809ba98f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6825","nonce":1985793423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6827","nonce":1985793423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6831","nonce":1985793423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1985793423},{"type":"v1","addr":"172.21.15.16:6829","nonce":1985793423}]},"public_addr":"172.21.15.16:6825/1985793423","cluster_addr":"172.21.15.16:6827/1985793423","heartbeat_back_addr":"172.21.15.16:6831/1985793423","heartbeat_front_addr":"172.21.15.16:6829/1985793423","state":["exists","up"]},{"osd":11,"uuid":"2d7bab10-d9ad-4917-9a11-a4b4bd363e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6825","nonce":3375809659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6827","nonce":3375809659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6831","nonce":3375809659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3375809659},{"type":"v1","addr":"172.21.15.170:6829","nonce":3375809659}]},"public_addr":"172.21.15.170:6825/3375809659","cluster_addr":"172.21.15.170:6827/3375809659","heartbeat_back_addr":"172.21.15.170:6831/3375809659","heartbeat_front_addr":"172.21.15.170:6829/3375809659","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:17:39.455307+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:03.501729+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:26.562020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:18:52.318169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:17.501858+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:19:42.441222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:08.439343+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:34.621993+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:20:59.084930+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:26.162286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:21:52.408657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:22:17.541009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1454579683":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6800/2168557623":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/545109421":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/1153421462":"2024-05-10T09:16:04.466876+0000","172.21.15.12:0/1731765200":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/2508497448":"2024-05-10T09:15:36.182637+0000","172.21.15.12:6800/1876427478":"2024-05-10T09:15:36.182637+0000","172.21.15.12:0/3479121957":"2024-05-10T09:16:04.466876+0000","172.21.15.12:6801/1876427478":"2024-05-10T09:15:36.182637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:30:21.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.510+0000 7f936ce38640 1 -- 172.21.15.12:0/4244002486 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f93540a2b90 msgr2=0x7f93540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:21.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.510+0000 7f936ce38640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f93540a2b90 0x7f93540a5050 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f93681028f0 tx=0x7f9358005f90 comp rx=0 tx=0).stop 2024-05-09T09:30:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.510+0000 7f936ce38640 1 -- 172.21.15.12:0/4244002486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9368102a90 msgr2=0x7f9368073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.510+0000 7f936ce38640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9368102a90 0x7f9368073c20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9350009230 tx=0x7f9350031ac0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.510+0000 7f936ce38640 1 -- 172.21.15.12:0/4244002486 shutdown_connections 2024-05-09T09:30:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.510+0000 7f936ce38640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f93540a2b90 0x7f93540a5050 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.510+0000 7f936ce38640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9368103450 0x7f9368074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.510+0000 7f936ce38640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9368102a90 0x7f9368073c20 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.510+0000 7f936ce38640 1 --2- 172.21.15.12:0/4244002486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93681018a0 0x7f93680755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.510+0000 7f936ce38640 1 -- 172.21.15.12:0/4244002486 >> 172.21.15.12:0/4244002486 conn(0x7f93680fce30 msgr2=0x7f9368105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.510+0000 7f936ce38640 1 -- 172.21.15.12:0/4244002486 shutdown_connections 2024-05-09T09:30:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.510+0000 7f936ce38640 1 -- 172.21.15.12:0/4244002486 wait complete. 2024-05-09T09:30:21.523 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T09:30:21.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.629+0000 7f5c06676640 1 -- 172.21.15.12:0/528810713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c00107610 msgr2=0x7f5c00109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:21.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.629+0000 7f5c06676640 1 --2- 172.21.15.12:0/528810713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c00107610 0x7f5c00109a00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009ca0 tx=0x7f5bf802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.629+0000 7f5c06676640 1 -- 172.21.15.12:0/528810713 shutdown_connections 2024-05-09T09:30:21.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.629+0000 7f5c06676640 1 --2- 172.21.15.12:0/528810713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c00107610 0x7f5c00109a00 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.629+0000 7f5c06676640 1 --2- 172.21.15.12:0/528810713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c00104ce0 0x7f5c001070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.629+0000 7f5c06676640 1 --2- 172.21.15.12:0/528810713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c001023b0 0x7f5c001047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.629+0000 7f5c06676640 1 -- 172.21.15.12:0/528810713 >> 172.21.15.12:0/528810713 conn(0x7f5c000fbd20 msgr2=0x7f5c000fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:21.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.629+0000 7f5c06676640 1 -- 172.21.15.12:0/528810713 shutdown_connections 2024-05-09T09:30:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.629+0000 7f5c06676640 1 -- 172.21.15.12:0/528810713 wait complete. 2024-05-09T09:30:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.629+0000 7f5c06676640 1 Processor -- start 2024-05-09T09:30:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.629+0000 7f5c06676640 1 -- start start 2024-05-09T09:30:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c06676640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c001023b0 0x7f5c00197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c06676640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c00104ce0 0x7f5c001980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c06676640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c00107610 0x7f5c0019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c06676640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0010d510 con 0x7f5c00107610 2024-05-09T09:30:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c06676640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0010d390 con 0x7f5c001023b0 2024-05-09T09:30:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c06676640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0010d690 con 0x7f5c00104ce0 2024-05-09T09:30:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c04c13640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c00107610 0x7f5c0019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c04c13640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c00107610 0x7f5c0019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58888/0 (socket says 172.21.15.12:58888) 2024-05-09T09:30:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c04c13640 1 -- 172.21.15.12:0/3479990869 learned_addr learned my addr 172.21.15.12:0/3479990869 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5bf7fff640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c00104ce0 0x7f5c001980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5bfffff640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c001023b0 0x7f5c00197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c04c13640 1 -- 172.21.15.12:0/3479990869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c00104ce0 msgr2=0x7f5c001980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c04c13640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c00104ce0 0x7f5c001980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c04c13640 1 -- 172.21.15.12:0/3479990869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c001023b0 msgr2=0x7f5c00197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c04c13640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c001023b0 0x7f5c00197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c04c13640 1 -- 172.21.15.12:0/3479990869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0019f7a0 con 0x7f5c00107610 2024-05-09T09:30:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c04c13640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c00107610 0x7f5c0019f130 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8004ad0 tx=0x7f5bf802f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5bfdffb640 1 -- 172.21.15.12:0/3479990869 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8003cf0 con 0x7f5c00107610 2024-05-09T09:30:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.630+0000 7f5c06676640 1 -- 172.21.15.12:0/3479990869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0019fa30 con 0x7f5c00107610 2024-05-09T09:30:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.631+0000 7f5bfdffb640 1 -- 172.21.15.12:0/3479990869 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5bf8004510 con 0x7f5c00107610 2024-05-09T09:30:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.631+0000 7f5bfdffb640 1 -- 172.21.15.12:0/3479990869 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf802fd10 con 0x7f5c00107610 2024-05-09T09:30:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.631+0000 7f5c06676640 1 -- 172.21.15.12:0/3479990869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c0019ff80 con 0x7f5c00107610 2024-05-09T09:30:21.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.632+0000 7f5c06676640 1 -- 172.21.15.12:0/3479990869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c000fd810 con 0x7f5c00107610 2024-05-09T09:30:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.633+0000 7f5bfdffb640 1 -- 172.21.15.12:0/3479990869 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5bf8004040 con 0x7f5c00107610 2024-05-09T09:30:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.633+0000 7f5bfdffb640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5bec0a2b90 0x7f5bec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.634+0000 7f5bfffff640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5bec0a2b90 0x7f5bec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.634+0000 7f5bfdffb640 1 -- 172.21.15.12:0/3479990869 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f5bf80f0c50 con 0x7f5c00107610 2024-05-09T09:30:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.634+0000 7f5bfffff640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5bec0a2b90 0x7f5bec0a5050 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0005d00 tx=0x7f5bf0005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:21.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.638+0000 7f5bfdffb640 1 -- 172.21.15.12:0/3479990869 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5bf8004330 con 0x7f5c00107610 2024-05-09T09:30:21.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:21 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3231681062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:30:21.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:21 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/4203575813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:30:21.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:21 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3231681062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:30:21.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:21 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/4203575813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:30:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.862+0000 7f5c06676640 1 -- 172.21.15.12:0/3479990869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5c00062430 con 0x7f5c00107610 2024-05-09T09:30:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.864+0000 7f5bfdffb640 1 -- 172.21.15.12:0/3479990869 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7f5bf80bd780 con 0x7f5c00107610 2024-05-09T09:30:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T09:30:21.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.868+0000 7f5c06676640 1 -- 172.21.15.12:0/3479990869 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5bec0a2b90 msgr2=0x7f5bec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:21.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.868+0000 7f5c06676640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5bec0a2b90 0x7f5bec0a5050 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0005d00 tx=0x7f5bf0005c50 comp rx=0 tx=0).stop 2024-05-09T09:30:21.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.868+0000 7f5c06676640 1 -- 172.21.15.12:0/3479990869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c00107610 msgr2=0x7f5c0019f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:21.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.868+0000 7f5c06676640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c00107610 0x7f5c0019f130 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8004ad0 tx=0x7f5bf802f450 comp rx=0 tx=0).stop 2024-05-09T09:30:21.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.868+0000 7f5c06676640 1 -- 172.21.15.12:0/3479990869 shutdown_connections 2024-05-09T09:30:21.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.868+0000 7f5c06676640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f5bec0a2b90 0x7f5bec0a5050 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.868+0000 7f5c06676640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c00107610 0x7f5c0019f130 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.868+0000 7f5c06676640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c00104ce0 0x7f5c001980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.868+0000 7f5c06676640 1 --2- 172.21.15.12:0/3479990869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c001023b0 0x7f5c00197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:21.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.868+0000 7f5c06676640 1 -- 172.21.15.12:0/3479990869 >> 172.21.15.12:0/3479990869 conn(0x7f5c000fbd20 msgr2=0x7f5c00102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:21.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.869+0000 7f5c06676640 1 -- 172.21.15.12:0/3479990869 shutdown_connections 2024-05-09T09:30:21.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:21.869+0000 7f5c06676640 1 -- 172.21.15.12:0/3479990869 wait complete. 2024-05-09T09:30:21.887 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T09:30:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.000+0000 7f34afc8c640 1 -- 172.21.15.12:0/3294643669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a81075f0 msgr2=0x7f34a81099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.000+0000 7f34afc8c640 1 --2- 172.21.15.12:0/3294643669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a81075f0 0x7f34a81099e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009ca0 tx=0x7f34a002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:30:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.000+0000 7f34afc8c640 1 -- 172.21.15.12:0/3294643669 shutdown_connections 2024-05-09T09:30:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.000+0000 7f34afc8c640 1 --2- 172.21.15.12:0/3294643669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a81075f0 0x7f34a81099e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.000+0000 7f34afc8c640 1 --2- 172.21.15.12:0/3294643669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a8104cc0 0x7f34a81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.000+0000 7f34afc8c640 1 --2- 172.21.15.12:0/3294643669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34a8102390 0x7f34a8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.000+0000 7f34afc8c640 1 -- 172.21.15.12:0/3294643669 >> 172.21.15.12:0/3294643669 conn(0x7f34a80fbce0 msgr2=0x7f34a80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.000+0000 7f34afc8c640 1 -- 172.21.15.12:0/3294643669 shutdown_connections 2024-05-09T09:30:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.000+0000 7f34afc8c640 1 -- 172.21.15.12:0/3294643669 wait complete. 2024-05-09T09:30:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34afc8c640 1 Processor -- start 2024-05-09T09:30:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34afc8c640 1 -- start start 2024-05-09T09:30:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34afc8c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a8102390 0x7f34a8101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34afc8c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34a8104cc0 0x7f34a8100340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34afc8c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a81075f0 0x7f34a8100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34afc8c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f34a810d4f0 con 0x7f34a8102390 2024-05-09T09:30:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34afc8c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34a810d370 con 0x7f34a81075f0 2024-05-09T09:30:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34afc8c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f34a810d670 con 0x7f34a8104cc0 2024-05-09T09:30:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34ada28640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a8102390 0x7f34a8101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34ada28640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a8102390 0x7f34a8101d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49318/0 (socket says 172.21.15.12:49318) 2024-05-09T09:30:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34ada28640 1 -- 172.21.15.12:0/2169280859 learned_addr learned my addr 172.21.15.12:0/2169280859 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34ae229640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a81075f0 0x7f34a8100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34ad227640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34a8104cc0 0x7f34a8100340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34ada28640 1 -- 172.21.15.12:0/2169280859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34a8104cc0 msgr2=0x7f34a8100340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34ada28640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34a8104cc0 0x7f34a8100340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34ada28640 1 -- 172.21.15.12:0/2169280859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a81075f0 msgr2=0x7f34a8100880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34ada28640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a81075f0 0x7f34a8100880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34ada28640 1 -- 172.21.15.12:0/2169280859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a8101110 con 0x7f34a8102390 2024-05-09T09:30:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34ae229640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a81075f0 0x7f34a8100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:30:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.001+0000 7f34ada28640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a8102390 0x7f34a8101d00 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f34a400d770 tx=0x7f34a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.002+0000 7f349effd640 1 -- 172.21.15.12:0/2169280859 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4017070 con 0x7f34a8102390 2024-05-09T09:30:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.002+0000 7f349effd640 1 -- 172.21.15.12:0/2169280859 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f34a40054f0 con 0x7f34a8102390 2024-05-09T09:30:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.002+0000 7f34afc8c640 1 -- 172.21.15.12:0/2169280859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a81a1c90 con 0x7f34a8102390 2024-05-09T09:30:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.002+0000 7f349effd640 1 -- 172.21.15.12:0/2169280859 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4004d60 con 0x7f34a8102390 2024-05-09T09:30:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.002+0000 7f34afc8c640 1 -- 172.21.15.12:0/2169280859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a81a2190 con 0x7f34a8102390 2024-05-09T09:30:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.003+0000 7f34afc8c640 1 -- 172.21.15.12:0/2169280859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a80fd7d0 con 0x7f34a8102390 2024-05-09T09:30:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.004+0000 7f349effd640 1 -- 172.21.15.12:0/2169280859 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f34a4004f00 con 0x7f34a8102390 2024-05-09T09:30:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.004+0000 7f349effd640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f34940a2b90 0x7f34940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.005+0000 7f34ad227640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f34940a2b90 0x7f34940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.005+0000 7f349effd640 1 -- 172.21.15.12:0/2169280859 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9638+0+0 (secure 0 0 0) 0x7f34a40cf110 con 0x7f34a8102390 2024-05-09T09:30:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.005+0000 7f34ad227640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f34940a2b90 0x7f34940a5050 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3498005fd0 tx=0x7f3498009470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:22.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.009+0000 7f349effd640 1 -- 172.21.15.12:0/2169280859 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f34a409baf0 con 0x7f34a8102390 2024-05-09T09:30:22.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a[30136]: 2024-05-09T09:30:21.862+0000 7f1109020700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T09:30:22.234 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:30:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-g[72174]: ignoring --setuser ceph since I am not root 2024-05-09T09:30:22.235 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:30:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-g[72174]: ignoring --setgroup ceph since I am not root 2024-05-09T09:30:22.235 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:30:21 smithi012 ceph-mds[72178]: -- 172.21.15.12:0/3253970293 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564827a87a40 con 0x564827a5a800 2024-05-09T09:30:22.235 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:30:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-g[72174]: starting mds.g at 2024-05-09T09:30:22.235 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:30:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-a[72523]: ignoring --setuser ceph since I am not root 2024-05-09T09:30:22.235 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:30:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-a[72523]: ignoring --setgroup ceph since I am not root 2024-05-09T09:30:22.235 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:30:21 smithi012 ceph-mds[72527]: -- 172.21.15.12:0/2448820931 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b6f27fa40 con 0x560b6f252800 2024-05-09T09:30:22.235 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:30:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-a[72523]: starting mds.a at 2024-05-09T09:30:22.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.236+0000 7f34afc8c640 1 -- 172.21.15.12:0/2169280859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f34a8062460 con 0x7f34a8102390 2024-05-09T09:30:22.243 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:30:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-l[63938]: ignoring --setuser ceph since I am not root 2024-05-09T09:30:22.243 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:30:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-l[63938]: ignoring --setgroup ceph since I am not root 2024-05-09T09:30:22.243 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:30:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-l[63938]: starting mds.l at 2024-05-09T09:30:22.244 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:30:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i[64878]: ignoring --setuser ceph since I am not root 2024-05-09T09:30:22.244 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:30:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i[64878]: ignoring --setgroup ceph since I am not root 2024-05-09T09:30:22.244 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:30:21 smithi170 ceph-mds[64882]: -- 172.21.15.170:0/2560328604 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55de6e36ba40 con 0x55de6e33e800 2024-05-09T09:30:22.244 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:30:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i[64878]: starting mds.i at 2024-05-09T09:30:22.244 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:30:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-c[64568]: ignoring --setuser ceph since I am not root 2024-05-09T09:30:22.244 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:30:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-c[64568]: ignoring --setgroup ceph since I am not root 2024-05-09T09:30:22.244 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:30:21 smithi170 ceph-mds[64572]: -- 172.21.15.170:0/2444020595 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557001fada40 con 0x557001f80800 2024-05-09T09:30:22.244 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:30:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-c[64568]: starting mds.c at 2024-05-09T09:30:22.245 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:30:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-f[64252]: ignoring --setuser ceph since I am not root 2024-05-09T09:30:22.245 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:30:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-f[64252]: ignoring --setgroup ceph since I am not root 2024-05-09T09:30:22.245 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:30:21 smithi170 ceph-mds[64261]: -- 172.21.15.170:0/1022318939 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559198cdda40 con 0x559198cb0c00 2024-05-09T09:30:22.245 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:30:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-f[64252]: starting mds.f at 2024-05-09T09:30:22.253 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:30:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h[63713]: ignoring --setuser ceph since I am not root 2024-05-09T09:30:22.253 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:30:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h[63713]: ignoring --setgroup ceph since I am not root 2024-05-09T09:30:22.253 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:30:21 smithi016 ceph-mds[63717]: -- 172.21.15.16:0/2176688119 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564415947a40 con 0x56441591ac00 2024-05-09T09:30:22.253 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:30:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h[63713]: starting mds.h at 2024-05-09T09:30:22.253 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:30:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-b[64018]: ignoring --setuser ceph since I am not root 2024-05-09T09:30:22.254 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:30:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-b[64018]: ignoring --setgroup ceph since I am not root 2024-05-09T09:30:22.254 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:30:21 smithi016 ceph-mds[64022]: -- 172.21.15.16:0/939786957 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55808ef8ba40 con 0x55808ef5e800 2024-05-09T09:30:22.254 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:30:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-b[64018]: starting mds.b at 2024-05-09T09:30:22.254 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:30:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-e[63398]: ignoring --setuser ceph since I am not root 2024-05-09T09:30:22.254 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:30:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-e[63398]: ignoring --setgroup ceph since I am not root 2024-05-09T09:30:22.254 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:30:21 smithi016 ceph-mds[63403]: -- 172.21.15.16:0/1493562940 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e75f8f5a40 con 0x55e75f8c8800 2024-05-09T09:30:22.254 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:30:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-e[63398]: starting mds.e at 2024-05-09T09:30:22.254 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:30:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-k[63080]: ignoring --setuser ceph since I am not root 2024-05-09T09:30:22.255 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:30:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-k[63080]: ignoring --setgroup ceph since I am not root 2024-05-09T09:30:22.255 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:30:21 smithi016 ceph-mds[63084]: -- 172.21.15.16:0/4041372025 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557ccb81da40 con 0x557ccb7f0800 2024-05-09T09:30:22.255 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:30:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-k[63080]: starting mds.k at 2024-05-09T09:30:22.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:22 smithi012 ceph-mon[30140]: pgmap v512: 129 pgs: 129 active+clean; 56 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-09T09:30:22.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:22 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4244002486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:30:22.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:22 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3479990869' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T09:30:22.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:22 smithi012 ceph-mon[30140]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T09:30:22.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:22 smithi012 ceph-mon[30140]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T09:30:22.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:22 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/3479990869' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T09:30:22.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:22 smithi012 ceph-mon[30140]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T09:30:22.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:22 smithi012 ceph-mon[30140]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T09:30:22.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:22 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2169280859' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:30:22.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:22 smithi170 ceph-mon[33632]: pgmap v512: 129 pgs: 129 active+clean; 56 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-09T09:30:22.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:22 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/4244002486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:30:22.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:22 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3479990869' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T09:30:22.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:22 smithi170 ceph-mon[33632]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T09:30:22.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:22 smithi170 ceph-mon[33632]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T09:30:22.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:22 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/3479990869' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T09:30:22.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:22 smithi170 ceph-mon[33632]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T09:30:22.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:22 smithi170 ceph-mon[33632]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T09:30:22.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:22 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2169280859' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:30:22.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:22 smithi016 ceph-mon[33335]: pgmap v512: 129 pgs: 129 active+clean; 56 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-09T09:30:22.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:22 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/4244002486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:30:22.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:22 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3479990869' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T09:30:22.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:22 smithi016 ceph-mon[33335]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T09:30:22.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:22 smithi016 ceph-mon[33335]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T09:30:22.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:22 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/3479990869' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T09:30:22.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:22 smithi016 ceph-mon[33335]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T09:30:22.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:22 smithi016 ceph-mon[33335]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T09:30:22.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:22 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2169280859' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:30:22.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.868+0000 7f349effd640 1 -- 172.21.15.12:0/2169280859 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) ==== 104+0+0 (secure 0 0 0) 0x7f34a409b3f0 con 0x7f34a8102390 2024-05-09T09:30:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.871+0000 7f34afc8c640 1 -- 172.21.15.12:0/2169280859 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f34940a2b90 msgr2=0x7f34940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.871+0000 7f34afc8c640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f34940a2b90 0x7f34940a5050 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3498005fd0 tx=0x7f3498009470 comp rx=0 tx=0).stop 2024-05-09T09:30:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.871+0000 7f34afc8c640 1 -- 172.21.15.12:0/2169280859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a8102390 msgr2=0x7f34a8101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.871+0000 7f34afc8c640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a8102390 0x7f34a8101d00 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f34a400d770 tx=0x7f34a400dc30 comp rx=0 tx=0).stop 2024-05-09T09:30:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.871+0000 7f34afc8c640 1 -- 172.21.15.12:0/2169280859 shutdown_connections 2024-05-09T09:30:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.871+0000 7f34afc8c640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f34940a2b90 0x7f34940a5050 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.871+0000 7f34afc8c640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a81075f0 0x7f34a8100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.871+0000 7f34afc8c640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34a8104cc0 0x7f34a8100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.871+0000 7f34afc8c640 1 --2- 172.21.15.12:0/2169280859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a8102390 0x7f34a8101d00 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.871+0000 7f34afc8c640 1 -- 172.21.15.12:0/2169280859 >> 172.21.15.12:0/2169280859 conn(0x7f34a80fbce0 msgr2=0x7f34a8102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:22.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.871+0000 7f34afc8c640 1 -- 172.21.15.12:0/2169280859 shutdown_connections 2024-05-09T09:30:22.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:22.871+0000 7f34afc8c640 1 -- 172.21.15.12:0/2169280859 wait complete. 2024-05-09T09:30:22.890 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T09:30:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.007+0000 7fbb524f5640 1 -- 172.21.15.12:0/163495726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb4c1075d0 msgr2=0x7fbb4c1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.007+0000 7fbb524f5640 1 --2- 172.21.15.12:0/163495726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb4c1075d0 0x7fbb4c1099c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009ca0 tx=0x7fbb4402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:30:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.007+0000 7fbb524f5640 1 -- 172.21.15.12:0/163495726 shutdown_connections 2024-05-09T09:30:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.007+0000 7fbb524f5640 1 --2- 172.21.15.12:0/163495726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb4c1075d0 0x7fbb4c1099c0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.007+0000 7fbb524f5640 1 --2- 172.21.15.12:0/163495726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb4c104ca0 0x7fbb4c107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.007+0000 7fbb524f5640 1 --2- 172.21.15.12:0/163495726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c102370 0x7fbb4c104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.007+0000 7fbb524f5640 1 -- 172.21.15.12:0/163495726 >> 172.21.15.12:0/163495726 conn(0x7fbb4c0fbce0 msgr2=0x7fbb4c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.007+0000 7fbb524f5640 1 -- 172.21.15.12:0/163495726 shutdown_connections 2024-05-09T09:30:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.007+0000 7fbb524f5640 1 -- 172.21.15.12:0/163495726 wait complete. 2024-05-09T09:30:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.008+0000 7fbb524f5640 1 Processor -- start 2024-05-09T09:30:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.008+0000 7fbb524f5640 1 -- start start 2024-05-09T09:30:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.008+0000 7fbb524f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb4c102370 0x7fbb4c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.008+0000 7fbb524f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb4c104ca0 0x7fbb4c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.008+0000 7fbb524f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c1075d0 0x7fbb4c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.008+0000 7fbb524f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c10d4f0 con 0x7fbb4c102370 2024-05-09T09:30:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.008+0000 7fbb524f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c10d370 con 0x7fbb4c1075d0 2024-05-09T09:30:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.008+0000 7fbb524f5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c10d670 con 0x7fbb4c104ca0 2024-05-09T09:30:23.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.008+0000 7fbb4bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb4c102370 0x7fbb4c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:23.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.008+0000 7fbb4bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb4c102370 0x7fbb4c197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49336/0 (socket says 172.21.15.12:49336) 2024-05-09T09:30:23.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.008+0000 7fbb4bfff640 1 -- 172.21.15.12:0/4174094599 learned_addr learned my addr 172.21.15.12:0/4174094599 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:23.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.008+0000 7fbb50a92640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c1075d0 0x7fbb4c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:23.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.008+0000 7fbb4b7fe640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb4c104ca0 0x7fbb4c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:23.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.009+0000 7fbb4bfff640 1 -- 172.21.15.12:0/4174094599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb4c104ca0 msgr2=0x7fbb4c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:23.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.009+0000 7fbb4bfff640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb4c104ca0 0x7fbb4c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:23.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.009+0000 7fbb4bfff640 1 -- 172.21.15.12:0/4174094599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c1075d0 msgr2=0x7fbb4c19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:23.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.009+0000 7fbb4bfff640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c1075d0 0x7fbb4c19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.009+0000 7fbb4bfff640 1 -- 172.21.15.12:0/4174094599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4c19f790 con 0x7fbb4c102370 2024-05-09T09:30:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.009+0000 7fbb4bfff640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb4c102370 0x7fbb4c197b70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00d950 tx=0x7fbb3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.009+0000 7fbb497fa640 1 -- 172.21.15.12:0/4174094599 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c017030 con 0x7fbb4c102370 2024-05-09T09:30:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.009+0000 7fbb524f5640 1 -- 172.21.15.12:0/4174094599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4c19fa80 con 0x7fbb4c102370 2024-05-09T09:30:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.009+0000 7fbb497fa640 1 -- 172.21.15.12:0/4174094599 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbb3c004020 con 0x7fbb4c102370 2024-05-09T09:30:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.009+0000 7fbb497fa640 1 -- 172.21.15.12:0/4174094599 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c0047a0 con 0x7fbb4c102370 2024-05-09T09:30:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.009+0000 7fbb524f5640 1 -- 172.21.15.12:0/4174094599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb4c1a0020 con 0x7fbb4c102370 2024-05-09T09:30:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.010+0000 7fbb524f5640 1 -- 172.21.15.12:0/4174094599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb4c0fd7d0 con 0x7fbb4c102370 2024-05-09T09:30:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.011+0000 7fbb497fa640 1 -- 172.21.15.12:0/4174094599 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbb3c0041d0 con 0x7fbb4c102370 2024-05-09T09:30:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.012+0000 7fbb497fa640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbb340a2b90 0x7fbb340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.013+0000 7fbb4b7fe640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbb340a2b90 0x7fbb340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.013+0000 7fbb497fa640 1 -- 172.21.15.12:0/4174094599 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9638+0+0 (secure 0 0 0) 0x7fbb3c0cf650 con 0x7fbb4c102370 2024-05-09T09:30:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.013+0000 7fbb4b7fe640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbb340a2b90 0x7fbb340a5050 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c199080 tx=0x7fbb4000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:23.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.016+0000 7fbb497fa640 1 -- 172.21.15.12:0/4174094599 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbb3c09c030 con 0x7fbb4c102370 2024-05-09T09:30:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.205+0000 7fbb524f5640 1 -- 172.21.15.12:0/4174094599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbb4c02d1c0 con 0x7fbb4c102370 2024-05-09T09:30:23.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T09:30:23.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T09:30:23.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: Cluster is now healthy 2024-05-09T09:30:23.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2169280859' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T09:30:23.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.170:6832/3344700576,v1:172.21.15.170:6833/3344700576] up:boot 2024-05-09T09:30:23.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.12:6838/2169265604,v1:172.21.15.12:6839/2169265604] up:boot 2024-05-09T09:30:23.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.16:6832/1559156991,v1:172.21.15.16:6836/1559156991] up:boot 2024-05-09T09:30:23.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.16:6834/4139197842,v1:172.21.15.16:6839/4139197842] up:boot 2024-05-09T09:30:23.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.12:6840/3410349650,v1:172.21.15.12:6841/3410349650] up:boot 2024-05-09T09:30:23.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.16:6833/2117527193,v1:172.21.15.16:6837/2117527193] up:boot 2024-05-09T09:30:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.16:6835/4217411329,v1:172.21.15.16:6838/4217411329] up:boot 2024-05-09T09:30:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.170:6835/929935203,v1:172.21.15.170:6837/929935203] up:boot 2024-05-09T09:30:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.170:6834/894426536,v1:172.21.15.170:6836/894426536] up:boot 2024-05-09T09:30:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: mds.? [v2:172.21.15.170:6838/3870177811,v1:172.21.15.170:6839/3870177811] up:boot 2024-05-09T09:30:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: fsmap 12 up:standby 2024-05-09T09:30:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T09:30:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T09:30:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T09:30:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T09:30:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T09:30:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T09:30:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T09:30:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T09:30:23.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:30:23.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T09:30:23.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:23 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4174094599' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:30:23.553 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T09:30:23.553 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T09:30:23.553 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: Cluster is now healthy 2024-05-09T09:30:23.553 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2169280859' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T09:30:23.553 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.170:6832/3344700576,v1:172.21.15.170:6833/3344700576] up:boot 2024-05-09T09:30:23.553 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.12:6838/2169265604,v1:172.21.15.12:6839/2169265604] up:boot 2024-05-09T09:30:23.553 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.16:6832/1559156991,v1:172.21.15.16:6836/1559156991] up:boot 2024-05-09T09:30:23.553 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.16:6834/4139197842,v1:172.21.15.16:6839/4139197842] up:boot 2024-05-09T09:30:23.553 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.12:6840/3410349650,v1:172.21.15.12:6841/3410349650] up:boot 2024-05-09T09:30:23.553 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.16:6833/2117527193,v1:172.21.15.16:6837/2117527193] up:boot 2024-05-09T09:30:23.554 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.16:6835/4217411329,v1:172.21.15.16:6838/4217411329] up:boot 2024-05-09T09:30:23.554 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.170:6835/929935203,v1:172.21.15.170:6837/929935203] up:boot 2024-05-09T09:30:23.554 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.170:6834/894426536,v1:172.21.15.170:6836/894426536] up:boot 2024-05-09T09:30:23.554 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: mds.? [v2:172.21.15.170:6838/3870177811,v1:172.21.15.170:6839/3870177811] up:boot 2024-05-09T09:30:23.554 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: fsmap 12 up:standby 2024-05-09T09:30:23.554 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T09:30:23.554 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T09:30:23.554 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T09:30:23.554 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T09:30:23.554 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T09:30:23.554 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T09:30:23.554 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T09:30:23.554 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T09:30:23.555 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:30:23.555 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T09:30:23.555 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:23 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/4174094599' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:30:23.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T09:30:23.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T09:30:23.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: Cluster is now healthy 2024-05-09T09:30:23.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2169280859' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T09:30:23.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.170:6832/3344700576,v1:172.21.15.170:6833/3344700576] up:boot 2024-05-09T09:30:23.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.12:6838/2169265604,v1:172.21.15.12:6839/2169265604] up:boot 2024-05-09T09:30:23.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.16:6832/1559156991,v1:172.21.15.16:6836/1559156991] up:boot 2024-05-09T09:30:23.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.16:6834/4139197842,v1:172.21.15.16:6839/4139197842] up:boot 2024-05-09T09:30:23.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.12:6840/3410349650,v1:172.21.15.12:6841/3410349650] up:boot 2024-05-09T09:30:23.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.16:6833/2117527193,v1:172.21.15.16:6837/2117527193] up:boot 2024-05-09T09:30:23.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.16:6835/4217411329,v1:172.21.15.16:6838/4217411329] up:boot 2024-05-09T09:30:23.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.170:6835/929935203,v1:172.21.15.170:6837/929935203] up:boot 2024-05-09T09:30:23.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.170:6834/894426536,v1:172.21.15.170:6836/894426536] up:boot 2024-05-09T09:30:23.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: mds.? [v2:172.21.15.170:6838/3870177811,v1:172.21.15.170:6839/3870177811] up:boot 2024-05-09T09:30:23.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: fsmap 12 up:standby 2024-05-09T09:30:23.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T09:30:23.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T09:30:23.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T09:30:23.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T09:30:23.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T09:30:23.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T09:30:23.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T09:30:23.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T09:30:23.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:30:23.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: from='mgr.14150 172.21.15.12:0/4126546770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T09:30:23.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:23 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/4174094599' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:30:23.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.871+0000 7fbb497fa640 1 -- 172.21.15.12:0/4174094599 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7fbb3c09b930 con 0x7fbb4c102370 2024-05-09T09:30:23.872 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' removed 2024-05-09T09:30:23.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.873+0000 7fbb524f5640 1 -- 172.21.15.12:0/4174094599 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbb340a2b90 msgr2=0x7fbb340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:23.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.873+0000 7fbb524f5640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbb340a2b90 0x7fbb340a5050 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c199080 tx=0x7fbb4000b390 comp rx=0 tx=0).stop 2024-05-09T09:30:23.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.873+0000 7fbb524f5640 1 -- 172.21.15.12:0/4174094599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb4c102370 msgr2=0x7fbb4c197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:23.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.873+0000 7fbb524f5640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb4c102370 0x7fbb4c197b70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00d950 tx=0x7fbb3c00de10 comp rx=0 tx=0).stop 2024-05-09T09:30:23.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.873+0000 7fbb524f5640 1 -- 172.21.15.12:0/4174094599 shutdown_connections 2024-05-09T09:30:23.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.873+0000 7fbb524f5640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fbb340a2b90 0x7fbb340a5050 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:23.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.873+0000 7fbb524f5640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c1075d0 0x7fbb4c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:23.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.873+0000 7fbb524f5640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb4c104ca0 0x7fbb4c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:23.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.873+0000 7fbb524f5640 1 --2- 172.21.15.12:0/4174094599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb4c102370 0x7fbb4c197b70 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:23.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.873+0000 7fbb524f5640 1 -- 172.21.15.12:0/4174094599 >> 172.21.15.12:0/4174094599 conn(0x7fbb4c0fbce0 msgr2=0x7fbb4c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:23.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.873+0000 7fbb524f5640 1 -- 172.21.15.12:0/4174094599 shutdown_connections 2024-05-09T09:30:23.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.873+0000 7fbb524f5640 1 -- 172.21.15.12:0/4174094599 wait complete. 2024-05-09T09:30:23.890 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T09:30:24.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.999+0000 7fa2badb0640 1 -- 172.21.15.12:0/1977441067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2b4100770 msgr2=0x7fa2b4100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:24.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:23.999+0000 7fa2badb0640 1 --2- 172.21.15.12:0/1977441067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2b4100770 0x7fa2b4100b90 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009900 tx=0x7fa29c02a410 comp rx=0 tx=0).stop 2024-05-09T09:30:24.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 -- 172.21.15.12:0/1977441067 shutdown_connections 2024-05-09T09:30:24.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 --2- 172.21.15.12:0/1977441067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2b4102320 0x7fa2b4108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 --2- 172.21.15.12:0/1977441067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b4101960 0x7fa2b4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 --2- 172.21.15.12:0/1977441067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2b4100770 0x7fa2b4100b90 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 -- 172.21.15.12:0/1977441067 >> 172.21.15.12:0/1977441067 conn(0x7fa2b40fbce0 msgr2=0x7fa2b40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 -- 172.21.15.12:0/1977441067 shutdown_connections 2024-05-09T09:30:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 -- 172.21.15.12:0/1977441067 wait complete. 2024-05-09T09:30:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 Processor -- start 2024-05-09T09:30:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 -- start start 2024-05-09T09:30:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2b4100770 0x7fa2b4197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b4101960 0x7fa2b4198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2b4102320 0x7fa2b419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b410d520 con 0x7fa2b4102320 2024-05-09T09:30:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b410d3a0 con 0x7fa2b4101960 2024-05-09T09:30:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.000+0000 7fa2badb0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b410d6a0 con 0x7fa2b4100770 2024-05-09T09:30:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b934d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2b4102320 0x7fa2b419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b934d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2b4102320 0x7fa2b419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49352/0 (socket says 172.21.15.12:49352) 2024-05-09T09:30:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b934d640 1 -- 172.21.15.12:0/2615947457 learned_addr learned my addr 172.21.15.12:0/2615947457 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b3fff640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b4101960 0x7fa2b4198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b8b4c640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2b4100770 0x7fa2b4197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b3fff640 1 -- 172.21.15.12:0/2615947457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2b4100770 msgr2=0x7fa2b4197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b3fff640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2b4100770 0x7fa2b4197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b3fff640 1 -- 172.21.15.12:0/2615947457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2b4102320 msgr2=0x7fa2b419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b3fff640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2b4102320 0x7fa2b419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b3fff640 1 -- 172.21.15.12:0/2615947457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b419f810 con 0x7fa2b4101960 2024-05-09T09:30:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b934d640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2b4102320 0x7fa2b419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:30:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b3fff640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b4101960 0x7fa2b4198130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400eb00 tx=0x7fa2a400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b1ffb640 1 -- 172.21.15.12:0/2615947457 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4010430 con 0x7fa2b4101960 2024-05-09T09:30:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2badb0640 1 -- 172.21.15.12:0/2615947457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b419fb00 con 0x7fa2b4101960 2024-05-09T09:30:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b1ffb640 1 -- 172.21.15.12:0/2615947457 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa2a4010aa0 con 0x7fa2b4101960 2024-05-09T09:30:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2b1ffb640 1 -- 172.21.15.12:0/2615947457 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4017b30 con 0x7fa2b4101960 2024-05-09T09:30:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.001+0000 7fa2badb0640 1 -- 172.21.15.12:0/2615947457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b41a0050 con 0x7fa2b4101960 2024-05-09T09:30:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.003+0000 7fa2badb0640 1 -- 172.21.15.12:0/2615947457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b4107ae0 con 0x7fa2b4101960 2024-05-09T09:30:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.003+0000 7fa2b1ffb640 1 -- 172.21.15.12:0/2615947457 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa2a4017d10 con 0x7fa2b4101960 2024-05-09T09:30:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.004+0000 7fa2b1ffb640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa2a00a2b90 0x7fa2a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.004+0000 7fa2b8b4c640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa2a00a2b90 0x7fa2a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.004+0000 7fa2b1ffb640 1 -- 172.21.15.12:0/2615947457 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9142+0+0 (secure 0 0 0) 0x7fa2a40cfe50 con 0x7fa2b4101960 2024-05-09T09:30:24.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.005+0000 7fa2b8b4c640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa2a00a2b90 0x7fa2a00a5050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa29c004770 tx=0x7fa29c031530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:24.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.009+0000 7fa2b1ffb640 1 -- 172.21.15.12:0/2615947457 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa2a409ca20 con 0x7fa2b4101960 2024-05-09T09:30:24.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.196+0000 7fa2badb0640 1 -- 172.21.15.12:0/2615947457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa2b4062430 con 0x7fa2b4101960 2024-05-09T09:30:24.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:24 smithi012 ceph-mon[30140]: pgmap v514: 129 pgs: 129 active+clean; 56 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-09T09:30:24.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:24 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/4174094599' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T09:30:24.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:24 smithi012 ceph-mon[30140]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T09:30:24.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:24 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2615947457' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:30:24.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:24 smithi012 ceph-mon[30140]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:30:24.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:24 smithi170 ceph-mon[33632]: pgmap v514: 129 pgs: 129 active+clean; 56 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-09T09:30:24.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:24 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/4174094599' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T09:30:24.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:24 smithi170 ceph-mon[33632]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T09:30:24.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:24 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2615947457' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:30:24.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:24 smithi170 ceph-mon[33632]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:30:24.754 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:24 smithi016 ceph-mon[33335]: pgmap v514: 129 pgs: 129 active+clean; 56 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-09T09:30:24.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:24 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/4174094599' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T09:30:24.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:24 smithi016 ceph-mon[33335]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T09:30:24.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:24 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2615947457' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:30:24.756 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:24 smithi016 ceph-mon[33335]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:30:24.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.876+0000 7fa2b1ffb640 1 -- 172.21.15.12:0/2615947457 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7fa2a409c320 con 0x7fa2b4101960 2024-05-09T09:30:24.878 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' removed 2024-05-09T09:30:24.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.880+0000 7fa2badb0640 1 -- 172.21.15.12:0/2615947457 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa2a00a2b90 msgr2=0x7fa2a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:24.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.880+0000 7fa2badb0640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa2a00a2b90 0x7fa2a00a5050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa29c004770 tx=0x7fa29c031530 comp rx=0 tx=0).stop 2024-05-09T09:30:24.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.880+0000 7fa2badb0640 1 -- 172.21.15.12:0/2615947457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b4101960 msgr2=0x7fa2b4198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:24.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.880+0000 7fa2badb0640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b4101960 0x7fa2b4198130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400eb00 tx=0x7fa2a400c2d0 comp rx=0 tx=0).stop 2024-05-09T09:30:24.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.881+0000 7fa2badb0640 1 -- 172.21.15.12:0/2615947457 shutdown_connections 2024-05-09T09:30:24.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.881+0000 7fa2badb0640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7fa2a00a2b90 0x7fa2a00a5050 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:24.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.881+0000 7fa2badb0640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2b4102320 0x7fa2b419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:24.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.881+0000 7fa2badb0640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b4101960 0x7fa2b4198130 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:24.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.881+0000 7fa2badb0640 1 --2- 172.21.15.12:0/2615947457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2b4100770 0x7fa2b4197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:24.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.881+0000 7fa2badb0640 1 -- 172.21.15.12:0/2615947457 >> 172.21.15.12:0/2615947457 conn(0x7fa2b40fbce0 msgr2=0x7fa2b4104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:24.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.881+0000 7fa2badb0640 1 -- 172.21.15.12:0/2615947457 shutdown_connections 2024-05-09T09:30:24.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:24.881+0000 7fa2badb0640 1 -- 172.21.15.12:0/2615947457 wait complete. 2024-05-09T09:30:24.898 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-09T09:30:25.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.014+0000 7f73fd7c8640 1 -- 172.21.15.12:0/4026742682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73f8102320 msgr2=0x7f73f8108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:25.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.014+0000 7f73fd7c8640 1 --2- 172.21.15.12:0/4026742682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73f8102320 0x7f73f8108bb0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009ca0 tx=0x7f73f002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:30:25.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.014+0000 7f73fd7c8640 1 -- 172.21.15.12:0/4026742682 shutdown_connections 2024-05-09T09:30:25.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.014+0000 7f73fd7c8640 1 --2- 172.21.15.12:0/4026742682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73f8102320 0x7f73f8108bb0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:25.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.014+0000 7f73fd7c8640 1 --2- 172.21.15.12:0/4026742682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73f8101960 0x7f73f8101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:25.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.014+0000 7f73fd7c8640 1 --2- 172.21.15.12:0/4026742682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f8100770 0x7f73f8100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:25.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.014+0000 7f73fd7c8640 1 -- 172.21.15.12:0/4026742682 >> 172.21.15.12:0/4026742682 conn(0x7f73f80fbce0 msgr2=0x7f73f80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:25.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.014+0000 7f73fd7c8640 1 -- 172.21.15.12:0/4026742682 shutdown_connections 2024-05-09T09:30:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.014+0000 7f73fd7c8640 1 -- 172.21.15.12:0/4026742682 wait complete. 2024-05-09T09:30:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.014+0000 7f73fd7c8640 1 Processor -- start 2024-05-09T09:30:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73fd7c8640 1 -- start start 2024-05-09T09:30:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73fd7c8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73f8100770 0x7f73f8196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73fd7c8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73f8101960 0x7f73f8196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73fd7c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f8102320 0x7f73f819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73fd7c8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73f810c3d0 con 0x7f73f8100770 2024-05-09T09:30:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73fd7c8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73f810c250 con 0x7f73f8102320 2024-05-09T09:30:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73fd7c8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f73f810c550 con 0x7f73f8101960 2024-05-09T09:30:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73f6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73f8100770 0x7f73f8196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73f6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73f8100770 0x7f73f8196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49372/0 (socket says 172.21.15.12:49372) 2024-05-09T09:30:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73f6ffd640 1 -- 172.21.15.12:0/2851300114 learned_addr learned my addr 172.21.15.12:0/2851300114 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T09:30:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73f77fe640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f8102320 0x7f73f819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73f67fc640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73f8101960 0x7f73f8196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73f6ffd640 1 -- 172.21.15.12:0/2851300114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73f8101960 msgr2=0x7f73f8196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73f6ffd640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73f8101960 0x7f73f8196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73f6ffd640 1 -- 172.21.15.12:0/2851300114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f8102320 msgr2=0x7f73f819e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73f6ffd640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f8102320 0x7f73f819e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73f6ffd640 1 -- 172.21.15.12:0/2851300114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f819e670 con 0x7f73f8100770 2024-05-09T09:30:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.015+0000 7f73f6ffd640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73f8100770 0x7f73f8196a50 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f73e800ead0 tx=0x7f73e800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.016+0000 7f73dffff640 1 -- 172.21.15.12:0/2851300114 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e800bbd0 con 0x7f73f8100770 2024-05-09T09:30:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.016+0000 7f73fd7c8640 1 -- 172.21.15.12:0/2851300114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f8068a20 con 0x7f73f8100770 2024-05-09T09:30:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.016+0000 7f73dffff640 1 -- 172.21.15.12:0/2851300114 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f73e800bd70 con 0x7f73f8100770 2024-05-09T09:30:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.016+0000 7f73dffff640 1 -- 172.21.15.12:0/2851300114 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8017410 con 0x7f73f8100770 2024-05-09T09:30:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.016+0000 7f73fd7c8640 1 -- 172.21.15.12:0/2851300114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f8068fc0 con 0x7f73f8100770 2024-05-09T09:30:25.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.017+0000 7f73fd7c8640 1 -- 172.21.15.12:0/2851300114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f8068650 con 0x7f73f8100770 2024-05-09T09:30:25.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.018+0000 7f73dffff640 1 -- 172.21.15.12:0/2851300114 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f73e801e020 con 0x7f73f8100770 2024-05-09T09:30:25.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.018+0000 7f73dffff640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f73e00a2c10 0x7f73e00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:30:25.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.019+0000 7f73dffff640 1 -- 172.21.15.12:0/2851300114 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8496+0+0 (secure 0 0 0) 0x7f73e80d0810 con 0x7f73f8100770 2024-05-09T09:30:25.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.019+0000 7f73f67fc640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f73e00a2c10 0x7f73e00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:30:25.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.019+0000 7f73f67fc640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f73e00a2c10 0x7f73e00a50d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f73f8197f60 tx=0x7f73ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:30:25.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.023+0000 7f73dffff640 1 -- 172.21.15.12:0/2851300114 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f73e809d670 con 0x7f73f8100770 2024-05-09T09:30:25.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.226+0000 7f73fd7c8640 1 -- 172.21.15.12:0/2851300114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f73f819e900 con 0x7f73f8100770 2024-05-09T09:30:25.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.875+0000 7f73dffff640 1 -- 172.21.15.12:0/2851300114 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) ==== 178+0+0 (secure 0 0 0) 0x7f73e809cf70 con 0x7f73f8100770 2024-05-09T09:30:25.876 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data_ec' removed 2024-05-09T09:30:25.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.879+0000 7f73fd7c8640 1 -- 172.21.15.12:0/2851300114 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f73e00a2c10 msgr2=0x7f73e00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:25.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.879+0000 7f73fd7c8640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f73e00a2c10 0x7f73e00a50d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f73f8197f60 tx=0x7f73ec005f90 comp rx=0 tx=0).stop 2024-05-09T09:30:25.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.879+0000 7f73fd7c8640 1 -- 172.21.15.12:0/2851300114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73f8100770 msgr2=0x7f73f8196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:30:25.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.879+0000 7f73fd7c8640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73f8100770 0x7f73f8196a50 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f73e800ead0 tx=0x7f73e800ef90 comp rx=0 tx=0).stop 2024-05-09T09:30:25.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.879+0000 7f73fd7c8640 1 -- 172.21.15.12:0/2851300114 shutdown_connections 2024-05-09T09:30:25.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.879+0000 7f73fd7c8640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.12:6800/3809515815,v1:172.21.15.12:6801/3809515815] conn(0x7f73e00a2c10 0x7f73e00a50d0 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f73f8197f60 tx=0x7f73ec005f90 comp rx=0 tx=0).stop 2024-05-09T09:30:25.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.879+0000 7f73fd7c8640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f8102320 0x7f73f819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:25.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.879+0000 7f73fd7c8640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73f8101960 0x7f73f8196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:25.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.879+0000 7f73fd7c8640 1 --2- 172.21.15.12:0/2851300114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73f8100770 0x7f73f8196a50 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:30:25.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.879+0000 7f73fd7c8640 1 -- 172.21.15.12:0/2851300114 >> 172.21.15.12:0/2851300114 conn(0x7f73f80fbce0 msgr2=0x7f73f8104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:30:25.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.879+0000 7f73fd7c8640 1 -- 172.21.15.12:0/2851300114 shutdown_connections 2024-05-09T09:30:25.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T09:30:25.879+0000 7f73fd7c8640 1 -- 172.21.15.12:0/2851300114 wait complete. 2024-05-09T09:30:25.902 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:30:25.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:30:25.965 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:30:25.998 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-09T09:30:25.998 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T09:30:26.016 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T09:30:26.034 DEBUG:teuthology.orchestra.run.smithi170:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T09:30:26.054 INFO:tasks.cephadm:Stopping all daemons... 2024-05-09T09:30:26.055 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-09T09:30:26.055 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.a 2024-05-09T09:30:26.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:25 smithi012 ceph-mon[30140]: pgmap v516: 97 pgs: 97 active+clean; 577 KiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:30:26.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:25 smithi012 ceph-mon[30140]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T09:30:26.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:25 smithi012 ceph-mon[30140]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T09:30:26.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:25 smithi012 ceph-mon[30140]: from='client.? 172.21.15.12:0/2851300114' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:30:26.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:26 smithi012 systemd[1]: Stopping Ceph mon.a for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:30:26.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:25 smithi170 ceph-mon[33632]: pgmap v516: 97 pgs: 97 active+clean; 577 KiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:30:26.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:25 smithi170 ceph-mon[33632]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T09:30:26.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:25 smithi170 ceph-mon[33632]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T09:30:26.243 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:25 smithi170 ceph-mon[33632]: from='client.? 172.21.15.12:0/2851300114' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:30:26.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:25 smithi016 ceph-mon[33335]: pgmap v516: 97 pgs: 97 active+clean; 577 KiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:30:26.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:25 smithi016 ceph-mon[33335]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T09:30:26.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:25 smithi016 ceph-mon[33335]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T09:30:26.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:25 smithi016 ceph-mon[33335]: from='client.? 172.21.15.12:0/2851300114' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:30:26.703 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:26 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a[30136]: 2024-05-09T09:30:26.372+0000 7f110e82b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:30:26.703 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:26 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a[30136]: 2024-05-09T09:30:26.372+0000 7f110e82b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-09T09:30:26.703 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:26 smithi012 podman[94582]: 2024-05-09 09:30:26.404676253 +0000 UTC m=+0.148185265 container died 1e0b9433ce3ef6566664cf618b2ea8d6339ecf7e5aa402f846b4df79e75c0cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-05-09T09:30:26.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:26 smithi012 podman[94582]: 2024-05-09 09:30:26.890942806 +0000 UTC m=+0.634451821 container cleanup 1e0b9433ce3ef6566664cf618b2ea8d6339ecf7e5aa402f846b4df79e75c0cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-09T09:30:26.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 09:30:26 smithi012 bash[94582]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-a 2024-05-09T09:30:27.473 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.a.service' 2024-05-09T09:30:27.519 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:30:27.520 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-09T09:30:27.520 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-09T09:30:27.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.b 2024-05-09T09:30:27.834 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:27 smithi016 systemd[1]: Stopping Ceph mon.b for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:30:28.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:27 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-b[33331]: 2024-05-09T09:30:27.834+0000 7f95956ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:30:28.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:27 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-b[33331]: 2024-05-09T09:30:27.834+0000 7f95956ce700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-09T09:30:28.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:28 smithi016 podman[67444]: 2024-05-09 09:30:28.038976579 +0000 UTC m=+0.315445799 container died f6666148f06493c709d485869a759f8ba89346c8036892f557136023e5cb327b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-09T09:30:28.618 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:28 smithi016 podman[67444]: 2024-05-09 09:30:28.418075078 +0000 UTC m=+0.694544299 container cleanup f6666148f06493c709d485869a759f8ba89346c8036892f557136023e5cb327b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:30:28.619 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:28 smithi016 bash[67444]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-b 2024-05-09T09:30:28.852 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.b.service' 2024-05-09T09:30:28.877 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:28 smithi016 podman[67458]: 2024-05-09 09:30:28.61960093 +0000 UTC m=+0.579258203 container remove f6666148f06493c709d485869a759f8ba89346c8036892f557136023e5cb327b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-b, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-09T09:30:28.877 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:28 smithi016 systemd[1]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.b.service: Deactivated successfully. 2024-05-09T09:30:28.877 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:28 smithi016 systemd[1]: Stopped Ceph mon.b for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:30:28.877 INFO:journalctl@ceph.mon.b.smithi016.stdout:May 09 09:30:28 smithi016 systemd[1]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.b.service: Consumed 10.319s CPU time. 2024-05-09T09:30:28.907 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:30:28.907 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-09T09:30:28.907 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-09T09:30:28.907 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.c 2024-05-09T09:30:29.216 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:28 smithi170 systemd[1]: Stopping Ceph mon.c for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:30:29.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:29 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-c[33615]: 2024-05-09T09:30:29.216+0000 7f4e02a4e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:30:29.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:29 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-c[33615]: 2024-05-09T09:30:29.216+0000 7f4e02a4e700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-09T09:30:29.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:29 smithi170 podman[68230]: 2024-05-09 09:30:29.483063374 +0000 UTC m=+0.375505229 container died 84836b4b5320d46ffd4dbbb262bed3bd09d19771e3cc6bb17d859a137f865c82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-c, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:30:30.060 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:29 smithi170 podman[68230]: 2024-05-09 09:30:29.868048329 +0000 UTC m=+0.760490198 container cleanup 84836b4b5320d46ffd4dbbb262bed3bd09d19771e3cc6bb17d859a137f865c82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T09:30:30.060 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:29 smithi170 bash[68230]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-c 2024-05-09T09:30:30.292 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.c.service' 2024-05-09T09:30:30.314 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:30 smithi170 podman[68244]: 2024-05-09 09:30:30.061354763 +0000 UTC m=+0.570956917 container remove 84836b4b5320d46ffd4dbbb262bed3bd09d19771e3cc6bb17d859a137f865c82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux ) 2024-05-09T09:30:30.314 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:30 smithi170 systemd[1]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.c.service: Deactivated successfully. 2024-05-09T09:30:30.314 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:30 smithi170 systemd[1]: Stopped Ceph mon.c for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:30:30.314 INFO:journalctl@ceph.mon.c.smithi170.stdout:May 09 09:30:30 smithi170 systemd[1]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mon.c.service: Consumed 9.904s CPU time. 2024-05-09T09:30:30.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:30:30.342 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-09T09:30:30.342 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-09T09:30:30.342 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.x 2024-05-09T09:30:30.663 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:30:30 smithi012 systemd[1]: Stopping Ceph mgr.x for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:30:30.985 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:30:30 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:30:30.661+0000 7f04579cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:30:30.985 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:30:30 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x[30371]: 2024-05-09T09:30:30.661+0000 7f04579cf700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-09T09:30:30.985 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:30:30 smithi012 podman[94704]: 2024-05-09 09:30:30.693815937 +0000 UTC m=+0.147600722 container died b9494772fc036e67aeb3e9c675f4840c27e4fcd846cd3be48da6148195033629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:30:31.464 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:30:31 smithi012 podman[94704]: 2024-05-09 09:30:31.197011438 +0000 UTC m=+0.650796213 container cleanup b9494772fc036e67aeb3e9c675f4840c27e4fcd846cd3be48da6148195033629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:30:31.465 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 09:30:31 smithi012 bash[94704]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-x 2024-05-09T09:30:31.639 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-09T09:30:31.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:30:31.678 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-09T09:30:31.678 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-09T09:30:31.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.y 2024-05-09T09:30:32.001 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:30:31 smithi016 systemd[1]: Stopping Ceph mgr.y for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:30:32.253 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:30:32 smithi016 podman[67566]: 2024-05-09 09:30:32.001896828 +0000 UTC m=+0.122829145 container died b92a20322afa34af15a4dcdbd37a074fc22eafb48c1a029dbe26780d9553e16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T09:30:32.565 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:30:32 smithi016 podman[67566]: 2024-05-09 09:30:32.34812259 +0000 UTC m=+0.469054907 container cleanup b92a20322afa34af15a4dcdbd37a074fc22eafb48c1a029dbe26780d9553e16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-09T09:30:32.565 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:30:32 smithi016 bash[67566]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y 2024-05-09T09:30:32.824 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:30:32 smithi016 podman[67580]: 2024-05-09 09:30:32.566612985 +0000 UTC m=+0.555745725 container remove b92a20322afa34af15a4dcdbd37a074fc22eafb48c1a029dbe26780d9553e16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-09T09:30:32.824 INFO:journalctl@ceph.mgr.y.smithi016.stdout:May 09 09:30:32 smithi016 systemd[1]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-09T09:30:32.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-09T09:30:32.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:30:32.867 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-09T09:30:32.867 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-09T09:30:32.867 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.z 2024-05-09T09:30:33.201 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:30:32 smithi170 systemd[1]: Stopping Ceph mgr.z for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:30:33.483 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:30:33 smithi170 podman[68356]: 2024-05-09 09:30:33.20250362 +0000 UTC m=+0.137399640 container died 8a125eab5f62942727f2831ef747bb99c92e8f35eeef73c48a3f4705004cddfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-09T09:30:33.743 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:30:33 smithi170 podman[68356]: 2024-05-09 09:30:33.574262075 +0000 UTC m=+0.509158092 container cleanup 8a125eab5f62942727f2831ef747bb99c92e8f35eeef73c48a3f4705004cddfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-09T09:30:33.743 INFO:journalctl@ceph.mgr.z.smithi170.stdout:May 09 09:30:33 smithi170 bash[68356]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mgr-z 2024-05-09T09:30:33.997 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-09T09:30:34.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:30:34.033 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-09T09:30:34.033 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-09T09:30:34.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.0 2024-05-09T09:30:34.359 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:34 smithi012 systemd[1]: Stopping Ceph osd.0 for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:30:34.734 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0[45217]: 2024-05-09T09:30:34.359+0000 7fd59c994700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:30:34.734 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0[45217]: 2024-05-09T09:30:34.359+0000 7fd59c994700 -1 osd.0 98 *** Got signal Terminated *** 2024-05-09T09:30:34.734 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0[45217]: 2024-05-09T09:30:34.359+0000 7fd59c994700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:30:40.176 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:39 smithi012 podman[94827]: 2024-05-09 09:30:39.844543952 +0000 UTC m=+5.607723320 container died c82ef4d0239529f94145ff91c062447ed77da7fbddc35be7b9fc03b1d9bbb6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-09T09:30:40.484 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:40 smithi012 podman[94827]: 2024-05-09 09:30:40.276781843 +0000 UTC m=+6.039961210 container cleanup c82ef4d0239529f94145ff91c062447ed77da7fbddc35be7b9fc03b1d9bbb6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:30:40.484 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:40 smithi012 bash[94827]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0 2024-05-09T09:30:40.756 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:40 smithi012 podman[94846]: 2024-05-09 09:30:40.494755347 +0000 UTC m=+0.643016163 container remove c82ef4d0239529f94145ff91c062447ed77da7fbddc35be7b9fc03b1d9bbb6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0, RELEASE=squid-96805dc, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T09:30:41.098 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:40 smithi012 podman[94938]: 2024-05-09 09:30:40.818685177 +0000 UTC m=+0.018483308 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:30:41.098 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:40 smithi012 podman[94938]: 2024-05-09 09:30:40.929340845 +0000 UTC m=+0.129139009 container create eeea61d98cf46ab500eb980fa374fd398cf42a71b46182ec595ac6b01d2cc948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-09T09:30:41.485 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:41 smithi012 podman[94938]: 2024-05-09 09:30:41.2241141 +0000 UTC m=+0.423912224 container init eeea61d98cf46ab500eb980fa374fd398cf42a71b46182ec595ac6b01d2cc948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux ) 2024-05-09T09:30:41.485 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:41 smithi012 podman[94938]: 2024-05-09 09:30:41.228104994 +0000 UTC m=+0.427903113 container start eeea61d98cf46ab500eb980fa374fd398cf42a71b46182ec595ac6b01d2cc948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T09:30:41.485 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:41 smithi012 podman[94938]: 2024-05-09 09:30:41.323845716 +0000 UTC m=+0.523643858 container attach eeea61d98cf46ab500eb980fa374fd398cf42a71b46182ec595ac6b01d2cc948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:30:41.852 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 09:30:41 smithi012 podman[94938]: 2024-05-09 09:30:41.535875617 +0000 UTC m=+0.735673744 container died eeea61d98cf46ab500eb980fa374fd398cf42a71b46182ec595ac6b01d2cc948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T09:30:42.215 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.0.service' 2024-05-09T09:30:42.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:30:42.261 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-09T09:30:42.261 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-09T09:30:42.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.1 2024-05-09T09:30:42.586 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:42 smithi016 systemd[1]: Stopping Ceph osd.1 for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:30:43.003 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:42 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1[37878]: 2024-05-09T09:30:42.586+0000 7f008d4b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:30:43.003 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:42 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1[37878]: 2024-05-09T09:30:42.586+0000 7f008d4b2700 -1 osd.1 98 *** Got signal Terminated *** 2024-05-09T09:30:43.003 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:42 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1[37878]: 2024-05-09T09:30:42.586+0000 7f008d4b2700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:30:48.161 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:47 smithi016 podman[67693]: 2024-05-09 09:30:47.91117987 +0000 UTC m=+5.450245406 container died 839098fb8fed9724f32744ab70df65889f724df4d331c69e9c90b718526712fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True) 2024-05-09T09:30:48.486 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:48 smithi016 podman[67693]: 2024-05-09 09:30:48.253376748 +0000 UTC m=+5.792442287 container cleanup 839098fb8fed9724f32744ab70df65889f724df4d331c69e9c90b718526712fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc) 2024-05-09T09:30:48.487 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:48 smithi016 bash[67693]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1 2024-05-09T09:30:48.753 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:48 smithi016 podman[67711]: 2024-05-09 09:30:48.487643962 +0000 UTC m=+0.565221006 container remove 839098fb8fed9724f32744ab70df65889f724df4d331c69e9c90b718526712fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5) 2024-05-09T09:30:49.253 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:48 smithi016 podman[67801]: 2024-05-09 09:30:48.837206969 +0000 UTC m=+0.018806366 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:30:49.253 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:48 smithi016 podman[67801]: 2024-05-09 09:30:48.947009432 +0000 UTC m=+0.128608856 container create 9a90e779e5df36b792c2fb2ccddf2db920c98277e8695bcdcd0f80737bab84f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_BRANCH=HEAD) 2024-05-09T09:30:49.253 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:49 smithi016 podman[67801]: 2024-05-09 09:30:49.175291705 +0000 UTC m=+0.356891098 container init 9a90e779e5df36b792c2fb2ccddf2db920c98277e8695bcdcd0f80737bab84f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc) 2024-05-09T09:30:49.253 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:49 smithi016 podman[67801]: 2024-05-09 09:30:49.178827908 +0000 UTC m=+0.360427297 container start 9a90e779e5df36b792c2fb2ccddf2db920c98277e8695bcdcd0f80737bab84f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:30:49.753 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:49 smithi016 podman[67801]: 2024-05-09 09:30:49.284170368 +0000 UTC m=+0.465769765 container attach 9a90e779e5df36b792c2fb2ccddf2db920c98277e8695bcdcd0f80737bab84f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1-deactivate, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-09T09:30:49.753 INFO:journalctl@ceph.osd.1.smithi016.stdout:May 09 09:30:49 smithi016 podman[67801]: 2024-05-09 09:30:49.4832633 +0000 UTC m=+0.664862673 container died 9a90e779e5df36b792c2fb2ccddf2db920c98277e8695bcdcd0f80737bab84f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-1-deactivate, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:30:50.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.1.service' 2024-05-09T09:30:50.264 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:30:50.264 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-09T09:30:50.264 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-09T09:30:50.264 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.2 2024-05-09T09:30:50.586 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:50 smithi170 systemd[1]: Stopping Ceph osd.2 for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:30:50.857 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:50 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2[38770]: 2024-05-09T09:30:50.586+0000 7f0c909d3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:30:50.857 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:50 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2[38770]: 2024-05-09T09:30:50.586+0000 7f0c909d3700 -1 osd.2 98 *** Got signal Terminated *** 2024-05-09T09:30:50.857 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:50 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2[38770]: 2024-05-09T09:30:50.586+0000 7f0c909d3700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:30:56.379 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:56 smithi170 podman[68480]: 2024-05-09 09:30:56.078395032 +0000 UTC m=+5.618604949 container died 2f7d263810f7be4765e9b8bf0ff6949490ed2fd717eff55965896e49ac9a69f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=squid-96805dc, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:30:56.696 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:56 smithi170 podman[68480]: 2024-05-09 09:30:56.479292456 +0000 UTC m=+6.019502370 container cleanup 2f7d263810f7be4765e9b8bf0ff6949490ed2fd717eff55965896e49ac9a69f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:30:56.696 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:56 smithi170 bash[68480]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2 2024-05-09T09:30:56.993 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:56 smithi170 podman[68509]: 2024-05-09 09:30:56.697416056 +0000 UTC m=+0.618749064 container remove 2f7d263810f7be4765e9b8bf0ff6949490ed2fd717eff55965896e49ac9a69f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T09:30:57.327 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:57 smithi170 podman[68602]: 2024-05-09 09:30:56.929617247 +0000 UTC m=+0.018311660 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:30:57.328 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:57 smithi170 podman[68602]: 2024-05-09 09:30:57.04019204 +0000 UTC m=+0.128886499 container create 68c5990719531f1fa50b8bdc713391ed51398019f4b43bd6fb89c48fea455e6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-09T09:30:57.618 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:57 smithi170 podman[68602]: 2024-05-09 09:30:57.328664733 +0000 UTC m=+0.417359145 container init 68c5990719531f1fa50b8bdc713391ed51398019f4b43bd6fb89c48fea455e6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:30:57.618 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:57 smithi170 podman[68602]: 2024-05-09 09:30:57.332065257 +0000 UTC m=+0.420759673 container start 68c5990719531f1fa50b8bdc713391ed51398019f4b43bd6fb89c48fea455e6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc) 2024-05-09T09:30:57.618 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:57 smithi170 podman[68602]: 2024-05-09 09:30:57.428684336 +0000 UTC m=+0.517378754 container attach 68c5990719531f1fa50b8bdc713391ed51398019f4b43bd6fb89c48fea455e6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T09:30:57.907 INFO:journalctl@ceph.osd.2.smithi170.stdout:May 09 09:30:57 smithi170 podman[68602]: 2024-05-09 09:30:57.618318815 +0000 UTC m=+0.707013235 container died 68c5990719531f1fa50b8bdc713391ed51398019f4b43bd6fb89c48fea455e6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.license=GPLv2) 2024-05-09T09:30:58.231 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.2.service' 2024-05-09T09:30:58.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:30:58.273 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-09T09:30:58.273 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-09T09:30:58.273 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.3 2024-05-09T09:30:58.605 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:30:58 smithi012 systemd[1]: Stopping Ceph osd.3 for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:30:58.984 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:30:58 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3[52176]: 2024-05-09T09:30:58.604+0000 7f1273b7d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:30:58.984 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:30:58 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3[52176]: 2024-05-09T09:30:58.604+0000 7f1273b7d700 -1 osd.3 98 *** Got signal Terminated *** 2024-05-09T09:30:58.984 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:30:58 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3[52176]: 2024-05-09T09:30:58.604+0000 7f1273b7d700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:31:01.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:00 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:00.977+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:01.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:01 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:01.133+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:01.993 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:01 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:01.740+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:02.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:02 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:02.167+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:02.498 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:02 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:02.011+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:02.993 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:02 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:02.711+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:03.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:02 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:02.746+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:03.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:03 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:03.175+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:03.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:03 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:03.036+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:03.993 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:03 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:03.678+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:04.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:03 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:03.719+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:04.137 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:31:03 smithi012 podman[95043]: 2024-05-09 09:31:03.967065747 +0000 UTC m=+5.490848070 container died 13d309c55ae530cf60a561b2307751937b766863a04ef327e6fe6a0fb32611ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-09T09:31:04.397 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:04 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:04.137+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:04.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:04 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:04.017+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:04.698 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:31:04 smithi012 podman[95043]: 2024-05-09 09:31:04.496689678 +0000 UTC m=+6.020472000 container cleanup 13d309c55ae530cf60a561b2307751937b766863a04ef327e6fe6a0fb32611ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T09:31:04.698 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:31:04 smithi012 bash[95043]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3 2024-05-09T09:31:04.984 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:31:04 smithi012 podman[95072]: 2024-05-09 09:31:04.698155816 +0000 UTC m=+0.733716327 container remove 13d309c55ae530cf60a561b2307751937b766863a04ef327e6fe6a0fb32611ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T09:31:04.994 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:04 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:04.640+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:05.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:04 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:04.718+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:05.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:05 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:05.157+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:05.484 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:31:05 smithi012 podman[95161]: 2024-05-09 09:31:05.030468883 +0000 UTC m=+0.018430771 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:31:05.484 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:31:05 smithi012 podman[95161]: 2024-05-09 09:31:05.149269035 +0000 UTC m=+0.137230904 container create 010d3852378854f61f46b746ef1c97fe139916b2b379a9b9cf13336fcbef476a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-96805dc, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T09:31:05.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:05 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:05.046+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:05.829 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:31:05 smithi012 podman[95161]: 2024-05-09 09:31:05.523838678 +0000 UTC m=+0.511800553 container init 010d3852378854f61f46b746ef1c97fe139916b2b379a9b9cf13336fcbef476a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-09T09:31:05.829 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:31:05 smithi012 podman[95161]: 2024-05-09 09:31:05.527830262 +0000 UTC m=+0.515792139 container start 010d3852378854f61f46b746ef1c97fe139916b2b379a9b9cf13336fcbef476a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T09:31:05.829 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:31:05 smithi012 podman[95161]: 2024-05-09 09:31:05.635327369 +0000 UTC m=+0.623289245 container attach 010d3852378854f61f46b746ef1c97fe139916b2b379a9b9cf13336fcbef476a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-09T09:31:05.993 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:05 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:05.644+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:06.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:05 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:05.721+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:06.179 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 09:31:05 smithi012 podman[95161]: 2024-05-09 09:31:05.830250819 +0000 UTC m=+0.818212701 container died 010d3852378854f61f46b746ef1c97fe139916b2b379a9b9cf13336fcbef476a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-09T09:31:06.456 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:06 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:06.190+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:06.470 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.3.service' 2024-05-09T09:31:06.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:06 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:06.015+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:06.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:31:06.515 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-09T09:31:06.515 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-09T09:31:06.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.4 2024-05-09T09:31:06.834 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:06 smithi016 systemd[1]: Stopping Ceph osd.4 for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:31:06.834 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:06 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:06.739+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:06.976 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:06 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:06.595+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:07.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:06 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:06.976+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:07.253 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:06 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4[44768]: 2024-05-09T09:31:06.833+0000 7f2d6b074700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:31:07.253 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:06 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4[44768]: 2024-05-09T09:31:06.833+0000 7f2d6b074700 -1 osd.4 98 *** Got signal Terminated *** 2024-05-09T09:31:07.253 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:06 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4[44768]: 2024-05-09T09:31:06.833+0000 7f2d6b074700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:31:07.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:07 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:07.201+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:07.965 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:07 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:07.577+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:08.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:07 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:07.740+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:08.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:07 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:07.966+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:08.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:08 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:08.163+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:08.743 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:08 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:08.528+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:09.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:08 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:08.975+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:09.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:08 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:08.975+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:09.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:08 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:08.760+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:09.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:09 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:09.170+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:09.485 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:09 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:09.084+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:09.987 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:09 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:09.558+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:10.243 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:10 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:10.032+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:10.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:09 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:09.988+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:10.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:09 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:09.988+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:10.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:09 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:09.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:10.486 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:10 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:10.080+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:10.486 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:10 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:10.156+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:10.943 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:10 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:10.547+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:11.003 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:10 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4[44768]: 2024-05-09T09:31:10.866+0000 7f2d63660700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.354543+0000 front 2024-05-09T09:30:45.354586+0000 (oldest deadline 2024-05-09T09:31:10.654822+0000) 2024-05-09T09:31:11.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:10 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:10.744+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:11.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:10 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:10.744+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:11.243 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:11 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:11.059+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:11.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:10 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:10.943+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:11.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:10 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:10.943+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:11.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:11 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:11.052+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:11.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:11 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:11.138+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:11.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:11 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:11.138+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:11.994 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:11 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:11.574+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:11.995 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:11 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:11.574+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.943338+0000 front 2024-05-09T09:30:45.943362+0000 (oldest deadline 2024-05-09T09:31:10.643474+0000) 2024-05-09T09:31:12.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:11 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:11.664+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:12.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:11 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:11.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:12.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:11 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:11.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:12.243 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:12 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:12.076+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:12.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:11 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:11.992+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:12.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:11 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:11.992+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:12.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:12 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:12.044+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:12.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:12 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:12.107+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:12.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:12 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:12.107+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:12.492 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:12 smithi016 podman[67923]: 2024-05-09 09:31:12.204887658 +0000 UTC m=+5.486169353 container died f3341ccd5d2a61a083e07494b7f7e7f0dc31172976ffb2623c03caea50a662e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.build-date=20240507) 2024-05-09T09:31:12.753 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:12 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:12.692+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:12.753 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:12 smithi016 podman[67923]: 2024-05-09 09:31:12.58465752 +0000 UTC m=+5.865939213 container cleanup f3341ccd5d2a61a083e07494b7f7e7f0dc31172976ffb2623c03caea50a662e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-09T09:31:12.753 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:12 smithi016 bash[67923]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4 2024-05-09T09:31:12.993 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:12 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:12.580+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:12.993 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:12 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:12.580+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.943338+0000 front 2024-05-09T09:30:45.943362+0000 (oldest deadline 2024-05-09T09:31:10.643474+0000) 2024-05-09T09:31:13.220 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:12 smithi016 podman[67940]: 2024-05-09 09:31:12.761109548 +0000 UTC m=+0.553391898 container remove f3341ccd5d2a61a083e07494b7f7e7f0dc31172976ffb2623c03caea50a662e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:31:13.220 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:12 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:12.804+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:13.220 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:12 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:12.804+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:13.347 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:13 smithi016 podman[68029]: 2024-05-09 09:31:13.124036079 +0000 UTC m=+0.018614298 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:31:13.347 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:13 smithi016 podman[68029]: 2024-05-09 09:31:13.237210219 +0000 UTC m=+0.131788423 container create 878550899fd595e8fc25a29240a2144521453f9d746ee58ab210c591d4411908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-09T09:31:13.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:13 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:13.069+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:13.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:13 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:13.092+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:13.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:13 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:13.092+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:13.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:13 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:13.104+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:13.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:13 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:13.031+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:13.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:13 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:13.031+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:13.733 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:13 smithi016 podman[68029]: 2024-05-09 09:31:13.473489604 +0000 UTC m=+0.368067821 container init 878550899fd595e8fc25a29240a2144521453f9d746ee58ab210c591d4411908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4-deactivate, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T09:31:13.733 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:13 smithi016 podman[68029]: 2024-05-09 09:31:13.477168738 +0000 UTC m=+0.371746945 container start 878550899fd595e8fc25a29240a2144521453f9d746ee58ab210c591d4411908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T09:31:13.734 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:13 smithi016 podman[68029]: 2024-05-09 09:31:13.556693965 +0000 UTC m=+0.451272170 container attach 878550899fd595e8fc25a29240a2144521453f9d746ee58ab210c591d4411908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:31:13.993 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:13 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:13.578+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:13.993 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:13 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:13.578+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.943338+0000 front 2024-05-09T09:30:45.943362+0000 (oldest deadline 2024-05-09T09:31:10.643474+0000) 2024-05-09T09:31:14.003 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:13 smithi016 podman[68029]: 2024-05-09 09:31:13.78755911 +0000 UTC m=+0.682137308 container died 878550899fd595e8fc25a29240a2144521453f9d746ee58ab210c591d4411908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T09:31:14.004 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:13 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:13.732+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:14.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:13 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:13.810+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:14.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:13 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:13.810+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:14.352 INFO:journalctl@ceph.osd.4.smithi016.stdout:May 09 09:31:14 smithi016 podman[68055]: 2024-05-09 09:31:14.344221535 +0000 UTC m=+0.546988520 container remove 878550899fd595e8fc25a29240a2144521453f9d746ee58ab210c591d4411908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-05-09T09:31:14.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.4.service' 2024-05-09T09:31:14.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:31:14.403 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-09T09:31:14.403 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-09T09:31:14.403 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.5 2024-05-09T09:31:14.431 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:14 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:14.073+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:14.431 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:14 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:14.002+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:14.431 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:14 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:14.002+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:14.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:14 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:14.105+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:14.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:14 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:14.128+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:14.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:14 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:14.128+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:14.704 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:14 smithi170 systemd[1]: Stopping Ceph osd.5 for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:31:14.704 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:14 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:14.583+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:14.704 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:14 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:14.583+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.943338+0000 front 2024-05-09T09:30:45.943362+0000 (oldest deadline 2024-05-09T09:31:10.643474+0000) 2024-05-09T09:31:14.704 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:14 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:14.703+0000 7fc945f40700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:31:14.704 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:14 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:14.703+0000 7fc945f40700 -1 osd.5 98 *** Got signal Terminated *** 2024-05-09T09:31:14.704 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:14 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:14.703+0000 7fc945f40700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:31:15.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:14 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:14.808+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:15.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:14 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:14.808+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:15.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:14 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:14.736+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:15.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:15 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:15.109+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:15.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:15 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:15.119+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:15.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:15 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:15.119+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:15.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:15 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:15.038+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:15.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:15 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:15.007+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:15.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:15 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:15.007+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:15.968 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:15 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:15.539+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:15.968 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:15 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:15.539+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.943338+0000 front 2024-05-09T09:30:45.943362+0000 (oldest deadline 2024-05-09T09:31:10.643474+0000) 2024-05-09T09:31:16.243 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:16 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:16.049+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:16.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:15 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:15.969+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:16.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:15 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:15.969+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:16.253 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:15 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:15.755+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:16.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:15 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:15.813+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:16.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:15 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:15.813+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:16.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:16 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:16.082+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:16.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:16 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:16.118+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:16.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:16 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:16.118+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:16.993 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:16 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:16.544+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:16.993 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:16 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:16.544+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.943338+0000 front 2024-05-09T09:30:45.943362+0000 (oldest deadline 2024-05-09T09:31:10.643474+0000) 2024-05-09T09:31:17.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:16 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:16.723+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:17.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:16 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:16.772+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:17.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:16 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:16.772+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:17.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:16 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:16.772+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:17.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:17 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:17.068+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:17.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:17 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:17.154+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:17.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:17 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:17.154+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:17.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:17 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:17.154+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:17.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:17 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:17.094+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:17.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:17 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:17.012+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:17.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:17 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:17.012+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:17.982 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:17 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:17.566+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:17.983 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:17 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:17.566+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.943338+0000 front 2024-05-09T09:30:45.943362+0000 (oldest deadline 2024-05-09T09:31:10.643474+0000) 2024-05-09T09:31:18.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:17 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:17.748+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:18.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:17 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:17.808+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:18.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:17 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:17.808+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:18.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:17 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:17.808+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:18.243 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:18 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:18.116+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:18.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:17 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:17.983+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:18.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:17 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:17.983+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:18.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:18 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:18.059+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:18.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:18 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:18.150+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:18.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:18 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:18.150+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:18.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:18 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:18.150+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:18.989 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:18 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:18.571+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:18.990 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:18 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:18.571+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.943338+0000 front 2024-05-09T09:30:45.943362+0000 (oldest deadline 2024-05-09T09:31:10.643474+0000) 2024-05-09T09:31:19.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:18 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:18.724+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:19.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:18 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:18.814+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:19.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:18 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:18.814+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:19.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:18 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:18.814+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:19.243 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:19 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:19.140+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:19.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:18 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:18.989+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:19.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:18 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:18.989+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:19.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:19 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:19.018+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:19.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:19 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:19.167+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:19.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:19 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:19.167+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:19.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:19 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:19.167+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:19.968 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:19 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:19.580+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.242010+0000 front 2024-05-09T09:30:37.241935+0000 (oldest deadline 2024-05-09T09:31:01.341772+0000) 2024-05-09T09:31:19.968 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:19 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5[45264]: 2024-05-09T09:31:19.580+0000 7fc942534700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.943338+0000 front 2024-05-09T09:30:45.943362+0000 (oldest deadline 2024-05-09T09:31:10.643474+0000) 2024-05-09T09:31:20.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:19 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:19.726+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:20.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:19 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:19.782+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:20.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:19 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:19.782+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:20.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:19 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:19.782+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:20.243 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:20 smithi170 podman[68705]: 2024-05-09 09:31:20.143094714 +0000 UTC m=+5.547824019 container died ec65f60c30c7574c4b38d771879e2b59b797cdd7fde5527ddc88b7579066701a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:31:20.244 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:20 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:20.188+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:20.244 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:20 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:20.188+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:20.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:19 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:19.968+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:20.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:19 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:19.968+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:20.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:19 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:19.968+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:20.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:20 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:20.010+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:20.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:20 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:20.159+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:20.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:20 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:20.159+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:20.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:20 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:20.159+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:20.815 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:20 smithi170 podman[68705]: 2024-05-09 09:31:20.513606757 +0000 UTC m=+5.918336059 container cleanup ec65f60c30c7574c4b38d771879e2b59b797cdd7fde5527ddc88b7579066701a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T09:31:20.815 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:20 smithi170 bash[68705]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5 2024-05-09T09:31:21.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:20 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:20.712+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:21.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:20 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:20.712+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:21.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:20 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:20.791+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:21.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:20 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:20.791+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:21.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:20 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:20.791+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:21.166 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:20 smithi170 podman[68723]: 2024-05-09 09:31:20.816099366 +0000 UTC m=+0.666595804 container remove ec65f60c30c7574c4b38d771879e2b59b797cdd7fde5527ddc88b7579066701a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-09T09:31:21.166 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:20 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:20.932+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:21.166 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:20 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:20.932+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:21.166 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:20 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:20.932+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:21.427 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:21 smithi170 podman[68814]: 2024-05-09 09:31:21.167008558 +0000 UTC m=+0.103987022 container create 27d8249ae58f9d998f2eab8fa02464a93b297c24e4aa38743032a7dbc88ab500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:31:21.428 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:21 smithi170 podman[68814]: 2024-05-09 09:31:21.081192279 +0000 UTC m=+0.018170744 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:31:21.428 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:21.188+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:21.428 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:21.188+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:21.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:21.007+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:21.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:21.007+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.967205+0000 front 2024-05-09T09:30:54.967080+0000 (oldest deadline 2024-05-09T09:31:20.867074+0000) 2024-05-09T09:31:21.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:21.206+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:21.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:21.206+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:21.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:21 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:21.206+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:21.725 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:21 smithi170 podman[68814]: 2024-05-09 09:31:21.428785427 +0000 UTC m=+0.365763891 container init 27d8249ae58f9d998f2eab8fa02464a93b297c24e4aa38743032a7dbc88ab500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5-deactivate, RELEASE=squid-96805dc, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:31:21.725 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:21 smithi170 podman[68814]: 2024-05-09 09:31:21.432163441 +0000 UTC m=+0.369141905 container start 27d8249ae58f9d998f2eab8fa02464a93b297c24e4aa38743032a7dbc88ab500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5-deactivate, RELEASE=squid-96805dc, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T09:31:21.725 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:21 smithi170 podman[68814]: 2024-05-09 09:31:21.487191727 +0000 UTC m=+0.424170211 container attach 27d8249ae58f9d998f2eab8fa02464a93b297c24e4aa38743032a7dbc88ab500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-09T09:31:21.993 INFO:journalctl@ceph.osd.5.smithi170.stdout:May 09 09:31:21 smithi170 podman[68814]: 2024-05-09 09:31:21.725900189 +0000 UTC m=+0.662878662 container died 27d8249ae58f9d998f2eab8fa02464a93b297c24e4aa38743032a7dbc88ab500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:31:21.993 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:21.948+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:21.993 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:21.948+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:21.993 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:21 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:21.948+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:22.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:21.727+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:22.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:21.727+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:22.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:21.834+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:22.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:21.834+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:22.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:21 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:21.834+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:22.357 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:22 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:22.204+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:22.357 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:22 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:22.204+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:22.382 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.5.service' 2024-05-09T09:31:22.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:31:22.465 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-09T09:31:22.465 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-09T09:31:22.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.6 2024-05-09T09:31:22.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:22 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:22.016+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:22.485 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:22 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:22.016+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.967205+0000 front 2024-05-09T09:30:54.967080+0000 (oldest deadline 2024-05-09T09:31:20.867074+0000) 2024-05-09T09:31:22.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:22 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:22.205+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:22.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:22 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:22.205+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:22.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:22 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:22.205+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:22.807 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:22 smithi012 systemd[1]: Stopping Ceph osd.6 for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:31:23.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:22 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:22.705+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:23.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:22 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:22.705+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:23.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:22 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:22.830+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:23.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:22 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:22.830+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:23.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:22 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:22.830+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:23.233 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:22 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:22.807+0000 7f9a7ae35700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:31:23.234 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:22 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:22.807+0000 7f9a7ae35700 -1 osd.6 98 *** Got signal Terminated *** 2024-05-09T09:31:23.234 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:22 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:22.807+0000 7f9a7ae35700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:31:23.234 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:23 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:23.010+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:23.234 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:23 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:23.010+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.967205+0000 front 2024-05-09T09:30:54.967080+0000 (oldest deadline 2024-05-09T09:31:20.867074+0000) 2024-05-09T09:31:23.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:22 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:22.985+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:23.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:22 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:22.985+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:23.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:22 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:22.985+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:23.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:23 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:23.233+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:23.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:23 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:23.233+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:23.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:23 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:23.233+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:23.743 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:23 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:23.249+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:23.743 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:23 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:23.249+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:24.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:23 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:23.713+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:24.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:23 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:23.713+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:24.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:23 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:23.845+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:24.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:23 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:23.845+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:24.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:23 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:23.845+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:24.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:24 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:24.050+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:24.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:24 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:24.050+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.967205+0000 front 2024-05-09T09:30:54.967080+0000 (oldest deadline 2024-05-09T09:31:20.867074+0000) 2024-05-09T09:31:24.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:24 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:24.213+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:24.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:24 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:24.213+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:24.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:24 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:24.213+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:24.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:24 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:24.234+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:24.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:24 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:24.234+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:24.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:24 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:24.009+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:24.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:24 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:24.009+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:24.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:24 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:24.009+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:25.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:24 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:24.709+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:25.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:24 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:24.709+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:25.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:24 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:24.839+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:25.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:24 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:24.839+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:25.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:24 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:24.839+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:25.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:25 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:25.099+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:25.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:25 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:25.099+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.967205+0000 front 2024-05-09T09:30:54.967080+0000 (oldest deadline 2024-05-09T09:31:20.867074+0000) 2024-05-09T09:31:25.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:25 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:25.219+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:25.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:25 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:25.219+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:25.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:25 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:25.219+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:25.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:25 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:25.247+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:25.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:25 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:25.247+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:25.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:25 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:25.247+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:25.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:25 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:25.043+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:25.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:25 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:25.043+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:25.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:25 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:25.043+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:26.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:25 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:25.744+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:26.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:25 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:25.744+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:26.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:25 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:25.744+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.943940+0000 front 2024-05-09T09:31:02.944284+0000 (oldest deadline 2024-05-09T09:31:25.244062+0000) 2024-05-09T09:31:26.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:25 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:25.821+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:26.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:25 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:25.821+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:26.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:25 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:25.821+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:26.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:26 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:26.118+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:26.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:26 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:26.118+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.967205+0000 front 2024-05-09T09:30:54.967080+0000 (oldest deadline 2024-05-09T09:31:20.867074+0000) 2024-05-09T09:31:26.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:26 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:26.195+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:26.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:26 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:26.195+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:26.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:26 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:26.195+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:26.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:26 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:26.089+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:26.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:26 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:26.089+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:26.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:26 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:26.089+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:26.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:26 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:26.204+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:26.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:26 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:26.204+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:26.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:26 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:26.204+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:27.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:26 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:26.728+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:27.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:26 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:26.728+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:27.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:26 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:26.728+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.943940+0000 front 2024-05-09T09:31:02.944284+0000 (oldest deadline 2024-05-09T09:31:25.244062+0000) 2024-05-09T09:31:27.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:26 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:26.785+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:27.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:26 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:26.785+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:27.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:26 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:26.785+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:27.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:27 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:27.118+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:43.563651+0000 front 2024-05-09T09:30:43.563586+0000 (oldest deadline 2024-05-09T09:31:08.164158+0000) 2024-05-09T09:31:27.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:27 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:27.118+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.967205+0000 front 2024-05-09T09:30:54.967080+0000 (oldest deadline 2024-05-09T09:31:20.867074+0000) 2024-05-09T09:31:27.485 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:27 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6[58853]: 2024-05-09T09:31:27.118+0000 7f9a73421700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:00.867363+0000 front 2024-05-09T09:31:00.867423+0000 (oldest deadline 2024-05-09T09:31:26.767582+0000) 2024-05-09T09:31:27.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:27 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:27.235+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:27.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:27 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:27.235+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:27.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:27 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:27.235+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:27.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:27 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:27.235+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:27.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:27 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:27.085+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:27.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:27 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:27.085+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:27.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:27 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:27.085+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:27.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:27 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:27.207+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:27.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:27 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:27.207+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:27.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:27 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:27.207+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:28.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:27 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:27.695+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:28.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:27 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:27.695+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:28.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:27 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:27.695+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.943940+0000 front 2024-05-09T09:31:02.944284+0000 (oldest deadline 2024-05-09T09:31:25.244062+0000) 2024-05-09T09:31:28.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:27 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:27.799+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:28.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:27 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:27.799+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:28.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:27 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:27.799+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:28.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:28 smithi012 podman[95265]: 2024-05-09 09:31:28.20844306 +0000 UTC m=+5.543469009 container died 275ea822210c083bbf2b460ba1dd9fffc48fcc69bafd45713a62eb1b636618b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc) 2024-05-09T09:31:28.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:28 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:28.274+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:28.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:28 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:28.274+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:28.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:28 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:28.274+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:28.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:28 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:28.274+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:28.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:28 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:28.166+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:28.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:28 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:28.166+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:28.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:28 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:28.166+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:28.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:28 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:28.098+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:28.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:28 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:28.098+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:28.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:28 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:28.098+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:28.984 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:28 smithi012 podman[95265]: 2024-05-09 09:31:28.548613384 +0000 UTC m=+5.883639328 container cleanup 275ea822210c083bbf2b460ba1dd9fffc48fcc69bafd45713a62eb1b636618b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:31:28.984 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:28 smithi012 bash[95265]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6 2024-05-09T09:31:28.984 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:28 smithi012 podman[95282]: 2024-05-09 09:31:28.767026815 +0000 UTC m=+0.560039490 container remove 275ea822210c083bbf2b460ba1dd9fffc48fcc69bafd45713a62eb1b636618b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507) 2024-05-09T09:31:29.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:28 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:28.687+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:29.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:28 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:28.687+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:29.004 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:28 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:28.687+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.943940+0000 front 2024-05-09T09:31:02.944284+0000 (oldest deadline 2024-05-09T09:31:25.244062+0000) 2024-05-09T09:31:29.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:28 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:28.833+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:29.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:28 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:28.833+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:29.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:28 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:28.833+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:29.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:28 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:28.833+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:29.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:29 smithi012 podman[95376]: 2024-05-09 09:31:29.167971192 +0000 UTC m=+0.103998227 container create 5e8f88433f723f3f4aae569429da7ba2e160430eebfce1c81faac05b912363df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:31:29.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:29 smithi012 podman[95376]: 2024-05-09 09:31:29.082590737 +0000 UTC m=+0.018617791 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:31:29.484 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:29 smithi012 podman[95376]: 2024-05-09 09:31:29.411837453 +0000 UTC m=+0.347864488 container init 5e8f88433f723f3f4aae569429da7ba2e160430eebfce1c81faac05b912363df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:31:29.485 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:29 smithi012 podman[95376]: 2024-05-09 09:31:29.415169597 +0000 UTC m=+0.351196632 container start 5e8f88433f723f3f4aae569429da7ba2e160430eebfce1c81faac05b912363df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-09T09:31:29.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:29 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:29.316+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:29.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:29 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:29.316+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:29.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:29 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:29.316+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:29.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:29 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:29.316+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:29.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:29 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:29.156+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:29.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:29 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:29.156+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:29.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:29 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:29.156+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:29.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:29 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:29.059+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:29.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:29 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:29.059+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:29.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:29 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:29.059+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:29.981 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:29 smithi012 podman[95376]: 2024-05-09 09:31:29.522569378 +0000 UTC m=+0.458596441 container attach 5e8f88433f723f3f4aae569429da7ba2e160430eebfce1c81faac05b912363df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-09T09:31:29.981 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 09:31:29 smithi012 podman[95376]: 2024-05-09 09:31:29.724041745 +0000 UTC m=+0.660068776 container died 5e8f88433f723f3f4aae569429da7ba2e160430eebfce1c81faac05b912363df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-6-deactivate, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T09:31:30.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:29 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:29.716+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:30.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:29 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:29.716+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:30.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:29 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:29.716+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.943940+0000 front 2024-05-09T09:31:02.944284+0000 (oldest deadline 2024-05-09T09:31:25.244062+0000) 2024-05-09T09:31:30.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:29 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:29.872+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:30.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:29 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:29.872+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:30.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:29 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:29.872+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:30.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:29 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:29.872+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:30.318 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.6.service' 2024-05-09T09:31:30.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:31:30.408 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-09T09:31:30.408 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-09T09:31:30.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.7 2024-05-09T09:31:30.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:30 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:30.166+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:30.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:30 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:30.166+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:30.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:30 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:30.166+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:30.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:30 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:30.043+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:30.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:30 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:30.043+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:30.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:30 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:30.043+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:30.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:30 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:30.332+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:30.753 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:30 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:30.332+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:30.753 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:30 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:30.332+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:30.754 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:30 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:30.332+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:30.754 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:30 smithi016 systemd[1]: Stopping Ceph osd.7 for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:31:31.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:30 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:30.829+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:31.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:30 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:30.829+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:31.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:30 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:30.829+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:31.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:30 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:30.829+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:31.004 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:30 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:30.737+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:31.004 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:30 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:30.737+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:31.004 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:30 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:30.737+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.943940+0000 front 2024-05-09T09:31:02.944284+0000 (oldest deadline 2024-05-09T09:31:25.244062+0000) 2024-05-09T09:31:31.004 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:30 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:30.747+0000 7f94a7b9b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:31:31.004 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:30 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:30.747+0000 7f94a7b9b700 -1 osd.7 98 *** Got signal Terminated *** 2024-05-09T09:31:31.004 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:30 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:30.747+0000 7f94a7b9b700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:31:31.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:31 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:31.207+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:31.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:31 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:31.207+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:31.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:31 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:31.207+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:31.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:31 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:31.060+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:31.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:31 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:31.060+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:31.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:31 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:31.060+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:31.637 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:31 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:31.289+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:31.638 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:31 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:31.289+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:31.638 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:31 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:31.289+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:31.638 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:31 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:31.289+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:32.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:31 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:31.825+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:32.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:31 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:31.825+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:32.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:31 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:31.825+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:32.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:31 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:31.825+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:32.004 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:31 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:31.743+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:32.004 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:31 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:31.743+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:32.004 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:31 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:31.743+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.943940+0000 front 2024-05-09T09:31:02.944284+0000 (oldest deadline 2024-05-09T09:31:25.244062+0000) 2024-05-09T09:31:32.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:32 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:32.227+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:32.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:32 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:32.227+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:32.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:32 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:32.227+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:32.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:32 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:32.102+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:32.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:32 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:32.102+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:32.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:32 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:32.102+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:32.654 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:32 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:32.325+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:32.654 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:32 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:32.325+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:32.654 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:32 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:32.325+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:32.654 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:32 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:32.325+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:32.654 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:32 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:32.325+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:33.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:32 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:32.752+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:33.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:32 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:32.752+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:33.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:32 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:32.752+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.943940+0000 front 2024-05-09T09:31:02.944284+0000 (oldest deadline 2024-05-09T09:31:25.244062+0000) 2024-05-09T09:31:33.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:32 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:32.847+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:33.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:32 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:32.847+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:33.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:32 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:32.847+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:33.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:32 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:32.847+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:33.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:32 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:32.847+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:33.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:33 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:33.235+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:33.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:33 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:33.235+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:33.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:33 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:33.235+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:33.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:33 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:33.116+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:33.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:33 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:33.116+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:33.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:33 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:33.116+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:33.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:33 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:33.320+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:33.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:33 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:33.320+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:33.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:33 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:33.320+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:33.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:33 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:33.320+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:33.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:33 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:33.320+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:34.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:33 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:33.719+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:34.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:33 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:33.719+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:34.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:33 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:33.719+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.943940+0000 front 2024-05-09T09:31:02.944284+0000 (oldest deadline 2024-05-09T09:31:25.244062+0000) 2024-05-09T09:31:34.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:33 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:33.801+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:34.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:33 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:33.801+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:34.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:33 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:33.801+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:34.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:33 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:33.801+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:34.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:33 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:33.801+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:34.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:34 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:34.086+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:34.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:34 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:34.086+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:34.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:34 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:34.086+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:34.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:34 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:34.202+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:34.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:34 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:34.202+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:34.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:34 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:34.202+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:34.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:34 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:34.202+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.221243+0000 front 2024-05-09T09:31:08.221372+0000 (oldest deadline 2024-05-09T09:31:33.521143+0000) 2024-05-09T09:31:34.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:34.293+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:34.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:34.293+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:34.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:34.293+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:34.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:34.293+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:34.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:34 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:34.293+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:35.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:34 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:34.673+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:35.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:34 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:34.673+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:35.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:34 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:34.673+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.943940+0000 front 2024-05-09T09:31:02.944284+0000 (oldest deadline 2024-05-09T09:31:25.244062+0000) 2024-05-09T09:31:35.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:34 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:34.780+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:35.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:34 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:34.780+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:35.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:34 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:34.780+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:35.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:34 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:34.780+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:35.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:34 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:34.780+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:35.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:35 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:35.113+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:35.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:35 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:35.113+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:35.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:35 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:35.113+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:35.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:35 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:35.238+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:35.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:35 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:35.238+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:35.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:35 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:35.238+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:35.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:35 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:35.238+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.221243+0000 front 2024-05-09T09:31:08.221372+0000 (oldest deadline 2024-05-09T09:31:33.521143+0000) 2024-05-09T09:31:35.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:35 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:35.276+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:35.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:35 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:35.276+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:35.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:35 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:35.276+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:35.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:35 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:35.276+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:35.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:35 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:35.276+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:36.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:35 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:35.713+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.441114+0000 front 2024-05-09T09:30:45.441136+0000 (oldest deadline 2024-05-09T09:31:10.741240+0000) 2024-05-09T09:31:36.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:35 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:35.713+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:50.741711+0000 front 2024-05-09T09:30:50.741651+0000 (oldest deadline 2024-05-09T09:31:20.142339+0000) 2024-05-09T09:31:36.003 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:35 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:35.713+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.943940+0000 front 2024-05-09T09:31:02.944284+0000 (oldest deadline 2024-05-09T09:31:25.244062+0000) 2024-05-09T09:31:36.004 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:35 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7[51772]: 2024-05-09T09:31:35.713+0000 7f94a418f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:09.146345+0000 front 2024-05-09T09:31:09.146255+0000 (oldest deadline 2024-05-09T09:31:35.046436+0000) 2024-05-09T09:31:36.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:35 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:35.814+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:36.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:35 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:35.814+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:36.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:35 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:35.814+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:36.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:35 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:35.814+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:36.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:35 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:35.814+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:36.381 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:36 smithi016 podman[68134]: 2024-05-09 09:31:36.10184116 +0000 UTC m=+5.477373893 container died 3020c7f30ffb18df9e802f357b76e3773588afb45b8b54873d064a65b93789dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:31:36.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:36 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:36.110+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:36.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:36 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:36.110+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:36.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:36 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:36.110+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:36.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:36 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:36.213+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:36.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:36 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:36.213+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:36.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:36 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:36.213+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:36.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:36 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:36.213+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.221243+0000 front 2024-05-09T09:31:08.221372+0000 (oldest deadline 2024-05-09T09:31:33.521143+0000) 2024-05-09T09:31:36.736 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:36 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:36.323+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:36.736 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:36 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:36.323+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:36.736 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:36 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:36.323+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:36.736 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:36 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:36.323+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:36.737 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:36 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:36.323+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:36.753 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:36 smithi016 podman[68134]: 2024-05-09 09:31:36.472330949 +0000 UTC m=+5.847863682 container cleanup 3020c7f30ffb18df9e802f357b76e3773588afb45b8b54873d064a65b93789dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:31:36.753 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:36 smithi016 bash[68134]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7 2024-05-09T09:31:37.065 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:36 smithi016 podman[68151]: 2024-05-09 09:31:36.765421136 +0000 UTC m=+0.653403382 container remove 3020c7f30ffb18df9e802f357b76e3773588afb45b8b54873d064a65b93789dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T09:31:37.065 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:36 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:36.808+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:37.066 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:36 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:36.808+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:37.066 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:36 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:36.808+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:37.066 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:36 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:36.808+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:37.066 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:36 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:36.808+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:37.393 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:37 smithi016 podman[68239]: 2024-05-09 09:31:37.066172061 +0000 UTC m=+0.097565151 container create 522abddabbc008885c88338538c66599b59dacd11781f8a652c19b2c214113c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7-deactivate, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T09:31:37.393 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:37 smithi016 podman[68239]: 2024-05-09 09:31:36.987306806 +0000 UTC m=+0.018699849 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:31:37.393 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:37 smithi016 podman[68239]: 2024-05-09 09:31:37.31079147 +0000 UTC m=+0.342184509 container init 522abddabbc008885c88338538c66599b59dacd11781f8a652c19b2c214113c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:31:37.393 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:37 smithi016 podman[68239]: 2024-05-09 09:31:37.314319192 +0000 UTC m=+0.345712221 container start 522abddabbc008885c88338538c66599b59dacd11781f8a652c19b2c214113c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7-deactivate, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-09T09:31:37.393 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:37 smithi016 podman[68239]: 2024-05-09 09:31:37.39398917 +0000 UTC m=+0.425382199 container attach 522abddabbc008885c88338538c66599b59dacd11781f8a652c19b2c214113c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-09T09:31:37.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:37 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:37.183+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:37.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:37 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:37.183+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:37.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:37 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:37.183+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:37.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:37 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:37.183+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.221243+0000 front 2024-05-09T09:31:08.221372+0000 (oldest deadline 2024-05-09T09:31:33.521143+0000) 2024-05-09T09:31:37.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:37 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:37.139+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:37.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:37 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:37.139+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:37.495 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:37 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:37.139+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:37.495 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:37 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:37.139+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:37.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:37 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:37.288+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:37.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:37 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:37.288+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:37.736 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:37 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:37.288+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:37.736 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:37 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:37.288+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:37.736 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:37 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:37.288+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:37.753 INFO:journalctl@ceph.osd.7.smithi016.stdout:May 09 09:31:37 smithi016 podman[68239]: 2024-05-09 09:31:37.614852692 +0000 UTC m=+0.646245720 container died 522abddabbc008885c88338538c66599b59dacd11781f8a652c19b2c214113c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_CLEAN=True) 2024-05-09T09:31:38.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:37 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:37.782+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:38.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:37 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:37.782+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:38.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:37 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:37.782+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:38.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:37 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:37.782+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:38.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:37 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:37.782+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:38.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.7.service' 2024-05-09T09:31:38.440 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:31:38.440 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-09T09:31:38.440 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-09T09:31:38.440 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.8 2024-05-09T09:31:38.468 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:38 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:38.148+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:38.469 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:38 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:38.148+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:38.469 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:38 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:38.148+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:38.469 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:38 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:38.148+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.221243+0000 front 2024-05-09T09:31:08.221372+0000 (oldest deadline 2024-05-09T09:31:33.521143+0000) 2024-05-09T09:31:38.469 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:38 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:38.166+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:38.470 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:38 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:38.166+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:38.470 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:38 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:38.166+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:38.470 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:38 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:38.166+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:38.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:38 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:38.274+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:38.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:38 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:38.274+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:38.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:38 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:38.274+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:38.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:38 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:38.274+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:38.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:38 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:38.274+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:38.735 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:38 smithi170 systemd[1]: Stopping Ceph osd.8 for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:31:38.993 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:38 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:38.735+0000 7f39ccea8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:31:38.994 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:38 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:38.735+0000 7f39ccea8700 -1 osd.8 98 *** Got signal Terminated *** 2024-05-09T09:31:38.994 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:38 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:38.735+0000 7f39ccea8700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:31:39.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:38 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:38.755+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:39.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:38 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:38.755+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:39.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:38 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:38.755+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:39.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:38 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:38.755+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:39.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:38 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:38.755+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:39.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:39 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:39.144+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:39.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:39 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:39.144+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:39.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:39 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:39.144+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:39.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:39 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:39.144+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.221243+0000 front 2024-05-09T09:31:08.221372+0000 (oldest deadline 2024-05-09T09:31:33.521143+0000) 2024-05-09T09:31:39.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:39 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:39.139+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:39.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:39 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:39.139+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:39.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:39 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:39.139+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:39.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:39 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:39.139+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:39.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:39 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:39.292+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:39.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:39 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:39.292+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:39.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:39 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:39.292+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:39.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:39 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:39.292+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:39.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:39 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:39.292+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:40.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:39 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:39.736+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:40.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:39 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:39.736+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:40.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:39 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:39.736+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:40.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:39 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:39.736+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:40.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:39 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:39.736+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:40.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:40 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:40.191+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:40.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:40 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:40.191+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:40.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:40 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:40.191+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:40.493 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:40 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:40.191+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.221243+0000 front 2024-05-09T09:31:08.221372+0000 (oldest deadline 2024-05-09T09:31:33.521143+0000) 2024-05-09T09:31:40.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:40 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:40.161+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:40.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:40 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:40.161+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:40.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:40 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:40.161+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:40.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:40 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:40.161+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:40.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:40 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:40.294+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:40.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:40 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:40.294+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:40.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:40 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:40.294+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:40.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:40 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:40.294+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:40.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:40 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:40.294+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:41.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:40 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:40.770+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:41.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:40 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:40.770+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:41.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:40 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:40.770+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:41.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:40 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:40.770+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:41.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:40 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:40.770+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:41.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:40 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:40.770+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:41.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:41 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:41.128+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:41.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:41 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:41.128+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:41.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:41 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:41.128+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:41.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:41 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:41.128+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:41.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:41 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:41.158+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:41.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:41 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:41.158+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:41.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:41 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:41.158+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:41.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:41 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:41.158+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.221243+0000 front 2024-05-09T09:31:08.221372+0000 (oldest deadline 2024-05-09T09:31:33.521143+0000) 2024-05-09T09:31:41.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:41 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:41.158+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.123660+0000 front 2024-05-09T09:31:19.123687+0000 (oldest deadline 2024-05-09T09:31:40.223728+0000) 2024-05-09T09:31:41.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:41.322+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:41.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:41.322+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:41.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:41.322+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:41.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:41.322+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:41.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:41.322+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:41.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:41 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:41.322+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.408174+0000 front 2024-05-09T09:31:16.408190+0000 (oldest deadline 2024-05-09T09:31:41.108021+0000) 2024-05-09T09:31:42.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:41 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:41.784+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:42.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:41 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:41.784+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:42.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:41 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:41.784+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:42.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:41 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:41.784+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:42.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:41 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:41.784+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:42.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:41 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:41.784+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:42.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:42 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:42.087+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:42.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:42 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:42.087+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:42.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:42 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:42.087+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:42.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:42 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:42.087+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:42.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:42 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:42.125+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:42.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:42 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:42.125+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:42.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:42 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:42.125+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:42.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:42 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:42.125+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.221243+0000 front 2024-05-09T09:31:08.221372+0000 (oldest deadline 2024-05-09T09:31:33.521143+0000) 2024-05-09T09:31:42.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:42 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:42.125+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.123660+0000 front 2024-05-09T09:31:19.123687+0000 (oldest deadline 2024-05-09T09:31:40.223728+0000) 2024-05-09T09:31:42.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:42 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:42.299+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:42.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:42 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:42.299+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:42.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:42 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:42.299+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:42.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:42 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:42.299+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:42.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:42 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:42.299+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:42.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:42 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:42.299+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.408174+0000 front 2024-05-09T09:31:16.408190+0000 (oldest deadline 2024-05-09T09:31:41.108021+0000) 2024-05-09T09:31:43.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:42 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:42.753+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:43.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:42 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:42.753+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:43.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:42 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:42.753+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:43.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:42 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:42.753+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:43.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:42 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:42.753+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:43.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:42 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:42.753+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:43.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:43 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:43.069+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:43.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:43 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:43.069+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:43.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:43 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:43.069+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:43.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:43 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:43.069+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:43.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:43 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:43.105+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:47.317663+0000 front 2024-05-09T09:30:47.317616+0000 (oldest deadline 2024-05-09T09:31:09.617425+0000) 2024-05-09T09:31:43.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:43 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:43.105+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:55.319310+0000 front 2024-05-09T09:30:55.319363+0000 (oldest deadline 2024-05-09T09:31:19.419422+0000) 2024-05-09T09:31:43.494 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:43 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:43.105+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:30:59.419899+0000 front 2024-05-09T09:30:59.419879+0000 (oldest deadline 2024-05-09T09:31:24.719924+0000) 2024-05-09T09:31:43.495 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:43 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:43.105+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.221243+0000 front 2024-05-09T09:31:08.221372+0000 (oldest deadline 2024-05-09T09:31:33.521143+0000) 2024-05-09T09:31:43.495 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:43 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-09T09:31:43.105+0000 7f39c5494700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.123660+0000 front 2024-05-09T09:31:19.123687+0000 (oldest deadline 2024-05-09T09:31:40.223728+0000) 2024-05-09T09:31:43.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:43 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:43.263+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:43.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:43 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:43.263+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:43.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:43 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:43.263+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:43.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:43 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:43.263+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:43.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:43 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:43.263+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:43.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:43 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:43.263+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.408174+0000 front 2024-05-09T09:31:16.408190+0000 (oldest deadline 2024-05-09T09:31:41.108021+0000) 2024-05-09T09:31:44.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:43 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:43.793+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:44.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:43 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:43.793+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:44.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:43 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:43.793+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:44.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:43 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:43.793+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:44.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:43 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:43.793+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:44.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:43 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:43.793+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:44.478 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:44 smithi170 podman[68919]: 2024-05-09 09:31:44.209738748 +0000 UTC m=+5.577906378 container died e6fade36a9007aa7e947b57668f7b04e91651cfce7469d141b190b3e17aae964 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, io.buildah.version=1.33.5) 2024-05-09T09:31:44.478 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:44 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:44.028+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:44.479 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:44 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:44.028+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:44.479 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:44 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:44.028+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:44.479 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:44 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:44.028+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:44.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:44 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:44.218+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:44.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:44 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:44.218+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:44.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:44 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:44.218+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:44.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:44 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:44.218+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:44.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:44 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:44.218+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:44.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:44 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:44.218+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.408174+0000 front 2024-05-09T09:31:16.408190+0000 (oldest deadline 2024-05-09T09:31:41.108021+0000) 2024-05-09T09:31:44.743 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:44 smithi170 podman[68919]: 2024-05-09 09:31:44.568927101 +0000 UTC m=+5.937094738 container cleanup e6fade36a9007aa7e947b57668f7b04e91651cfce7469d141b190b3e17aae964 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-09T09:31:44.743 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:44 smithi170 bash[68919]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8 2024-05-09T09:31:45.184 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:44 smithi170 podman[68936]: 2024-05-09 09:31:44.853562133 +0000 UTC m=+0.640736505 container remove e6fade36a9007aa7e947b57668f7b04e91651cfce7469d141b190b3e17aae964 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T09:31:45.184 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:45 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:45.040+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:45.184 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:45 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:45.040+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:45.185 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:45 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:45.040+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:45.185 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:45 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:45.040+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:45.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:44 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:44.777+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:45.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:44 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:44.777+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:45.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:44 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:44.777+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:45.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:44 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:44.777+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:45.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:44 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:44.777+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:45.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:44 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:44.777+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:45.447 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:45 smithi170 podman[69030]: 2024-05-09 09:31:45.087628523 +0000 UTC m=+0.017880895 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:31:45.448 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:45 smithi170 podman[69030]: 2024-05-09 09:31:45.213069217 +0000 UTC m=+0.143321593 container create 4331a68bf9b4f59ebfe9291b165c36003ebfada481fda369e9c7176d059e573e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T09:31:45.448 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:45 smithi170 podman[69030]: 2024-05-09 09:31:45.448886828 +0000 UTC m=+0.379139210 container init 4331a68bf9b4f59ebfe9291b165c36003ebfada481fda369e9c7176d059e573e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:31:45.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:45 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:45.204+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:45.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:45 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:45.204+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:45.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:45 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:45.204+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:45.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:45 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:45.204+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:45.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:45 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:45.204+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:45.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:45 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:45.204+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.408174+0000 front 2024-05-09T09:31:16.408190+0000 (oldest deadline 2024-05-09T09:31:41.108021+0000) 2024-05-09T09:31:45.744 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:45 smithi170 podman[69030]: 2024-05-09 09:31:45.452409092 +0000 UTC m=+0.382661473 container start 4331a68bf9b4f59ebfe9291b165c36003ebfada481fda369e9c7176d059e573e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T09:31:45.744 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:45 smithi170 podman[69030]: 2024-05-09 09:31:45.507216491 +0000 UTC m=+0.437468881 container attach 4331a68bf9b4f59ebfe9291b165c36003ebfada481fda369e9c7176d059e573e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T09:31:46.034 INFO:journalctl@ceph.osd.8.smithi170.stdout:May 09 09:31:45 smithi170 podman[69030]: 2024-05-09 09:31:45.748806474 +0000 UTC m=+0.679058851 container died 4331a68bf9b4f59ebfe9291b165c36003ebfada481fda369e9c7176d059e573e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-09T09:31:46.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:45 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:45.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:46.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:45 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:45.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:46.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:45 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:45.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:46.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:45 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:45.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:46.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:45 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:45.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:46.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:45 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:45.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:46.343 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:46 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:46.058+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:46.343 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:46 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:46.058+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:46.343 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:46 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:46.058+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:46.344 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:46 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:46.058+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:46.344 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:46 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:46.058+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:46.360 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.8.service' 2024-05-09T09:31:46.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:31:46.441 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-09T09:31:46.441 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-09T09:31:46.441 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.9 2024-05-09T09:31:46.472 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:46 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:46.198+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:46.472 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:46 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:46.198+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:46.472 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:46 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:46.198+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:46.472 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:46 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:46.198+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:46.472 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:46 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:46.198+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:46.472 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:46 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:46.198+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.408174+0000 front 2024-05-09T09:31:16.408190+0000 (oldest deadline 2024-05-09T09:31:41.108021+0000) 2024-05-09T09:31:46.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:46 smithi012 systemd[1]: Stopping Ceph osd.9 for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:31:47.188 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:46 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:46.767+0000 7f7de0640700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:31:47.188 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:46 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:46.767+0000 7f7de0640700 -1 osd.9 98 *** Got signal Terminated *** 2024-05-09T09:31:47.188 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:46 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:46.767+0000 7f7de0640700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:31:47.188 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:47 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:47.187+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:47.188 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:47 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:47.187+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:47.188 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:47 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:47.187+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:47.188 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:47 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:47.187+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:47.188 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:47 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:47.187+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:47.188 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:47 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:47.187+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.408174+0000 front 2024-05-09T09:31:16.408190+0000 (oldest deadline 2024-05-09T09:31:41.108021+0000) 2024-05-09T09:31:47.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:46 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:46.771+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:47.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:46 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:46.771+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:47.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:46 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:46.771+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:47.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:46 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:46.771+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:47.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:46 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:46.771+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:47.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:46 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:46.771+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:47.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:47 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:47.041+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:47.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:47 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:47.041+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:47.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:47 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:47.041+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:47.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:47 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:47.041+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:47.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:47 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:47.041+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:48.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:47 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:47.816+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:48.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:47 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:47.816+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:48.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:47 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:47.816+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:48.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:47 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:47.816+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:48.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:47 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:47.816+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:48.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:47 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:47.816+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:48.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:48 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:48.192+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:48.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:48 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:48.192+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:48.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:48 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:48.192+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:48.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:48 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:48.192+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:48.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:48 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:48.192+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:48.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:48 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:48.192+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.408174+0000 front 2024-05-09T09:31:16.408190+0000 (oldest deadline 2024-05-09T09:31:41.108021+0000) 2024-05-09T09:31:48.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:48 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:48.004+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:48.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:48 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:48.004+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:48.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:48 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:48.004+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:48.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:48 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:48.004+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:48.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:48 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:48.004+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:49.129 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:48 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:48.831+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:49.129 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:48 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:48.831+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:49.130 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:48 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:48.831+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:49.130 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:48 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:48.831+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:49.130 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:48 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:48.831+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:49.130 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:48 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:48.831+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:49.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:48 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:48.979+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:49.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:48 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:48.979+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:49.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:48 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:48.979+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:49.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:48 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:48.979+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:49.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:48 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:48.979+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:49.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:49 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:49.205+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:49.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:49 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:49.205+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:49.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:49 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:49.205+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:49.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:49 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:49.205+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:49.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:49 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:49.205+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:49.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:49 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:49.205+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.408174+0000 front 2024-05-09T09:31:16.408190+0000 (oldest deadline 2024-05-09T09:31:41.108021+0000) 2024-05-09T09:31:50.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:49 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:49.848+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:50.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:49 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:49.848+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:50.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:49 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:49.848+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:50.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:49 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:49.848+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:50.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:49 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:49.848+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:50.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:49 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:49.848+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:50.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:50 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:50.206+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:50.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:50 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:50.206+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:50.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:50 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:50.206+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:50.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:50 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:50.206+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:50.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:50 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:50.206+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:50.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:50 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:50.206+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.408174+0000 front 2024-05-09T09:31:16.408190+0000 (oldest deadline 2024-05-09T09:31:41.108021+0000) 2024-05-09T09:31:50.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:50 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:50.007+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:50.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:50 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:50.007+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:50.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:50 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:50.007+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:50.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:50 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:50.007+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:50.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:50 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:50.007+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:51.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:50 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:50.819+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:51.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:50 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:50.819+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:51.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:50 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:50.819+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:51.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:50 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:50.819+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:51.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:50 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:50.819+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:51.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:50 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:50.819+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:51.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:51 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:51.037+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:51.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:51 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:51.037+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:51.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:51 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:51.037+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:51.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:51 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:51.037+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:51.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:51 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:51.037+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:51.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:51 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:51.246+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:36.002624+0000 front 2024-05-09T09:30:36.002582+0000 (oldest deadline 2024-05-09T09:31:00.702679+0000) 2024-05-09T09:31:51.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:51 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:51.246+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.204769+0000 front 2024-05-09T09:30:45.204883+0000 (oldest deadline 2024-05-09T09:31:11.104568+0000) 2024-05-09T09:31:51.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:51 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:51.246+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:51.105100+0000 front 2024-05-09T09:30:51.105138+0000 (oldest deadline 2024-05-09T09:31:16.405288+0000) 2024-05-09T09:31:51.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:51 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:51.246+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:01.106257+0000 front 2024-05-09T09:31:01.106291+0000 (oldest deadline 2024-05-09T09:31:27.006455+0000) 2024-05-09T09:31:51.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:51 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:51.246+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:07.007014+0000 front 2024-05-09T09:31:07.007112+0000 (oldest deadline 2024-05-09T09:31:32.307111+0000) 2024-05-09T09:31:51.735 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:51 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9[65863]: 2024-05-09T09:31:51.246+0000 7f7ddcc34700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.408174+0000 front 2024-05-09T09:31:16.408190+0000 (oldest deadline 2024-05-09T09:31:41.108021+0000) 2024-05-09T09:31:52.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:51 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:51.810+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:52.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:51 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:51.810+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:52.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:51 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:51.810+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:52.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:51 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:51.810+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:52.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:51 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:51.810+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:52.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:51 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:51.810+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:52.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:51 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:51.810+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.621437+0000 front 2024-05-09T09:31:25.621573+0000 (oldest deadline 2024-05-09T09:31:51.521260+0000) 2024-05-09T09:31:52.475 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:52 smithi012 podman[95495]: 2024-05-09 09:31:52.173436942 +0000 UTC m=+5.526339792 container died 6bde3e41e342cc1b3b522aad82dbc87ec5c042b62c38f4ffe8ee1601f1026f10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-09T09:31:52.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:52 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:52.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:52.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:52 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:52.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:52.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:52 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:52.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:52.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:52 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:52.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:52.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:52 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:52.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:52.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:52 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:52.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:31:52.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:52 smithi012 podman[95495]: 2024-05-09 09:31:52.583274738 +0000 UTC m=+5.936177583 container cleanup 6bde3e41e342cc1b3b522aad82dbc87ec5c042b62c38f4ffe8ee1601f1026f10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T09:31:52.734 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:52 smithi012 bash[95495]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9 2024-05-09T09:31:53.198 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:52 smithi012 podman[95512]: 2024-05-09 09:31:52.784686783 +0000 UTC m=+0.606144720 container remove 6bde3e41e342cc1b3b522aad82dbc87ec5c042b62c38f4ffe8ee1601f1026f10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-09T09:31:53.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:52 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:52.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:53.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:52 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:52.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:53.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:52 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:52.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:53.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:52 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:52.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:53.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:52 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:52.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:53.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:52 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:52.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:53.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:52 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:52.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.621437+0000 front 2024-05-09T09:31:25.621573+0000 (oldest deadline 2024-05-09T09:31:51.521260+0000) 2024-05-09T09:31:53.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:53 smithi012 podman[95602]: 2024-05-09 09:31:53.101990596 +0000 UTC m=+0.018212903 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:31:53.484 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:53 smithi012 podman[95602]: 2024-05-09 09:31:53.202124058 +0000 UTC m=+0.118346391 container create 7e221255ab20ab569cc64fa568f8521db098423d054775e6e09d6cb10e79da9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9-deactivate, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-09T09:31:53.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:53 smithi012 podman[95602]: 2024-05-09 09:31:53.421447197 +0000 UTC m=+0.337669488 container init 7e221255ab20ab569cc64fa568f8521db098423d054775e6e09d6cb10e79da9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-09T09:31:53.485 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:53 smithi012 podman[95602]: 2024-05-09 09:31:53.425019086 +0000 UTC m=+0.341241380 container start 7e221255ab20ab569cc64fa568f8521db098423d054775e6e09d6cb10e79da9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9-deactivate, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-09T09:31:53.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:53 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:53.040+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:53.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:53 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:53.040+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:53.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:53 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:53.040+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:53.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:53 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:53.040+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:53.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:53 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:53.040+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:53.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:53 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:53.040+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:31:53.984 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:53 smithi012 podman[95602]: 2024-05-09 09:31:53.487975343 +0000 UTC m=+0.404197660 container attach 7e221255ab20ab569cc64fa568f8521db098423d054775e6e09d6cb10e79da9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-09T09:31:53.985 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 09:31:53 smithi012 podman[95602]: 2024-05-09 09:31:53.731577992 +0000 UTC m=+0.647800283 container died 7e221255ab20ab569cc64fa568f8521db098423d054775e6e09d6cb10e79da9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:31:54.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:53 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:53.818+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:54.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:53 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:53.818+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:54.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:53 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:53.818+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:54.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:53 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:53.818+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:54.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:53 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:53.818+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:54.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:53 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:53.818+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:54.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:53 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:53.818+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.621437+0000 front 2024-05-09T09:31:25.621573+0000 (oldest deadline 2024-05-09T09:31:51.521260+0000) 2024-05-09T09:31:54.452 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.9.service' 2024-05-09T09:31:54.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:54 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:54.022+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:54.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:54 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:54.022+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:54.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:54 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:54.022+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:54.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:54 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:54.022+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:54.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:54 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:54.022+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:54.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:54 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:54.022+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:31:54.537 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:31:54.537 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-09T09:31:54.537 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-09T09:31:54.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.10 2024-05-09T09:31:54.885 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:54 smithi016 systemd[1]: Stopping Ceph osd.10 for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:31:54.885 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:54 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:54.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:54.885 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:54 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:54.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:54.885 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:54 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:54.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:54.885 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:54 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:54.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:54.885 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:54 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:54.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:54.885 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:54 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:54.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:54.885 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:54 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:54.769+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.621437+0000 front 2024-05-09T09:31:25.621573+0000 (oldest deadline 2024-05-09T09:31:51.521260+0000) 2024-05-09T09:31:55.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:54 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:54.980+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:55.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:54 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:54.980+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:55.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:54 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:54.980+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:55.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:54 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:54.980+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:55.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:54 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:54.980+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:55.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:54 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:54.980+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:31:55.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:54 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:54.884+0000 7f1eb5749700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:31:55.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:54 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:54.884+0000 7f1eb5749700 -1 osd.10 98 *** Got signal Terminated *** 2024-05-09T09:31:55.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:54 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:54.884+0000 7f1eb5749700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:31:56.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:55 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:55.746+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:56.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:55 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:55.746+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:56.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:55 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:55.746+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:56.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:55 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:55.746+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:56.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:55 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:55.746+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:56.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:55 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:55.746+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:56.004 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:55 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:55.746+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.621437+0000 front 2024-05-09T09:31:25.621573+0000 (oldest deadline 2024-05-09T09:31:51.521260+0000) 2024-05-09T09:31:56.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:55 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:55.984+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:56.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:55 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:55.984+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:56.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:55 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:55.984+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:56.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:55 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:55.984+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:56.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:55 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:55.984+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:56.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:55 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:55.984+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:31:57.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:56 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:56.966+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:57.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:56 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:56.966+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:57.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:56 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:56.966+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:57.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:56 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:56.966+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:57.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:56 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:56.966+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:57.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:56 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:56.966+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:31:57.245 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:56 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:56.966+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-05-09T09:31:33.872953+0000 front 2024-05-09T09:31:33.873083+0000 (oldest deadline 2024-05-09T09:31:56.172774+0000) 2024-05-09T09:31:57.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:56 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:56.789+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:57.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:56 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:56.789+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:57.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:56 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:56.789+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:57.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:56 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:56.789+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:57.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:56 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:56.789+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:57.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:56 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:56.789+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:57.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:56 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:56.789+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.621437+0000 front 2024-05-09T09:31:25.621573+0000 (oldest deadline 2024-05-09T09:31:51.521260+0000) 2024-05-09T09:31:58.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:57 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:57.945+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:58.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:57 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:57.945+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:58.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:57 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:57.945+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:58.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:57 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:57.945+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:58.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:57 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:57.945+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:58.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:57 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:57.945+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:31:58.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:57 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:57.945+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-05-09T09:31:33.872953+0000 front 2024-05-09T09:31:33.873083+0000 (oldest deadline 2024-05-09T09:31:56.172774+0000) 2024-05-09T09:31:58.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:57 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:57.819+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:58.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:57 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:57.819+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:58.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:57 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:57.819+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:58.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:57 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:57.819+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:58.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:57 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:57.819+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:58.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:57 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:57.819+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:58.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:57 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:57.819+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.621437+0000 front 2024-05-09T09:31:25.621573+0000 (oldest deadline 2024-05-09T09:31:51.521260+0000) 2024-05-09T09:31:59.243 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:58 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:58.936+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:31:59.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:58 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:58.936+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:31:59.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:58 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:58.936+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:31:59.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:58 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:58.936+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:31:59.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:58 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:58.936+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:31:59.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:58 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:58.936+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:31:59.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:58 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:58.936+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-05-09T09:31:33.872953+0000 front 2024-05-09T09:31:33.873083+0000 (oldest deadline 2024-05-09T09:31:56.172774+0000) 2024-05-09T09:31:59.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:58 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:58.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:31:59.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:58 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:58.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:31:59.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:58 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:58.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:31:59.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:58 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:58.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:31:59.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:58 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:58.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:31:59.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:58 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:58.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:31:59.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:58 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:58.788+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.621437+0000 front 2024-05-09T09:31:25.621573+0000 (oldest deadline 2024-05-09T09:31:51.521260+0000) 2024-05-09T09:32:00.246 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:59 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:59.977+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:32:00.246 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:59 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:59.977+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:32:00.246 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:59 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:59.977+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:32:00.246 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:59 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:59.977+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:32:00.246 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:59 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:59.977+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:32:00.246 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:59 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:59.977+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:32:00.247 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:31:59 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:31:59.977+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-05-09T09:31:33.872953+0000 front 2024-05-09T09:31:33.873083+0000 (oldest deadline 2024-05-09T09:31:56.172774+0000) 2024-05-09T09:32:00.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:59 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:59.786+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:37.813910+0000 front 2024-05-09T09:30:37.814106+0000 (oldest deadline 2024-05-09T09:31:02.514019+0000) 2024-05-09T09:32:00.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:59 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:59.786+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:44.814993+0000 front 2024-05-09T09:30:44.815040+0000 (oldest deadline 2024-05-09T09:31:10.115072+0000) 2024-05-09T09:32:00.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:59 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:59.786+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:53.616258+0000 front 2024-05-09T09:30:53.616373+0000 (oldest deadline 2024-05-09T09:31:16.516095+0000) 2024-05-09T09:32:00.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:59 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:59.786+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T09:31:02.818212+0000 front 2024-05-09T09:31:02.818164+0000 (oldest deadline 2024-05-09T09:31:28.118203+0000) 2024-05-09T09:32:00.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:59 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:59.786+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:08.118701+0000 front 2024-05-09T09:31:08.118659+0000 (oldest deadline 2024-05-09T09:31:32.218756+0000) 2024-05-09T09:32:00.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:59 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:59.786+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:16.919909+0000 front 2024-05-09T09:31:16.919811+0000 (oldest deadline 2024-05-09T09:31:40.419797+0000) 2024-05-09T09:32:00.254 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:31:59 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10[58510]: 2024-05-09T09:31:59.786+0000 7f1eb1d3d700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.621437+0000 front 2024-05-09T09:31:25.621573+0000 (oldest deadline 2024-05-09T09:31:51.521260+0000) 2024-05-09T09:32:00.701 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:32:00 smithi016 podman[68357]: 2024-05-09 09:32:00.424199433 +0000 UTC m=+5.678059571 container died 2ada98632da790225a62d9b9aa39f1e9fbeb62151ac6773ef6159922cb77d6af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:32:01.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:32:00 smithi016 podman[68357]: 2024-05-09 09:32:00.792796953 +0000 UTC m=+6.046657081 container cleanup 2ada98632da790225a62d9b9aa39f1e9fbeb62151ac6773ef6159922cb77d6af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:32:01.003 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:32:00 smithi016 bash[68357]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10 2024-05-09T09:32:01.475 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:32:01 smithi016 podman[68378]: 2024-05-09 09:32:01.135528778 +0000 UTC m=+0.698985737 container remove 2ada98632da790225a62d9b9aa39f1e9fbeb62151ac6773ef6159922cb77d6af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-09T09:32:01.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:01 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:01.015+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:32:01.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:01 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:01.015+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:32:01.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:01 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:01.015+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:32:01.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:01 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:01.015+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:32:01.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:01 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:01.015+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:32:01.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:01 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:01.015+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:32:01.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:01 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:01.015+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-05-09T09:31:33.872953+0000 front 2024-05-09T09:31:33.873083+0000 (oldest deadline 2024-05-09T09:31:56.172774+0000) 2024-05-09T09:32:01.753 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:32:01 smithi016 podman[68469]: 2024-05-09 09:32:01.379686574 +0000 UTC m=+0.018296892 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:32:01.754 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:32:01 smithi016 podman[68469]: 2024-05-09 09:32:01.486550441 +0000 UTC m=+0.125160746 container create b3044a907f59388babfa72dad20e84d7f007840aa89eb3ceb0410674b02eba36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T09:32:01.754 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:32:01 smithi016 podman[68469]: 2024-05-09 09:32:01.705597468 +0000 UTC m=+0.344207781 container init b3044a907f59388babfa72dad20e84d7f007840aa89eb3ceb0410674b02eba36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc) 2024-05-09T09:32:01.754 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:32:01 smithi016 podman[68469]: 2024-05-09 09:32:01.708944038 +0000 UTC m=+0.347554353 container start b3044a907f59388babfa72dad20e84d7f007840aa89eb3ceb0410674b02eba36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-09T09:32:02.042 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:32:01 smithi016 podman[68469]: 2024-05-09 09:32:01.780336092 +0000 UTC m=+0.418946400 container attach b3044a907f59388babfa72dad20e84d7f007840aa89eb3ceb0410674b02eba36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-96805dc) 2024-05-09T09:32:02.042 INFO:journalctl@ceph.osd.10.smithi016.stdout:May 09 09:32:02 smithi016 podman[68469]: 2024-05-09 09:32:02.017920977 +0000 UTC m=+0.656531275 container died b3044a907f59388babfa72dad20e84d7f007840aa89eb3ceb0410674b02eba36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-10-deactivate, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-09T09:32:02.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:02 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:02.008+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:32:02.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:02 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:02.008+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:32:02.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:02 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:02.008+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:32:02.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:02 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:02.008+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:32:02.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:02 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:02.008+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:32:02.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:02 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:02.008+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:32:02.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:02 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:02.008+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-05-09T09:31:33.872953+0000 front 2024-05-09T09:31:33.873083+0000 (oldest deadline 2024-05-09T09:31:56.172774+0000) 2024-05-09T09:32:02.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.10.service' 2024-05-09T09:32:02.686 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:02.687 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-09T09:32:02.687 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-09T09:32:02.687 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.11 2024-05-09T09:32:02.993 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:02 smithi170 systemd[1]: Stopping Ceph osd.11 for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:32:03.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:03 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:03.000+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:32:03.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:03 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:03.000+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:32:03.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:03 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:03.000+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:32:03.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:03 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:03.000+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:32:03.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:03 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:03.000+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:32:03.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:03 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:03.000+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:32:03.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:03 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:03.000+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-05-09T09:31:33.872953+0000 front 2024-05-09T09:31:33.873083+0000 (oldest deadline 2024-05-09T09:31:56.172774+0000) 2024-05-09T09:32:03.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:03 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:03.004+0000 7f23eef65700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:32:03.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:03 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:03.004+0000 7f23eef65700 -1 osd.11 98 *** Got signal Terminated *** 2024-05-09T09:32:03.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:03 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:03.004+0000 7f23eef65700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:32:04.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:04 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:04.050+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:32:04.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:04 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:04.050+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:32:04.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:04 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:04.050+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:32:04.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:04 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:04.050+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:32:04.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:04 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:04.050+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:32:04.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:04 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:04.050+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:32:04.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:04 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:04.050+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-05-09T09:31:33.872953+0000 front 2024-05-09T09:31:33.873083+0000 (oldest deadline 2024-05-09T09:31:56.172774+0000) 2024-05-09T09:32:05.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:05 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:05.048+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:32:05.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:05 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:05.048+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:32:05.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:05 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:05.048+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:32:05.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:05 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:05.048+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:32:05.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:05 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:05.048+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:32:05.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:05 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:05.048+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:32:05.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:05 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:05.048+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-05-09T09:31:33.872953+0000 front 2024-05-09T09:31:33.873083+0000 (oldest deadline 2024-05-09T09:31:56.172774+0000) 2024-05-09T09:32:06.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:06 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:06.075+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:32:06.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:06 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:06.075+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:32:06.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:06 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:06.075+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:32:06.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:06 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:06.075+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:32:06.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:06 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:06.075+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:32:06.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:06 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:06.075+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:32:06.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:06 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:06.075+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-05-09T09:31:33.872953+0000 front 2024-05-09T09:31:33.873083+0000 (oldest deadline 2024-05-09T09:31:56.172774+0000) 2024-05-09T09:32:06.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:06 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:06.075+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6820 osd.8 since back 2024-05-09T09:31:39.673839+0000 front 2024-05-09T09:31:39.673885+0000 (oldest deadline 2024-05-09T09:32:05.573864+0000) 2024-05-09T09:32:07.493 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:07 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:07.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T09:30:35.562606+0000 front 2024-05-09T09:30:35.562710+0000 (oldest deadline 2024-05-09T09:31:00.862834+0000) 2024-05-09T09:32:07.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:07 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:07.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-05-09T09:30:45.965322+0000 front 2024-05-09T09:30:45.965121+0000 (oldest deadline 2024-05-09T09:31:08.865145+0000) 2024-05-09T09:32:07.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:07 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:07.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-05-09T09:30:54.666641+0000 front 2024-05-09T09:30:54.666661+0000 (oldest deadline 2024-05-09T09:31:19.366876+0000) 2024-05-09T09:32:07.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:07 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:07.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-05-09T09:31:11.669707+0000 front 2024-05-09T09:31:11.669616+0000 (oldest deadline 2024-05-09T09:31:36.369507+0000) 2024-05-09T09:32:07.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:07 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:07.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-05-09T09:31:19.270637+0000 front 2024-05-09T09:31:19.270658+0000 (oldest deadline 2024-05-09T09:31:45.170764+0000) 2024-05-09T09:32:07.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:07 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:07.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T09:31:25.672106+0000 front 2024-05-09T09:31:25.672031+0000 (oldest deadline 2024-05-09T09:31:51.571803+0000) 2024-05-09T09:32:07.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:07 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:07.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-05-09T09:31:33.872953+0000 front 2024-05-09T09:31:33.873083+0000 (oldest deadline 2024-05-09T09:31:56.172774+0000) 2024-05-09T09:32:07.494 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:07 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11[59234]: 2024-05-09T09:32:07.070+0000 7f23e7551700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.170:6820 osd.8 since back 2024-05-09T09:31:39.673839+0000 front 2024-05-09T09:31:39.673885+0000 (oldest deadline 2024-05-09T09:32:05.573864+0000) 2024-05-09T09:32:08.771 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:08 smithi170 podman[69148]: 2024-05-09 09:32:08.501804769 +0000 UTC m=+5.615167494 container stop de818b5f651eda8244d35f169ea01dfddfe5abe673959b9992ddce8adf010083 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:32:08.771 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:08 smithi170 podman[69148]: 2024-05-09 09:32:08.52738964 +0000 UTC m=+5.640752375 container died de818b5f651eda8244d35f169ea01dfddfe5abe673959b9992ddce8adf010083 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-09T09:32:09.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:08 smithi170 podman[69148]: 2024-05-09 09:32:08.846355711 +0000 UTC m=+5.959718436 container cleanup de818b5f651eda8244d35f169ea01dfddfe5abe673959b9992ddce8adf010083 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:32:09.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:08 smithi170 bash[69148]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11 2024-05-09T09:32:09.244 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:08 smithi170 podman[69165]: 2024-05-09 09:32:08.997671895 +0000 UTC m=+0.488927226 container remove de818b5f651eda8244d35f169ea01dfddfe5abe673959b9992ddce8adf010083 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc) 2024-05-09T09:32:09.683 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:09 smithi170 podman[69254]: 2024-05-09 09:32:09.415111118 +0000 UTC m=+0.115066281 container create c7a84b1e665484a0f9878030deb958ec3421641538daf193620da48f2a39f760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:32:09.683 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:09 smithi170 podman[69254]: 2024-05-09 09:32:09.31815792 +0000 UTC m=+0.018113103 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:32:09.683 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:09 smithi170 podman[69254]: 2024-05-09 09:32:09.609383425 +0000 UTC m=+0.309338586 container init c7a84b1e665484a0f9878030deb958ec3421641538daf193620da48f2a39f760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T09:32:09.683 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:09 smithi170 podman[69254]: 2024-05-09 09:32:09.612658267 +0000 UTC m=+0.312613430 container start c7a84b1e665484a0f9878030deb958ec3421641538daf193620da48f2a39f760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11-deactivate, RELEASE=squid-96805dc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5) 2024-05-09T09:32:09.684 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:09 smithi170 podman[69254]: 2024-05-09 09:32:09.684307667 +0000 UTC m=+0.384262842 container attach c7a84b1e665484a0f9878030deb958ec3421641538daf193620da48f2a39f760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-09T09:32:09.993 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:09 smithi170 podman[69254]: 2024-05-09 09:32:09.90432061 +0000 UTC m=+0.604275774 container died c7a84b1e665484a0f9878030deb958ec3421641538daf193620da48f2a39f760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11-deactivate, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-09T09:32:10.428 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.11.service' 2024-05-09T09:32:10.451 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:10 smithi170 podman[69280]: 2024-05-09 09:32:10.404307641 +0000 UTC m=+0.490904285 container remove c7a84b1e665484a0f9878030deb958ec3421641538daf193620da48f2a39f760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507) 2024-05-09T09:32:10.451 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:10 smithi170 systemd[1]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.11.service: Deactivated successfully. 2024-05-09T09:32:10.451 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:10 smithi170 systemd[1]: Stopped Ceph osd.11 for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:32:10.452 INFO:journalctl@ceph.osd.11.smithi170.stdout:May 09 09:32:10 smithi170 systemd[1]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968@osd.11.service: Consumed 28.595s CPU time. 2024-05-09T09:32:10.467 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:10.467 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-09T09:32:10.468 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-09T09:32:10.468 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.a 2024-05-09T09:32:10.776 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:32:10 smithi012 systemd[1]: Stopping Ceph mds.a for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:32:11.234 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:32:10 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-a[72523]: 2024-05-09T09:32:10.776+0000 7fcf5345d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:32:11.234 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:32:10 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-a[72523]: 2024-05-09T09:32:10.776+0000 7fcf5345d700 -1 mds.a *** got signal Terminated *** 2024-05-09T09:32:14.193 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:32:13 smithi012 podman[95722]: 2024-05-09 09:32:13.938601658 +0000 UTC m=+3.261523565 container died 512799093bbe69d806a619485538f6e80226fd5130309c0d9a4974e296226263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:32:14.484 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:32:14 smithi012 podman[95722]: 2024-05-09 09:32:14.292237078 +0000 UTC m=+3.615158974 container cleanup 512799093bbe69d806a619485538f6e80226fd5130309c0d9a4974e296226263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:32:14.484 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 09:32:14 smithi012 bash[95722]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-a 2024-05-09T09:32:14.678 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.a.service' 2024-05-09T09:32:14.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:14.714 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-09T09:32:14.715 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-09T09:32:14.715 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.d 2024-05-09T09:32:15.234 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:32:14 smithi012 systemd[1]: Stopping Ceph mds.d for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:32:15.235 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:32:15 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-d[71567]: 2024-05-09T09:32:15.045+0000 7eff90815700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:32:15.235 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:32:15 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-d[71567]: 2024-05-09T09:32:15.045+0000 7eff90815700 -1 mds.d *** got signal Terminated *** 2024-05-09T09:32:17.062 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:32:16 smithi012 podman[95842]: 2024-05-09 09:32:16.802771447 +0000 UTC m=+1.858717382 container died b27c3a53d84589eac833b97c70c75f620c09353a6dcf13b9b3d53ee02ffeb554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, ceph=True, org.label-schema.schema-version=1.0) 2024-05-09T09:32:17.312 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:32:17 smithi012 podman[95842]: 2024-05-09 09:32:17.137609252 +0000 UTC m=+2.193555183 container cleanup b27c3a53d84589eac833b97c70c75f620c09353a6dcf13b9b3d53ee02ffeb554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True) 2024-05-09T09:32:17.313 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:32:17 smithi012 bash[95842]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-d 2024-05-09T09:32:17.565 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 09:32:17 smithi012 podman[95856]: 2024-05-09 09:32:17.313611197 +0000 UTC m=+0.509529537 container remove b27c3a53d84589eac833b97c70c75f620c09353a6dcf13b9b3d53ee02ffeb554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS) 2024-05-09T09:32:17.569 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.d.service' 2024-05-09T09:32:17.606 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:17.607 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-09T09:32:17.608 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-09T09:32:17.610 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.g 2024-05-09T09:32:17.984 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:32:17 smithi012 systemd[1]: Stopping Ceph mds.g for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:32:18.484 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:32:18 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-g[72174]: 2024-05-09T09:32:18.025+0000 7f9d52eed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:32:18.484 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:32:18 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-g[72174]: 2024-05-09T09:32:18.025+0000 7f9d52eed700 -1 mds.g *** got signal Terminated *** 2024-05-09T09:32:22.199 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:32:21 smithi012 podman[95963]: 2024-05-09 09:32:21.938788559 +0000 UTC m=+4.100204035 container died 087bd705b6e3329a6cd246ab493d57e0291890f904ac14609e84063c37297f48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc) 2024-05-09T09:32:22.484 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:32:22 smithi012 podman[95963]: 2024-05-09 09:32:22.291114651 +0000 UTC m=+4.452530119 container cleanup 087bd705b6e3329a6cd246ab493d57e0291890f904ac14609e84063c37297f48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-g, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T09:32:22.484 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 09:32:22 smithi012 bash[95963]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-g 2024-05-09T09:32:22.722 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.g.service' 2024-05-09T09:32:22.757 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:22.757 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-09T09:32:22.757 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-09T09:32:22.758 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.j 2024-05-09T09:32:23.234 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 09:32:22 smithi012 systemd[1]: Stopping Ceph mds.j for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:32:23.235 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 09:32:23 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-j[71216]: 2024-05-09T09:32:23.068+0000 7f08e80b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:32:23.235 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 09:32:23 smithi012 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-j[71216]: 2024-05-09T09:32:23.068+0000 7f08e80b6700 -1 mds.j *** got signal Terminated *** 2024-05-09T09:32:26.152 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 09:32:25 smithi012 podman[96082]: 2024-05-09 09:32:25.882561229 +0000 UTC m=+2.905605276 container died b9e335ae8c04a148b475ee394944130de70cad5d9cbe809bb83e04069bf6c840 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-j, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:32:26.419 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 09:32:26 smithi012 podman[96082]: 2024-05-09 09:32:26.227885196 +0000 UTC m=+3.250929231 container cleanup b9e335ae8c04a148b475ee394944130de70cad5d9cbe809bb83e04069bf6c840 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.build-date=20240507) 2024-05-09T09:32:26.420 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 09:32:26 smithi012 bash[96082]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-j 2024-05-09T09:32:26.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.j.service' 2024-05-09T09:32:26.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:26.680 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-09T09:32:26.680 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-09T09:32:26.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.b 2024-05-09T09:32:26.968 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:32:26 smithi016 systemd[1]: Stopping Ceph mds.b for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:32:27.253 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:32:26 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-b[64018]: 2024-05-09T09:32:26.968+0000 7f98b9275700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:32:27.253 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:32:26 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-b[64018]: 2024-05-09T09:32:26.968+0000 7f98b9275700 -1 mds.b *** got signal Terminated *** 2024-05-09T09:32:30.209 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:32:29 smithi016 podman[68588]: 2024-05-09 09:32:29.934436131 +0000 UTC m=+3.055052357 container died 19bdab9e4e0c64a83533b4d2eab29354ac91c7f2bd01fd90a5fd3af5047917e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-b, RELEASE=squid-96805dc, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:32:30.484 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:32:30 smithi016 podman[68588]: 2024-05-09 09:32:30.28456953 +0000 UTC m=+3.405185763 container cleanup 19bdab9e4e0c64a83533b4d2eab29354ac91c7f2bd01fd90a5fd3af5047917e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-09T09:32:30.485 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:32:30 smithi016 bash[68588]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-b 2024-05-09T09:32:30.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.b.service' 2024-05-09T09:32:30.748 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:32:30 smithi016 podman[68602]: 2024-05-09 09:32:30.485564698 +0000 UTC m=+0.542221912 container remove 19bdab9e4e0c64a83533b4d2eab29354ac91c7f2bd01fd90a5fd3af5047917e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:32:30.748 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:32:30 smithi016 systemd[1]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.b.service: Deactivated successfully. 2024-05-09T09:32:30.749 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:32:30 smithi016 systemd[1]: Stopped Ceph mds.b for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:32:30.749 INFO:journalctl@ceph.mds.b.smithi016.stdout:May 09 09:32:30 smithi016 systemd[1]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.b.service: Consumed 24.845s CPU time. 2024-05-09T09:32:30.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:30.762 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-09T09:32:30.762 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-09T09:32:30.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.e 2024-05-09T09:32:31.003 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:32:30 smithi016 systemd[1]: Stopping Ceph mds.e for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:32:31.503 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:32:31 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-e[63398]: 2024-05-09T09:32:31.096+0000 7f031268b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:32:31.503 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:32:31 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-e[63398]: 2024-05-09T09:32:31.096+0000 7f031268b700 -1 mds.e *** got signal Terminated *** 2024-05-09T09:32:34.253 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:32:33 smithi016 podman[68708]: 2024-05-09 09:32:33.936059676 +0000 UTC m=+2.940715215 container died c2904ab19e3e8b0f272f1336553c4df10a5d156094ac92bcea0bfff143408229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-e, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-09T09:32:34.663 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:32:34 smithi016 podman[68708]: 2024-05-09 09:32:34.429660077 +0000 UTC m=+3.434315606 container cleanup c2904ab19e3e8b0f272f1336553c4df10a5d156094ac92bcea0bfff143408229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-09T09:32:34.663 INFO:journalctl@ceph.mds.e.smithi016.stdout:May 09 09:32:34 smithi016 bash[68708]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-e 2024-05-09T09:32:34.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.e.service' 2024-05-09T09:32:34.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:34.862 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-09T09:32:34.862 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-09T09:32:34.862 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.h 2024-05-09T09:32:35.192 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:32:34 smithi016 systemd[1]: Stopping Ceph mds.h for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:32:35.503 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:32:35 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h[63713]: 2024-05-09T09:32:35.192+0000 7f5b995b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:32:35.503 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:32:35 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h[63713]: 2024-05-09T09:32:35.192+0000 7f5b995b2700 -1 mds.h *** got signal Terminated *** 2024-05-09T09:32:38.225 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:32:37 smithi016 podman[68843]: 2024-05-09 09:32:37.93597109 +0000 UTC m=+2.843394029 container died 881bf909381e7c4c7e76b79f821364d0e6f47466f2b8be5dd3138d8b78c7815b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-96805dc) 2024-05-09T09:32:38.503 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:32:38 smithi016 podman[68843]: 2024-05-09 09:32:38.300199678 +0000 UTC m=+3.207622614 container cleanup 881bf909381e7c4c7e76b79f821364d0e6f47466f2b8be5dd3138d8b78c7815b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T09:32:38.505 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:32:38 smithi016 bash[68843]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h 2024-05-09T09:32:38.505 INFO:journalctl@ceph.mds.h.smithi016.stdout:May 09 09:32:38 smithi016 podman[68857]: 2024-05-09 09:32:38.459663826 +0000 UTC m=+0.519018943 container remove 881bf909381e7c4c7e76b79f821364d0e6f47466f2b8be5dd3138d8b78c7815b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-09T09:32:38.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.h.service' 2024-05-09T09:32:38.718 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:38.719 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-09T09:32:38.719 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-09T09:32:38.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.k 2024-05-09T09:32:39.003 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:32:38 smithi016 systemd[1]: Stopping Ceph mds.k for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:32:39.503 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:32:39 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-k[63080]: 2024-05-09T09:32:39.071+0000 7f8856210700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:32:39.503 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:32:39 smithi016 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-k[63080]: 2024-05-09T09:32:39.071+0000 7f8856210700 -1 mds.k *** got signal Terminated *** 2024-05-09T09:32:42.195 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:32:41 smithi016 podman[68965]: 2024-05-09 09:32:41.93441429 +0000 UTC m=+2.993227704 container died e88be52650b2ec17e3b89ee7191d71da0171385df2031999ab40e20642c10476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-k, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-09T09:32:42.503 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:32:42 smithi016 podman[68965]: 2024-05-09 09:32:42.295674463 +0000 UTC m=+3.354487878 container cleanup e88be52650b2ec17e3b89ee7191d71da0171385df2031999ab40e20642c10476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-09T09:32:42.503 INFO:journalctl@ceph.mds.k.smithi016.stdout:May 09 09:32:42 smithi016 bash[68965]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-k 2024-05-09T09:32:42.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.k.service' 2024-05-09T09:32:42.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:42.764 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-09T09:32:42.764 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-09T09:32:42.764 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.c 2024-05-09T09:32:43.177 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:32:42 smithi170 systemd[1]: Stopping Ceph mds.c for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:32:43.450 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:32:43 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-c[64568]: 2024-05-09T09:32:43.387+0000 7f1915850700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:32:43.450 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:32:43 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-c[64568]: 2024-05-09T09:32:43.387+0000 7f1915850700 -1 mds.c *** got signal Terminated *** 2024-05-09T09:32:46.243 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:32:45 smithi170 podman[69359]: 2024-05-09 09:32:45.951065386 +0000 UTC m=+2.991979908 container died 552f76904383249cd56aa58991f00dde4b771bb171bb877655c4997fd08046aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-09T09:32:46.587 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:32:46 smithi170 podman[69359]: 2024-05-09 09:32:46.378427352 +0000 UTC m=+3.419341864 container cleanup 552f76904383249cd56aa58991f00dde4b771bb171bb877655c4997fd08046aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc) 2024-05-09T09:32:46.587 INFO:journalctl@ceph.mds.c.smithi170.stdout:May 09 09:32:46 smithi170 bash[69359]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-c 2024-05-09T09:32:46.805 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.c.service' 2024-05-09T09:32:46.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:46.842 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-09T09:32:46.842 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-09T09:32:46.843 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.f 2024-05-09T09:32:47.223 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:32:46 smithi170 systemd[1]: Stopping Ceph mds.f for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:32:47.493 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:32:47 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-f[64252]: 2024-05-09T09:32:47.223+0000 7fb6ab83a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:32:47.493 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:32:47 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-f[64252]: 2024-05-09T09:32:47.223+0000 7fb6ab83a700 -1 mds.f *** got signal Terminated *** 2024-05-09T09:32:50.190 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:32:49 smithi170 podman[69494]: 2024-05-09 09:32:49.935961696 +0000 UTC m=+2.876167632 container died d55293678f8990ff47ea365f0398ebcb339abdda236120f1cade439c01602a16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T09:32:50.474 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:32:50 smithi170 podman[69494]: 2024-05-09 09:32:50.282110322 +0000 UTC m=+3.222316263 container cleanup d55293678f8990ff47ea365f0398ebcb339abdda236120f1cade439c01602a16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-09T09:32:50.474 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:32:50 smithi170 bash[69494]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-f 2024-05-09T09:32:50.704 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.f.service' 2024-05-09T09:32:50.725 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:32:50 smithi170 podman[69508]: 2024-05-09 09:32:50.475036572 +0000 UTC m=+0.534344091 container remove d55293678f8990ff47ea365f0398ebcb339abdda236120f1cade439c01602a16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T09:32:50.726 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:32:50 smithi170 systemd[1]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.f.service: Deactivated successfully. 2024-05-09T09:32:50.726 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:32:50 smithi170 systemd[1]: Stopped Ceph mds.f for 770e12d0-0de4-11ef-bc98-c7b262605968. 2024-05-09T09:32:50.726 INFO:journalctl@ceph.mds.f.smithi170.stdout:May 09 09:32:50 smithi170 systemd[1]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.f.service: Consumed 28.548s CPU time. 2024-05-09T09:32:50.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:50.737 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-09T09:32:50.737 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-09T09:32:50.737 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.i 2024-05-09T09:32:50.993 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:32:50 smithi170 systemd[1]: Stopping Ceph mds.i for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:32:51.493 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:32:51 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i[64878]: 2024-05-09T09:32:51.077+0000 7f9b128f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:32:51.493 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:32:51 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i[64878]: 2024-05-09T09:32:51.077+0000 7f9b128f1700 -1 mds.i *** got signal Terminated *** 2024-05-09T09:32:54.203 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:32:53 smithi170 podman[69617]: 2024-05-09 09:32:53.944622677 +0000 UTC m=+2.979158678 container died 0aa652887d7e121377e0ed01150a2e34db3742164167b0e7527582cc53abc8f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:32:54.466 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:32:54 smithi170 podman[69617]: 2024-05-09 09:32:54.286348508 +0000 UTC m=+3.320884516 container cleanup 0aa652887d7e121377e0ed01150a2e34db3742164167b0e7527582cc53abc8f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T09:32:54.466 INFO:journalctl@ceph.mds.i.smithi170.stdout:May 09 09:32:54 smithi170 bash[69617]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-i 2024-05-09T09:32:54.677 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.i.service' 2024-05-09T09:32:54.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:54.710 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-09T09:32:54.710 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-09T09:32:54.711 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.l 2024-05-09T09:32:55.031 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:32:54 smithi170 systemd[1]: Stopping Ceph mds.l for 770e12d0-0de4-11ef-bc98-c7b262605968... 2024-05-09T09:32:55.493 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:32:55 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-l[63938]: 2024-05-09T09:32:55.031+0000 7f5df8f5c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:32:55.493 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:32:55 smithi170 ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-l[63938]: 2024-05-09T09:32:55.031+0000 7f5df8f5c700 -1 mds.l *** got signal Terminated *** 2024-05-09T09:32:58.231 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:32:57 smithi170 podman[69739]: 2024-05-09 09:32:57.941069674 +0000 UTC m=+3.007739007 container died 415b7f396a939f0327e29254712785229eeec599a22211a6eac42d426341905a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-l, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-09T09:32:58.493 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:32:58 smithi170 podman[69739]: 2024-05-09 09:32:58.298261895 +0000 UTC m=+3.364931230 container cleanup 415b7f396a939f0327e29254712785229eeec599a22211a6eac42d426341905a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-09T09:32:58.493 INFO:journalctl@ceph.mds.l.smithi170.stdout:May 09 09:32:58 smithi170 bash[69739]: ceph-770e12d0-0de4-11ef-bc98-c7b262605968-mds-l 2024-05-09T09:32:58.727 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-770e12d0-0de4-11ef-bc98-c7b262605968@mds.l.service' 2024-05-09T09:32:58.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:32:58.760 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-09T09:32:58.760 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-09T09:32:58.962 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:33:01.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-09T09:33:01.636 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:33:04.216 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-09T09:33:04.407 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:33:06.796 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:33:06.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:33:06.865 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:33:06.892 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-09T09:33:06.895 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722/remote/smithi012/crash 2024-05-09T09:33:06.895 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/crash -- . 2024-05-09T09:33:06.925 INFO:teuthology.orchestra.run.smithi012.stderr:tar: /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T09:33:06.926 INFO:teuthology.orchestra.run.smithi012.stderr:tar: Error is not recoverable: exiting now 2024-05-09T09:33:06.927 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722/remote/smithi016/crash 2024-05-09T09:33:06.928 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/crash -- . 2024-05-09T09:33:06.969 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T09:33:06.969 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-05-09T09:33:06.971 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722/remote/smithi170/crash 2024-05-09T09:33:06.972 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/crash -- . 2024-05-09T09:33:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T09:33:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now 2024-05-09T09:33:07.010 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-09T09:33:07.011 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-09T09:33:07.052 INFO:tasks.cephadm:Compressing logs... 2024-05-09T09:33:07.052 DEBUG:teuthology.orchestra.run.smithi012:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T09:33:07.094 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T09:33:07.097 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T09:33:07.120 INFO:teuthology.orchestra.run.smithi012.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.log 2024-05-09T09:33:07.120 INFO:teuthology.orchestra.run.smithi012.stderr:: No such file or directory 2024-05-09T09:33:07.120 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-09T09:33:07.131 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.log 2024-05-09T09:33:07.131 INFO:teuthology.orchestra.run.smithi170.stderr:: No such file or directory 2024-05-09T09:33:07.131 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-09T09:33:07.132 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-09T09:33:07.132 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-09T09:33:07.132 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-09T09:33:07.132 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-09T09:33:07.132 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-09T09:33:07.132 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-09T09:33:07.132 INFO:teuthology.orchestra.run.smithi170.stderr: 86.0% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-09T09:33:07.132 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-09T09:33:07.132 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-09T09:33:07.132 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.i.log: 90.2% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-09T09:33:07.132 INFO:teuthology.orchestra.run.smithi170.stderr: 92.5% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-09T09:33:07.133 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-09T09:33:07.133 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-09T09:33:07.133 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-09T09:33:07.133 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-09T09:33:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.j.log: /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-09T09:33:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mon.a.log 2024-05-09T09:33:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.6.log 2024-05-09T09:33:07.134 INFO:teuthology.orchestra.run.smithi012.stderr: 86.0% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-09T09:33:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mon.a.log: 90.0%gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-09T09:33:07.134 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-09T09:33:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-09T09:33:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.cephadm.log: 83.1% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-09T09:33:07.134 INFO:teuthology.orchestra.run.smithi012.stderr: 94.8% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.j.log.gzgzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-09T09:33:07.134 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-05-09T09:33:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-09T09:33:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.log 2024-05-09T09:33:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:: No such file or directory 2024-05-09T09:33:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-09T09:33:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-09T09:33:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-09T09:33:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-09T09:33:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-09T09:33:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-09T09:33:07.138 INFO:teuthology.orchestra.run.smithi016.stderr: 86.0% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-09T09:33:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-09T09:33:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-09T09:33:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.h.log: 90.1% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.audit.log.gzgzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-09T09:33:07.138 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-05-09T09:33:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-09T09:33:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.10.log: /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.cephadm.log: 83.0% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-09T09:33:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-09T09:33:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.e.log: 92.4% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-09T09:33:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-09T09:33:07.141 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-09T09:33:07.141 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-09T09:33:07.141 INFO:teuthology.orchestra.run.smithi170.stderr: 82.9% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-09T09:33:07.141 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-09T09:33:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-09T09:33:07.142 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-09T09:33:07.143 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-09T09:33:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.d.log: 94.8% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-09T09:33:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-09T09:33:07.148 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-09T09:33:07.149 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T09:33:07.149 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-09T09:33:07.155 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T09:33:07.157 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T09:33:07.170 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-09T09:33:07.172 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T09:33:07.174 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T09:33:07.176 INFO:teuthology.orchestra.run.smithi170.stderr: 91.0% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-09T09:33:07.184 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/cephadm.log: 93.3%/var/log/ceph/ceph-client.admin.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T09:33:07.185 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-09T09:33:07.351 INFO:teuthology.orchestra.run.smithi012.stderr: 90.0% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-09T09:33:07.471 INFO:teuthology.orchestra.run.smithi170.stderr: 91.9% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-09T09:33:07.552 INFO:teuthology.orchestra.run.smithi012.stderr: 95.0% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-09T09:33:07.730 INFO:teuthology.orchestra.run.smithi016.stderr: 91.9% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-09T09:33:07.875 INFO:teuthology.orchestra.run.smithi016.stderr: 95.3% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-09T09:33:07.907 INFO:teuthology.orchestra.run.smithi012.stderr: 90.6% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-09T09:33:07.927 INFO:teuthology.orchestra.run.smithi016.stderr: 95.3% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-09T09:33:07.961 INFO:teuthology.orchestra.run.smithi170.stderr: 95.4% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-09T09:33:09.378 INFO:teuthology.orchestra.run.smithi016.stderr: 95.4% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-09T09:33:11.319 INFO:teuthology.orchestra.run.smithi016.stderr: 91.9% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-09T09:33:12.836 INFO:teuthology.orchestra.run.smithi170.stderr: 91.9% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-09T09:33:13.099 INFO:teuthology.orchestra.run.smithi016.stderr: 92.2% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-09T09:33:13.663 INFO:teuthology.orchestra.run.smithi012.stderr: 94.1% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-09T09:33:13.844 INFO:teuthology.orchestra.run.smithi016.stderr: 91.6% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-09T09:33:14.321 INFO:teuthology.orchestra.run.smithi170.stderr: 92.1% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-09T09:33:14.608 INFO:teuthology.orchestra.run.smithi012.stderr: 91.6% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-09T09:33:14.817 INFO:teuthology.orchestra.run.smithi016.stderr: 93.9% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-09T09:33:15.200 INFO:teuthology.orchestra.run.smithi012.stderr: 91.8% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-09T09:33:15.951 INFO:teuthology.orchestra.run.smithi012.stderr: 91.6% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-09T09:33:16.230 INFO:teuthology.orchestra.run.smithi170.stderr: 91.8% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-09T09:33:16.280 INFO:teuthology.orchestra.run.smithi012.stderr: 91.9% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-09T09:33:16.396 INFO:teuthology.orchestra.run.smithi170.stderr: 93.9% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-09T09:33:16.456 INFO:teuthology.orchestra.run.smithi016.stderr: 91.8% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-09T09:33:16.458 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-05-09T09:33:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m9.347s 2024-05-09T09:33:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m36.592s 2024-05-09T09:33:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m1.065s 2024-05-09T09:33:17.539 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-09T09:33:17.668 INFO:teuthology.orchestra.run.smithi170.stderr: 93.9% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-09T09:33:20.531 INFO:teuthology.orchestra.run.smithi012.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-09T09:33:20.533 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-05-09T09:33:20.533 INFO:teuthology.orchestra.run.smithi012.stderr:real 0m13.424s 2024-05-09T09:33:20.533 INFO:teuthology.orchestra.run.smithi012.stderr:user 0m53.131s 2024-05-09T09:33:20.533 INFO:teuthology.orchestra.run.smithi012.stderr:sys 0m1.499s 2024-05-09T09:33:31.234 INFO:teuthology.orchestra.run.smithi170.stderr: 93.6% -- replaced with /var/log/ceph/770e12d0-0de4-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-09T09:33:31.236 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-05-09T09:33:31.236 INFO:teuthology.orchestra.run.smithi170.stderr:real 0m24.125s 2024-05-09T09:33:31.236 INFO:teuthology.orchestra.run.smithi170.stderr:user 1m14.862s 2024-05-09T09:33:31.237 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m2.196s 2024-05-09T09:33:31.237 INFO:tasks.cephadm:Archiving logs... 2024-05-09T09:33:31.237 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722/remote/smithi012/log 2024-05-09T09:33:31.238 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T09:33:38.249 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722/remote/smithi016/log 2024-05-09T09:33:38.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T09:33:43.372 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722/remote/smithi170/log 2024-05-09T09:33:43.373 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T09:33:53.062 INFO:tasks.cephadm:Removing cluster... 2024-05-09T09:33:53.063 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 --force 2024-05-09T09:33:53.266 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:33:53.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 --force 2024-05-09T09:33:53.784 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:33:54.095 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 770e12d0-0de4-11ef-bc98-c7b262605968 --force 2024-05-09T09:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 770e12d0-0de4-11ef-bc98-c7b262605968 2024-05-09T09:33:54.625 INFO:tasks.cephadm:Removing cephadm ... 2024-05-09T09:33:54.626 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T09:33:54.646 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T09:33:54.664 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T09:33:54.681 INFO:tasks.cephadm:Teardown complete 2024-05-09T09:33:54.681 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T09:33:54.694 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T09:33:54.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T09:33:54.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T09:33:54.708 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T09:33:54.818 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T09:33:54.829 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T09:33:54.829 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T09:33:54.832 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T09:33:54.836 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T09:33:54.848 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-05-09T09:33:54.850 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-05-09T09:33:54.851 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-05-09T09:33:54.854 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T09:33:54.854 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-05-09T09:33:54.854 INFO:teuthology.orchestra.run.smithi012.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 +457us[ +477us] +/- 52ms 2024-05-09T09:33:54.854 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 -775us[ -755us] +/- 55ms 2024-05-09T09:33:54.854 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +407us[ +427us] +/- 73ms 2024-05-09T09:33:54.855 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:33:54.859 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T09:33:54.859 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-05-09T09:33:54.859 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 +425us[ +425us] +/- 52ms 2024-05-09T09:33:54.859 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 69 -789us[ -790us] +/- 55ms 2024-05-09T09:33:54.859 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 67 +392us[ +392us] +/- 72ms 2024-05-09T09:33:54.860 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:33:54.860 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T09:33:54.860 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-05-09T09:33:54.860 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 +453us[ +441us] +/- 52ms 2024-05-09T09:33:54.860 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -778us[ -778us] +/- 55ms 2024-05-09T09:33:54.861 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +400us[ +400us] +/- 73ms 2024-05-09T09:33:54.861 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:33:54.861 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T09:33:54.880 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T09:33:54.881 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T09:33:54.897 DEBUG:teuthology.orchestra.run.smithi012:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T09:33:54.985 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T09:33:55.060 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T09:33:55.125 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T09:33:55.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:33:55.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T09:33:55.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:33:55.188 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T09:33:55.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:33:55.216 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T09:33:55.227 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T09:33:55.259 INFO:teuthology.task.internal:Duration was 1572.859477 seconds 2024-05-09T09:33:55.260 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T09:33:55.324 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T09:33:55.325 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T09:33:55.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T09:33:55.330 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T09:33:55.369 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T09:33:55.370 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T09:33:55.372 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T09:33:55.798 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T09:33:55.798 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-05-09T09:33:55.798 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T09:33:55.822 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-05-09T09:33:55.822 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T09:33:55.851 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-05-09T09:33:55.851 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T09:33:55.879 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T09:33:55.879 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T09:33:55.881 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T09:33:55.896 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T09:33:55.946 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T09:33:55.947 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T09:33:55.951 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T09:33:55.968 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T09:33:56.352 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T09:33:56.364 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T09:33:56.365 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T09:33:56.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T09:33:56.417 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T09:33:56.443 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T09:33:56.457 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:33:56.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:33:56.463 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:33:56.482 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-05-09T09:33:56.484 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-05-09T09:33:56.507 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-05-09T09:33:56.551 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:33:56.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:33:56.567 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:33:56.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:33:56.582 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:33:56.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:33:56.606 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T09:33:56.618 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T09:33:56.619 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722/remote/smithi012 2024-05-09T09:33:56.619 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T09:33:56.663 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722/remote/smithi016 2024-05-09T09:33:56.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T09:33:56.706 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698722/remote/smithi170 2024-05-09T09:33:56.706 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T09:33:56.743 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T09:33:56.744 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T09:33:56.746 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T09:33:56.748 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T09:33:56.793 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T09:33:56.807 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T09:33:56.807 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T09:33:56.881 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T09:33:56.882 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T09:33:56.884 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T09:33:56.886 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T09:33:56.898 INFO:teuthology.orchestra.run.smithi012.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 09:33 /home/ubuntu/cephtest 2024-05-09T09:33:56.900 INFO:teuthology.orchestra.run.smithi016.stdout: 265295 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 09:33 /home/ubuntu/cephtest 2024-05-09T09:33:56.902 INFO:teuthology.orchestra.run.smithi170.stdout: 265834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 09:33 /home/ubuntu/cephtest 2024-05-09T09:33:56.903 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T09:33:56.920 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T09:33:56.981 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/norstats}} duration: 1572.8594772815704 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-05-09T09:33:56.981 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T09:33:57.076 INFO:teuthology.run:pass